2007 Meeting of GRP Working Group on Data Management and Analysis
The City College of New York, New York, NY, USA
5 - 7 September 2007
The WGDMA Chairman opened the meeting with an overview of the agenda and highlights of some of the meeting objectives: (1) continue work towards a coordinated re-processing of all the data products (specifically improvements in radiance calibrations and analysis algorithms), (2) develop criteria for selecting common ancillary data products to be used by all analyses and (3) defining the features of common data products. The meeting sessions were organized into Plenary Sessions on the first half-day and during the last 1.5 hr of the third morning with the remainder of the 3.5 day schedule taken up with separate meetings of the GPCP and ISCCP (with GACP, SRB and SeaFlux) groups.
The chairman also summarized salient discussions at GRP, GEWEX and WCRP. The new chairman of GRP, starting in 2008, is Chris Kummerow from Colorado State University. In preparing for the transition, an effort is being made to collect brief summaries from each of the global data projects stating their goals/objectives, organization, summary of data products and achievements. A representative of the SeaFlux activity has now joined WGDMA as it evolves towards producing a global data product (possibly more than one) within the next year or two: in the nearer-term, this study can contribute to selecting a common atmospheric temperature-humidity product as well as providing a common SST product. The newer LandFlux activity is following a similar path but may, in the nearer term, contribute to obtaining common land surface products (albedo and skin temperature). The chairman also noted several other international activities concerning organization of data systems by WOAP across WCRP activities and by GEOSS across all Earth observation activities.
W. Rossow summarized the status of the International Satellite Cloud Climatology Project (ISCCP). Radiance calibration is complete through December 2006 and geostationary normalization to polar orbiter radiances is complete through May 2007. Radiance and ancillary datasets have been delivered through December 2006 (23.5 years) and cloud products have been delivered through June 2006 (23 years). Processing is underway for the last six months of 2006 but it has been delayed by problems associated with the introduction of several new satellites. The products from July 2005 through 2006 do not yet include the FY-2C data as the radiance data had only recently been delivered. Funding for the ISCCP centers is in place except for the Global Processing Center; a renewal proposal has been submitted to NASA (after the meeting, approval for GPC funding was received). If this proposal is accepted, then work will go forward to switch the derivation of the cloud products from the B3 dataset (30 km sampling) to the B1 dataset (10 km sampling, within a factor of 2 of full infrared resolution), which would make the products much more useful for cloud process studies. Such a switch would delay the beginning of ISCCP re-processing by about one year to early 2009. Also, it was noted that two proposals had been submitted to support re-engineering the ISCCP processing software to allow its conversion to an operational system: NOAA funding has been received to transfer the ISCCP processing to the NOAA National Climatic Data Center (NCDC) and NASA funding is pending. The 23-yr data record for ISCCP shows a partial recovery of global mean cloud cover from its minimum in 1999-2000 of about 64.5% to 66.5% (the maximum of 69% occurred in 1986-1987) and a small increase (about 0.2) in global mean cloud optical thickness over the last six years. The list of products now produced by ISCCP includes the radiance products (B3 and B1U), the ancillary atmosphere and snow-ice products, the main cloud products (DX, D1 and D2) and several diagnostic products (radiative flux profiles, FD, mesoscale convective tracking, CT, tropical and midlatitude “weather states”, WS, midlatitude cyclone tracking composites, CY, and cloud particle sizes, RE).
M. Mishchenko summarized the status of the Global Aerosol Climatology Project (GACP). Employing the two solar wavelength channels on the afternoon polar orbiting satellites from the ISCCP DX product, a climatology of monthly mean aerosol optical depth (AOD) and Angstrom coefficient (size index) over global oceans has been produced for the period August 1981 through June 2005 (soon to be extended through 2006). Recent corrections were made to allow larger AOD values to occur and to account for the annual variation of solar illumination because of sun-Earth distance changes. The record clearly shows the two major volcanic events (El Chichon and Mt Pinatubo) and the subsequent decrease of total AOD; subtracting the SAGE stratospheric aerosol record can isolate the tropospheric aerosol component. Also notable now in the longer record is a gradual decline of total AOD from about 1995: since SAGE indicates that the stratospheric AOD is < 0.01 during this time period, this change is caused by a nearly 30% decrease in tropospheric aerosols over the last decade. This trend is also apparent in the new PATMOS product. Future plans include a detailed analysis of the retrieval uncertainties by comparing results from GACP, MODIS and MISR over the full range of space-time scales. A key aspect of this investigation is application of the GACP, MODIS and MISR retrievals to data from the aircraft RSP instrument, which makes measurements similar to all three types of instruments, to identify specific problem situations. The launch of the precision polarimeter (APS) on the Glory mission provides another opportunity to investigate similarities and differences among these products globally. Finally, a study will be made to see whether these products can be integrated into a single consistent climatology. Funding for the GACP processing center was renewed (after a 1-yr hiatus) through 2010.
C. Clayson summarized the activities of the SeaFlux project. The main focus of activity in the past year was completing a detailed comparison of available global surface turbulent flux products for the year 1999, where advanced satellite instruments and a large collection of in situ measurements were also available. Comparisons are also being made with NWP reanalysis products and moored array results from other years. Results are being compiled after a workshop in Amsterdam and a paper is being written. The global products include GSSTF2 (not currently in production but a renewal proposal has been submitted to NASA), INFREMER, J-OFURO, HOAPS-3 and OAFlux (the latter relying on ship and buoy measurements). The global comparisons show a spread of the flux products of about 30 Wm-2 for latent heat flux and 20-40 Wm-2 for the sensible heat flux in the tropics and at high latitudes; the ranges are larger if the reanalyses are included. The reanalyses show more latitudinal variation of sensible heat fluxes than most of the data products. A particular need to obtain the desired fluxes is for a skin-SST product that describes the diurnal cycle and its dependence on windspeed and cloud-radiative effects; no such product exists but the first version of an experimental product has been completed using SRB surface shortwave fluxes (based on ISCCP cloud properties). The best surface windspeed dataset appears to be the NOAA Blended product; studies are still underway to select the best estimates of near-surface air temperature and humidity. Production of a new product is planned for spring 2008.
R. Adler summarized the status of the Global Precipitation Climatology Project (GPCP). Production continued extending the monthly and pentad datasets to cover the period February 1979 through June 2007; the daily product now covers 10 years, October 1996 through June 2007. GPCP is cited in over 700 papers now. The GRP Assessment Report has been completed in is now being published. Analysis of the long-term variations of tropical precipitation shows a small trend (about 3% over almost 30 years), if proper account is taken of other signals in the record associated with volcanic and ENSO events. This systematic variation appears to be limited to lower latitudes. There is also some evidence of partially offsetting changes over ocean and land. Work continues on a Version 3 analysis to solve several problems with the current products: (1) bias errors in the gauge analysis in mountainous areas, (2) updating the microwave algorithm over ocean to a more modern method that allows for finer time resolution, (3) inclusion of more recent instruments in the merged product and (4) eliminating the “boundary” in the gauge analysis by using a more homogeneous sample population. In addition, changes are being investigated to add rain/snow discrimination and increase the space-time resolution over the whole record. The goal for this meeting is to make firm plans for implementation of these changes by the end of next year.
P. Stackhouse summarized the status of the Surface Radiation Budget project (SRB). Processing is complete from July 1983 through June 2005. The major change introduced in the products over the past year was the switch from GEOS-1 to GEOS-4, which allowed for the extension of the data record. Additional small changes corrected for filling and angle interpolation problems associated with shortwave fluxes and missing sea ice in the ISCCP products. Evaluation activities continued, focusing on more extensive comparisons with the BSRN (and GEBA) and the new CERES products, as well as participating in the on-going Radiation Assessment. The GEBA shortwave flux comparisons show a bias of SRB of less than 1 Wm-2 (well within the uncertainty of GEBA data) and rms differences of monthly mean values of 23 Wm-2. The time series of the interannual anomalies of colocated values are very well correlated (coefficient > 0.8). The shortwave bias with the BSRN site measurements is somewhat larger (8 Wm-2) but still within the uncertainty; the longwave bias is about 2 Wm-2. Rms differences of monthly mean values are 15-25 Wm-2 (shortwave) and 12-17 Wm-2 (longwave). Similar agreement is found with the shorter CERES record. Some algorithm modifications are being worked on to reduce effects of unrealistic aerosol assumptions in the polar regions and flaws in the angle correction models being used (these will be updated to the newer CERES models in the future). The near-term focus of efforts are on the Radiation Assessment. Funding to continue production is pending. If funding is renewed, then all the input data records will be reviewed to identify any anomalous interannual variations, the angle models will be revised, the cloud-layering scheme in the longwave calculations will be revised after study of CloudSat and Calipso data and the products revised to exploit the higher resolution from the B1-version of ISCCP (after the meeting funding approval was received). New products are also being considered, including UV and “window” fluxes. Re-processing could commence in late 2009.
K. Knapp from NOAA NCDC reported on lessons learned in the rescue of the alternate ISCCP radiance dataset (Stage B1 with 10 km sampling instead of 30 km); this effort also involves going back to the operational archives to find images missed in the original ISCCP collection and datasets from before the start of ISCCP in July 1983 (an article about this was published in BAMS). This effort required deciphering multiple data formats (finding format documentation, finding or writing READ programs) and multiple navigation algorithms and finding all the needed metadata; the formats and navigation procedures are not only different from one agency to another but change with time even for a single satellite operator. Some satellite datasets include information for different navigation procedures without explanation of what to do if the results disagree, which they sometimes do. Data covering the period from late 1979 to current day comes from 29 different geostationary satellites and 14 different polar orbiting satellites. This effort highlighted the critical importance of storing complete metadata within the dataset and for documenting the reasons why the data is structured the way it is. Frequent use of datasets can prevent the loss of important information. K. Knapp also recommended the creation of a common set of datasets from the set including ISCCP, GPCP, SRB and GACP and the use of a modern, widely-used format such as netCDF.
W. Rossow listed the common (used by at least two projects) ancillary datasets that are needed: (1) topography and land/water mask, (2) ozone (probably TOMS, except recently), (3) snow/sea ice (probably the NSIDC products), (4) surface albedo (for ocean, albedo is based on theory with empirical correction, for land the MODIS spectral-angle models applied to ISCCP visible/near-IR radiances), (5) surface infrared emissivity (check new AIRS products), (6) surface skin temperature, (7) aerosol climatology or GACP product, and (8) atmospheric temperature and humidity profiles. He also illustrated some of the issues in choosing a common dataset using the case for the atmospheric temperature and humidity profiles. In this case, although the magnitude of the differences among the various products is relatively small (2-3 K temperature, 1-2 mm precipitable water), the differences are systematic. Notably, none of the available atmospheric products reproduces the diurnal temperature amplitudes observed at surface weather stations. There are also systematic differences in the upper tropospheric temperatures over oceans and over high topography. Moreover, there are interannual discontinuities in some of the products produced by changes of input or analysis methodology. All of these flaws introduce corresponding flaws in the cloud and radiation products particularly. Hence, the best choice of a common dataset must consider these systematic problems as well as general accuracy.
Summary of GPCP Sessions
R. Adler opened the GPCP Sessions by summarizing the project status and outlining the issues and goals for the meeting. The key issues are: (1) reducing bias errors in gauge analysis in mountainous regions, (2) revising the ocean-microwave algorithm to provide higher time resolution results, (3) eliminating the data boundaries caused by changing gauge and satellite analysis methods and (4) considering the inclusion of AMSR, AMSU and TRMM data in the product. The goal is to produce a plan of work towards Version 3.
R. Ferraro summarized activities at the Microwave-Land Center. Production of products has been routine; however, there are problems with a new noise source in one channel of the SSM/I on F15 and difficulties with the calibration (and change of frequency for one channel) for the newer SSMIS instruments on F16 and F17 (the latter two have not been used in the GPCP products as yet). An AMSU algorithm is operational on all polar orbiters, including METOP-A; recent improvements include addition of an emission-based estimate where the scattering-based method does not converge or makes no detection, adjustments for view-angle bias and a AMSU-B-based estimate of rainrates in coastal areas. These changes make the AMSU-based results more similar to SSM/I-based results. Work continues on an AMSU-based snowfall algorithm. He also reported on studies of enhancements to the GPROF algorithm for cold season precipitation and of developments of an SSMIS-based precipitation regime classification scheme. L. Chiu summarized activities at the Microwave-Ocean Center. Production of products has been routine. New versions of the product have been introduced because of the availability of updated SSM/I calibrations; the latest version tends to increase the rainfall rate in regions with already high rainfall rates and to change precipitation near coastlines. There are also significant differences in the long-term trends obtained with the latest version. J. Janowiak summarized activities at the Geosynchronous Center. Production of products has been routine. He reported on the investigation of the use of the ISCCP B1 (10 km spatially sampled radiances) in the IR-precipitation algorithm; using this product would provide GPCP with uniform set of IR radiances from geosynchronous satellites back to 1983, instead of the four different versions used until now, which would help reduce inhomogeneities in the GPCP product. The 10km sampling captures nearly all of the detail shown in the original 4km “resolution” data and produces products very similar to the current ones. U. Schneider summarized activities at the Gauge Center (GPCC). Production of products has been routine. To meet the range of user requirements, the center now provides two more rapid products (5-day and 2-month latency), the second for GPCP, and two retrospective products, a “full data” reanalysis (maximizing density) and a climatology (maximizing sampling homogeneity), the latter now covering a 50-yr period. Notable changes to the products in the past year include the introduction of a weather-dependent gauge correction (instead of using climatology) and discrimination of the precipitation amounts into three types (liquid, ice, mixed). New versions (reanalyses with added stations) of the last three products are planned for the next year. G. Huffman summarized activities at the Merge Center. Production of products has been routine. The high latitude portion of the merged product is now based on AIRS instead of HIRS. In addition to continued analysis of the long-term variations shown in the GPCP product (especially comparisons with estimates of evaporation), the main focus of activities in the next year will be on preparing to implement revised algorithms and products for Version 3.
A. Gruber provided an update on the recently completed Precipitation Product Assessment. Now that it has been reviewed by the full GRP membership and revised accordingly, it has been submitted for publication by WMO. The Assessment Report has been posted online (http://cics.umd.edu/GPCP and http://www.isac.cnr.it/~ipwg/reports.html); a short summary has been published in the GEWEX News and another has been submitted to GRL. There was a discussion about the need for and timing of a second assessment: it seems reasonable to expect that the PEHRPP activity, if it goes as planned, might address some of the issues not addressed in this assessment and that the next formal assessment of GPCP should follow the release of Version 3.
U. Schneider led off the discussion of gauge datasets by describing the new products being produced at Deutcher Wetterdienst (Global Precipitation Climatology Centre). A major effort is still on-going to expand the number of stations reporting and the number of reports obtained from them. The average number of reporting stations now available over the period 1951-2000 is over 25,000; there is a period from 1986-1995, where more than 40,000 stations are available. The 50-yr Climatology Product currently uses the most continuous records from a subset that is about 9,000 stations and the Monitoring Product supplied to GPCP (with a 2-month lag) uses about 5,000-6,000 stations. The total holdings continue to grow, now exceeding 19 million station-months; the holdings have been extended back to 1901 as well. Version 2 of the Monitoring Product has now been extended back to 1982 and expanded to 6,000-7,500 stations. J. Janowiak reported on a new NOAA CPC gauge analysis procedure. Currently there are about 15,000 stations reporting in real-time over the GTS system. A revised QC procedure looks for spurious zero values and values that are too large, as well as checking against nearest neighbor values, historical values from each station and concurrent estimates from satellite products and model forecasts. The newly developed operational analysis is an optimum interpolation approach that consistently out-performs other fast methods.
C. Kummerow presented an update of ongoing development and evaluation of the GPROF algorithm used to analyze passive microwave measurements for TRMM. He started by noting that, although the current version of GPROF is used to produce a number of precipitation products, these products differ because of different attributes and calibration of the various microwave instruments. The main upgrade to the GPROF algorithm being worked on replaces the cloud resolving model generated a priori database with one based on TRMM PR and TMI measurements. This new algorithm will be ready in time to be used by GPCP in its planned reprocessing. However, this still leaves problems with warm rain over land and cold precipitation unaddressed. Also being developed are parametric algorithms that do not require tuning (although radiance calibration becomes more important). Such an analysis will also work over land and sea ice if accurate surface emissivity models exist; empirical models can be developed from non-precipitating observations. R. Ferraro reported on continuing studies to improve a microwave-based snowfall algorithm and to develop regime-classification approaches for microwave precipitation algorithms that include snowfall cases. The former are based on the higher frequency measurements by the microwave sounders; the latter focuses on use of all the channels available on SSMIS to determine relational patterns that signify different kinds of precipitating systems.
L. Chiu proposed continuing the simple emission-based microwave precipitation product as a heritage baseline for newer products; this would be especially useful if a uniformly calibrated microwave dataset were to be available, like the one being produced at Colorado State University. The baseline could be provided at 2.5 and 5 degree resolution but, using calibration to TRMM starting in 1998, a 1 degree product could also be produced.
Several different “high” resolution precipitation products have been developed and are now being evaluated. P. Arking described a systematic evaluation of these new products (PEHRPP, Program for Evaluation of High Resolution Precipitation Products) proposed by the IPWG. These products generally rely on combining passive microwave measurements from polar orbiting satellites (lower time-space resolutions) with higher time-space resolution infrared measurements from geostationary satellites. This evaluation would be based on dense rain gauge and precipitation radar networks; it is expected that these products can represent the space-time variations of precipitation realistically, even if the absolute precipitation rates are dependent on different assumptions about precipitation characteristics. The first workshop is planned for 3-5 December 2007 in Geneva. G. Huffman described the TRMM Multi-satellite Precipitation Analysis developed at NASA GSFC (25 km, 3 hr). This method first obtains the maximum space-time sampling possible by combining results from all available microwave instruments, calibrated against TRMM; the use of infrared measurements is still necessary but minimized. The infrared measurements are calibrated against matched passive microwave using monthly statistics. Although the monthly statistical matches look good, instantaneous skill is only modest; hence biases are reasonably small (10-20%) but rms differences are larger (30-50%). Retrievals at higher latitudes are more problematic and harder to evaluate. The full complement of “A-train” satellites, especially with CloudSat, are being studied to improve an AIRS-based approach. K. Hsu described the PERSIANN product developed at UC Irvine (25 km, 1 hr), focusing on new training activities in the tropical Americas region. Comparing collocated microwave- and infrared-based products mapped at different spatial resolutions shows that the correspondence of the two products increases significantly at spatial scales > 100 km for daily results. Aggregating results at smaller spatial scales over time (e.g., 10 days) does not improve the correspondence as much as aggregation over spatial scale. He also showed that the diurnal cycle of precipitation obtained by PERSIANN is significantly improved by a TRMM-based adjustment. He also presented a snowfall estimation methodology based on the same IR satellite measurements (precipitation is considered to be snow when surface air temperatures are below 0°C). J. Janowiak first described improvements made to AMSU-based retrievals over land and ocean and work to incorporate METOP-A data into the CMORPH product (25 km, 1 hr), which increases the available time sampling by passive microwave. He also described using CMORPH to disaggregate gauge data that has been interpolated to a full map grid. NOAA CPC is working towards a fully-unified set of products from disparate sources that are consistent with each other. T. Smith described another effort to merge the maximum amount of microwave-based precipitation estimates, including SSM/I, AMSR-E, AMSU and TMI.; however, each of these products currently uses a different algorithm. Five years of data (2002-2006) have been processed to date after solving a number of minor data quality, software and hardware problems.
The GPCP breakout session ended with a summary discussion of progress and remaining issues; this summary is given below as part of the final plenary session.
Summary of ISCCP, GACP, SRB and SEAFLUX Sessions
C. Clayson reiterated the results of the comparison by SeaFlux of the available global ocean surface turbulent flux datasets that indicate the need for improved quality SST and near-surface meteorology data to reduce the uncertainties. Studies of the latter issue, which SeaFlux shares in common with SRB, will be coordinated with SRB studies of the same issue. She described efforts to obtain a better SST product: the need is for a product that estimates true skin SST, instead of so-called bulk SST, which is a temperature more representative of the ocean mixed layer, and describes its diurnal cycle and its dependence on windspeed and cloud-radiative effects. Using bulk SST values can cause systematic errors in the fluxes of up to 10% (a 1 K temperature error can cause a 30 Wm-2 flux error); diurnal variations can reach 3 K. Accounting for the diurnal variations of skin SST can reduce the rms errors by a factor of three. A methodology based on bulk SST products that modifies them to account for skin differences using surface windspeed at solar radiation inputs shows quite good success at capturing the skin SST variations as compared with detailed experimental (ship-based measurements) The most significant changes that this analysis produces is a decrease in the east-west temperature gradient in the tropical Pacific but increased SST values during El Nino events. It looks as though the SeaFlux SST values should be used by all the other projects.
M. Mishchenko described validation studies of GACP in more detail. Comparisons of the aerosol optical depth values over oceans show excellent agreement but also demonstrate the sensitivity of the results to assumed scattering in the ocean by particulates. Similar comparisons with other satellite-based products from MODIS and MISR do not show any reduction in the scatter of the results. Detailed regional comparisons among GACP, MODIS and MISR also show that the similarities and differences in these products exhibit strong regional dependence. He outlined in more detail future plans for an in-depth analysis of the retrieval uncertainties by comparing results from GACP, MODIS and MISR and for investigating whether these results can be combined into a single consistent climatology. No dependencies on ancillary products employed by the other projects were identified, although plans are to merge the GACP product with the new ISCCP cloud particle size product.
P. Stackhouse provided more detail on the status of SRB processing, focusing on the effects of ancillary datasets. Changes to aerosol optical depth values, particularly in the polar regions produced improved shortwave fluxes. A revised treatment of the near-surface air and land skin temperature differences also improved downwelling longwave fluxes, especially in arid regions. The effects of the switch of atmospheric data from GEOS-1 to GEOS-4 were illustrated; the new MERRA product being produced by NASA is being studied as a replacement for GEOS-4. This activity is being coordinated with comparison studies using the new CERES-based surface radiation flux product. Coordinated evaluations of the atmospheric ancillary data with ISCCP, SeaFlux and LandFlux were planned in discussion. Also, solar “constant”, aerosol and trace gas ancillary products are being re-considered. Re-processing should commence in late 2009. P. Stackhouse also provided more detail on the completion of the Radiative Flux Assessment activity: writing of the report is now underway but also final statistical analyses are being conducted based on an online version of all the major datasets, which will be released to the public afterwards. One study compared the two surface radiation products (SRB and ISCCP-FD) with the IPCC models showing that the differences between the two data products are three times smaller than the differences among the IPCC models.
W. Rossow opened the ISCCP part of the meeting by summarizing the operational status of the data collection and processing and by identifying several issues to be discussed. All elements of the project are operating routinely with data deliveries on or close to schedule: final cloud products through June 2006 have been archived and processing of the next 6 months is underway. The beginning of processing of Chinese geostationary satellite data (from FY-2C) raises the issue of how to handle an unprecedented situation for ISCCP, namely the availability of more than the minimum amount of data: there will soon be 7 geostationary and 3 polar orbiting satellites contributing data. Newer instruments have also raised new issues with regard to calibration that need to be investigated, including much stronger non-linearities and scan angle dependencies. Having evaluated most of the currently available atmospheric temperature and humidity products, there is no obviously superior replacement for the operational TOVS product in the ISCCP processing; although the reanalyses probably have smaller interannual artifacts, other features of these datasets are still problems. If the GPC funding renewal is approved (which it was after the meeting), the switch from the B2 to the B1 radiance dataset will necessitate some changes of the cloud algorithm, those that involve spatial contrast tests, but the nature of the cloud products needs to be re-considered in terms of whether the gridded products should now be reported at a finer latitude-longitude interval.
K. Knapp led off the reports of the Satellite Processing Centers by describing NOAA NCDC processing of polar orbiting data over the past year. There are currently 6 polar orbiting satellites providing data; only NOAA-17 and NOAA-18 provide data to ISCCP but METOP-2 could. The launch of NOAA-N’ to replace NOAA-18 is planned for 2009 and the replacement of METOP-2 is planned for 2011. There are currently 4 GOES satellites in orbit, GOES-11 and 12 provide data to ISCCP. The next launches of the remaining GOES satellites in the current series are planned for 2008 and 2009; the new series satellites, starting with GOES-R, are planned to begin launching in 2012. He reported on the truncation of the calibration coefficients for the non-linear correction to IR channels that was not corrected for the period 2000-2005, which affects NOAA-15, 16, 17 and 18 IR radiance data quality, primarily at the cold end of the range. This problem also affected the whole ISCCP calibration when NOAA-16 became the reference standard (and NOAA-18 for a time). This problem, equivalent to temperature errors at 220 K of 0.5-1.0 K, was discovered during a comparison of AVHRR and HIRS radiance statistics over the whole record. The radiance data have now been re-processed but the ISCCP IR calibrations will have to be re-done (later) to fix this. All other aspects of data processing are routine and up to date. W. Rossow presented a report on behalf of the JMA representative, A. Okuyama, who could not attend the meeting. There are currently two MTSATs in orbit, MTSAT-1R is providing data to ISCCP and MTSAT-2 is backup. Processing and delivery for ISCCP and GPCP has been routine with a 99.6% capture rate. During a brief outage of MTSAT-1R for instrument outgassing, data were successfully obtained from MTSAT-2. Copies of GMS-1 radiance data from December 1978 through November 1979 were discovered at the University of Wisconsin and provided to JMA by NOAA; these data will be processed for ISCCP and GPCP. K. Holmlund reported on status and activities at EUMETSAT. METOP-A (now METOP-2) was launched in October 2006, has completed check out and entered full operational service in October 2007 (some products are still being evaluated). A review of instrument performance shows that all instruments are within planned specifications. Validation activities continue for the new, advanced instruments such as IASI, GOME, GRAS and ASCAT In addition to producing the ISCCP radiance products from the AVHRR instrument, new products being developed include soil moisture and AVHRR-winds. Four METEOSAT satellites are in orbit with METEOSAT-9 and METEOSAT-7 providing data to ISCCP, the later located over the Indian Ocean. A new definition of radiance units will be forthcoming, which will necessitate a re-processing of the datasets for METEOSAT-8 and METEOSAT-9; the differences for the channels used by ISCCP are very small (< 1%). No representative from the Canadian Meteorological Satellite Center, which processes GOES-12 data, attended the meeting and no report on status was received. J. Forsythe reported on the status and activities at Colorado State University, which processes GOES-11 data. Processing has been more highly automated over the past year and has run routinely. CSU participating in testing GOES-13 measurements (currently this satellite is a backup); this exercise showed that several improvements in data quality compared with GOES-11. CSU also conducted another in a series of field experiments (CLEX) to evaluate remote sensing of middle-level, mixed phase clouds, this time using CloudSat and Calipso data. All of these results are being employed to evaluate the ISCCP products. Y. Liu presented the status, activities and plans of CMA, now processing FY-2C data for ISCCP; FY-2C has been operational since the beginning of 2005 and FY-2D has been the backup satellite since mid-2007. The collection of data has become routine and the capture rate is very close to 100%. Since mid-2007 when FY-2D became operational, staggered image collection from the two satellites, separated in longitude by about 18°, has been conducted to produce a 15-min sampling interval during June, July and August and a 30-min sampling the rest of the year. Deliveries of B1/B2 radiance data from FY-2C are now up to date; the possibility exists to supply FY-2D data in addition. Launch of FY-2E is planned for early 2009; two more satellites in this series are planned before transition to the next generation (FY-4 planned for launch around 2017). FY-4 will include sounder and Earth Radiation Budget instruments, as well as increasing the number of spectral bands on the imager. Also, this new generation of geostationary satellite will be expanded to a two-satellite system, similar to GOES. Also in development is the next generation polar orbiting weather satellites (FY-3) with expanded capabilities.
A special report was presented by R. Liu concerning comparisons of ISCCP cloud products with in situ (surface weather stations) and other satellite (MODIS) analyses over China. The most noticeable differences appeared over Tibet and other mountainous regions. Generally the datasets are more similar in summertime than wintertime; some wintertime disagreements appear to be related to snow-cloud discrimination problems. Satellite cloud amounts on average are larger than surface reports but the magnitude of variations is smaller for satellites. The ISCCP cloud amounts seem to correspond to the surface results more closely than do the MODIS-based results.
K. Knapp reported on the ISCCP Central Archives activities in the past year. Data collection, checking and archiving functions are now much more automated and have been routine. As of the time of the meeting, deliveries of B1 radiance data were up to date for all satellites. Work continues to look for older, pre-ISCCP datasets: data from GOES-1, 2, 3 and 4, GMS-1 and SMS-1, 2 have been recovered and recovery of METEOSAT-1 is being investigated. The collection contains data from 29 geostationary satellites. Work also continues to provide online access to the complete set of GEWEX global data products using a THREDDS server approach. Next the GOES B1 data holdings will be reviewed and an attempt made to recover the larger (> 4 days) data gaps; later this effort may be extended to filling smaller gaps. A calibration comparison between AVHRR and HIRS has been completed (leading to the discovery of the coefficient problem for the later AVHRRs). One of the first uses of the re-calibrated, refurbished B1 IR radiances was to re-do the hurricane intensity climatology (still ongoing) that was based on operational versions of these radiances, which were not inter-calibrated. A common format for all the B1 data will be investigated.
C. Bishop summarized the status of radiance calibrations, starting with the report from the Satellite Calibration Center on behalf of Y. Desormeaux, who could not attend the meeting. Normalization of the geostationary radiances to the afternoon polar orbiter has proceeded routinely over the past year; results are up to date for all satellites through May 2007 at the time of the meeting. Satellites processed include MTSAT-1R, FY-2C, GOES-11 and 12 and METEOSAT-5, 7, 8 and 9. Specialized software was created to process METEOSAT-8; results have been obtained for the whole period from April 2005 through May 2007. The transition to METEOSAT-9 occurred smoothly. Processing of METEOSAT-7 over the Indian Ocean commenced in October 2006; METEOSAT-5 processing continued until April 2007. These results, together with the parallel processing of all satellites during a nearly-year-long overlap of NOAA-16 and NOAA-18, provide extensive overlapping of calibration results that make possible a much more detailed cross-checking of normalization coefficients. During the past year, the normalization of the water vapor channels was stopped because the needed channel on HIRS aboard NOAA-18 was bad quality. Calibrating the visible channel on the new AVHRRs (starting with NOAA-16) became more problematic with the introduction of a bi-linear response function; investigations are underway to evaluate and confirm the current results in more detail. The problem of the incorrect IR non-linear coefficients is illustrated by the remaining small discrepancy (about 2 K at 240 K) at the cold end of the instrument range; this arises because the normalization of each AVHRR to the reference climatology from NOAA-9 is done linearly, so although the warm-end calibration is unaffected, the cold-end values are slightly biased (colder). This also shows up as a small cloud top pressure bias of about 10 mb. Other open items being investigated are associated with the new MTSAT and METEOSAT-8 radiometers, which do not yet appear consistent with past climatology.
W. Rossow summarized activities at the Global Processing Center during the past year. Algorithm evaluation and revision studies focused on the polar regions, exploiting the detailed datasets available from the SHEBA experiment (surface-based radar/lidar, meteorology and radiative flux measurements). Although the ISCCP seasonal total cloud amounts appear to be consistent with other estimates to within about 10-20% (somewhat too high in wintertime and too low in summertime), these amounts turn out (in comparison with SHEBA) to result from a near cancellation of two errors: missed detections and false detections; about 40% of the data are erroneously labeled. The seasonal variation of the bias is explained by twice as many missed as false detections in summertime and twice as may false as missed detections in wintertime. Implementation of additional multi-spectral algorithm tests (using the “split-window” contrast) as investigated by other researchers appears to alleviate these problems fairly well. The availability of Calipso data now makes it possible to refine the algorithm changes not only over the whole Arctic (including land areas at lower latitudes) but also over the Antarctic where conditions may differ from the SHEBA-case. After completing the Calipso comparison, this algorithm change will be implemented for the planned re-processing. The other major activity is the continued development of more specialized products to facilitate more focused research on specific cloud types and meteorological situations. A special subset of tropical convective clouds with cloud tops that penetrate into the stratosphere was developed for a joint SPARC-GCSS workshop on the tropical tropopause and lower stratosphere. The cloud top pressure – optical thickness histograms reported in the D1 dataset have been analyzed to produce identify characteristic patterns that indicate distinct meteorological states: results are now available for the complete ISCCP record for the whole tropics (±15° latitude), for an extended low-latitude zone (±35°) encompassing the Hadley circulation regime and for northern and southern midlatitudes (30°-65°). The latter is in addition to the cyclone-tracking composites released the year before. Planned revisions of the ISCCP processing for the next re-processing included revised polar cloud detection, refined radiative retrievals to account better for surface and aerosol effects as well as to revise the treatment of ice clouds slightly, replacement of ancillary datasets with more homogeneous and more accurate products and switching the analysis from the 30-km sampled B3 radiance data to the 10-km sampled B1 radiance data.
W. Rossow also summarized the status of the Cloud Product Assessment activity on behalf of C. Stubenrauch, who could not attend the meeting. As a reminder of the overall situation, the agreement among the various, long-term, global products on basic quantities such as the global mean cloud amount and the mean seasonal variation amplitude is quantitatively quite good: global mean cloud amount is 70±5% (about 8±3% more over oceans than land) and stable over the past two decades to within about 5% and the seasonal variations range from about 4±1% over southern midlatitudes to 12±3% over the tropics-subtropics. This good agreement also extends to the amounts of high, middle and low clouds (the latter two from the top-down viewpoint of satellites): there is 27±3% low clouds (with no obscuring upper level clouds), about 11±3% more over oceans than land, and about 33% high-level clouds (the less sensitive ISCCP sensors find 5-10% less cirrus than the more sensitive IR sounders). The seasonal variation of low clouds over midlatitude oceans is only about 5%, whereas it is about 10% over low-latitude oceans. The seasonal variation of high clouds over land is over 40% in the southern low latitudes; much of this variation appears in the thinnest cirrus, which is missed by ISCCP. The seasonal variations over other land areas is about 10-20%, again somewhat stronger for the thinnest cirrus. Work continues to document and investigate the differences among the cloud products; these results will help place accurate uncertainty estimates on the results.
There was a brief review of available ancillary data products. The snow and sea ice products can now be changed to the standard products produced by NSIDC – these products are being used now but were not used in the earlier part of the record. It was noted that the TOMS ozone record had ended but that such data were being continued by OMI measurements; the availability of AIRS-based results needs to be investigated. Currently, there is no obvious replacement for the operational TOVS product to provide atmospheric temperature and humidity profiles; the available reanalyses may be continued and might be more homogeneous than the TOVS product but comparison studies also showed some systematic differences with sounder products. More study is needed. The breakout session ended with a summary discussion of progress and remaining issues; this summary is given below as part of the final plenary session.
Final Plenary Session
A recommendation from the GPCP breakout is to continue the heritage products even if newer (higher resolution, TRMM-anchored) versions are produced. These can be used for calibration purposes at least and may be ended after more experience is gained with the new products. Thus, the tentative suite of GPCP products would be: (a) 3-hr, 25-km starting in 1998 (possibly limited to ±50° latitude), (b) global, daily, 50-km starting in 1983 using ISCCP B1 IR radiances, (c) global, pentad, 50-km (maybe) starting in 1979 and (d) global, monthly, 50-km (maybe) starting in 1979. A test year (probably 2004) will be processed using all available algorithms. The likely passive microwave combination is the so-called L1C version of the calibration and the GPROF (V2007) but this is still being examined. The gauge dataset that will be used is the GPCC V4 Full Re-analysis; this implies a lag of 1-3 years in final products but a “quick-look” product will be produced for users needed more recent data. A snow/rain discrimination will be added, at least one based on near-surface air temperatures but snow algorithms are still being studied. The results in mountainous regions should be improved by the new gauge analysis; no global adjustment scheme is available. For periods when no geostationary satellite IR radiances are available over the Indian Ocean, AVHRR data will be used to fill in. No decision on the map grid for the products was made.
Several decisions were made regarding a new B1-based ISCCP product and how to handle the availability of more satellite data than the minimum required for global coverage: the preferred constellation is 2 polar orbiters and 5 geostationary satellites, 3 polar orbiters and 7 geostationary satellites are now available and there may be 2 more of each type available in the next few years. The decisions were to collect the maximum amount of available data (with all available spectral channels) and to release the so-called DS dataset – this is gridded data like D1 but separate by satellite including all the data usually discarded when the satellites are merged into D1. Thus, even if a significant portion of the data is discarded to make D1, the DS product will be available for regional studies. Also, when the switch to B1 radiances is made, the production of B2/B3 will cease. The new design for D1/D2 based on the B1 data can allow for reporting of results at finer resolution; during the coming year, the structure of the 2-dimensional histograms of cloud top pressure and optical thickness as a function of spatial scale will be investigated. A possible idea would be to report the histograms and cloud type information on the current 2.5°-equivalent equal-area grid but report the area-mean cloud properties on a finer grid (0.5°). Note that the precision of the cloud amount values depends on the number of satellite pixels used to determine it, so using too fine a grid can degrade the results.
The discussion of a new atmospheric temperature-humidity product among the projects identified three new products that might be used (SRB proposed the new NASA reanalysis, MERRA, SeaFlux is developing its own near-surface temperature-humidity datasets over ocean that might be merged with a global product, NOAA is developing a new HIRS-based analysis). Moreover, the SeaFlux project is producing a more accurate skin SST product that could be used by the other projects. These products will be examined during the next year.
All projects will continue to produce a suite of products that best serve their purposes but one product from each project will be designed to have a common space-time sampling interval, common grid and definition of day. The target will be 3-hr, 1° but also common daily, monthly products will also be considered. Higher resolution products, if possible for part of the time record, will also be produced. Moreover, common radiances will use the same calibrations. The plan is to commence re-processing of all products in 2009.
The next meeting of WGDMA will be held in eastern Asia on 22-24 September 2008. Invitations have been received for a meeting venue in either Beijing or Hong Kong.
2007 Meeting of GRP Working Group on Data Management and Analysis
Dates: 5 - 7 September, New York, NY, USA
Location: The City College of New York (Exhibit Room)
Contact: William B. Rossow E-mail: wbrossow@ccny.cuny.edu
Tel: 1-212-650-5389 Fax: 1-212-650-8097
Main Agenda
05 September 2007 Wednesday AM — Exhibit Room
0830-0845: Welcome, Meeting Organization and Goals (Barba, Rossow)
0845-0900: Report on GEWEX/WOAP/WCRP (Rossow)
0900-0915: Project Status Report – ISCCP (Rossow)
0915-0930: Project Status Report – GACP (Mishchenko)
0930-1000: Project Status Report – SeaFlux (Clayson)
1000-1030: Break
1030-1100: Project Status Report – GPCP (Adler)
1100-1130: Project Status Report – SRB (Stackhouse)
1130-1200: Data Archaeology (Knapp)
1200-1230: Issues for Choosing Common Atmospheric Dataset (Rossow)
1230-1345: Lunch – Exhibit Room
Wednesday PM — Exhibit Room and Room 105
1345-1515: Project Sessions #1
1515-1545: Break
1545-1745: Project Sessions #2
1745: Adjourn
1800: Reception – Rossow apartment (20 West 64th St, #29L)
06 September 2007 Thursday AM — Exhibit Room and Room 105
0830-1000: Project Sessions #3
1000-1030: Break
1030-1200: Project Sessions #4
1200-1330: Lunch – Exhibit Room
Thursday PM — Exhibit Room and Room 105
1330-1500: Project Sessions #5
1500-1530: Break
1530-1730: Project Sessions #6
1730: Adjourn
07 September 2007 Friday AM
0830-1000: Project Sessions #7 – Exhibit Room and Room 105
1000-1030: Break
1030-1230: Plenary Session – Exhibit Room
Breakout Summaries, Common Datasets, Wrap-up
1230: Adjourn
Project Sessions for GPCP
Room 105
05 September 2007 Session #1
1345-1425: Project Status and Issues/Goals for the Meeting (Adler)
1425-1445: Project Data Center Report (Microwave-Land Center -Ferraro)
1445-1515: Project Data Center Report (Microwave-Ocean - Chiu)
1515-1545: Break – Exhibit Room
Session #2
1545-1605: Project Data Center Report (Geosynchronous Center - Janowiak)
1605-1625: Project Data Center Report (Gauge Center - Schneider)
1625-1645: Project Data Center Report (Merge Center-Huffman)
1645-1745: Discussion
1745: Adjourn
06 September 2007 Session #3
0830-0900: Data Product Assessment Report: Precipitation (Gruber)
0900-0945: Presentations on gauge data sets for GPCP (Schneider, Xie, Other)
0945-1000: Discussion of gauge data set selection
1000-1030: Break – Exhibit Room
Session #4
1030-1130: Microwave algorithms/products (Kummerow, Ferraro, Other)
1130-1150: Plans for heritage microwave input (Chiu)
1150-1200: Discussion
1200-1330: Lunch – Exhibit Room
Session #5
1330-1420: High-time resolution algorithms/products
(Arkin, Huffman, Janowiak, Other)
1420-1500: Discussion of Version 3 attributes, methods, choices
1500-1530: Break – Exhibit Room
Session #6
1530-1630: Additional presentations on Version 3 topics
1630-1730: Draft synthesis (from discussion) of Version 3 approach (Adler)
1730-1745: Final discussion with choices/directions, schedule for Version 3
1745: Adjourn
07 September 2007 Session #7
0830-1000 Final discussion of plans for Version 3
Project Sessions for ISCCP, GACP, SRB and SEAFLUX
Exhibit Room
05 September 2007 Session #1
1345-1415: Project Report: (SeaFlux – Clayson)
1415-1445: Project Report (GACP – Mishchenko)
1445-1515: Project Report (SRB - Stackhouse)
1515-1545: Break
Session #2
1545-1615: Data Product Assessment Report: Radiation (Stackhouse/Wielicki)
1615-1645: Discussion of Product Improvements and Re-processing
1645-1730: Discussion of Atmospheric (and Other Ancillary) Data Selection
1730: Adjourn
06 September 2007 Session #3
0830-0900: ISCCP Status and Issues (Rossow)
0900-0930: Project Data Center Report (ISCCP SPC-NOA - Knapp)
0930-1000: Project Data Center Report (ISCCP SPC-JMA - Okuyama/Rossow)
1000-1030: Break
Session #4
1030-1100: Project Data Center Report (ISCCP SPC-EUM - Holmlund)
1100-1130: Project Data Center Report (ISCCP SPC-MSC - Megyes)
1130-1200: Project Data Center Report (ISCCP SPC-CSU - Forsythe)
1200-1330: Lunch
Session #5
1330-1400: Project Data Center Report (ISCCP SPC-CMA - Liu)
1400-1430: Project Data Center Report (ISCCP ICA - Knapp)
1430-1500: SCC Report (Desormeaux/Bishop)
1500-1530: Break
Session #6
1530-1615: Project Data Center Report (ISCCP GPC - Rossow)
1615-1630: Data Product Assessment Report: Clouds (Baum/Stubenrauch/Rossow)
1630-1700: Discussion of Product Improvements and Re-processing
1700-1730: Discussion of Atmospheric (and Other Ancillary) Data Selection
1730: Adjourn
Participants
Robert Adler (NASA Goddard Space Flight Center)
Phil Arkin (U Maryland)
Chris Bishop (Columbia U at NASA GISS)
Long Chiu (George Mason University)
Carol Anne Clayson (Florida State U)
Steve Cox (NASA Langley Research Center)
Ralph Ferraro (NOAA)
John Forsythe (Colorado State U)
Violeta Golea (SSP at NASA GISS)
Arnold Gruber (U. Maryland)
Ken Holmlund (EUMETSAT)
Kuolin Hsu (UC Irvine)
George Huffman (NASA Goddard Space Flight Center)
John Janowiak (NOAA)
Ken Knapp (NOAA NCDC)
Chris Kummerow (CSU)
Ruixia Liu (CMA)
Yujie Liu (CMA)
Michael Mishchenko (NASA GISS)
William Rossow (City College of New York)
Udo Schneider (Deutche Wetterdienst)
Tom Smith (NOAA)
Paul Stackhouse (NASA Langley Research Center)
Pingping Xie (NOAA)
|
