EMDI Experimental Design

The Ecosystem Model/Data Intercomparison (EMDI) working group held in April, 1999 at the National Center for Ecological Analysis and Synthesis (NCEAS), Santa Barbara has formalized and experimental design and set of timelines and activities for the EMDI activity. As was outlined in a previous email (see =Call_details.rtf) the model/data comparisons will be made for point and gridded (spatial) NPP. The point measurements of NPP will be made for intensive or well-documented Class I/II sites (e.g. site-specific climate, soils information, etc.) and for extensive or Class III sites with less documentation and site-specific information.

The emphasis will be comparisons between individual models and data; therefore, it is not as essential for all models to use the same driver data. However, we will look for general patterns between models, e.g., do all models predict lower NPP values than data for boreal forests? Therefore, we have not provided a common dataset for all possible driver data and modelers may choose to use non-common driver data; however, we request that if a model use non-common data, that these data be provided to EMDI for use by other modelers.

EMDI recognizes that there may be problems in the field data and that feedback from the modelers is an important quality assurance step in compiling appropriate NPP field data for validation and model development. We also hope to state comparisons in neutral terms and not imply that the data are assumed to be truth.

All the NPP field data will be made publicly available in October to compare to model outputs and to the broader modeling community for continued model development. Therefore, only those modeling groups that submit their model simulations to EMDI by 1 October 1999 will be able to claim their model was "blindly" compared to this unique and new collection of NPP data.

The overall design is to initially focus on spatial comparisons with temporal comparisons to be conducted at a second EMDI workshop in spring of 2000. We will be able to initially incorporate a few flux tower sites with NPP data and provide detailed driving variables for many sites. Later this year we plan to assemble flux data from flux towers to compare additional carbon (e.g. NEE) output variables from those models able to simulate a carbon budget. The next iteration of our experimental design will address specific details of suggested spin-up protocols and model runs.

Analysis of output will be performed at the University of New Hampshire by a Master’s-level technician supported by the National Center for Ecological Analysis and Synthesis (NCEAS). At the current time, EMDI is working on providing travel funds for one person to attend the November workshop at the University of New Hampshire from each modeling group, HOWEVER, we would like to emphasize that travel funds are not yet secured, and travel may have to be encumbered by individual groups.

An EMDI Web page (http://www.nceas.ucsb.edu/fmt/doc?/cgi-bin/ppage.pl?project=137), email listserver (emdi@nceas.ucsb.edu), and a controlled-access "Members Only" data server has been established at NCEAS to provide general information and access to the data.
Potentially non-representative NPP data will be flagged (e.g., observed NPP data that lies outside +- 2 stdev within biome groups will be flagged as outliers in the database); however, these data will NOT be removed. In addition, managed systems will be flagged, but not removed.
Biomass to carbon unit conversions: grasslands: 2.4; forests and all litter: 2.0
We ask that all modeling groups use the provided climatologies to drive their models.
An ‘official’ global map of observed nitrogen deposition (NDEP) is unavailable at this time, therefore, the modeling groups will be asked to handle NDEP after their own methods. We will ask that the modeling groups provide their strategies for handling NDEP for analyses purposes.
Atmospheric CO₂ concentrations will be provided as a single global value for each year, from 1900-1990.
If alternative data sets are used for model calculations (e.g. climatologies, soils, AVHRR, etc.), we ask that the data sets are identified and provided to all groups (e.g., be provided so they can be posted on the NCEAS Web server).
Results will be made publicly available only for groups that agree to public release.

Basic site information
Site information includes site ID, latitude, longitude, and elevation. Other site information to be provided includes (as available): country, stand origin, dominant species, stand age, disturbance history, year(s) of observed NPP.

Vegtype
Although the vegetation type is provided in the measured data sets, they are inconsistent with respect to a standardized vegetation classification. ORNL will provide the vegetation description from the site as well as the appropriate classification for the 8km grid cell containing the site from the DeFries et al. (1998) classification (http://www.geog.umd.edu/landcover/8km-map.html). Classification will be 1: Evergreen Needleleaf Forests, 2: Evergreen Broadleaf Forests, 3: Deciduous Needleleaf Forests, 4: Deciduous Broadleaf Forests, 5: Mixed Forests, 6: Woodlands, 7: Wooded Grasslands/Shrubs, 8: Closed Bushlands or Shrublands, 9: Open Shrublands, 10: Grasses, 11: Croplands, 12: Bare, and 13: Mosses and Lichens.

Soils
Soils information will be provided from the newly developed database developed by Scholes et al. (1995) and distributed by the IGBP-DIS.

ORNL will provide a soil description from the site (when available) as well as the following soils data for the 0.5 grid cell containing the site from the Scholes database for each datapoint:

Climate
Long-term and transient monthly climatologies will be extracted by Wolfgang Cramer from the University of East Anglia Climatic Research Unit dataset (http://www.cru.uea.ac.uk/cru/data/). Long-term and transient mean monthly data will be provided for all datapoints (Class I/II, III, grids, flux sites). Actual climate data from those sites with 20+ years of climate history will be used to test the sensitivity of NPP predictions between actual versus long-term climatologies (see Question #2 below).

Pathfinder AVHRR
For those models that are driven by remotely-sensed inputs, an average, or representative year from the Pathfinder data set will be provided as well as the entire series (approximately 13 years) from the satellite archive for monthly NDVI and thermal channels. These data will be extracted by Rob Braswell.

Others who have responded, but not completed the questionnaire, or are assumed to participate:





	EXPERIMENTAL DESIGN AND ANTICIPATED TIMELINES FOR THE ECOSYSTEM MODEL/DATA INTERCOMPARISON (EMDI) May 24, 1999 The Ecosystem Model/Data Intercomparison (EMDI) working group held in April, 1999 at the National Center for Ecological Analysis and Synthesis (NCEAS), Santa Barbara has formalized and experimental design and set of timelines and activities for the EMDI activity. As was outlined in a previous email (see =Call_details.rtf) the model/data comparisons will be made for point and gridded (spatial) NPP. The point measurements of NPP will be made for intensive or well-documented Class I/II sites (e.g. site-specific climate, soils information, etc.) and for extensive or Class III sites with less documentation and site-specific information. The emphasis will be comparisons between individual models and data; therefore, it is not as essential for all models to use the same driver data. However, we will look for general patterns between models, e.g., do all models predict lower NPP values than data for boreal forests? Therefore, we have not provided a common dataset for all possible driver data and modelers may choose to use non-common driver data; however, we request that if a model use non-common data, that these data be provided to EMDI for use by other modelers. EMDI recognizes that there may be problems in the field data and that feedback from the modelers is an important quality assurance step in compiling appropriate NPP field data for validation and model development. We also hope to state comparisons in neutral terms and not imply that the data are assumed to be truth. All the NPP field data will be made publicly available in October to compare to model outputs and to the broader modeling community for continued model development. Therefore, only those modeling groups that submit their model simulations to EMDI by 1 October 1999 will be able to claim their model was "blindly" compared to this unique and new collection of NPP data. The overall design is to initially focus on spatial comparisons with temporal comparisons to be conducted at a second EMDI workshop in spring of 2000. We will be able to initially incorporate a few flux tower sites with NPP data and provide detailed driving variables for many sites. Later this year we plan to assemble flux data from flux towers to compare additional carbon (e.g. NEE) output variables from those models able to simulate a carbon budget. The next iteration of our experimental design will address specific details of suggested spin-up protocols and model runs. Analysis of output will be performed at the University of New Hampshire by a Master’s-level technician supported by the National Center for Ecological Analysis and Synthesis (NCEAS). At the current time, EMDI is working on providing travel funds for one person to attend the November workshop at the University of New Hampshire from each modeling group, HOWEVER, we would like to emphasize that travel funds are not yet secured, and travel may have to be encumbered by individual groups. Scientific Questions: How does the simulated NPP compare with the data? This is a simple question that can be answered by this exercise. We anticipate that analyses will be performed by comparing simulated to observed by biome/ecosystem type, climate space with simple scatter plots, regression analyses (Wilmott-D statistics (RMS, MSE, ME); investigating model deviates from the dataà do they act in concert? We will also examine abiotic (e.g. AET) versus NPP from individual models. How sensitive are models to site-specific climate? A sensitivity analysis will be performed to determine whether model results differ if driven by long-term average versus transient climate/NDVI inputs. Does additional mechanistic detail in models result in a better match with field measurements? Or, if modeled ‘total’ NPP matches field data, how does the allocation patterns with respect to above and belowground match the field data? In the context of question #1, an analyses of simple versus mechanistic models to identify key ‘lumped’ or summary parameters in the models that can be used to evaluate light, water, and nutrient constraints on canopy processes, photosynthesis, and hydrology. This is a simple versus complex question. How useful are the measures of NPP for evaluating model predictions? One of the objectives of the GPPDI was to develop a robust technique where sites that had measured total NPP could be used to estimate total NPP for sites with incomplete (e.g. lacking belowground) measures of NPP. Similarly, GPPDI addressed how well is peak standing crop as generally used for a proxy for observed NPP in grassland? This question is designed is to test the integrity of the actual measured data against simulations. Components would include above and belowground biomass, litterfall, plant functional type from biogeography models and biomass in wood production as well as soil C, soil respiration, NEP, LAI, and ET from the flux tower sites. How well do models represent regional patterns of NPP? Model output and regional data for 0.5º grid cells will be compared to address this question. Regions include the US eastern deciduous forest; US Great Plains; Queensland, Australia; and potentially other regions. Additional Protocols: An EMDI Web page (http://www.nceas.ucsb.edu/fmt/doc?/cgi-bin/ppage.pl?project=137), email listserver (emdi@nceas.ucsb.edu), and a controlled-access "Members Only" data server has been established at NCEAS to provide general information and access to the data. Potentially non-representative NPP data will be flagged (e.g., observed NPP data that lies outside +- 2 stdev within biome groups will be flagged as outliers in the database); however, these data will NOT be removed. In addition, managed systems will be flagged, but not removed. Biomass to carbon unit conversions: grasslands: 2.4; forests and all litter: 2.0 We ask that all modeling groups use the provided climatologies to drive their models. An ‘official’ global map of observed nitrogen deposition (NDEP) is unavailable at this time, therefore, the modeling groups will be asked to handle NDEP after their own methods. We will ask that the modeling groups provide their strategies for handling NDEP for analyses purposes. Atmospheric CO₂ concentrations will be provided as a single global value for each year, from 1900-1990. If alternative data sets are used for model calculations (e.g. climatologies, soils, AVHRR, etc.), we ask that the data sets are identified and provided to all groups (e.g., be provided so they can be posted on the NCEAS Web server). Results will be made publicly available only for groups that agree to public release. Input Data Sets: By 30 June, the Oak Ridge National Laboratory (ORNL) will release input data sets to drive the model simulations. These data sets include: site, vegtype, soils, climate, and Pathfinder information. Some sites that are included in the 30 June list may be dropped from the final list of sites. We are including all the flux towers and other potential sites with the hope that NPP values will be provided this summer (however, we realize that some sites will lack NPP for the initial comparisons). Data will be provided as ACII, comma separated value (.CSV) files. By early June, ORNL plans to release an example set of information for a typical site using the formats envisioned for the 30 June release. Basic site information Site information includes site ID, latitude, longitude, and elevation. Other site information to be provided includes (as available): country, stand origin, dominant species, stand age, disturbance history, year(s) of observed NPP. Vegtype Although the vegetation type is provided in the measured data sets, they are inconsistent with respect to a standardized vegetation classification. ORNL will provide the vegetation description from the site as well as the appropriate classification for the 8km grid cell containing the site from the DeFries et al. (1998) classification (http://www.geog.umd.edu/landcover/8km-map.html). Classification will be 1: Evergreen Needleleaf Forests, 2: Evergreen Broadleaf Forests, 3: Deciduous Needleleaf Forests, 4: Deciduous Broadleaf Forests, 5: Mixed Forests, 6: Woodlands, 7: Wooded Grasslands/Shrubs, 8: Closed Bushlands or Shrublands, 9: Open Shrublands, 10: Grasses, 11: Croplands, 12: Bare, and 13: Mosses and Lichens. Soils Soils information will be provided from the newly developed database developed by Scholes et al. (1995) and distributed by the IGBP-DIS. ORNL will provide a soil description from the site (when available) as well as the following soils data for the 0.5 grid cell containing the site from the Scholes database for each datapoint: 0-30 cm: % sand, % silt, % clay, % rock fragments Water Holding Capacity Saturated Hydraulic Capacity pH measured in water Bulk Density Thermal Diffusivity at 0,15, 100% WHC 0-20 cm, 0-30 cm, 0-100 cm: Soil Carbon Soil Nitrogen Climate Long-term and transient monthly climatologies will be extracted by Wolfgang Cramer from the University of East Anglia Climatic Research Unit dataset (http://www.cru.uea.ac.uk/cru/data/). Long-term and transient mean monthly data will be provided for all datapoints (Class I/II, III, grids, flux sites). Actual climate data from those sites with 20+ years of climate history will be used to test the sensitivity of NPP predictions between actual versus long-term climatologies (see Question #2 below). Pathfinder AVHRR For those models that are driven by remotely-sensed inputs, an average, or representative year from the Pathfinder data set will be provided as well as the entire series (approximately 13 years) from the satellite archive for monthly NDVI and thermal channels. These data will be extracted by Rob Braswell. Output from Models: All output variables provided for models will be annual. The only "required" output variable is NPP in gC/m²/y. However, the following list of variables are proposed to address the scientific questions: Gross Primary Production (GPP) Aboveground NPP (ANPP) Belowground NPP (BNPP) Wood Increment Litterfall Fine Root NPP Peak Aboveground Standing Live Biomass Leaf Area Index (LAI) Net Ecosystem Productivity (NEP) Net Biome Productivity (NBP) Absorbed Photosynthetically Active Radiation (APAR) Soil Carbon Total Autotrophic Respiration (R_a) Aboveground Autotrophic Respiration Belowground Autotrophic Respiration Heterotrophic Respiration (R_h) Soil Respiration (R_s) Evapotranspiration (ET) Evaporation (E) Transpiration (T_s) Latent Heat of Evaporation (LE) Actual Evapotranspiration (AET) Timelines: 30 June: Release driver data sets for Class I/II and Class III sites to modelers (climatologies, soils, vegtypes) 31 August: release 50% of observed NPP for Class I/II and Class III sites to modelers who have turned in blind simulations (enforced). No model results will be released to other groups. Ancillary data sets for regions will be released to modelers at this time. 1 October: Strict drop-dead date for model results to be turned in. We also request that the modeling groups turn in any documentation of model adjustments, alternative driver data sets and any comments about possible outliers in the data sets they have received (NPP or ancillary). All of the observed NPP data will be made publicly available at this time. 25 October: Graphs of model results will be available on servers (password-protected) Week of 8 November: EMDI I workshop, tentative location, University of New Hampshire, Durham, NH. Workshop Products: A series of peer-reviewed papers associated with each of the scientific questions will be written based on the results of November meeting, there is a preference for the papers to be published together as a special issue. In addition, the documented data will be publicly distributed. Current list of Participants: If you do not see your model group listed below, or, if you would like to have your group removed, please contact Kathy Hibbard. COUNTRY Model RESPONDENT OF ORIGIN STOMATE Pierre Friedlingston/Laurent KergoatFrance Unnamed/DOLY Bill Emanuel U.S. CASA Chris Field U.S. PNet John Aber U.S. LPJ Stephen Sitch/Ben Smith U.K. AVIM Jinjun Ji Japan HRBM Tim Reichenau Germany SIB2 w/Carbon Chris Field/Jörg Kaduk U.S. CARAIB Bernard Nemry Belgium SILVAN 2.4 Jörg Kaduk Germany/U.S. TEM Dave Kicklighter U.S. VECODE Victor Broukin Germany GDAY Ross McMurtrie Australia *** Others who have responded, but not completed the questionnaire, or are assumed to participate: BGC Peter Thornton U.S. CENTURY Becky McKeown/Bill Parton U.S. HYBRID Andrew Friend U.K. GLO-PEM Steve Prince U.S. IBIS Jon Foley U.S. **** Recent call sent to: TRIFFID Peter Cox Germany MC1 Ron Neilson U.S. More Information: Contact Kathy Hibbard, GAIM. Literature Cited:** DeFries, R.S., Hansen, M., Townshend, J.R.G., and Sohlberg, R., 1998, Global land cover classifications at 8km spatial resolution: the use of training data derived from Landsat imagery in decision tree classifiers, International Journal of Remote Sensing, 19(16), 3141-3168. Scholes, R.J., Skole, D. and JS Ingram 1995. A global database of soil properties: proposal for implementation. IGBP-DIS Working Paper #10.