Gain insight into Regrow's methodology that powers outcome quantification in Sustainability Insights
Introduction
DNDC is a process-based, soil biogeochemical model designed to assess the impact of agricultural management practices on soil carbon (C) and nitrogen (N) dynamics (Li et al. 1992; Li et al. 1994; Li 2000). The model runs on a daily time step and is capable of simulating both aerobic and anaerobic soil conditions. The DNDC model has been applied across a wide range of agro-ecosystems globally, extensively validated and peer reviewed in over 200 peer-reviewed publications (Giltrap et al. 2010; Gilhespy et al. 2014; Yeluripati et al. 2015).
DNDC supports over 100 crop types, received the first ever ‘General Approval’ from the Climate Action Reserve for measuring soil GHG dynamics, and consistently received minimal (roughly 5%) deductions for model uncertainty when predicting soil organic carbon levels (per 100 fields, within .5mt/acre).
Regrow has a dedicated team that is responsible for ensuring that DNDC is calibrated for each crop, field, and region that we model. This team is responsible for finding the best available emissions data for a particular crop/geo combination, and using this data to ensure that the DNDC model is subsequently calibrated, and producing accurate results.
Additionally, Regrow has a package of publicly available peer-reviewed literature that a client can use in order to demonstrate DNDC’s accuracy for a particular context. CAR SEP also produced a validation report for DNDC (CAR SEP Validation of DNDC) that a customer can point an auditor towards.
DNDC conducts a full accounting of C and N cycling by simulating the impacts of major ecological drivers (climate, soil, vegetation, management) on soil climate conditions, plant growth, and decomposition (Figure 1). The model is built using classical laws of physics, chemistry, and biology, as well as empirical equations generated from laboratory studies to parameterize specific biogeochemical processes.
Figure 1: DNDC drivers (green boxes) and process based sub-modules (dashed boxes). Agricultural management practices directly modify the soil climate and plant growth sub-modules and indirectly alter organic matter decomposition and nutrient cycling (denitrification, nitrification, fermentation).
Model Calibration and Validation
DNDC is validated globally, and calibrated for different crop/geo combinations.
Calibration
Regrow maintains a large database of experimental studies used for regular calibration and validation of DNDC. The database is populated through literature review of peer-reviewed studies and datasets that report changes in emission sources of interest within targeted validation domains. Additionally, the database is populated with thousands of soil samples that contribute the required ‘ground-truth’ information used to generate DNDC estimates. The database is then used to build and run DNDC simulations for relevant experimental treatments (i.e. longitudinal measurements of a target emissions source, like soil organic carbon, or nitrous oxide flux).
Independent Validation
Validation of DNDC simulations is demonstrated through the description of an uncertainty model, allowing for the propagation of the uncertainty quantified through the validation data to new modeling units in the validation domain. The uncertainty model is an empirical model that estimates the lack of fit between model estimates and measured values of differences (i.e. calibration data). According to the Climate Action Reserve, a leading third party independent validator of soil carbon claims, DNDC effectively meets the bias and error requirements needed for validation within specific geographies and for specific crops.
Accounting for Model Uncertainty
When Regrow estimates a change in soil organic carbon (SOC), our model delivers a distribution of values as the distribution of estimated changes in soil carbon, reflecting model uncertainty. Most values fall near the middle value, but some fall far from the middle. This pattern (distribution) describes the model's uncertainty.
Uncertainty decreases as the number of fields or acreage size increases. Statistically, the more a sample size is expanded, the more likely it is that individual results align closely to the projected mean. Regrow has performed analyses that show our average uncertainty is quite small for simulations of a large number of fields.
Main input parameters into the DNDC
1. Soil texture
Soil texture is a classification of the inorganic, or abiotic weathered rock, material within a given soil - a summary of the sand, silt, and clay particles within a soil. Soil texture is an important soil variable as it dictates important soil characteristics as well as being a controlling factor in various soil health metrics. As soil texture dictates soil drainage, water holding capacity, soil erosion as well as interacting with other abiotic factors (e.g. soil temperature, soil fertility, etc) - it is important to accurately reflect soil texture and conditions of a soil as these resulting conditions will have impacts on both soil carbon (SOC) and greenhouse gas emissions (GHGs). Bulk density, aeration, and water holding capacity are some of the primary parameters of interest that will result in divergent results from a model like DNDC based on the inputs.
For example, sandy soils where there are more large sandy particles allows for high aeration and good drainage due to the particle size. On the other hand, they can leach out nutrients more readily and are thus less fertile soils. Further, the “good” drainage also results in low water holding capacity, and thus plant available water may be limited for crops, thus restricting otherwise potential growth.
Accurately reflecting these site specific soil conditions will thus impact water and nutrient flow, and resulting soil carbon sequestration (dSOC) and GHG emissions from a site - thus why accurate site level soil information is important. For example, given those structural factors, coarse soils are more susceptible to losing carbon than fine or medium soils. The soil aggregation ability, along with other factors already mentioned, thus have a strong influence on how much soil carbon can be stored within a soil.
2. Cover crops
At a high level, the purpose of cover crops is to protect the soil. This is realized by wind and water erosion prevention, by having a protective cover on the soil. Further, a bare soil does not have any plants photosynthesizing, and thus a bare (or fallow) soil has a missed opportunity in incorporating carbon within plants. When cover crops die or are incorporated into soil, this newly incorporated carbon can potentially be sequestered into soil organic matter, benefiting the soil.
The duration of the cover crop and the type of species directly influences the amount of carbon sequestered into the soil.
3. Tillage
Tillage has become a common tool for farmers for controlling weeds, residues management, and seedbed preparation. However, tillage also has unintended consequences that can harm a soil and its climate benefits. Tillage is a disturbance and mixing practice that can move residue throughout soil but also alters soil structure and exposes soil substrates. Tillage breaks up soil aggregates and soil structure when an implement is pulled through - this is important as aggregate size plays an important role in both water holding capacity as well as soil carbon protection. Larger agg regates have greater pore space, where more organic matter (carbon) and water can be held than in a tilled system with smaller aggregates. Think of a sponge with its greater water holding capacity compared to that of a towel of similar size - the sponge has larger and more aggregates and thus water holding capacity than that of the towel.
When tillage breaks these aggregates and disturbs soil, it exposes previously protected organic matter to oxygen and thus decomposition increases, seen in CO2 emissions peaks in the immediate time period after tillage events. Similarly, N2O emissions can similarly increase after tillage as well. These pulses of CO2/N2O after tillage are generally short lived, though increased decomposition and altered soil structure can continue in the tilled soil.
4. Residue management
More residue generally increases dSOC as you need carbon inputs to achieve carbon storage. There are numerous variations in what make crops unique. This ranges from calendar dates (plant/harvest dates and duration on field), varying biomass amounts, to more internally complex factors like water and nitrogen demand, carbon to nitrogen ratio, etc.
For example, rough carbon to nitrogen ratios of some crops are described in Table 1.
Material |
C:N Ratio |
Rye Straw |
82:1 |
Wheat Straw |
80:1 |
Oat Straw |
70:1 |
Corn Stover |
57:1 |
Rye Cover Crop (Anthesis) |
37:1 |
Pea Straw |
29:1 |
Rye Cover Crop (Vegetative) |
26:1 |
Mature Alfalfa Hay |
25:1 |
Ideal Microbial Diet |
24:1 |
Rotted Barnyard Manure |
20:1 |
Beef Manure |
17:1 |
Legume Hay |
17:1 |
Young Alfalfa Hay |
13:1 |
Hairy Vetch Cover Crop |
11:1 |
Soil Microbes (Average) |
8:1 |
Table 1: Carbon to Nitrogen Ratios in Cropping Systems; Source: (nrcs.usda.gov)
Why do C:N ratios matter?
The stability of soil carbon is inseparably linked with both carbon and nitrogen (amongst other factors) in soils. Climatic factors as well as available substrates have a direct impact in terms of whether a soil is more likely to sequester residue carbon or emit it. Soil organic matter will generally have a C:N ratio of 8-15 (ideally 10). When new residue is added to a soil, a "priming" effect occurs where microbes are stimulated, reacting and trying to decompose the residue. This priming effect differs based on the initial soil conditions, climate factors, as well as quantity/quality of residue inputs, etc.
An example, if a soil has a C:N ratio of 14 (low nitrogen) and we dump a lot of residue with C:N=70 (even lower nitrogen) - microbes may not be able to decompose the residue well as the residue (microbe energy) is inadequate for microbial growth. Microbes will try to take more nitrogen from the soil (N immobilization), depriving subsequent plant growth by taking accessible nitrogen out of the system. Slowly, as carbon does break down, microbial die off and N release from residue into soil can occur, allowing soil C:N to decrease again.
5. Weather and rainfall
It is worth mentioning that as soil carbon is strongly correlated to plant biomass (NPP - net primary productivity), arid soils are thus generally, unsurprisingly lower in soil carbon than soils in wetter climates. Put another way, in order to have soil carbon, you need to have carbon residue inputs into soil as well as water to decompose them. In arid environments where that soil carbon input is limited, there is only so much carbon that can be stored in soils, thus initial and potential dSOC is constrained by the limiting precipitation. Otherwise, it's a trade-off between increasing soil fertility through more carbon inputs (residue, manure, etc.) and potentially increasing emissions in the process of carbon decomposition that are occurring within the soil.
On N2O, emissions are driven by microbes that are linked to soil moisture, temperature, and substrate availability - warmer and wetter soils are likely to increase emissions. However, timing (e.g., when the precipitation falls, duration of elevated soil moisture, etc) as well as other management factors play a role in this as well (e.g., if more nitrogen is applied, then that will likely increase N2O emissions as well).
Figure 2: Denitrification and nitrification fluxes against water capacity
6. Nutrient Management (4Rs)
This practice encompasses fertilizer, manure, organic matter additions (OMAD), residue management and others.
Per the name (DNDC = DeNitrification DeComposition) the model renders both the denitrification (NO3 nitrate -> N2) and nitrification (NH4 ammonium -> NO3) cycles. Or put another way, DNDC tracks soil conditions (moisture and temperature), carbon and nitrogen substrates (and their form e.g., nitrogen levels of NO3 and NH4), on a daily time step.
Working on this daily time step, the model traces or has fluxes from these respective pools. For example, you can see in Figure 2 that N2O primarily occurs at 50-80% WFPS - so the model is constantly updating the soil environment and the nitrogen is being transformed between different nitrogen forms based on these soil conditions, leading to condition specific fluxes of N2O, N2, NO3 or leaching, depending on substrates available and the soil conditions.
So, not only is the model reflecting conditions that affect plant growth, but it is also reflecting the nitrogen transformations that lead to N2O emissions (and other N losses), and does so on a field-condition specific basis.
This is also why it's hard to give a straight-forward answer comparing two different sets of practices, as we can quickly compare two annual values for instance, but we can't readily/easily compare 365 daily time steps and the underlying soil conditions, N use, climate, etc that will result in different results between two simulations or two years of data.
7. Soil sampling & stratification
Protocols require that DNDC is initialized with estimates of soil organic carbon percentage (SOC) and bulk density based on measured data from the project domain.
To obtain these initial soil estimates, a stratified soil sampling design that uses auxiliary data correlated to SOC and bulk density is developed across the entire project. This method is more efficient (i.e. it reduces the number of samples needed to estimate a given change) than simple random sampling over the entire land area or grid/systematic sampling because there is less unexplained variability. The strata (auxiliary data layers) account for some portion of the variability over the entire land area.
For the stratification, field boundaries are discretized to 30m grids. Publicly available auxiliary data such as remotely sensed maps of vegetation, soil properties, topography, and crop history are recorded at each of the discretized locations. Pre-existing soil properties maps that include estimates of organic matter or specifically soil organic carbon are used to obtain mean and variance estimates at various spatial units. The sample size is determined as the minimum number of samples needed to restrict the error of the mean for a particular spatial unit to be no more than 5% of the mean. Examples of spatial units include the entire project domain, soil texture class areas, individual fields (i.e. homogeneous management units). The choice of particular spatial unit on which to base the final sample size depends on the available budget. Once the sample size is determined, the location of samples are selected from the discretized locations across the entire project domain using conditional latin hypercube sampling of the auxiliary data.
Measured SOC and bulk density at the sampled locations are then related back to the auxiliary data layers/strata using a digital soil mapping approach. The digital soil map is the output of a predictive model that estimates field level soc and bulk density with predictive error. The table below shows output data from 3rd party soils sampling providers.
Description |
Level of requirement |
Units |
GPS coordinates of planned sampled |
required |
long, lat |
GPS coordinates of actual core sample |
required |
long, lat |
Planned Depth |
required |
30 cm |
actual depth of measurement (part of sample maybe lost or difficult to obtain due to rocks) |
required |
cm |
diameter of core |
required |
metric |
Fine Soil Bulk density |
required |
g/cm^3 |
Total Carbon |
required |
% |
Organic Carbon |
required |
% |
Inorganic Carbon |
required |
% |
sampled date |
required |
YYYY-MM-DD |
data received by Lab Date |
required |
YYYY-MM-DD |
Field notes about field conditions (eg. field was recently tilled, snow in field, issues with the planned location) |
required (or provide image) |
|
photo image of field location |
nice to have |
|
sample mass dried |
nice to have |
g |
subsample mass pre-oven |
nice to have |
g |
subsample gravimetric water content |
nice to have |
% |
soil mass after flail mill |
nice to have |
g |
farm/field information included in the original design csv (such as field name, sampling point-id, etc) |
nice to have |
|
Table 2: Soil sampling requirements; source https://www.virescosolutions.com/.
8. Outcomes calculation
Outcome payments are calculated using the DNDC model to simulate greenhouse gas emissions and carbon sequestration associated with the practice changes contracted by the farmer. DNDC is the most globally-recognized, extensively peer-reviewed, model for soil carbon and GHG emission simulations. Model outcomes are influenced by many factors, including; soil, weather, crop choice, cover crops, tillage and other management practices. DNDC models changes in Soil organic Carbon (SOC), nitrous oxide (N2O) and methane (CH4), with respective pathways converted to carbon dioxide equivalents (CO2eq) using the global warming potentials, or climate impact, of the respective emissions.
The abatement potential is the difference in GHG emissions and soil carbon sequestration (dSOC) between a baseline scenario (reflecting current management practices) and an intervention scenario (reflecting the regenerative practice). Both scenarios utilize identical management information for the previous 3-5 years of historical practices, a requirement for improving model accuracy as well as providing identical soil conditions and ‘baseline’ management for protocols to ensure new regenerative practice changes were implemented. The project intervention scenario is then simulated and compared against the baseline scenario, with results provided as an annualized intervention compared to baseline reduction.
In an MRV program the outcomes provided at enrollment are an estimate of the potential practice change based on average outcomes of the same practice change across representative fields in the region of interest, regional soil and weather. Final calculations at the end of the season in the measurement phase are actual DNDC simulations based on incorporation of the full suite of information for the provided field, including: soils, weather, yield, and management practices for the time period of interest.
Conclusion
Reflecting on all of this, it is more than a simple 'X' crop or intervention will result in 'Y' outcome. It is too intertwined with…
- site conditions (soil texture, initial SOC, etc.),
- climate factors (affecting decomposition rates and microbes, water to oxygen ratio in soil affecting anaerobic vs aerobic systems and thus nitrogen cycle),
- nitrogen and crop histories (nitrogen available in soil and previous residues)
…for clear descriptive suggestions for crop management at this scale.
A meta-analysis of the sensitivity of the DNDC model outputs, SOC and N2O, to model parameters
Methodology
In order to identify the impact on different model parameters, particularly input parameters, on the outputs generated by the DNDC model, a comprehensive survey of the available scientific literature was conducted. A total of 27 studies (Appendix A) were identified with sensitivity analyses that focused on Nitrous Oxide (N2O) emissions and/or change in Soil Organic Carbon (SOC) storage using the DNDC model. We grouped the parameters tested in these sensitivity analyses into four categories: climate, soil, management, and crop. In the studies identified here, we determined that initial SOC was the parameter that was most commonly tested in the sensitivity analyses for both SOC and N2O (see Tables 3 and 4).
We further attempted to identify the most significant parameter for N2O and SOC in each study. Typically, the most straightforward method used to quantify or compare the sensitivity of SOC and/or N2O to changes of selected parameters is using a sensitivity index where the range of the response variable is weighted over the range in the sensitivity such that the higher the absolute value indicates greater sensitivity of the model to the parameter and the sign of the index represents whether simulations are positively or negatively correlated with the given input parameter. While this method was used in some of the studies identified here, it was not utilized by the majority.
In the majority of studies, changes in percentage value of outputs or the magnitude of changes in output values were used instead to report the sensitivity of the tested parameters. For these studies, the most sensitive parameters were determined based on the conclusions in the studies or best judgment. While this review attempted to determine the most sensitive parameter in each study, it is important to note that other parameters could still be highly influential and/or influence could change based on site conditions.
For the studies that reported the baseline, minimum, and maximum simulated N2O emissions and/or change in SOC storage (dSOC) with the corresponding baseline, minimum, and maximum input parameters, we also summarized the percentage of the input parameter change along with the percentage of the output change for N2O emissions in Table 3 and dSOC in Table 4.
Results and Discussion
Nitrous oxide emissions:
Of the 27 studies collected, a total of 19 studies with a sensitivity analysis of model parameters in DNDC to N2O emissions were identified. The numbers of tested parameters in each study ranged from 4 to 27 (median 6). There was only 1 study that investigated all 4 parameter categories of climate, soil, management, and crop (Qin et al. 2013), and also only 1 study that investigated only 1 category (management).
Initial SOC was the most tested parameter, followed by air temperature, precipitation, rate of N application, and soil pH. 4.5* out of 13 studies and 3.5* out of 16 studies that tested the rate of N application and initial SOC respectively found them to be the most sensitive. With the rate of N application and initial SOC changing from -67% to 450% and -80% to 1100%, the output annual N2O emissions ranged from -83% to 145% and -82% to 300%, respectively. Other parameters determined to be most sensitive to N2O fluxes included soil bulk density (2 out of 10 tested studies), soil pH (2 out of 13 tested studies), precipitation (2 out of 14 tested studies), field capacity (1 out of 3 tested studies), soil clay content (1 out of 12 tested parameters), timing of N application (1 out of 1 tested study), frequency of N application (1 out of 3 tested studies), and residue fraction of being left on the field after harvest (1 out of 5 tested studies).
The most common parameters that were tested (i.e. tested more than once) as well as identified as the most sensitive to N2O emissions are listed in Table 3 below, together with their percentages of the input parameter and output result changes. However, it should be noted that in some cases, while a single parameter was identified to be the most sensitive, other parameters also had a significant impact. For example, fraction of residue being left on the field after harvest was determined as the most sensitive parameter controlling N2O emissions from Chen et al. 2020; however other variables, such as pH, air temperature, rate of N application, and initial SOC, were also shown to be highly influential on N2O emissions (see Figure 2 in Chen et al. 2020). Further it is important to point out that sensitivity can also be influenced by site conditions. Using Chen et al. 2020 as an example again, depth of residue incorporation only had a sizable impact on N2O results at the high end of initial SOC spectrum.
Parameters |
Tested times (out of 19 studies) |
Most sensitive times |
Percentage of the input parameter change |
Percentage of the annual N2O emission change |
initial SOC |
16 |
3.5* |
-80% to 1100% |
-82% to 300% |
air temperature |
14 |
0 |
-102% to 44% |
-43% to 62% |
precipitation |
14 |
2 |
-100% to 76% |
-50% to 65% |
rate of N application |
13 |
4.5* |
-67% to 450% |
-83% to 145% |
soil pH |
13 |
2 |
-32% to 30% |
-64% to 56% |
soil clay content |
12 |
1 |
-84% to 600% |
-87% to 76% |
soil bulk density |
10 |
2 |
-30% to 30% |
-59% to 163% |
residue fraction (left on the field after harvest) |
5 |
1 |
-100% to 500% |
-26% to 25% |
depth of N application |
4 |
0 |
0 to 30 cm |
-100% to 0% |
N concentration in rainfall |
4 |
0 |
-69% to 300% |
-15% to 53% |
initial soil NO3 |
4 |
0 |
-25% to 25% |
-5% to 2% |
rate of organic amendment application |
4 |
0 |
-67% to 67%** |
-8% to 83% |
depth of tillage |
3 |
0 |
0 to 20 cm |
-45% to 0% |
frequency of N application |
3 |
1 |
1 to 3 N applications |
-26% to 138% |
initial soil NH4 |
3 |
0 |
-20% to 20% |
0% to 1% |
field capacity |
3 |
1 |
-25% to 25% |
-36% to 530% |
wilting point |
3 |
0 |
-25% to 25% |
0% to 17% |
soil porosity |
3 |
0 |
-25% to 25% |
0% to 2% |
rate of irrigation |
3 |
0 |
-67% to 67% |
-52% to 67% |
hydro-conductivity |
2 |
0 |
-25% to 25% |
0% to 2% |
maximum crop yield |
2 |
0 |
-25% to 25% |
0% to 5% |
form of N fertilizers |
2 |
0 |
ammonium sulfate, calcium ammonium nitrate, urea, and anhydrous ammonia |
-36% to 55% |
animal grazing intensity |
2 |
0 |
0% to 133% |
not reported |
timing of N application |
1 |
1 |
Apr 25, Jun 15, Jul 15, Aug 15, and Sep 15 |
0% to 81% |
* 0.5 means that this parameter was tested to be the most sensitive under some treatments in a study.
** Some studies were excluded, because of a baseline of no organic amendment.
Table 3: Results of the meta-analysis on the sensitivity of DNDC N2O emissions to model parameters.
Soil Organic Carbon:
Of the studies identified here, a total of 10 studies that had a sensitivity analysis of model parameters to SOC values were identified. The numbers of tested parameters in each SOC study ranged from 2 to 27 (median 4). There was only 1 study that investigated all 4 parameter categories of climate, soil, management, and crop (Qin et al. 2013), while 4 studies investigated only 1 category (climate for 2 studies, soil and management for 1 study each).
Six out of the seven studies that tested initial SOC reported that the changes in SOC were most sensitive to the initial SOC. With initial SOC changing from -95% to 700%, the output dSOC ranged from -111% to +1356%. However, there is one exception reported where SOC was more sensitive to the changes in soil bulk density than initial SOC (Khalil et al. 2020). Two out of the 4 studies that tested precipitation, reported that the SOC was most sensitive to the precipitation. However, it should be noted that the 2 studies identifying precipitation being the most sensitive parameter, only investigated the impact of climate parameters on SOC (i.e., precipitation could be the most sensitive climate parameter to SOC, but it might not be the most sensitive one when comparing to soil parameters). Other sensitive parameters identified in SOC studies are soil bulk density (1 out of 3 tested studies), fraction of residue left on the field after harvest (0.5* out of 3 tested studies), and rate of organic amendment application (0.5* out of 3 tested studies).
The most common parameters that were tested (i.e., tested more than once) as well as identified as the most sensitive to dSOC are listed in Table 4 below, together with their percentages of the input parameter and output result changes.
Parameters |
Tested times (out of 10 studies) |
Most sensitive times |
Percentage of the input parameter change |
Percentage of the annual dSOC change |
initial SOC |
7 |
6 |
-95% to 700% |
-111% to 1356% |
soil clay content |
6 |
0 |
-91% to 85% |
-133% to 81% |
rate of N application |
5 |
0 |
-50% to 100% |
-99% to 116% |
soil pH |
5 |
0 |
-33% to 33% |
-184% to 1% |
air temperature |
4 |
0 |
-44% to 44% |
-110% to 83% |
precipitation |
4 |
2 |
-74% to 74% |
-125% to 296% |
soil bulk density |
3 |
1 |
-69% to 79% |
-19% to 38% |
residue fraction (left on the field after harvest) |
3 |
0.5* |
0% to 500% |
-141% to 461% |
rate of organic amendment application |
3 |
0.5* |
0% to 2000%** |
-77% to 363% |
depth of tillage |
2 |
0 |
0 to 20 cm |
-121% to 26% |
depth of N application |
2 |
0 |
0 to 20 cm |
0 to 16% |
* 0.5 means that this parameter was tested to be the most sensitive under some treatments in a study.
** Some studies were excluded, because of a baseline of no organic amendment.
Table 4: Results of the meta-analysis on the sensitivity of DNDC dSOC outputs to DNDC model parameters.
Conclusions
In general, the following parameters were identified in this meta-analysis as the ones that DNDC model outputs (SOC and N2O emissions) are most sensitive to - initial SOC, soil pH, soil bulk density, soil clay content, and precipitation. In the case of N2O emissions, the model was also sensitive to the parameters governing the application of nitrogen compounds to the soil – rate, timing, and frequency of the N application.
However, it should be noted that the methods used in the studies identified in this meta-analysis for the sensitivity analysis are not always consistent. Future research could involve normalizing the methods across different studies through the use of sensitivity indices or other similar methods in order to further improve the accuracy of the results.