Introduction

African countries face the challenge of keeping pace with rapidly growing food needs, which are expected to triple by 2050 (Searchinger et al., 2015). Crop yields are currently far below their potential, pointing to a possible route for Africa to increase her food supply (Beltran-Peña et al., 2020). The shortfall in agricultural production is often referred to as "the yield gap", or the difference between actual and potential yields for a specific time and place (Beddow et al., 2015;Cassman and Grassini, 2020). There is substantial scope to increase agricultural production in many African countries through intensification and extensification of agro-climatically suitable land, which would enable the region to meet much of its growing food demand (Bhalla et al., 2021;Djoumessi, 2022;Searchinger et al., 2015). The Global Yield Gap Atlas (GYGA) shows that Sub-Saharan African countries have a gap that varies from 70% to 90% of their potential yield.foot_0 Specifically, countries like Ghana and Kenya have average maize yields of 1.6 MT/ha and 1.9 MT/ha respectively, while their potential maize yields are of >7.5 MT/ha (GYGA, 2018;Hoogenboom et al., 2019). According to our estimates, Zambia has one of the highest potential maize yields in Africa at 9.8 MT/ha, but currently averages just 1-2 Mt./ha. On the other hand, closing the yield gap may expose farmers to increased risk from weather shocks; risk that they may not want or be able to bear. In this article, we ask how much of the yield gap is driven by farmer choices in the face of weather risk.

This gap is driven by a variety of factors. Recent literature decomposes yield gaps in Sub-Saharan Africa (SSA) into biological and economic constraints (Affholder et al., 2013;Beza et al., 2017;Owusu and Bravo-Ureta, 2022;Van Dijk et al., 2017). Among the latter are institutional factors such as access to credit, market, and technology (Cassman and Grassini, 2020;Frankema, 2014;Silva and Ramisch, 2019), as well as the fact that maximizing yield is rarely economically efficient (Beddow et al., 2015;Jain et al., 2019). Because inputs are costly, farmers are expected to use less inputs than the amount that would maximize yield, instead choosing input levels that maximize expected profits (Mundlak and Butzer., 2016), which results in yields that are below agronomic potential. We argue that a second constraint to increasing yield is the fact that increased yields often come with increased risk. Van Dijk et al. (2017) decomposed the yield gap into technical efficiency, allocative efficiency, economic, and technological components, although their discussion of risk is limited to the technical efficiency portion of the gap. While risk aversion may affect managerial decisions, the authors suggest that this is due to an overuse of inputs in some instances. In other words, they are agnostic about how input choices affect the variance of yields.

The degree to which farmers' risk attitudes affect yield gaps has not been quantified. Recent studies explore how input use and productivity are tied to risk. Two field experiments examined how farmers' behavior changes when they are offered insurance to reduce risk (Karlan et al., 2014;Michler et al., 2022;Wu and Li, 2023). Farmers who adopted insurance invested in riskier inputs that made yields more sensitive to rainfall. However, these experiments also found that farmers will pay for insurance when it is highly subsidized yet purchases at market rates remain low. This low demand suggests that insurance does not cover all components of risk, or some other constraint prevents uptake, such as lack of trust or understanding (Bulte and Lensink, 2022). It is thus likely that these experiments do not fully capture how risk aversion affects farmer behavior and yield outcomes. To our knowledge, no other paper shows how risk affects the yield gap.

Farmers' responses to unpredictable weather and market conditions can contribute to yield gaps. For example, weather-related stresses often result in large losses (Farooq et al., 2022;Herrero et al., 2020) and risk can drive farmers to make conservative production choices that further restrict yields. Farmers know that an increase in expected profit may also increase their risk exposure, and weather variability can make it impossible for a farmer to simultaneously maximize profit and mitigate risk (Binswanger, 1980(Binswanger, , 1981;;Just and Pope, 1978;Just and Zilberman, 1983). For example, recent work shows that fertilizer both increases yield as well as yield variability (Zhu, 2018), which explains why farmers often use less than the yield-maximizing amount of fertilizer. If farmers are risk-averse, they may prefer to reduce inputs that pay off when the weather is good but increase losses when it is bad and opt instead for inputs that mitigate risk but reduce profit. In the context of the yield gap, if an input is risk-increasing, then closing the yield gap by increasing the use of this input would require the farmer to use a riskier input combination, which, in a setting without insurance, may be detrimental to their interests (op cit.). Further, we might expect this component of the yield gap to grow as weather outcomes become more variable with climate change.

Earlier research focused on farmers' adoption of risk-reducing technologies. Horowitz and Lichtenberg (1993) show that adoption of fertilizer and pesticides are not universally risk increasing or decreasing. Hence, farmers may choose agricultural inputs that are perceived to be risk decreasing to hedge against risk that may result in a lower but safer expected yield. Emerick et al. (2016) found that the adoption of floodtolerant rice varieties in India by farmers effectively reduces their downside risk, and farmers subsequently increase their investment in modern inputs, like fertilizer. Having seeds that are resistant to floods frees resources for farmers, so they can invest in fertilizer and laborintensive practices (op cit.).

The aim of this article is to decompose yield gaps in a manner that accounts for farmers' risk behavior. The effect of different input levels on the distribution of crop and profit outcomes will vary by location (e. g., soil type, market access) and by agricultural production technology used. Specifically, we want to understand the degree to which crop yields can be increased without subjecting farmers to higher weather risk. To undertake our assessment, we combine simulated yield estimates that represent all possible weather-input combinations with primary survey data, within a conceptual framework for estimating the risk component of the yield gap. The simulated data allows us to represent farmers' knowledge about how inputs and weather affect crop yields and profits in their area, while the survey data then allows us to know how farmers respond to this distribution of potential outcomes. We contribute to Van Dijk et al. (2017) by adding the risk-aversion gap component to their yield gap estimation. Further, we investigate whether fertilizer and seed maturity are risk increasing or decreasing.

The latter would allow us to determine whether closing the yield gap would increase risk exposure from farmers in the surveyed districts.

We focus on maize production in Zambia. Zambia is a valuable case study, because it has achieved substantial yield gains since 2000 (USDA-FAS, 2019) while facing a large variation in rainfall across the country. Since most agricultural production is rainfed (only 3% of the cultivated area is irrigated), Zambian agriculture is highly sensitive to weather risks (FAO, 2005). Moreover, climate variability is increasing in regions where rainfall is low, and maize is frequently a monocrop (Chapoto et al., 2018;FAO, 2019). The results of this work provide insight into the factors responsible for yield gaps, and the steps that can be taken to address them in a time of increasing climate uncertainty.

Theoretical framework

Expected utility, expected profits and yield gaps

To understand the risk-related dimension of yield gaps, it is necessary to detail the choices and associated uncertainties a farmer makes when choosing inputs and any other production-related decision (Binswanger, 1981;Just and Pope, 1978;Loehman and Nelson, 1992;Moschini and Hennessy, 2001). Before planting, farmers choose seed type and crop mix based on their expectations of future weather and prices, and their preferences. At that moment, weather and market conditions are unknown, although a farmer's previous experiences provide insight into how yields and financial returns vary with input choice. Because inputs are costly and weather poses substantial risks, a farmer's primary goal might be either to maximize profits or to reduce risk exposure or some combination, as opposed to simply maximize yields. As a result, the farmer may choose inputs that generate expected yields that are lower than their land's maximum potential. 2We illustrate the input choices that maximize expected utility versus expected profit in Fig. 1. We show the optimal profits by input combination, again, assuming inputs are risk increasing; however, inputs can be also risk-decreasing (Tang and Luo, 2021). Since farmers might be risk-averse, they get more utility from avoiding risk than by assuming more risk (Zhu, 2018). By doing that, they are choosing inputs that result in a lower profit than for a risk-neutral farmer. In other words, assuming they use seeds and fertilizer are risk-increasing, they may choose a smaller amount of inputs that a risk-neutral farmer. As a result of farmers choices, the maximum expected utility profit π Umax is lower than the maximum expected profit solution X πmax . If farmers care about risk, they will choose an input combination that yields a certainty equivalent to the maximum profit input combination. As a consequence, their yield and profits will be lower than the risk-tolerant combination explaining part of the yield gap.

For smallholder farmers, a bad year due to a crop failure or low crop prices plus high costs of inputs, might be disastrous. Farmers often do not have easy access to credit and many Zambian farmers live near the poverty line, making it hard to recover. Therefore, it makes sense that farmers might be willing to trade off some expected profit to avoid the risk of extreme yields. If using more fertilizer increases yield in good weather but increases the risk of bad outcomes in bad weather, a farmer might be tempted to use less fertilizer than they would to maximize profit as a form of self-insurance. This self-insurance effect is expected to be larger for farmers who are more risk averse. This action of using lower amounts of risk-increasing inputs, will lead to a lower yield than the yield associated with a farmer choosing to maximize expected profits, which in turn is lower than the maximum potential yield.

Analytical framework

To quantify risk as a contributor to the yield gap (Y g ), we decompose the gap following Van Dijk et al. (2017) and we identify four distinct gap components: the economic gap (E g ), the risk-aversion gap (RA g ), the allocative-efficiency gap (AE g ), and the technical efficiency gap (TE g ). 3The total yield gap is thus:

And each gap components is:

where Y p is feasible potential yield for a given location (e.g. Zambia), Y n is the risk-neutral yield, the yield that a profit-maximizing but non-risk averse farmer would obtain, Y a is the yield of a risk-averse farmer, Y te is the technically efficient maximum yield, or the yield that can be achieved through optimal allocation of input bundles, and Y i is observed yield. The relationship between these different yield definitions and the differing input levels is shown along a production frontier in Fig. 2.

Farmers may produce at any point on or below the frontier. Imagine we observe a number of different yields (Y i ) from different farmers (i) as illustrated by the dots in Fig. 2. The difference between Y p and Y i is the total yield gap for each farmer. The size of each of the components will depend on the farmer's input choices.

The Economic Gap (E g ) is caused by the decision to forego yield in favor of profit. Potential yield, Y p , which may be obtained in field trials, is only feasible at an input level X p , the maximum level of inputs that can be selected without regard for their costs. Risk-neutral yield, Y n is achieved under input level X n , which is the point at which profits are maximized, where farmers cut back on costly inputs to the point that the potential increase in economic return brought about by that input equals its cost.

The Risk-Aversion Gap (RA g ) is the difference between the riskneutral yield, Y n , and the lower but less volatile Y a than results from risk averse farmer opting for X a quantity of inputs. The gap represents the theoretical potential difference between farmers whose objective is to maximize profits versus farmers that have risk-preferences. If farmers are risk-averse and inputs can be risk-decreasing or risk-increasing, their potential yields should be different. If risk is not important, then the riskaversion gap may tend to zero.

When the inputs are constrained due to imperfect access to input markets, the resulting yield Y te is lower than Y a , reflecting the case in which farmers make optimal use of this restricted set of inputs (X te ). This difference in yield resulting from more abundant inputs used in a risk averse way, and less abundant but optimally used inputs, leads to the allocative efficiency gap (AE g ).

Lastly, farmers have varying managerial skills, and thus on average, apply their own set of inputs (X i ) which is less than optimally leading to the technically efficient yield gap (TE g ). This component shows how much yields can be improved by using best management practices without changing the bundle of inputs being employed.

Farmers can therefore improve their yields in four ways. One is to work to increase their input use efficiency, using X i to move from Y i to yield to feasible yield. Hence, we do not estimate a technology gap. We use average seed and fertilizer levels by district to adjust the biophysical potential yield to transform it into potential feasible yields, therefore, we are assuming technology is district specific.

Y te . Another is to change their input allocation entirely, either to X a , to achieve the maximum without increasing risk exposure (Y a ), or to X n , which increases expected yields further still, but with greater risk. Lastly, if cost is not an objective, yields could be increased further by using the optimal level of inputs that money can buy X p .

Data and methods

Data sources

To evaluate the size of each of the gap components, we combine three sets of data. We first needed to determine how much different levels and combinations of inputs affected yield under a range of feasible weather outcomes for different soil types, which we simulate using Decision Support System for Agrotechnology Transfer (DSSAT) (Jones et al., 2003). In the DSSAT model, we use historical rainfall and temperature data as a proxy for weather scenarios, and we obtain the yields that could be obtained by using combinations of seed maturities and fertilizer levels. Using data on local input and crop prices, we then translate these simulated input and crop outcomes into a distribution of possible profits to capture the range of financial risk faced by farmers in each district. This dataset provided the basis for estimating Y p , Y n , and Y a . We then compare that distribution of possible profits to primary panel data from farmer surveys. We then use these primary data on farmer behavior to understand what levels of inputs farmers use, and the level of yields they obtain, for the potential distribution of outcomes Fig. 2. Components of the yield gap. Source: own elaboration adapted from Van Dijk et al. (2017).

they might obtain in their area under different weather scenarios. These primary data provided information on farmers' actual input choices and yield and were used to derive Y te and Y i . Below, we describe how we use each dataset in some detail. 4To generate the first dataset, we established a factorial experimental design that runs DSSAT simulations for every permutation of weather, soil type, and management (fertilizer, planting date, cultivar) inputs for each district in Zambia. 5 To provide the necessary weather inputs for DSSAT, we extracted daily precipitation, temperature, short-wave radiation, wind, specific humidity, and pressure data from over 31 years each district from the gridded Multi-Source Weighted-Ensemble Precipitation (MSWEP), which has a 3-hourly temporal and 0.1 • spatial resolution (Beck et al., 2019;Vergopolan et al., 2021). The weather data were extracted for the centroid of each of Zambia's districts. We extracted soil data from the World Inventory of Soil Emission Potential (WISE) (Romero et al., 2012) to identify the three most common soil profiles located within each district. For management inputs, we tested Nitrogen inputs ranging in from 0 to 300 kg/ha, applied in 20 kg/ha increments, planting dates ranging from the earliest typical planting date (Nov. 1st) to the latest typical date (Dec. 15th), in 2-week intervals. These quantities correspond to the range of fertilizer used in our primary household data. Early, medium, and late maturing maize cultivars, calibrated for the growing season length in agro-ecological zones, were simulated. These resulting yields represent potential yields under each input-soils-weather combination and assume no losses due to disease or other limitations.

To develop the second dataset on prices, we used information collected as part of an agricultural input dealer survey conducted in 2016. These data contain agricultural input prices from 62 dealers on the main maize varieties sold and seed prices during the previous growing season. We combined these data with information about seed maturity to estimate prices for early, medium, and late maturing varieties (Waldman et al., 2017). Fertilizer prices were extracted from africa fertilizer.com, which provides retail-level fertilizer prices by type and district, including both commercial and subsidized prices. We use seed and fertilizer prices to estimate the expected profits and the expected tradeoff between the average profit and its variance. 6 We then use these price data combined with the simulations to generate the bundles of inputs that maximize expected profit and the risk-averse yields where farmers choose inputs to reduce yield variance based on an average level of risk aversion. The risk aversion parameter defines farmers preference for risk. Positive values of this parameter represent some degree of risk aversion. We follow Binswanger (1980) by selecting a parameter of 0.1 which is equivalent to slight-to-neutral risk aversion. 7 Our results will be a lower bound of the effect of risk aversion on yields because lower preferences for taking risk would increase the risk-aversion component of the gap.

To obtain data on farmer decisions and outcomes, we drew on primary household surveys conducted across 12 districts that cover the different agro-ecological regions in Zambia (Fig. 3) during 2015/16, 2016/17, 2017/18, and 2018/19 harvest seasons. 8 These surveys capture detailed information on household demographics, expenditures, food security, as well as data on input use and maize production, across a broad range of variability related to rainfall, soil quality, topography, and distance to regional markets. From these surveys, we constructed a balanced panel dataset on 749 farmers' input use and maize production. To prepare this dataset for estimating Y te and Y i using empirical models, we derived rainfall and soil variables from the same sources used for DSSAT, as soil quality indicators and weather data were not collected by the survey. For soil, we used the percentages of clay and soil organic carbon between 15 and 30 cm depth and soil pH. For rainfall, we extracted total growing season rainfall (October-May) from MSWEP (Beck et al., 2019;Vergopolan et al., 2021) for each year in the panel, using the household location as the point of extraction. We use the household soil and weather data to estimate a production function using an stochastic production frontier approach (Greene et al., 2015). 9

Yield gap estimation

We followed five separate analytical steps to estimate each component of the yield gap. In step 1, we used the DSSAT simulated yields together with the market price data on maize and inputs to compute the expected profit per hectare, as follows:

where E[PY it ] is the expected revenue from maize sales, scost it is seed cost, 10 and fcost it is fertilizer cost. Farmers' expected profits were calculated as the difference between expected revenue and expected variable input costs. Expected revenue comes from multiplying potential yield from DSSAT with maize farm gate prices, while expected variable costs were found by multiplying seed and fertilizer prices by their simulated quantities. Because DSSAT does not explicitly allow for seeding rates, and we need a seeding rate to calculate seed cost, we use a weighted average seeding rate by district. Because we combine a crop model estimation with primary data on household, it is imperative to calibrate the DSSAT model using feasible input use and yields. DSSAT include input combinations assuming farmers are perfectly efficient, and no unexpected shocks ever reduce yields. For instance, DSSAT would not take a crop pest such as fall armyworm into account, leaving the model to overestimate potential yields. Likewise, DSSAT assumes production technology is the same everywhere, hence, yields from the most productive areas are attainable by farmers in the less productive areas. To account for this yield overestimation, we use our primary data to estimate the average fertilizer level and seed use per district to find the associated potential yield from DSSAT. We used the distance between the latter yield and the DSSAT potential yield (Y p ) in each district as a correction factor to scale the yield gap components from DSSAT. This adjustment enabled a more realistic comparison between observed and feasible potential yields.

We estimate profit based on farmer choices of seed and fertilizer, the most consequential inputs. Yields typically increase with the amount of fertilizer applied and tend to increase with seed maturation period. Late maturing seed varieties are generally higher yielding than those requiring shorter periods, but need a longer rainy season, while early and medium maturity seeds can be planted later. Early and medium varieties are more common where the rainy season is shorter and are often used as a hedging strategy in which planting is delayed reducing the risk of losses caused by early season droughts. We excluded other costs, such as total farmland and labor decisions, from the profit function. The total land used by each farmer does not vary significantly between years because land rental markets are not well developed. Likewise, we assume capital can be considered a sunk costs since there are no major changes in assets from what we see from the survey data. 11 In step 2, I use the expected profit function as the main argument for constructing the expected utility function. To estimate the risk-averse level of production, we imposed a standard functional form of "expected utility" or benefit, that defines the tradeoffs between the expected outcome and the variance of that outcome, reduced to a parameter that determines the degree of risk aversion (Binswanger, 1980;Holt Charles and Laury, 2002;Schoemaker, 1982). We assume then used the expected profits as the primary input for an expected utility function that follows a negative exponential form U it = -e -ρE [πit] . This functional form has the desirable properties for measuring risk aversion: it is continuous on positive values of expected profits, and it produces a constant risk aversion coefficient because it is twice differentiable (U ′ > 0; U ′′ < 0). It is a standard practice in Economics to assume smallholder farmers have some level of risk aversion based on experimental studies. We assumed a value of 0.1, which represents lowrisk aversion (thus, relatively high tolerance to risk) in an experimental setting (Binswanger, 1980). This assumption would act as a lower bound for the incidence of risk in the total yield gap. 12 In step 3, we analyzed each input bundle by district to construct the risk-neutral (x n ) and the risk-aversion bundles (x a ). The risk-neutral bundle (x n ) contained all fertilizer and seed maturities that maximized the expected profits for a given district, while the risk-aversion bundle (x a ) contained the combinations that provided the optimal tradeoff between expected mean and variance given the risk aversion parameter.

In step 4, we used the cumulative distribution function of the expected profit functions (Eq. B.1) and the mean-variance expected tradeoffs function (B.2) to obtain the risk-neutral (y n ) and risk-aversion (y a ) yields, respectively. We used the distributions of y n to identify optimal inputs for a risk-neutral farmer (x n ), and those of y a to find optimal inputs for a risk-averse farmer (x a ).

In

Step 5, we used the resulting distribution of expected profit for all combinations of inputs and yearly weather to compute the variance in profits changes as a function of inputs 13 :

Where the dependent variable is the conditional variance of the natural logarithm of expected profit. We calculated the variance from 100 randomly selected outcomes for each district using simulations. We explore the relationship between the variance of expected profits and use of inputs to understand whether the use of fertilizer and seeds increase risk exposure. This step is not necessary for estimating the yield gap components; however, it has relevant policy implications regarding reducing the risk-aversion component of the yield gap. For instance, if an input is risk-increasing, then recommending its use may increase the yield gap when weather is more variable.

In step 6, we undertook a stochastic frontier analysis (SFA) using detailed input and output information from the 4-year household panel dataset. This entailed developing a regression-based crop production function, and then using the residuals to separate the effects of management from those of weather and other exogenous shocks. We compared a Cobb-Douglas (CD) and a Translog (TL) functional form for the frontier estimation. The latter is a more flexible functional form that can accommodate non-linear relationship between inputs. Using a Log-Fig. 3. Surveyed district being used for the yield gap estimation and rainfall regions in Zambia.

11 See appendix B for more details. 12 Experimental evidence for a subset of households shows that the riskversion coefficient goes from 0.4 to 2.1, indicating moderate risk-aversion. 13 Our expected profit variance estimation may face misspecification bias (Just and Pope, 1978;Moschini and Hennessy, 2001). However, we consider the most important agricultural input management choices available in the DSSAT modeling, so we expect that this problem is being minimized. Alternatively, we have planting date as a choice variable. Simulation results are similar and are available upon request. We do not consider simulations with both planting date and seed maturity as preferred because planting date and seed maturity have significant collinearity problems and both variables are proxies from each other.

Likelihood Ratio (LR) test to evaluate the best model, we choose the TL specification. Therefore, we estimated the technical efficiency gap and the efficient yields using the Translog (TL) production function specification:

where Y idt is the log of total maize production in kilograms from household i in district d for year t, X is a vector of inputs that includes the logarithms of maize area (ha), fertilizer (kg), and the total seed planted. Household characteristics and maize growth-limiting factors are given by Z idt . Household variables included the number of household members, the age, education level, and gender of the household head, distance to the nearest tarmac road, and three dummy variables: one indicates whether the household was able to borrow 2500 Kwacha, the second indicates whether the household has a water pump, and the third indicates whether the household was enrolled for election vouchers under the Farmer Input Support Programme (FISP). 14 Maize growthlimiting factors included the logarithm of growing-season rainfall, a dummy that indicates whether the farmer uses recycled maize seed, the percentage of sand, clay and carbon content in the first 15 cm of the soil, and its ph level. Yield is assumed to also be affected by a random error v it that is assumed to follow a normal distribution with mean zero and variance (v it ~iid N(0, σ v 2 )) and μ it which is the non-negative unobservable random error associated with the technical inefficiency of the i-th household, and w i which is the random effect for each household (Belotti et al., 2013;Greene et al., 2015). 15

Implications of DSSAT and SFA assumptions

Comparing outcomes from different models is challenging and it should be done with caution. DSSAT and SFA have different assumptions that have implications over the yield gap results. While the range of values of fertilizer use and seed maturities in the simulations are feasible and are informed by our primary data, DSSAT assumes there are no inefficiencies in the use of inputs and there are no limitations such as diseases or pests. Farmers might have different expectations and results in trying to deal with these problems. For instance, the model does not consider the recent armyworm infestation in maize production in Zambia (Hadunka, 2019). Thus, changes in farmers' behavior and yield reductions due to armyworms were not incorporated in the DSSAT model.

The estimation of the technical efficiency gap relies on estimating a production frontier that represents the maize production process. Omitting relevant inputs might underestimate efficiency. For example, we include total number of household members as a proxy of family labor and having water pumps as a proxy for capital. The fact that we do not know intra-household time allocation, the quality of that labor or capital uses, these might lead to attribute a portion of the efficiency gap to the missing inputs. While we include the main agricultural inputs, it is important to have this caveat in mind.

Lastly, using the SFA to obtain optimal seed and fertilizer levels could also be calculated using the input coefficients and prices; however, it does not account for how input levels relate to every possible weather outcome and the non-linear agronomic relationship between inputs. DSSAT accommodate these issues by considering how input combinations and weather realizations affect yields. Therefore, optimal input levels from SFA are underestimating the effect of weather on the yield gap.

Results

Potential yields, risk and input choices

In Table 1, we present the theoretical optimal input bundle to maximize expected profit (risk-neutral) versus the bundle that maximizes the tradeoff risk averse farmers would make between the mean and variance of profit by district. Our simulations suggest that to maximize expected profit, a risk-neutral farmer may use from 180 to 300 kg/ha of fertilizer, while a risk-averse farmer might use between 80 and 280 kg/ha depending on their location. In terms of seed maturity, a farmer that aims to tradeoff mean and variance of profit, in general, would be expected to choose early or medium maturing seed over latematuring varieties. Risk-neutral farmers tend to choose medium maturity seeds to maximize expected profits. We do observe some locations where risk-averse farmers would be expected to select later maturing seed than their risk-neutral counterparts, but in all of those cases, they also use far less fertilizer.

Components of the yield gap

In Table 2, we present the average actual yields (Y i ), technically efficient yields (Y te ), risk-averse yields (Y a ), risk-neutral yields (Y n ), and feasible potential yields (Y p ) by surveyed district in Zambia. 16 Observed yields were substantially lower than the feasible potential yields, and the average gap is 91%. In places like Choma and Petauke, with worse agroecological conditions for maize production than the rest of the districts under consideration, the observed yields are higher relative to yields in other districts but remain far from technically efficient levels. The The FISP has been in place as a national program for >10 years. So, we include the dummy for e-voucher participation to control for a demand shock to the input markets that affect both recipients and non-recipients alike. 15 To empirically estimate the production function, we used the True Random Effect (TRE) Stochastic Frontier first introduced in Greene ( 2005) and implemented in STATA by Belotti et al. (2013). The TRE model is an adaptation of the standard fixed effects model augmented by the inefficiency effect. The model is estimated using maximum likelihood and allow to separate the effect of individual characteristics from management differences. 16 See the supplementary material for more detail about the estimation of the yield gap components.

N. Gatti et al.

technically efficient yields are significantly lower than the risk aversion yields showing that it is still feasible to reallocate inputs without increasing exposure to risk, possibly through the development of agricultural input markets. There is a correspondence between risk-averse yields, risk-neutral yields, and feasible potential yields. Mpongwe, Mkushi and Chinsali farmers are the ones with more potential to increase productivity if risk and economic constraints are removed. We used the results from DSSAT to calculate and compare riskaversion, risk-neutral, and feasible potential yields for each surveyed district (Fig. 4). Risk-averse yields ranged from 4000 to 9000 kg/ha and were substantially lower than risk-neutral yields (5800-10,800 kg/ha). Risk-averse yields were higher in the more agro-climatically favorable central districts, and lowest (4000-6000 kg/ha) in the southern districts of Choma, Petauke, and Namwala, which are riskier because of higher inter-annual precipitation variability (Waldman et al., 2019). In contrast, risk-neutral yields were highest in the northwestern districts, where rainfall and thus feasible potential yield is highest. Compared to southern districts, there is less variability in rainfall and providing a more stable environment for production that translates into lower risk.

Farmers with different risk profiles are expected to select different input bundles to optimize mean-variance tradeoffs, as implied by the differences between the fertilizer levels and cultivar choices relative to those that maximized expected profits (Table 1). We see that fertilizer levels are lower for farmers who trade off mean and variance compared to expected profit maximizers and no clear pattern arises in terms of seed choices. However, when studying the marginal effects of seed on expected profits, we find that the profitability of fertilizer and seed maturities depends on the district, and earlier maturities can reduce expected profits. These results highlight why it is problematic to label inputs as universally risk increasing or risk decreasing. Depending on the location, the input level, and the combination, fertilizer and seed maturity can be either.

Examining the yield gap component as a share of the overall average gap between actual and feasible potential yield by district, the most important component in order of magnitude is the technical efficiency gap which is 33% on average, implying that farmers can improve productivity by increasing efficiency in the use of their actual combination of inputs. The second component is the allocative efficiency gap (32%), suggesting that farmers could increase yield by changing their input mix and without increasing risk (Fig. 4). We calculate a risk aversion gap of 24% on average showing that there is potential to reduce the gap from increasing insurance uptake. Lastly, the economic gap is on average 10% showing that the profit maximizing yield is not far from the maximum agronomic yield.

Discussion

Main results

While we estimate the risk-aversion gap is 24%, this is likely an underestimate since we use a relatively low degree of risk aversion. That said, this gap could be reduced if farmers had access to products such as crop insurance allowing farmers to invest in riskier, but more productive, input combinations. However, the solution to close the riskaversion gap might not be necessarily to increase the use of fertilizer or early maturities varieties everywhere in Zambia. For instance, originally Zambia's input subsidy program provided the same input packages across different areas having a low impact on productivity (Mason et al., 2020). With the subsidy, farmers are using seed maturities that have a similar expected yield, but lower variance droughts are becoming more frequent. In the south of Zambia, farmers report to be choosing combinations of early and late maturity seeds as well as recycled seeds in It is worth noticing that the allocative efficiency gap is the second most important component, which indicates that farmers may face constraints to accessing high-yield agricultural inputs and markets (Satyasai and Pereira, 2019;Holden, 2019). Silva et al. (2017) studied the yield gaps with a similar approach although for rice production. Their resource gap might be equivalent to the summation of our risk and allocative components because both are input allocation related gaps. They find that the technical efficiency gap is nearly 40% of the gap while the resource gap varies between 31% and 40% of the yield gap depending on the rice season. In our case, allocative and risk taken together is 58% on average. While the gap is higher in our case, the advantage of our estimation is that we can identify how much of the gap is related to risk management embedded in input choices as opposed to limitations due to market conditions. The comparison with Van Dijk et al. ( 2017) is more straightforward. The authors find that the most important component in Tanzania is the TE gap (52%), followed by the AE gap (47%), the Economic Gap (34%), and lastly the Technology gap (33%). At the national level, we find that the allocative portion is most relevant portion. If we add up the risk aversion gap and the allocative gap, Zambia has higher gaps due to input allocation than Tanzania.

Our results in terms of a 33% technical efficiency gap are in line with other studies in Zambia. Ng'ombe (2017) finds that smallholder farmers from Zambia have an average of 30% of technical efficiency gap.foot_6 In a cross-sectional analysis, Chiona et al. (2014) find that farmers are 50% technically efficient on average in the Central province of Zambia. Both studies use data prior to our sample, which suggests that efficiency in the use of inputs has either remained stable or decreased compared to the 30-50% estimation in the literature.

Policy implications

Promoting policies based on a 'one-size-fits-all' solution overestimates the potential for reducing the yield gap. Closing the yield gap is site specific. It requires to characterize the socio-economic conditions of farmers, their decision-making process and the potential yields that are feasible conditional on risk exposure. These set of conditions are variable by agro-ecological zones, district, village and even between neighboring farmers.

Overall, not accounting for farmers' risk exposure in different zones may increase the yield gap. For instance, policies promoting long maturity seeds where the planting dates are narrowing because of changes in rainfall patterns, may overestimate their potential role in increasing productivity. Instead, we may see an increase in the gap because these seeds are more vulnerable to negative yield shocks resulting from weather, increasing the risk of their use. In the districts of Petauke and Namwala, which have the largest risk-aversion component of the total gap promoting longer maturity seeds can induce higher potential losses due to weather variability. Hence, if weather risk is increasing, high-yield technology is increasing the gap, not decreasing it.

Developing insurance alternatives may encourage farmers to take risks and increase yields. Input choices are related to farmers' risk profile; hence, farmers may invest in risky inputs if they can access to insurance. Otherwise, farmers will continue to invest in inputs that reduces the expected variability of yield at the cost of obtaining a less than economically efficient yield. While the government can create incentives for the development of insurance markets, it is important to acknowledge that, formal insurance may crowd out informal channels through which farmers hedge against risk (Kramer et al., 2022;Bulte and Lensink, 2022). For instance, private arrangements or relational contracts with agro-dealers compete with traditional insurance in developing countries. Hence, it is possible that subsidizing access to formal mechanisms alone might not be an effective tool for agriculture development.

The allocative and technical efficient gaps require farmers' investments in physical and human capital. Assuming there is a movement towards better market access, these components of the gap might even increase in the short run. In other words, new technologies require longrun investments, farmers might be even more reticent to make them if they increased risk.

The technical and allocative gaps could be partially addressed with pro-market interventions. For instance, the 2017/18 version of the FISP program in Zambia moved towards changing the traditional input package delivery to a modern conditional cash transfer program (Kuteya, 2019). This means that, farmers are purchasing inputs that better fit their preferences instead of receiving a pre-stablished agronomic package. However, this is not the panacea because there are important structural deficiencies in rural areas that limit market expansion. Access to tarmac roads, markets and connections to district towns are necessary conditions that are more the norm than the exception in rural Africa (Satyasai and Pereira, 2019;Quium, 2019). Otherwise, agro-dealers expansion would be limited to zones that they already serve and not start new business in more distant areas.

Conclusions

A growing body of literature has targeted closing the agricultural yield gap as the primary policy objective to increase agricultural productivity, reduce food insecurity and improve household welfare. However, fully closing the yield gap poses a substantial risk for farmers because adopting high-yield technology expose farmers to greater variation in outcomes. In the context of climate change with increasing weather risks, farmers' input decisions are guided by the yield mean and variance tradeoff. The lack of insurance and the high dependence on rain-fed agricultural systems make farmers target low but safe expected yields. Our paper contributes by adding the role of risk as a determinant of the yield gap, presenting a case study for Zambia.

Our work contributes to identifying the potential for reducing the yield gap. Our approach provides a novel contribution by combining crop modeling and survey data to study its determinants. We incorporate behavioral assumptions to determine the role of risk aversion on agricultural productivity, which leads to identifying where and how yields gains can be achieved. Our results suggest that productivity gains should mainly come from improving the use of the available technologies and the development of input markets so that farmers can obtain the necessary amount of inputs to be economically efficient. Moreover, the reduction of yield gaps may come from incentivizing the development of insurance instruments. However, a pro-market initiative also requires infrastructure development that help input providers to reach distant rural areas.

While Zambia has favorable climatic and biophysical conditions for agricultural production, we found that increasing the use of high-yield technologies might not close the yield gap because it will underestimate the effect of risk mitigation on agricultural production. This current gap might be exacerbated in the context of future climate change, so targeting the areas where productivity improvements are possible without increasing risk will help to reduce the yield gap without exposing farmers to unnecessarily increased risk.