Second Impact Evaluation Report: Mango Results

Preliminary Draft Results

June 2017

www.rti.org

RTI International is a registered trademark and a trade name of Research Triangle Institute.

Mango Summary of Findings

2

• Long Season: Overall production numbers for both intervention and comparison farmers did not change significantly between 2015 and 2016. While total losses increased in absolute terms for the 2017 Long Season, intervention farmers saw a 1.6 % reduction in post-harvest loss, while comparison farmers observed a 7 % increase in post-harvest loss.
• Short Season: Production numbers increased dramatically between 2015 and 2016 for both intervention and comparison famers, however there was no significant difference in average increases between the intervention and comparison groups. Similarly, there were no significant differences in total loss, pre-harvest loss or post-harvest loss.
• Overall: Technology use, by means of scouting for fruit fly and using fruit fly traps, was significantly higher for the intervention group of farmers.

​

• For both the long and short seasons for 2016 intervention farmers are significantly more likely to have attended a training on harvesting, PHL, production or quality and more likely to report they had attended a TNS-led training.
• intervention farmers were more likely to be a member of a farmer group for 2016, as well as being a member of an active farmer group.

​

• Only a single question was asked about loans in 2015, which limited the number of comparisons that could be made between 2015 and 2016. Prospectively, from 2016 onward there will be more information gathered on access to credit. Overall, access to credit appears to be low for farmers regardless of intervention/comparison group, or long/short season.

​

• Brokers constitute the largest buyers for both groups of farmers. For the 2016 long season, there was no statistically significant indication intervention farmers had selling habits different from comparison farmers. In 2016 approximately 81 percent of the intervention farmer sales and 73 percent of the comparison farmer sales (for the sampled populations) went to brokers.
• For the 2016 short season, intervention farmers were significantly more likely to sell their harvest to a farmer group than were comparison farmers.

​

Understanding Crop Losses Across the Value Chains

3

Fruit fly traps, harvesting tools

Crates

Solar drying, cold storage

Crates, bins, trader training

Buyer Forums

Question to TNS: At what points of loss beyond farmer surveys are monitoring and evaluation efforts capturing losses?

Mango Conceptual Difference-in-Difference Analysis

DRAFT

4

We followed the same intervention and comparison farmers between 2015 and 2016. Based on these linkages over time, we calculated the difference between each farmer’s 2015 and 2016 results and then averaged the differences for each group. A significant difference between the groups suggests an intervention effect.

Long Season

DRAFT

5

Mango: Long Season Production & Loss

DRAFT

6

• Statistically significant differences in favor of the intervention group for % total loss and % post-harvest loss

​

• % Post-harvest loss was reduced by 1.6 % for intervention farmers, and increased by 7% for comparison farmers

Difference-in-Difference: Percent Total Loss

DRAFT

7

Total production loss attempts to capture the full percentage of losses farmers experience pre- and post-harvest loss, and is taken as a percentage of total production.

Difference-in-Difference: Percent Pre-Harvest Loss

DRAFT

8

Difference-in-Difference: Percent Post-Harvest Loss

DRAFT

9

Difference-in-Difference: Total Loss in Kilograms

DRAFT

10

Total production loss attempts to capture the full percentage of losses farmers experience pre- and post-harvest loss, and is taken as a percentage of total production.

Difference-in-Difference: Pre-Harvest Loss in Kilograms

DRAFT

11

Difference-in-Difference: Post-Harvest Loss in Kilograms

DRAFT

12

Short Season

DRAFT

13

Mango: Short Season Production & Losses

DRAFT

14

• No statistically significant differences in loss estimates between comparison and intervention groups

​

• Increases in loss estimates for both groups across categories

Difference-in-Difference: Percent Total Loss

DRAFT

15

Percent total loss for the short season was, on average, 114 percent higher in 2016 for the intervention group.

​

For the comparison group short season , % total loss was, on average, 208 percent higher in 2016 than in 2015.

​

There was a difference in% total loss for short season intervention and comparison farmers of 94 percentage points, favoring the intervention farmers, though all farmers’ total losses were greater in 2016.

Total production loss attempts to capture the full percentage of losses farmers experience pre- and post-harvest loss, and is taken as a percentage of total production.

Difference-in-Difference: Percent Pre-Harvest Loss

DRAFT

16

% pre-harvest loss for the short season was, on average, 188 percent higher in 2016 for the intervention group.

​

For the comparison group in the short season, % pre-harvest loss was, on average, 292 percent higher in 2016 than in 2015.

​

The short season difference between the % pre-harvest loss for intervention and comparison farmers was 104 percentage points, favoring the intervention farmers, though all farmers’ pre-harvest losses were greater in 2016.

Difference-in-Difference: Percent Post-Harvest Loss

DRAFT

17

Short season % post-harvest loss was, on average, 60 percent higher in 2016 for the intervention group.

​

For the short season comparison group, % post-harvest loss was, on average, 151 percent higher in 2016 than in 2015.

​

There was a difference of 92 percentage points between the intervention group and comparison group, favoring the intervention farmers, though both groups experienced greater losses in 2016.

Difference-in-Difference: Total Loss in Kilograms

DRAFT

18

Overall, short season production (kg) was much higher, so total losses were higher.

​

Total short season losses (kg) were , on average, 664 percent higher in 2016 for the intervention group.

​

For the comparison group, total short season losses (kg) were, on average, 421 percent higher in 2016 than in 2015.

​

The difference between the total (kg) losses for intervention and comparison farmers was 244 percentage points, with a lower increase favoring the comparison farmers. However, all farmers saw more total loss (kg) in 2016.

Total production loss attempts to capture the full percentage of losses farmers experience pre- and post-harvest loss, and is taken as a percentage of total production.

Difference-in-Difference: Pre-Harvest Loss in Kilograms

DRAFT

19

Overall, short season production (kg) was much higher, so pre-harvest losses were higher.

​

Pre-harvest short season loss (kg) was, on average, 822 percent higher in 2016 for the intervention group.

​

For the comparison group, pre-harvest short season loss (kg) was, on average, 385 percent higher in 2016 than in 2015.

​

The difference between the intervention and comparison farmers was 437 percentage points, with a lower increase in pre-harvest loss favoring the comparison farmers. However, all farmers saw more pre-harvest loss (kg) in 2016.

Difference-in-Difference: Post-Harvest Loss in Kilograms

DRAFT

20

Overall, short season production (kg) was much higher, so post-harvest losses were higher.

​

Short season post-harvest (kg) loss was, on average, 472 percent higher in 2016 for the intervention group.

​

For the comparison group, short season post-harvest loss (kg) was, on average, 451 percent higher in 2016 than in 2015.

​

The difference between the post-harvest loss (kg) for intervention and comparison farmers was a mere 21 percentage, indicating that the farmers saw more post-harvest loss (kg) in 2016.

Technical Documents

DRAFT

21

Mango Data Collection: Timing & Locations

DRAFT

22

• April 2016 survey was conducted by TechnoServe with data collection and analytical support from data collection firm VSAL.

​

• Important to note that the sample between comparison and intervention groups was not balanced, implicating levels of statistical significance

​

• Analysis on 2017 long season data forthcoming, but not included in this round of analysis due to time constraints

​

​

Mango Data Collection: Sample Sizes

DRAFT

23

1. 842 farmers were interviewed for the 2016 long season, but 655 were able to be linked to either a intervention or comparison designation.

Definitions of Loss

DRAFT

24

• Pre-harvest loss is the loss of mangos prior to harvest, due to diseases, theft, shriveling, being eaten by stray animals, diseases, and damage before harvesting. The percentage is these losses divided by the total produced, which is the sum of harvested mangos and all lost mangoes during production.

​

• Post-harvest losses occur after harvest, and include waste incurred after harvesting, being fed to livestock (presumably due to poor quality), and other losses incurred after the harvest. Post-harvest losses are taken as a percentage of the quantity of mangos harvested (not produced).

​

• Post-harvest losses occur after harvest, and include waste incurred after harvesting, being fed to livestock (presumably due to poor quality), and other losses incurred after the harvest. Post-harvest losses are taken as a percentage of the quantity of mangos harvested (not produced).

Mango Analysis Methods

DRAFT

25

Seasonal Differences: The panel datasets collected in three rounds between August 2015 and April 2016 allow us to evaluate the long season (September to April) separately from the short season (May to August)

Difference-in-Difference: Results are taking an individual farmer’s results minus his/her results from 2015, taking advantage of the panel dataset

Significant levels of data cleaning and management were required to construct the panel datasets for long and short seasons. The August 2015 questionnaire collected information on both the short and long seasons, which limited the number of intervention and comparison farmers that could be linked.

• From the Kenya long season data, it was possible to link 524 intervention farmers and 131 comparison farmers between the 2015 and 2016 seasons, though not every farmer had complete data in each indicator area.
• The short season data yielded 524 intervention farmers and 111 comparison farmers that could be linked between the 2015 and 2016 short season harvests.
• To analyze these data, certain assumptions had to be made about missing data; though it was possible to link large numbers of farmers, it was possible to analyze fewer than the full panel.

​

Mango: Percentage Losses

DRAFT

26

Mango: Technologies

DRAFT

27

• Statistically significantly higher level of uptake of scouting for fruit flies and use of fruit fly traps in both short and long season

Long Season

Short Season

Mango: Training & Aggregation

DRAFT

28

Long Season

Short Season

• Long season: for training on the topics of harvest, PHH, production and quality, the intervention group farmers finished the 2016 long season with a higher level of training than the comparison farmers, the results of which are statistically significant

​

• Short season follows a similar pattern: significantly higher continued or new attendance for TNS/YieldWise-led trainings, harvesting training, PHH, production and quality. Continued/new membership in farmers groups for 2016 were significantly higher for intervention group farmers, regardless of whether being asked if the farmer group was active or not

Other: Access to Finance, Markets

DRAFT

29

• Only a single question asked about loans in 2015, limiting the number of comparisons- no statistically significant results for long or short season

​

• Access to markets
• Long season: no statistically significant differences between farmers in either group selling to brokers, at roadside markets, through farmer groups, to driers, puree processors, or other buyers
• Short Season: Significant difference between intervention farmers and comparison farmers selling through farmer groups, in favor of intervention farmers.

​