Targeting, Personalization, and Engagement in an Agricultural Advisory Service

Why this paper is being considered

The paper reports iterative experiments on targeting calls in an agricultural advisory service serving over one million farmers in India. Policies designed with current data gave much smaller on-policy gains than off-policy evaluation suggested (up to 8 percent). It bears on how digital advisory services should target and personalise at scale. An evaluation would focus on the on-policy versus off-policy gap, the engagement outcome, and whether engagement translates into farm outcomes.

AI-generated criterion ratings and reasoning

Scores come from AI prioritization; the accompanying explanations include AI-assisted source checks. These are provisional judgments, separate from human ratings and commissioned evaluations.

What the paper says

We conduct a series of iterative experiments to evaluate approaches for optimally targeting calls for an agricultural advisory service serving over one million farmers in rural India. We estimate the value of alternative targeted policies using “off-policy” evaluation on data from randomized call assignments. When we evaluate off-policy using held-out data from the same time periods used to design the targeted policies, we find that targeted policies increase engagement by up to 8%. However, when we design a policy using current data and then implement the policy for randomly selected users in subsequent weeks, our “on-policy” evaluation estimates show realized gains that are substantially smaller. We develop tools to diagnose the causes of this underperformance and adopt a transfer-learning approach to policy learning and off-policy evaluation that accounts for temporal changes in farmer behavior, substantially improving off-policy estimates of performance in subsequent weeks. We further develop novel approaches to targeted policy design that respond to organizational objectives and resource constraints, including distributional goals (e.g., reaching women farmers) and prioritizing farmers likely to benefit most from the information on downstream outcomes (e.g., yields). We propose a method for organizations to quantify trade-offs between objectives, and we demonstrate the value of targeting for alleviating these trade-offs.

Claims to check

• Iterative randomized-call experiments plus off-policy evaluation can estimate the value of alternative targeting/personalization policies for an advisory service serving over one million Indian farmers.
• Optimally targeted and personalized calling policies increase farmer engagement relative to untargeted or default policies.
• Off-policy (counterfactual) evaluation on RCT call data yields credible value estimates for policies that were not themselves fully deployed.

Opus 5.5 re-evaluation (experimental)

Read with care. This is an experimental re-evaluation by a different model (Claude Opus 5.5, 2026-10-01). Across a calibration sample Opus scored about 15 points lower than GPT-5.5, so 15 points are added to compare it with GPT-5.5 scores. The calibration is a small sample (n=44) and is not human-validated, and the two models disagree about the order of papers within the top group. The AI score at the top of this page is the usual evaluation-priority score shifted by the difference between the adjusted Opus score and the earlier holistic score; the synthesis uses that shifted value, and the unshifted score is shown beside it. On the dashboard you can switch the Opus scores off. Read the methods note.

Different reference score. The earlier score for this paper came from an older Opus run (opus), not GPT-5.5, so no +15 offset is applied: the shift is the Opus 5.5 score minus that earlier score, which assumes the two Opus versions share a scale.

Dashboard details and provenance

Full AI dashboard scoring rationale

This is a high-prominence NBER working paper by Susan Athey (a leading figure in causal ML and off-policy evaluation), Shawn Cole (a well-known development/finance economist), and Shanjukta Nath, studying how to optimally target and personalize an agricultural advisory service reaching over one million smallholder farmers in rural India. It sits squarely in Unjournal's strongest cause area (development economics/LMICs) and speaks directly to a live operational question for agricultural extension programs: given scarce contact budgets, whom should you call, when, and with what content to maximize engagement and downstream farmer outcomes. The methodological contribution — iterative field experiments combined with off-policy (counterfactual policy) evaluation to value alternative targeting rules — is exactly the kind of ML-for-policy design where an independent evaluation adds value, because off-policy evaluation is notoriously sensitive to assumptions (propensity estimation, overlap/positivity, variance of importance weights) that referees at a general venue may not scrutinize closely. It is a working paper with no peer review yet, so feedback is timely and actionable. The main caution is honest calibration: the abstract emphasizes engagement/uptake of calls rather than final welfare outcomes (yields, income, farmer well-being), so decision-relevance hinges on how well engagement proxies for real impact — worth flagging but not disqualifying.

AI decision-relevance rationale

Agricultural advisory/extension services (e.g., Precision Agriculture for Development-style digital advisory, Digital Green, government extension programs) face a concrete allocation problem: contact capacity is far smaller than the farmer base, so targeting and personalization rules directly determine cost-effectiveness. Evidence on which targeted-calling policies raise engagement — and ideally downstream adoption — informs how funders and implementers spend limited outreach budgets across millions of farmers. Off-policy evaluation methods that let programs value new policies without running every arm as a fresh RCT are themselves decision-relevant, since they promise cheaper iteration for any org running a large advisory service.

AI timing assessment

Newly posted NBER working paper (w34951), not yet peer-reviewed, so this is the maximum-value window: independent evaluation of the identification and off-policy evaluation assumptions could inform revisions before publication, and the methods are directly reusable by implementers now.

Public attention and use

The paper is hosted by Harvard Kennedy School and Harvard Business School pages, and the broader program has generated practitioner writing. We found no independent critique of this paper.

This was a quick check (a few targeted searches). Treat the gaps below as especially uncertain.

What the search did not establish

• This was a quick check; treat the gaps as especially uncertain.
• No news, forum or direct policy use surfaced.
• The most decision-relevant next signal is whether advisory providers adopt the targeting methods.

Quick search checked 2026-10-01. Search scope: Targeted exact-title searches across the open web, institutional and author pages, news and public social-media results, EA Forum/LessWrong, and policy/white-paper contexts. Evidence records distinguish commissioning or report use from independent discussion, media attention, indexing, and post-publication policy use. A search miss is reported as uncertainty, not proof of absence. Entries with check_depth "quick" rest on roughly one to two searches and should be read as especially uncertain; entries checked on 2026-10-01 were added for the shortlist of papers without human ratings.

Human feedback so far

No human ratings have been submitted for this paper yet, so there is no human aggregate or synthesis score. If you know the work, rate or discuss it on the dashboard card; the team reviews feedback before it affects prioritization.