The Unjournal · Experimental methods pilot · September 12, 2026

From paper evidence to welfare-relevant decisions

Six detailed cases and 18 additional screens test a transparent, multiplicative protocol for tracing research through changed beliefs and decisions to outcomes and welfare.

Status: AI-assisted draft for human review. These calculations do not replace the current 0–10 impact ratings. None of the 24 papers yet supports a defensible total expected-welfare estimate for the research product.
Six-paper pilot 18-paper screen Calculation protocol Download pilot JSON Download expanded JSON
6detailed pilot cases
18additional screening cases
0defensible total welfare estimates
4 of 6conditional calculations; the other two have no welfare arithmetic

The protocol being tested

Research outputBeliefs or toolsPolicy, funding, or research decisionNatural outcomesWelfare
Funding pathway = relevant budget × research-caused allocation change × incremental outcome per dollar versus the displaced alternative × welfare per outcome − costs and harms
Coverage pathway = eligible population × research-caused coverage change × incremental outcome per covered unit × duration × welfare weight − costs and harms
Research-tool pathway = downstream decisions × P(tool improves evidence) × P(improved evidence changes a decision) × outcome difference × welfare weight − costs and harms

Every input is marked as a source estimate, analyst assumption, or unknown. Low, central, and high values are sensitivity scenarios unless a source explicitly supplies an interval. Natural outcomes remain visible when a welfare conversion is unavailable.

Distributional and moral conventions

Income cases use CRRA sensitivity at η = 1, 1.5, and 2 when beneficiary consumption is available. A first-pass health bridge treats one reference consumption-doubling-year as about 0.435 healthy-year equivalents. Animal cases use Rethink Priorities' sentience-adjusted welfare-range percentiles without applying sentience twice. Catastrophic-risk cases keep present-generation and long-future ledgers separate.

Why paper impact and evaluation value are separate

Paper impact compares decisions with the research against decisions without it. Evaluation value compares publication and use of the paper with an Unjournal evaluation against publication and use without that evaluation. An influential, well-scrutinized paper can have high welfare relevance and low value from another general evaluation.

Six-paper pilot

The detailed cases include two high-income-country papers that appeared high in the earlier ordinal ranking, a direct health intervention in India, an LMIC consumption intervention, an observed-choice animal-welfare case, and a catastrophic-risk research tool. Open each case to inspect the causal chain, inputs, arithmetic, and separate evaluation counterfactual.

Loading the six cases…

Eighteen additional paper screens

This lower-depth pass adds three cases in each of six areas. It records sourced quantities and decisive unknowns without manufacturing a common welfare score. The screening signal organizes follow-up work within a case; it is not a cross-cause ranking.

Loading the expanded assessments…

What this changes

  1. Large study populations or policy budgets no longer substitute for research-caused influence.
  2. Measured outcomes remain in auditable natural units before cross-cause conversion.
  3. High-income-country benefits use actual beneficiary circumstances rather than a broad label such as “disadvantaged.”
  4. Already-evaluated papers are assessed for the value of an update, implementation trace, or revised model rather than a duplicate review.
  5. An unknown multiplicative term stays unknown. It is neither silently set to zero nor replaced by an unexplained decimal.

Return to the full methods report for the audit of the current scores, historical human calibration, audio briefing, and framework sources.