# AI-assisted working evaluation: The Economics of Recursive Self-Improvement + What If Automating AI R&D Triggers an Intelligence Explosion?

**Evaluation date:** 2026-10-04

**Last attention check recorded:** 2026-10-05

**Storage reconciliation:** 2026-10-07


> **AI-assistance disclosure.** This working assessment supports The Unjournal’s prioritization process. Its judgments and ratings have not been adopted as a commissioned human evaluation or a team decision. A human evaluator should verify the factual claims, methods, and ratings before signing or submitting an official evaluation.

The substantive analysis below preserves the fuller existing draft. The attention date records the earlier public-brief check; storage reconciliation does not imply a new evidence search.

## Papers

- Cunningham et al., [*The Economics of Recursive Self-Improvement*](https://arxiv.org/abs/2609.15802), submitted 14 September 2026.
- Chan et al., [*What If Automating AI R&D Triggers an Intelligence Explosion?*](https://www.governance.ai/research-paper/what-if-automating-ai-r-d-triggers-an-intelligence-explosion), 28 September 2026.

## Executive assessment

The Cunningham et al. paper is a useful conceptual advance because it replaces the vague claim “AI can improve AI, therefore an intelligence explosion follows” with explicit feedback conditions. Its core point is correct and important: automation of AI R&D is not sufficient for self-sustaining acceleration. The relevant question is the total gain around the feedback loops, including diminishing returns and bottlenecks.

The paper's **formal framework is much stronger than its calibration**. The graph/elasticity representation is a clean way to organize a complicated system, but the empirical threshold is built from parameters that are only loosely measured. The paper itself is appropriately cautious, calling the calibration tentative and emphasizing large uncertainty and possible model misspecification.

The most decision-relevant number is the comparison between a roughly **15% AI-R&D-productivity uplift per one-unit capability increase** needed under the paper's current calibration and a rough **9% observed uplift** inferred from reported AI-engineer productivity since coding agents. This comparison should not be treated as a precise “distance to takeoff.” The 15% threshold depends on the mapping from algorithmic efficiency to the Epoch Capabilities Index, assumptions about effective R&D effort, and the production structure. The 9% estimate is not a clean causal elasticity: it combines changing models, tools, tasks, inference use, researcher behavior and lab processes.

The policy-facing Chan et al. paper is substantially more confident and urgent in tone. It points to fast-rising internal AI-R&D automation, including Anthropic's self-reported 26% “AI leads” share in August 2026, and argues that an intelligence explosion could compress years of AI progress into months or less. This is a plausible scenario that deserves analysis, but **AI R&D automation share is not the same object as feedback-loop gain**. A model can perform a larger share of current R&D tasks without causing proportionately faster frontier progress, especially if automated tasks are lower-leverage, if humans/experimental compute/data remain bottlenecks, or if productivity improvements substitute for rather than multiply total effort.

The paired evaluation therefore changes the weight I would place on the public argument: the technical paper strongly supports **measuring the relevant elasticities and bottlenecks**, while it does not establish that the threshold for self-sustaining acceleration is likely to be crossed on any particular timeline. The white paper's specific policy proposals rest on additional judgments about probability, timing, harms, institutional effectiveness and intervention costs that are not estimated by the Cunningham model.

## Claim map

### Claim 1: Full AI-R&D automation does not by itself imply self-sustaining acceleration

**Assessment: strong.**

Cunningham et al. explicitly distinguish R&D automatability from self-sustaining acceleration. In the simplest model, algorithmic efficiency can improve without its growth rate increasing if ideas become harder to find. In richer models, human labor, experimental compute, inference compute or data can bottleneck the feedback loop.

This is an important correction to popular formulations of recursive self-improvement. It is also broadly consistent with semi-endogenous growth theory: a technology can contribute to research without overcoming diminishing returns.

### Claim 2: Feedback strength can be represented as products and sums of local elasticities around loops

**Assessment: strong as a local organizing framework, with scope conditions.**

In the baseline model, the total elasticity of new algorithmic improvements with respect to the stock of algorithmic efficiency combines a direct self-feedback term with a capability-mediated loop. The capability-mediated contribution is the product of the elasticity from algorithmic efficiency to capability and the elasticity from capability to algorithmic improvements.

This is mathematically natural under differentiable production relationships. It is useful because it exposes where a seemingly strong loop can fail: one weak edge can reduce the whole product.

The limitation is that these are **local elasticities**. Their values can change sharply with scale, task mix, complementary inputs and time lags. The graph is therefore not by itself a forecast. When edges change endogenously or production has strong complementarities/non-separabilities, static local elasticities can be a poor guide to dynamics far from the calibration point.

### Claim 3: With the paper's current calibration, self-sustaining acceleration requires roughly a 15% increase in AI R&D productivity per one-unit increase in capability

**Assessment: useful back-of-the-envelope threshold, low-to-moderate empirical precision.**

The paper measures capability using the Epoch Capabilities Index and assumes algorithmic efficiency and training compute enter capability through “effective compute.” It estimates a capability elasticity with respect to algorithmic efficiency of roughly 6.5 and combines this with an estimate of returns to aggregate R&D effort.

The 15% threshold is not a structural constant. It moves if:

- algorithmic efficiency is not equivalent to training compute in its effect on capability;
- the Epoch index does not map stably to economically relevant research capability;
- the return to R&D changes as research scales;
- experimental compute, data or humans become binding complements;
- the mix of R&D tasks changes;
- time delays between research, experiments, training runs and deployment matter.

A particularly useful sensitivity figure would show the threshold for a broad grid of the three core elasticities rather than foregrounding one 15% crossing point.

### Claim 4: Current AI-R&D productivity uplift is roughly 9%, below the 15% threshold but increasing

**Assessment: directionally informative; numerically fragile.**

This is the weakest empirical link in the headline calibration. The paper itself says there is almost no evidence that maps cleanly to the elasticity of R&D effort with respect to AI capability.

A reported engineer-productivity uplift can differ from the needed elasticity for many reasons:

- tool adoption changes at the same time as model capability;
- inference compute per researcher rises;
- researchers select tasks suited to agents;
- “productivity” can mean task throughput rather than frontier algorithmic progress;
- automated work may be easier or lower-leverage than the remaining human work;
- human review, experiments and integration can bottleneck end-to-end progress;
- a lab's organizational response can amplify or dampen technical capability.

The 9%-versus-15% comparison is therefore best read as an order-of-magnitude diagnostic, not a measured margin of safety.

### Claim 5: Anthropic's 26% “AI leads” statistic is evidence that the feedback loop is strengthening

**Assessment: relevant evidence, but not a direct measure of the required elasticity.**

Anthropic's September disclosure says Claude “leads” 26% of measured internal AI R&D work as of August 2026, meaning the AI completes most of a task end-to-end from a high-level prompt under human supervision. Anthropic also reports that no measured category is fully autonomous. The measure is internally constructed and, as reported publicly, not an independent audit of frontier R&D productivity.

The trend is important because it shows rapid change in task-level automation. But three quantities must not be conflated:

1. share of tasks with substantial AI participation;
2. average researcher/task productivity uplift;
3. elasticity of frontier AI progress with respect to AI capability.

Only the third maps directly to the core feedback condition. A rise in the first can occur with a modest second, and a large second can still translate into a small third if automated tasks are not the bottleneck.

### Claim 6: If the threshold is crossed, years of AI progress could be compressed into months or less

**Assessment: possible within some model paths, but timing is not established by the threshold alone.**

A local elasticity above one can imply an increasing growth rate in simplified models. How quickly this becomes an “intelligence explosion” depends on how far above the threshold the system is, starting levels, lags, reinvestment, bottlenecks and whether elasticities remain high as the system changes.

The Cunningham paper defines “intelligence explosion” very strongly as capabilities going to infinity in finite time, while the Chan et al. paper uses the term more operationally for extreme acceleration. The paired evaluation should keep these definitions separate.

The technical model therefore supports the **logical possibility** of rapid acceleration under strong enough feedback. It does not itself establish a date or a high probability for the white paper's “years to months” scenario.

## Narrow versus broad capability: a major strength

One of the best features of Cunningham et al. is the explicit distinction between narrow AI-R&D capability and broad economically valuable capability. A system might become very effective at benchmarked algorithmic optimization without comparable gains in open-ended scientific judgment, robotics, deployment or social/economic impact.

This matters for the policy-facing paper because some downstream consequences require more than narrow R&D acceleration. Claims about large economic, cybersecurity, biosecurity or geopolitical effects need evidence about the transmission from narrow research capability to broad capability and real-world deployment.

The narrow/broad distinction should therefore be carried through more prominently into public summaries of “intelligence explosion.”

## Bottlenecks and substitution

The model explicitly includes important bottlenecks: humans, inference compute, experimental compute and data. This is a strength. The main empirical problem is knowing the relevant substitution elasticities.

If research inputs are strong complements, a cheap and abundant AI-researcher input can have sharply diminishing value once experiments, human approval, data collection or training infrastructure bind. If substitution is easy, the feedback can be much stronger.

The current calibration does not pin this down. A serious follow-up should prioritize lab-level production data: expenditure shares by input, task-level throughput, experiment queues, bottleneck time, inference usage, and attribution of frontier-relevant advances.

## Relation between the two papers

The two papers are complementary but should not be treated as if the technical model directly proves the policy paper's premises.

**Directly supported by the Cunningham framework:**

- R&D automation can create reinforcing feedback.
- Full automation is neither necessary nor sufficient for self-sustaining acceleration.
- The relevant feedback strength depends on measurable elasticities and bottlenecks.
- Available data leave large uncertainty.
- Current evidence can be consistent with a strengthening loop without showing that the threshold has been crossed.

**Additional claims in the Chan et al. paper requiring separate evidence:**

- AI R&D will be mostly or fully automated on a short timeline.
- Feedback strong enough for years-to-months acceleration is sufficiently probable to dominate current planning.
- Narrow R&D acceleration will rapidly translate into broad strategic capability.
- Particular institutional responses have net benefits relative to alternatives.
- International or organizational responses can be implemented effectively under the relevant conditions.

Those claims may be reasonable, but they are not outputs of the elasticity calibration alone.

## Current attention

The retained attention evidence includes the following coverage, published before the 3 October prioritization pass.

- [The Guardian](https://www.theguardian.com/technology/2026/sep/28/ai-godfathers-warn-of-runaway-intelligence-explosion) covered the release on 28 September.
- The *Wall Street Journal* covered calls by AI researchers for oversight of self-improving systems on 28 September.
- Axios covered the paper on 28 September.
- *Business Insider* published a follow-up interview/feature on 2 October discussing the possibility of very rapid AI-development cycles.

This is now clearly a high-attention public debate. That increases the value of checking the technical chain. It does **not** increase the evidentiary strength of the calibration.

## Highest-value robustness checks

1. Reproduce every algebraic threshold and verify the graph rules against the underlying production functions.
2. Publish a full sensitivity surface for the three core elasticities, not just the 15% benchmark.
3. Replace point estimates with distributions for each elasticity and propagate them to a probability distribution over feedback gain.
4. Separate task automation share, researcher productivity uplift and frontier-progress elasticity empirically.
5. Re-estimate the AI-R&D productivity elasticity using lab experiments that randomly vary model/tool access where feasible.
6. Measure lags between AI-assisted research, successful experiments, training runs and deployed capability gains.
7. Stress-test strong complementarity with experimental compute, human oversight and data.
8. Repeat the calibration under alternative capability measures, including METR time horizons and task-specific R&D benchmarks.
9. Examine whether gains are concentrated in lower-leverage coding/engineering tasks or in advances that causally improve frontier models.
10. Model the transmission from narrow R&D capability to broad capability separately.

## Questions for Cunningham et al.

1. How invariant is the headline threshold to alternative normalizations and alternative measures of capability?
2. Can you provide the full calculation behind the 9% uplift estimate, including which model transitions, tasks and dates enter it?
3. Which empirical input dominates uncertainty in the threshold? A probabilistic sensitivity analysis would be especially helpful.
4. How do time lags between research, experiments and training alter the condition or the time-to-acceleration?
5. How would the condition change under strong complementarity between AI research labor and experimental compute/human oversight?
6. Can the graph framework accommodate task heterogeneity where AI automates a large mass of low-leverage tasks but not a small set of decisive bottlenecks?
7. How should readers translate the Anthropic “26% AI leads” measure into, or deliberately not translate it into, your elasticity?
8. What evidence would convince you that the loop had crossed the self-sustaining threshold?

## Questions for Chan et al.

1. Which quantitative claims in the white paper rely on Cunningham et al. versus independent evidence?
2. What is the best current estimate or range for the probability and timing of a years-to-months acceleration, rather than simply the possibility?
3. How much weight should be placed on Anthropic's internally measured automation index before independent validation?
4. What evidence connects task-level R&D automation to frontier capability progress?
5. Which conclusions depend on narrow AI-R&D acceleration versus broad capability acceleration?
6. For each proposed institutional response, what evidence addresses effectiveness, implementation constraints, costs and unintended consequences? The technical RSI model does not estimate these.

## Evaluation ratings

### Cunningham et al.

- **Conceptual contribution: 4.5 / 5.** Clear framework that disciplines an otherwise vague debate.
- **Formal/model clarity: 4 / 5.** The elasticity-loop logic is strong and interpretable.
- **Empirical calibration: 2.5 / 5.** Useful as a back-of-the-envelope exercise, but key parameters are weakly measured.
- **Robustness currently demonstrated: 3 / 5.** The paper discusses bottlenecks and uncertainty well; quantitative sensitivity can go much further.
- **Value of further public evaluation: 5 / 5.** The claims are high-stakes, auditable and now central to a live debate.

### Chan et al.

- **Importance / decision relevance: 5 / 5.** The scenario is consequential and receiving substantial attention.
- **Evidence for rising AI-R&D automation: 3.5 / 5.** Multiple relevant indicators, but some key measures are internal/self-reported.
- **Evidence for imminent self-sustaining acceleration: 2.5 / 5.** Plausibility is established better than probability/timing.
- **Support for specific institutional responses from this evidence alone: 2 / 5.** The paper offers a policy agenda, but the RSI calibration does not estimate comparative intervention effects.
- **Value of independent evaluation: 5 / 5.** The gap between public salience and parameter uncertainty makes this an especially valuable paired evaluation.

## Bottom line

The strongest conclusion from the pair is not “an intelligence explosion is imminent,” nor “it is unlikely.” It is that the question can be decomposed into empirical objects that are much more precise than the usual debate suggests.

Cunningham et al. gives a good map of those objects. The present measurements are too weak to locate the system precisely relative to the threshold. Chan et al. is valuable for showing why the scenario matters and for assembling evidence of rapidly rising AI use in AI R&D, but some of its strongest implications require additional evidence about the feedback gain, timing, narrow-to-broad capability transmission and institutional effectiveness.

That makes the pair unusually well suited to a public Unjournal evaluation: the disagreement can be shifted from rhetoric about “takeoff” toward concrete measurements that labs and independent researchers can potentially supply.

## Sources

- [Cunningham et al., arXiv](https://arxiv.org/abs/2609.15802)
- [Chan et al., GovAI](https://www.governance.ai/research-paper/what-if-automating-ai-r-d-triggers-an-intelligence-explosion)
- [Foundation for American Innovation mirror / release](https://www.thefai.org/posts/what-if-automating-ai-r-and-d-triggers-an-intelligence-explosion)
- [Guardian coverage, 28 Sep 2026](https://www.theguardian.com/technology/2026/sep/28/ai-godfathers-warn-of-runaway-intelligence-explosion)
- [Epoch Capabilities Index](https://epoch.ai/)