AI research review
Key questions and evidence
What the model establishes, which conclusions depend on assumptions, and what still needs checking. Open a question for the finding, practical checks, and source links.
For a suggested evaluation approach, see the evaluator guide.
What reproduces, and what remains unavailable?
Tables 3, 5, and 6 reproduce from the public model code. The calibration construction, survey microdata and weights, and code that generated the tables are not deposited. Reproducing the model outputs therefore leaves parts of the empirical pipeline unchecked.
What to check. Check whether the reported reproduction covers the calculation you need. Prioritize the unverified calibration and survey pipeline rather than repeating already-reproduced cells.
Useful clarification. Provide the calibration construction, survey weights and coding, and table/figure generation scripts where sharing is possible; document restrictions otherwise.
Review evidence: Issue 3 · Issue 18.
Earlier assessment and review context
The earlier assessment treats computational reproduction as a major strength and recommends shifting effort away from another code replication. The later audit also reproduces Tables 3, 5, and 6, but separates those checks from the missing calibration code, survey microdata, weights, and table-generating code. Successful reproduction of model outputs does not clear the whole empirical pipeline.
Does the survey support the substantial scenario?
The survey is nearest the substantial scenario with the task-grid capability measure. With the paper’s alternative own-hours measure, it is nearer the modest scenario. The re-employment mapping and interpretation also change the unemployment comparison. Neither comparison establishes a scenario’s probability.
What to check. Check the capability measure and re-employment units separately. Distinguish the model at item medians from medians of respondent-level model runs; inspect non-response assumptions.
Useful clarification. Clarify which capability measure enters each result, how months map into model job-finding rates, and how weighting and missing answers are handled.
Review evidence: Issue 1 · Issue 2 · Issue 4.
Earlier assessment and review context
The earlier assessment flags the complete-case sample and fixed inputs. The later audit adds two material qualifications: the paper’s own-hours capability measure puts the survey nearer the modest scenario, and the re-employment answer’s mapping and interpretation change the unemployment comparison. This does not establish that either scenario is likely.
How robust are the unemployment and wage headlines?
The unemployment and wage magnitudes depend strongly on wage adjustment, occupational mobility, and where people replacing labor-force exits search for work. In the audit’s alternative routings, extreme cognitive unemployment remains in the teens, while the substantial scenario’s cognitive-wage comparison can change sign. Those routings are sensitivity checks, not measured central estimates.
What to check. Check the entrant/exit routing, the unemployment baseline, and the wage comparison date before interpreting a headline. Assess the evidence for the routing assumption.
Useful clarification. Explain where replacement workers search, report the relevant baseline and dates, and show the headline outcomes under defensible alternative routings.
Review evidence: Issue 5 · Issue 6 · Issue 11 · Issue 12.
Earlier assessment and review context
The earlier assessment emphasizes wage rigidity and occupational mobility. The later audit adds the treatment of people replacing cognitive workers who leave the labor force: redirecting these entrants changes unemployment and can reverse the substantial scenario’s cognitive-wage comparison. The routing variants are bounds, not measured central estimates.
What resources are available for compensation?
Potential compensation survives the model-consistent capital-financing cases tested. But the ‘nearly three times’ comparison uses gross GDP: charging for the extra capital reduces the available gain. The size of a transfer relative to the gain must also be distinguished from what the recipients and payers have after the transfer. This accounting does not establish an implementable welfare policy.
What to check. State the income or resource basis and capital-financing closure. Keep the gain, workers’ loss, and transfer separate, and check Table 3’s income accounting before treating GDP as spendable resources.
Useful clarification. Specify the denominator, capital charge, financing assumption, and transfer baseline. Separate potential compensation from an implementable policy or individual welfare claim.
Review evidence: Issue 7 · Issue 8 · Issue 17.
Earlier assessment and review context
The earlier assessment distinguishes aggregate compensation arithmetic from a feasible welfare policy. The later audit adds the cost of the extra capital and correct transfer accounting. Potential compensation survives the model-consistent capital closures tested, while the ‘nearly three times’ multiple is a gross-output comparison. Paying a transfer does not itself destroy the gain.
What does the small innovation effect establish?
The small short-horizon innovation effect follows from the paper’s research-funding rule and model structure. It does not establish that AI-driven research feedback would be small under other assumptions or over longer horizons.
What to check. Check the research-funding rule and horizon before interpreting the small modeled effect as evidence about recursive improvement.
Useful clarification. Make the assumed capability path, derived innovation cap, and limits of the interpretation explicit.
Review evidence: Issue 10.
Earlier assessment and review context
Both reviews distinguish the assumed capability path from the model’s innovation channel. The later audit finds that the small short-horizon innovation effect follows from the paper’s own research-funding rule. It does not establish that AI-driven research feedback is small outside this model.
What evidence could discriminate the scenarios?
The model supplies conditional paths rather than scenario probabilities. To discriminate between them, one needs observables, measurement dates, and a way to separate AI’s contribution from other changes. Benchmark comparisons also need matching horizons and income concepts.
What to check. Specify observable tests and dates; match horizons and denominators in benchmark comparisons. Distinguish an unrun test from evidence against the paper.
Useful clarification. State the measurements and thresholds that would favor one scenario over another, and align literature comparisons to common horizons and concepts.
Review evidence: Issue 13 · Issue 14 · Issue 16.
Earlier assessment and review context
The earlier assessment proposes calibration, sensitivity, and external forecast comparisons. The later audit asks which observables, by when, could distinguish scenarios once AI’s contribution is isolated. It also qualifies benchmark comparisons whose horizons or income concepts differ. No review establishes probabilities for the three scenarios.
How this guide was prepared
These summaries are an AI-generated editorial synthesis of the isitcredible review and earlier working assessment. They are not a new audit or human adjudication. The original reports and their differences are preserved in the source links.
The audit used earlier AI reports as leads, so their agreement is not independent confirmation. Replication remains OPEN: missing survey microdata and some unavailable published sources limit what was verified. Missing evidence is not proof that a claim is false.
Sources: working assessment, 4–7 October; deep-research explainer, 27 October; isitcredible review and audit, 28–29 October 2026. Full provenance.
Select a passage to annotate it, or open the Hypothes.is sidebar tab on the right edge. Public annotations can correct a number, challenge an inference or point to evidence.
How annotations may be used: Hypothes.is annotations on this page may be incorporated into The Unjournal’s knowledge base and may be quoted, summarized, or reposted on other Unjournal pages or public channels. Do not include private or confidential information.
Hypothes.is help