Remote Labor Index reportedly ranks Fable 5 at 16.1% human-accepted tasks

• Rohan Paul's RLI summary says CAIS and Scale put Fable 5 at 16.1% full automation on real remote-work projects, versus 8.3% for Opus 4.8 and 6.3% for GPT-5.5.
• Paul's score chart says the previous published leader was Opus 4.6 with the Claude Cowork scaffold at 4.17%, and the field topped out at 2.5% when RLI launched.
• Paul's judge screenshot says the automated judge overrated GPT-5.5 at 17.9% versus 6.25% human approval, and Opus 4.8 at 18.8% versus 8.33% human approval.
• Paul's time-horizon chart says model success did not fall as human completion time rose, weakening the clean “longer human task equals harder AI task” assumption.

The Remote Labor Index source link frames the benchmark around paid freelance-style projects with briefs, files, and professional deliverables. The judge analysis has the nasty eval gotcha: directionally useful rankings, bad absolute scores on newer models. The time-horizon result is the stranger finding for forecasters, because 60-hour human work did not reliably mean lower AI pass rates.

The headline number is the pass rate: the share of projects where the model's deliverable was judged at least as good as the professional baseline.

• Fable 5: 16.1%
• Opus 4.8: 8.3%
• GPT-5.5: 6.3%
• Opus 4.6 with Claude Cowork scaffold: 4.17%
• Manus 1.6 Max: 2.92%
• GPT-5.2: 2.50%
• Grok 4: 2.08%
• Gemini 3 Pro: 1.25%

That still leaves Fable 5 failing most projects. It also puts the new leader at roughly 4x the 4.17% previous published leader and more than 6x the 2.5% frontier from RLI's launch, according to Paul's chart.

RLI is closer to messy computer work than a chat benchmark. Paul's summary says each task includes a brief, files, and a professional deliverable used as the human baseline.

The task mix named in the summary spans:

• CAD
• Architecture
• Animation
• Audio
• Data analysis
• Web apps

Fable 5's strongest examples, according to Paul's summary, included ring modeling, animation, and bathroom design.

The automated judge tracked model ordering but missed absolute quality failures on newer systems. The screenshot reports Spearman ρ = 0.90 for ranking, then shows large score inflation once GPT-5.5 and Opus 4.8 entered the set.

• GPT-5.5: 6.25% human-approved, 17.9% automated, about 2.9x too high
• Opus 4.8: 8.33% human-approved, 18.8% automated, about 2.3x too high
• Earlier calibration models: 3.3% human-approved, about 3% automated, about 1x

Quality control is the hard wall in this result. The judge can point in the right direction, while still being wrong enough to change the story a buyer would tell themselves.

The time result is the least intuitive part of the update. Paul's chart says pass rates were flat to slightly rising as human completion time increased, rather than falling as a time-horizon model would predict.

• Fable 5 starts around 15% on short projects and rises above 20% around 64-hour human tasks.
• Opus 4.8 stays roughly in the 5% to 10% band.
• GPT-5.5 drops near 0% around the middle bins, then recovers around longer tasks.

Paul's thread gives the practical explanation: long professional projects can contain production work that tools compress, while short projects can require judgment models still lack.

The model leaderboard is also a harness leaderboard. Paul's summary attributes the gains to stronger agent setups around the model:

• Better tool use
• Full desktop environments
• Professional software access
• Longer runtimes
• Worker-critic loops, where one agent produces the work and another reviews it like a demanding client

That is the part AI engineers will recognize from production agents: capability shows up as model plus environment plus reviewer loop.

The update also sparked a model-identity and benchmark-design argument. Dan Shipper claimed the benchmark was measuring “the same model” with slightly more fallback to Opus 4.8, producing a mix of Fable and Opus behavior.

Zeeg's reply said the numbers they had seen were lower, “like 10-20%.” bridgemindai's response argued that a benchmark should not be altered to match what developers think a model should score.

Two primary link posts, and , helped carry the benchmark fight beyond the original RLI summary.