Anthropic's Fable 5.2 leads freelance benchmarks but fails most tasks

Leading the freelance index

Anthropic recently released results showing that its Fable 5.2 model currently leads the Remote Labor Index (RLI). This specific leaderboard, published by Scale on November 12, ranks Fable 5.2 at a 41.3% pass rate for completing freelance projects. This performance notably surpasses the previous high mark set by the Opus 5.5 model, which achieved a 21.25% success rate.

The RLI is designed to provide a practical test of AI capabilities by measuring complete deliverables against professional reference work. Instead of just checking if an AI can follow a simple instruction, the index looks at whether the AI can finish a project to a standard a human client would accept. This makes it a more rigorous test for businesses looking to automate specific remote work tasks.

Anthropic reported these results in an announcement on November 9, noting that the model was tested with its safeguards enabled. The company labeled the performance as "High effort," suggesting that the tasks required significant computational resources or complex reasoning. While Fable 5.2 is currently the leader on this list, the data shows that it still fails more than half of the projects it attempts.

How the RLI benchmark works

The Remote Labor Index was originally proposed in October 2025 by researchers from the Center for AI Safety and Scale AI. The benchmark consists of a dataset of 240 projects spanning 23 different categories of freelance work. Each project includes a brief, the necessary files to complete the work, and a professional human-reference deliverable to serve as the gold standard.

Because the dataset covers such a wide range of work, the AI must handle various file types including 3D models, audio, video, and computer-aided design files. This means the test evaluates not just the AI's ability to understand a client's request, but also its ability to produce high-quality finished products. The complexity of these files makes production quality a core component of the evaluation process.

To determine a pass, trained human reviewers inspect the AI's work against both the project brief and the human-made reference. The evaluation uses independent reviews and majority voting to decide the outcome. A project only passes if the AI's deliverable satisfies the brief at least as well as the reference. This means a useful prototype or a partial result is not enough to earn a passing score.

Limitations of the current data

While the RLI provides a demanding definition of success, the current results have specific boundaries that limit their scope. The original study explicitly excludes any work that requires physical labor or teamwork. It also excludes projects where the final outcome must be waited on before it can be evaluated, such as tasks with long processing times.

Because of these exclusions, the results may not fully represent the entire remote-work economy. For example, tasks that require constant client interaction or multi-person collaboration are not captured in this specific leaderboard. This means the RLI is a targeted look at independent, deliverable-based freelance tasks rather than a total map of all remote work.

Furthermore, Anthropic's release page does not provide a full picture of the execution environment for Fable 5.2. The company did not specify the precise spending per project or the amount of human support used during the testing. These missing details make it harder to determine the exact cost and autonomy of the model in a real-world business setting.

What this means for businesses

For a business assessing the usefulness of AI, the RLI provides a much more grounded metric than standard chat benchmarks. It focuses on carrying an assignment through to an inspectable product that meets a professional standard. This helps companies identify which types of freelance work are currently viable to outsource to AI and which still require human oversight.

The fact that Fable 5.2 passes 41.3% of projects suggests that AI is becoming capable of handling complex, multi-step production tasks. However, the 58% failure rate serves as a reminder that AI is not yet a perfect replacement for human freelancers. Businesses will likely need to maintain a human-in-the-loop system to review and refine the AI's output before it reaches a client.

As models continue to improve, the gap between a "useful prototype" and a "professional deliverable" is where the most important progress will happen. The RLI will likely remain a key metric for tracking this progress because it forces the AI to meet a specific quality bar. It moves the conversation from what an AI can do in a sandbox to what it can do for a paying customer.