The AI Writes the Code. Expertise Still Decides Who Ships It.

Across roughly 400,000 agentic coding sessions, humans made about 70% of the planning decisions — and only 20% of the execution decisions [Fact]. The AI did most of the typing. So did expertise stop mattering? Anthropic's own data says the opposite: the gap between novices and experts didn't close. It moved — to a place most of the "AI levels the playing field" arguments aren't looking.

What 400,000 sessions actually measured

Anthropic's Societal Impacts team analyzed roughly 400,000 Claude Code sessions from about 235,000 users between October 2025 and April 2026 [Fact]. The report — "Agentic coding and persistent returns to expertise," by Zoe Hitzig, Maxim Massenkoff, Eva Lyubich, Shaoyi Zhang, Ryan Heller, and Peter McCrory — was published June 16, 2026 as company research, not a peer-reviewed study. That distinction matters, and we'll come back to it.

The method is worth two sentences because the findings hang on it. Privacy-preserving classifiers built on Claude Sonnet 4.6 read session transcripts and labeled what happened: the task, the user's apparent expertise, whether the session succeeded. Occupations were mapped to the US Bureau of Labor Statistics SOC taxonomy, and task values were calibrated against freelance marketplace job postings [Fact]. One validation check the authors report: more than 90% of sessions the classifier labeled as writing or modifying code showed actual code changes in telemetry [Fact].

A typical session ran about four conversational turns, and each human prompt set off a chain of around 10 actions by Claude, producing on average 2,400 words of output [Fact]. That is the shape of agentic coding right now: the human steers occasionally; the machine executes continuously.

The success bar was deliberately strict. "Verified success" required both a judgment that the session succeeded and at least one hard signal — git commits or pull requests matching the work, passing test suites, or explicit user confirmation [Fact]. Keep that strictness in mind when the percentages look low.

The expertise premium didn't shrink. It relocated.

Here is the headline pattern. Sessions from users classified as novices reached verified success 15% of the time; intermediate through expert users landed between 28% and 33% [Fact]. On the looser partial-success measure, novices hit 77% against 91–92% for everyone else [Fact]. And novices abandoned 19% of their sessions, versus 5–7% for more experienced users [Fact].

The sharpest number sits in the wreckage. When a session ran into trouble, novices recovered it to verified success just 4% of the time. Experts recovered 15% of theirs [Fact].

Do the arithmetic the report doesn't do explicitly. On the smooth path, experts outperform novices by roughly a factor of two (15% vs. 28–33%). In troubled sessions, the ratio is nearly four to one (4% vs. 15%) [Estimate — our calculation from the reported figures]. The expertise premium is not spread evenly across the work. It concentrates precisely where the AI gets stuck — in the ambiguous, broken, half-diagnosed situations that agentic tools still can't climb out of alone.

Experts also simply extract more from the same tool. An expert's prompt triggered about 12 Claude actions and 3,200 words of output on average; a novice's prompt got 5 actions and 600 words — a gap the authors report at p < 0.001 [Fact]. Same subscription, same model, five times the output. The tool is not the equalizer. The operator is the variable.

One detail changes how you should read all of this: "expertise" here was not a job title or a years-of-experience field. The classifier scored behavior — how precisely users framed directions, what they asked Claude to verify, and whether the user tended to correct Claude or the other way around [Fact]. Expertise, in this dataset, is something you do in the session.

The five-point gap that complicates "learn to code first"

Now the other half of the tension. In code-producing sessions, users in software occupations reached verified success about 34% of the time. Users from other occupations: about 29% [Fact]. Partial success was nearly identical — 89% versus 88% — and every one of the ten largest occupation groups in the data landed within seven points of software engineers [Fact].

Put the two gaps side by side, which the report itself does not do: your occupation is worth about 5 percentage points; how expertly you operate the tool is worth 13 to 18 [Estimate — our calculation]. Working like an expert predicts roughly three times more than being employed as one.

That's the honest resolution of the apparent contradiction in this data. The barrier to producing working code is falling — marketers, analysts, and researchers in this dataset shipped verified code at rates close to professional engineers. But the returns to knowing how to direct, verify, and rescue machine work are holding firm, and at the failure boundary they are widening. Two different things called "expertise," moving in opposite directions.

Before anyone declares the entry barrier dead, though, look again at the abandonment number: nearly one in five novice sessions ended with the user simply walking away, almost triple the rate of everyone else [Fact]. The barrier didn't vanish. It changed shape — from "can you write the code?" to "can you tell whether the code is right, and what to do when it isn't?"

The work itself shifted inside six months

The composition of what people used the tool for moved fast. Debugging and fixing fell from 33% of sessions to 19% between October and April. Operating software grew from 14% to 21%. Writing and analysis roughly doubled, from about 10% to 20% [Fact]. The average value of a session's task, benchmarked against freelance-market pricing, rose 27% over the same window, with building tasks up around 43% [Fact].

Read as a labor signal rather than a product stat: the model absorbed a chunk of the fixing, and humans drifted toward running systems and communicating about them. If your mental model of a coding job is "mostly debugging," that model aged six months in this dataset. For the task-level view of how AI activity distributes across actual software occupations, see our pages on software developers, computer programmers, and web developers — the observed-usage tables there show which specific tasks the models are being applied to, release by release.

Read it with both eyes open

This is Anthropic studying Anthropic's product, published on Anthropic's blog. That doesn't make the numbers wrong, but it shapes what could be measured and what got emphasized [Claim]. The authors themselves list real limits: the analysis cannot observe whether code from a session was ever actually used; it excludes non-interactive usage, which they call a substantial share of activity; and every classification depends on a model reading a transcript, which remains hard to validate at scale [Fact].

Add the sampling problem: people who chose an agentic coding tool in 2025–2026 are early adopters, not a cross-section of the workforce. A marketer who installs Claude Code is already unusual. The 29% figure tells you what motivated non-engineers achieve, not what the median office worker would.

And "verified success" at 15–33% may read as damning until you recall the definition — a hard external signal like passing tests or a matching commit. Plenty of genuinely useful sessions never produce one. The partial-success rates of 77–92% are the ceiling; the verified rates are the floor. The truth about usefulness sits somewhere between, and this dataset cannot pin it down.

What to do with this

If you're not a software professional: the five-point gap is the most encouraging number in the report. The data does not support waiting until you've "learned to code properly" before using agentic tools on real problems.

If you're building skill: the premium lives in the troubled sessions. Practice the unglamorous parts — writing directions precise enough to be checkable, asking for verification (tests, reproduction steps) before believing output, and recognizing early whether a session is worth rescuing or restarting. Those are the behaviors the classifier read as expertise, and they doubled to quadrupled success rates in this data.

If you're a working developer: your occupation-specific edge in raw success is 5 points and the debugging share of the work is shrinking. The durable advantage this dataset shows is judgment at the failure boundary — the 15%-versus-4% number. That is the skill to sharpen, and it is the part of the job furthest from being absorbed.

Sources

• Hitzig, Z., Massenkoff, M., Lyubich, E., Zhang, S., Heller, R., & McCrory, P. (2026, June 16). Agentic coding and persistent returns to expertise. Anthropic. https://www.anthropic.com/research/claude-code-expertise

This article was produced with AI-assisted analysis. Figures were taken directly from the cited report and re-verified against the original text; calculations labeled as ours are derived from the report's published numbers. This site is an independent project, not affiliated with or endorsed by Anthropic.