[h] home [b] blog [n] notebook

Ideas are cheap; evidence is expensive.

Sure, it makes sense to me that AI would help its own development cycle. But when you think about what researchers are doing—writing code, exploring failures, tinkering with existing approaches—then it's quite reasonable to say that having better tooling should help. What seems less reasonable to me is that we should be able to map the work output of the tools to the rate of model progress.

My intuition is that there's an intermediate decision that needs to happen: somebody has to decide what to explore. Say you have a research group with compute for five experiments. If you give them a hundred variants of a nonviable research question, they'll be left doing a costly filter. If you give them a hundred new ideas, they'll have a chance to improve.

But now the demo results may be equally convincing, so things only get expensive when the research group opens its bank account.

I think it's worth paying more attention to the question of selection in recursive self-improvement. My rough intuition is that AI will more rapidly lower the bar for proposing reasonable ideas than it will for assessing their validity. In which case, whoever is able to do the second one better gets the edge.

My intuition also is that it's quite possible this decision-making role would be played by AI itself. There's no reason that the gatekeeper has to be human. The question of idea generation and the question of idea validation are different, and their costs are different, and knowing how fast one is going is surprisingly uninformative about the other.

Cunningham et al. model the way that better AI can cycle back to improve AI productivity. But either way, the really interesting question they ask is: when does the cycle become self-sustaining, such that it can maintain its own acceleration without more external resources? The framework is genuinely useful to me, but I wouldn't trust it to give me an accurate calibration of the parameters for any one research group. Anyway, the question of how much of an impact a better tool has depends on what's blocking the research group when they get it.

If a researcher has a list of experiments ready to be built, then a better coding agent is useful. If they're queued for compute, then you may just be adding to the backlog. If it's hard to read the results, then the bottleneck is interpreting them. But the same tool can be useful in the first case and disappointing in the latter two, so I'm wary of arguing about rates of research acceleration based on time saved.

In other words, if an hour saved unblocks your training pipeline, that can have massive impacts. If you save an hour of stuff that was going to sit untouched for a month, the short-term impact will be small. I want to know what the researcher can do now that they have this hour.

Closer to the metric we actually care about, there's some evidence. METR's November 2026 report estimated that the growth rate of Anthropic's capability index had increased by a factor of roughly 1.9 since October 2025 (1.5x to 2.8x under the primary specification).

Anthropic’s benchmark trend sped up. the estimate depends on the scale.

1×2×3×4×5×standard logistic1.9×probit1.8×cauchit2.8×heavy easy-task tail1.1×heavy hard-task tail2.9×learned tails2.6×rate of benchmark progress relative to earlier trend
figure 1 · points are fitted slope ratios. bars show reported 95% confidence intervals within each specification. the dashed line marks the historical rate. METR data and methods

That's a pretty big acceleration, and it's harder to pin down what caused it. METR honestly judged that most of the acceleration was due to AI by October 2026, although we also have to consider how benchmark scores were mapped to a capability index. Really, a lot of the evidence behind the evaluation was redacted by Anthropic.

But anyway, it's an interesting result worth investigating, although it's pretty uncertain whether this prediction will hold up over the next year.

I think a result like this could reflect a number of different things, all of which might be happening simultaneously. Maybe better tooling allowed a researcher to build out something that had been in a backlog for months. Maybe a new test convinced everyone that a dogmatically rejected strategy was worth exploring. And a new model was produced, but it's hard to tell which of these was responsible for the bump.

I would add that this is even harder to infer from self-reported productivity. METR's early-2025 randomized trial included 16 seasoned open-source developers who completed 246 tasks, and they estimated that AI would decrease their time to completion by 24%, and then they estimated that it had decreased their time to completion by 20%. Measurably, the time to completion actually increased by 19%.

developers expected to save time. the trial measured longer tasks.

-40%-20%0%+20%+40%expected before-24%estimated afterward-20%observed in trial+19%change in completion time with AI access
figure 2 · early-2025 study of 16 experienced open-source developers. the measured change was +19%, with a 95% confidence interval of +2% to +39%. METR randomized trial

I wouldn't use that study to draw conclusions about newer tools that came out long after, and I certainly wouldn't apply it to frontier research.

What sticks with me for this one is that experience didn't correct the participants' perceptions—they'd done the work, but they still got the sign of the effect wrong. That makes me quite wary of taking anyone's word for their time savings, including my own. The reason that individual improvements can be misleading is because they can be partial. Say you're working on a ten-day project, and eight days are spent on prep, and two days are spent training and reviewing.

If you make the prep step ten times faster, then you get to 2.8 days, which is a nice 3.6x speedup overall. But yeah, the prep takes up less; the other stuff takes up most of the project.

a 10× task speedup can buy a 3.57× project speedup

1×2×3×4×5×1×2×4×6×8×10×3.57× overall5× ceilingspeedup on the accelerable 80% of the project
figure 3 · illustrative model. overall speedup = 1 / (0.20 + 0.80 / task speedup). the vertical axis shows the speedup for the whole project.

The lab would therefore have a lot of motivation to compress that stage, and there are ways to work around that: basically you can do a smaller version of the experiment, check its results, and if it passes, move on. Some of that process could be automated. There doesn't need to be an obvious reason for any of this delay to be uncompressible. I don't think I'm misreading the intended meaning here; I think the point is that whenever you improve something, you change the problem you should focus on. A review process that was roughly fine when preparation took eight days could be the most urgent problem in the building.

Also I would expect that cheaper experiments would lead to more experiments. Researchers would probably already have a list of experiments they were eager to run, but couldn't justify because they were too expensive. But as the cost goes down, the set of experiments they are happy to do would expand. (And even when things are cheaper, it's possible for the overall expenditures to go up.) This is roughly consistent with how productivity growth works. I wonder if the extra experiments are a net benefit (more valuable than their cost).

But this is really how I read it: a small example. Suppose an experiment costs $10,000 and a big follow-up experiment costs $1 million. A reliable test that rules out one bad run in ten can be worth paying for. Now suppose a lab starts looking at ten times as many ideas. If almost none of that cost is in the idea-generation process, you can expect a huge jump in demand for that test.

The arithmetic also explains why I'm interested in rejection agents. If an agent can catch a bad experiment before it's run, that might be worth more than the agent that just runs experiments quickly (close enough). But it's hard to demonstrate, since the experiment never actually runs, so you can't see the data. But there's a risk of overfitting here. It's easy to reject weird ideas.

I'd probably actually want to see a lab run a sample of rejected ideas and see if its filter was getting too conservative. It's possible that an agent trained to prioritize projects similar to those that had been successful before would be really good at not making normal mistakes, but really bad at finding new directions. So "better judgment" could become "more of what we already know" in a quiet way. I'd ask a researcher at a cutting-edge lab: "What's the last experiment you dropped? When did you start to think it wasn't good, how much money did you spend, what decided it?"

Basically I think the details would matter (roughly). So a result that doesn't scale and a result that can't be reproduced are fundamentally different. An experiment that is carried on because some senior researcher thinks it's interesting is even more different. I'd learn more about the places where AI could help from hearing that story than from a generic feeling of "everyone feels like they're ten times faster."

Josh Lerner, my academic advisor, wrote with Jean Tirole on the economics of open source.

Basically their work has some relevance here: giving away useful software can be consistent with profiting from it. Contributors can build reputations. Companies can profit from a larger installed base of a shared tool because it increases the value of some other part of the business. Lerner and Tirole (give or take). And honestly, this is roughly how I'd think about a lab open-sourcing a model.

Other developers can extend its capabilities, find bugs, and try things the lab wouldn't fund. The lab then benefits from some of that work, while also retaining something that would be expensive to recreate (such as the data for deciding which improvements to pursue in the big version). Say two companies both use a common open-source model. One of them has years of useful experiment history, including failures at scales the other company has never reached. If they can write roughly the same code and come up with similar ideas, that history could make a huge difference in helping the one company spend its research money well.

So even if both companies have the same model weights, it doesn't really help with equalizing costs. So my real prediction would be that some labs would move towards greater openness with models and code, while keeping the most useful tests secret (close enough). They would benefit from a bigger pool of outside contributors, while preserving the ability to make better choices about which ideas to pursue. This argument is limited by the fact that private tests can be bad. Maybe the accumulated data isn't helpful in some new area.

Maybe a small team will discover a method that makes the incumbent's experience much less useful. The extent of the advantage is only the extent to which the secret information improves choices. But that's also why failed experiments are interesting: it's really easy to leave their value uncollected. Maybe a lab finds that a good technique doesn't work at scale. They spend a month on that.

If they describe it carefully, maybe a few other labs would save a month too. Some credit might go to the original lab, but the financial benefits would mostly go to others. They might be justified in keeping it secret, even if that would hurt the industry. If ideas get cheaper to generate, this could also waste more resources. Multiple agents could independently propose the same plausible idea, because they don't know about the expensive evidence against it.

Then everyone would spend a bunch to find out.

I would like more publications about failed experiments – negative results. Just "we tried it and it didn't work" is rarely helpful, but knowing the details about how it was implemented and what conditions it failed under would be useful to other researchers. (Though most of the time, simply failing to publish a sufficiently detailed report about a failed experiment risks misleading other researchers, who might blame the failure on a bug or bad hyperparameters.) Publishing enough useful detail about a failed experiment can take considerable time, which the lab would rather spend on their next project. (Roughly.)

This creates an awkward situation where failed experiments are more valuable to the research community exactly when the relevant researchers have more and more to do. Efforts to increase the number of proposals don't alleviate this incentive problem.

Also, there is the speed of experimentation. Suppose two labs both discover a useful but equally expensive improvement. One takes three months and the other one a week to validate and implement the change. The second lab can start using the improvement in other experiments even while the first lab is still pondering whether to adopt it. This is especially important when the improvement is itself an improvement to the research process, since implementing a superior method more quickly can improve the progress of several subsequent experiments. For instance, even though a cheap but accurate predictor of the outcome of an exhaustive experiment would not improve the benchmark score directly, it could be valuable.

And yet, a wrong adoption decision might not just affect one but several rounds of experiments. For some reason, if a lab's faulty assessment leads it to repeatedly adopt the wrong improvements, the correction would involve revisiting completed experiments. (Still, I would care about how long it takes to find out about mistakes, as well as how long it takes to adopt a successful change.)

One obvious counter-argument is that agents might end up being excellent at evaluating, which can be done by a separate agent. If the evaluation scheme is reliable, verification costs can be reduced, thereby reducing the costs of coming up with ideas.

Okay, maybe. This is something I'm open to, but I would want concrete evidence from past situations where there was initial doubt about the answer, and where mistakes were costly. The fact that multiple agents with similar models agree would be less convincing than their ability to accurately forecast the outcome of a new, larger-scale experiment.

This is also a practical indicator of trust: how much money will the lab trust the system to spend on its own judgement? It would be telling if an agent was trusted to code small things, but much more telling if it was trusted to decide which million-dollar experiment to run. It would be interesting to follow up on how those decisions went.

I would also change my mind if small-scale labs using public models and tests caught up to the research productivity of leading labs. This would be very strong evidence that AI made it cheaper to do frontier research, and that my expectation about the value of private evaluation was not quite right.

Right now, I want to see the evidence about the chain of effects between experiments. If a better research agent is used to achieve a similar level of improvement for the same cost, and the improved model enables a lab to achieve the next useful result in a shorter or cheaper fashion (roughly), and this pattern repeats itself, it would show that the feedback was improving.

I think AI will improve research productivity, but I'm less confident about what part of the gain will be kept through the choice between an interesting proposal and a deployed model. The leading labs, if they made the right choices, might be running fewer experiments and learning more from them (give or take).