This is the Executive Summary for a piece I co-authored with Jack Weisman in Inference Magazine on AI and explosive growth - read there for the remainder of this piece on explosive growth, and more content on AI industrial policy, technical explainers and forecasting.
Executive Summary
The view in San Francisco is that AI will far exceed the pace and depth of change in all previous technological revolutions. This is because of a belief that AI can automate the process of invention itself. Since Bacon, the march of science depended on the actions individual inventors and small groups of researchers; but perhaps in a few years, we can create AI systems that will be capable of performing research at the level of — or indeed, much better than — the best human researchers. We can put tech progress on autopilot.
How does this arise, according to this view? First, the AI labs create an AI system capable of performing AI research, on par with their top researchers. Next, millions of instances of the ‘digital AI researcher’ are run to make much faster research progress. These breakthroughs are applied to training the next generation of digital AI researchers, in a recursive self-improvement loop. This process leads to the creation of digital AI researchers which are much smarter than humans—this is ‘superintelligence’. In the dominant intellectual paradigm in San Francisco, this happens quickly. One important work on ‘takeoff speeds’ towards superintelligence argued that the time between AI systems capable of performing 20% of tasks humans do, and 100% of tasks humans do, was just four years.5
Superintelligence, as it is conceived, would have important implications for the economy: we could have an ‘explosion’ in R&D; and systems capable of performing 100% of the tasks that humans do could begin to automate the whole economy. (As the narrative goes, the superintelligence could figure out how to make robots which could perform as well as humans.) There is some academic work which investigates what happens to economic output when 100% of tasks are automatable, and many growth theory models show explosive economic growth (20% per year, or more).6 Under some conditions there is an economic singularity, which means growth models predict infinite output in finite time.7
On the contrary, most economists who study the impact of AI do not consider the prospect of recursive self-improvement. But most work on explosive economic growth does not deal with the microeconomic constraints of running an AI lab. This is a gap we hope to fill—providing a grounded view of what AI research automation will look like, and how this might come to affect R&D and cognitive labour automation in the near future.
AI research automation
The most important thing to understand about AI research automation is that the AI labs are constrained by computational power to run experiments, not by researchers. A researcher from the Gemini team at DeepMind has said, “I think the Gemini program would probably be maybe five times faster with 10 times more compute or something like that”. Whilst, the cloud providers are spending enormous amounts on compute—Microsoft just announced it would spend $80 billion this year on building AI datacentres, but most of this compute would be used to run inference for customers, and is unlikely to be for AI researchers to run experiments. The economics of inference for customers is very different from the economics of compute for R&D: compute for experiments and training needs to be amortised across all of the inference profit margins. As we shall see, there are strong headwinds to making money selling tokens!
One of the assumptions which proponents of the Explosive Growth view often make is that a digital AI researcher will be trained on a large compute cluster, and then millions of instances will be run on the same cluster. This seems irregular to us! If the point is to recursively self-improve the AI system, but the training compute is being used for inference, where is the next generation agent going to be trained? It seems much more reasonable to imagine that ~60% of the AI labs' compute goes on serving customers, ~30% goes on training the next model, and ~10% goes on experiments. (These numbers are extremely rough guesses.) If the AI lab wants to run instances of the digital AI researcher, they will need to trade this off against experimental compute; and remember, research output is bottlenecked by experimental compute. If the digital AI researcher has equivalently good or worse ideas to the best human researcher, it makes sense to run zero copies; for it make sense, the ideas have to be better.
AI research will be automated in the future. It is reasonable to imagine that, perhaps soon, we will create a ‘digital AI researcher’ whose research intuition—i.e. ability to predict which experiments will work—surpasses that of the best human researchers, but before then, digital AI researchers will have a bounded impact on research output, owing to the compute bottleneck. We discuss the practical challenges to increasing research output, as well as some reasons our mainline case could be wrong, in greater detail below.
R&D automation
Concurrent to our progress on AI research automation, we want to make progress in other fields of science and technology! The opportunity is enormous—for biomedical research, clean energy, materials, synthetic biology, nanotechnology, and robotics. As with AI research, the goal is to create systems which are capable of performing all steps of the research process—generating hypotheses, designing and running experiments, and interpreting results. There are a number of challenges to scientific automation, related to the availability of data, the necessity of real-world experimentation, and so forth. It also seems reasonable to believe that academia is poorly configured to take full advantage of the opportunity which AI automation is. We expand in greater detail on both points below.
We focus on three potential fields for automation—chip research because if we are compute bottlenecked, improving our chips would help to alleviate this; robotics as improvements here could begin to automate more physical labour, and biomedical research; for effects on human wellbeing. There are different challenges in each area to automation, though in general, experimental throughput is most likely to be rate-limiting.
Cognitive labour automation
Thus far, chatbots and ‘agents’ have struggled to meaningfully increase the productivity of human cognitive labour. Deploying systems is difficult right now—it requires specialised knowledge about how to build infrastructure for models. But as the models become increasingly capable of acting on long horizons, we expect most of the challenges to deployment to become diminished. We will still require people to have liability for AI systems, and in many professions, there are ‘embodied’ complements to cognitive tasks (e.g. when a doctor has a consultation, they are both doing the diagnosis, and tailoring their explanation to the patient, and expressing care and empathy, and so on) These factors together lead us to expect that people will be managing teams of agents in their jobs—it will look like ‘a promotion for everyone’—rather than a lot of job losses. However, there might be some areas where production is entirely substitutable, and so jobs might be lost. To estimate the increases to output from tasks being handed off to agents, we built a growth model that shows how many tasks might be automated, how much these tasks can replace other tasks, how cheap these AI systems are, and how concentrated this is within sectors. We find that growth will be quick by historical standards, but not explosive. We expect AI will provide a 3%-9% increase to economic growth per year in the near future, and we expect it will be in the lower end of this range due to bottlenecks we discuss further in the piece. This picture will seem conservative to some—but it is worth reiterating that we will develop intelligences greater than our own, and it will radically change almost all aspects of our lives, our analysis is limited to the near-term economic picture.
There are a few variables across this whole analysis for which different assumptions would produce very different technological and economic outcomes. The most obvious is what is the inference cost of running digital researchers and cognitive labourers—if it is cheap to run both, we should expect faster research progress and we should expect greater economic growth from normal sectors of the economy. We note that it is important not to have too much confidence in a specific vision of the future; but rather see the direction of travel.
Read the rest of the piece: