Artificial General Intelligence or AGI means having cognitive capabilities as good as or better than humans in every facet. As I opined long ago (see here), while there is no inherent physical law to prevent AGI from exceeding humans, I doubt it would make economic sense to do so for everything. Is there really that much utility for a machine to be able to tell jokes better than any human? It’s not clear to me how anyone or any thing could be funnier than the late Sam Kinison anyway. It would take a lot of reinforcement learning in front of a lot of audiences and even then it might fail because comedy in the 80’s was just funnier (I’m purposely dating myself).
The real issue for society is when AGI will be able to do most jobs better than humans. I think AGI capability may come reasonably soon, maybe in the next few years but I think it will take some time before it completely upends society. AI aficionados believe in something called the intelligence explosion, where they believe that once AGI reaches a certain level they will then design new AGIs and iterate to infinite intelligence rapidly. Yet, even with iterative self-improvement there is still a wide range of possible AGI take off speeds depending on how exactly new AGI will increase the capacity of the next generation.
We can make this quantitative with a simple growth analogy. For population growth, if each member of the population reproduces at some fixed rate then the rate of increase will scale as the size of the population, like bacteria dividing in two and thus doubling the population every generation. This is classic exponential growth and can be represented by the ordinary differential equation:

where
is the population size and
is a rate constant measured in units of inverse time, like per day. We need a differential equation to get
, the population size at any given time, because we only have local information, (see my last toolkit for the physical world post). The left hand side of the growth equation is the derivative of population size with respect to time, which is the slope or rate of population growth. The population size
is a number while
is a time and a number divided by time is a speed or rate. For the equation to make sense, the right hand side must have the same “dimensions” as the left so that is why we need to include a rate constant
, which must have units of inverse time. In physics/math parlance,
sets the scale. Usually, it’s easier to think in terms of time rather than rate so we often set
, where
is called a time constant. It’s like a doubling time or half life but in base e rather than base 2. It matters quite a bit if the time constant is a year or a century. The full solution to the growth equation is

where
is the initial time and
is the initial condition at time
. For every derivative in a differential equation, you need to specify a piece of information – initial conditions for time derivatives and boundary conditions for spatial derivatives. For simplicity, let
, you can always shift the time to call whenever you start zero, and let
, you can always rescale the population to think in terms of population compared to initial population, so we get
. In math notation,
means
. The rate constant determines everything. If you think that the time constant for AGI growth is between a year or two, like Moore’s law, then you’ll have really rapid growth. You’ll get a tenfold increase in less than a decade. If you believe it’s more like ten years then that same growth will take a century. If
is negative then you get decay. The exact same equation describes both exponential growth and exponential decay. It also explains exponential approach to an equilibrium. For example, if I combine linear growth with exponential decay I get

for which
will approach
exponentially with a rate constant of
. This equation can even be applied to body weight where the time constant is about a year. The general rule of thumb is that it will take about three time constants to get 95% there, so for body weight, you have to eat consistently for three years to know your eventual body weight, or you can use the NIH body weight planner, which incorporates some of the modeling work that I did with Kevin Hall.
In exponential growth, it will take infinite time to get to infinity. There is no singularity, which would require
go to infinity in a finite time, like a black hole. However, it is possible to have a growth equation exhibit a singularity by changing the equation to

where
is a measure of the gain in rate with population size. One way to think about this is to rewrite the right hand side of the equation as
and consider
as an
dependent rate constant. It gets larger for larger
if
is greater than one and smaller if it is less than one. Cities are an example where there is an increasing advantage with scale and
is greater than one. Economic growth would not be as fast if people were spread out rather than concentrated in one place. It is not a coincidence that almost all the AI companies and startups are located in San Francisco and most finance companies are in New York or London.
We can solve the differential equation by rearranging to obtain
and integrating both sides to get

where
is an integration constant determined by the initial condition. If
then if we choose the initial condition appropriately, we can write

Notice that we have
in the denominator. Suppose we start at
less than
. As
gets closer to
,
will get bigger and bigger and
will be infinity when
. In math, this is called a finite time blow up and is a bonafide singularity. Things start slow and then they explode. On the flip side if
is less than one then you’ll get slower than exponential growth.
The rate of AI takeoff thus boils down to the values of
and
. Both need not be constant either, they could change depending on circumstance. For AI,
would not correspond to a population size but rather some measure of AI capability. When AI capability is limited, like a year ago,
is near zero. AI improvement up to now has not been limited by how good AI is but rather by human ingenuity, data, and compute. The dramatic increase in performance we’ve seen recently rides on top of Moore’s law, cleverness of human AI researchers, and the audacity to keep going bigger.
I use AI tools daily and I’ve gotten pretty good at getting them to do what I need them to do but I need to be quite specific and I often have to iterate multiple times before I get something useful. The biggest problem I have is that the AI forgets what transpired a few interactions ago, so I have to explain the problem over and over again. It’s kind of like working with someone without a hippocampus who will forget everything once it’s out of their immediate attention. Part of this will be solved with longer context windows. But having a completely viable AGI helper may require having a dedicated model whose weights are updated as you interact with it. All of this will require a lot more compute and that will be a rate limiting step for at least the next short while.
AI will increase productivity of AI research but for growth to explode, AI cannot be limited by anything other than itself. Even if AI takes over all research including designing hardware and building foundries and power plants, it still has to obey the laws of physics. The size of transistors on GPUs are limited by the structure and properties of matter. How fast information can travel is limited by the speed of light. How large buildings can be are limited by the strength of gravity. AI optimists seem to think that all problems are limited by intelligence but intelligence can only do so much. The theory of computation shows that there are problems that are just plain hard to solve no matter how smart you are. Biological systems are complex and contingent on random evolutionary quirks that must simply be discovered. From my own experience, I don’t think intelligence is the rate limiting step for progress in medicine. Most drugs fail because you just don’t know how a drug will behave unless you actually test it on lots of people, and that is just going to be slow. Being smarter might make the odds better but it will still take time to test. I thus think the jury is still out on how fast AI will take off. I am sure that I will be replaced someday but I’m really not sure when.