Faster GPUs won’t conjure a world model from out of thin air we’re scaling mimicry, not understanding: that is my guess as to why the MAMLM frontier is spiky, with breathtaking benchmarks, baffling failures, real consequences…
Models ace PhD quizzes, then misclick a checkout button and hallucinate a blazer and tie. If these systems are about to become “better than humans at ~everything,” why can’t they keep time on a roasting turkey?Without being built from studs up around a world model—durable representations of time, causality, and goals—frontier MAMLMs systems are high‑speed supersummarizers of the human record and hypercompetent counters at scale, yet brittle in embodied context, interfaces, and long‑horizon tasks. Calling it a “jagged frontier” misidentifies this unevenness, except to the extent it leads to permanent acceptance of centaur workflows where humans supply judgment and guardrails.
Or so I guess.
I really do not grok things like this.
It seems obvious to me that “AI” as currently constituted—Modern Advanced Machine-Learning Models, MAMLMs, relying on scaling laws and bitter lessons—will not be “better than humans at ~everything” just with faster chips and properly-tuned GPUs and software stacks. Without world models, next‑token engines merely (merely!) draw on and summarize the real ASI, the Human Collective Mind Anthology Super‑Intelligence, and excel only where answers are clear or where counting at truly massive scale suffices: fast mimics—useful, but narrow.
That seems obvious to me. But not to Helen Toner.
And so Helen Toner stands across a vast gulf, among the serried ranks of AI-optimist singulatarians who believe that we are building our cognition betters, and face the “peak horse” problem that the steam engine, the internal-combustion engine, and the electric motor brought to the equine. And yet she is not on Team Artificial Super-Intelligence by 2030, not at all:
Helen Toner: Taking Jaggedness Seriously <https://helentoner.substack.com/p/taking-jaggedness-seriously>: ‘Why we should expect AI capabilities to keep being extremely uneven, and why that matters…. Two things are true…. AI models keep getting better and better… [and] AI models keep sucking, and the things they keep sucking at are kind of confusing…. Who here saw Project Vend from Anthropic? This was so good. For anyone who isn’t familiar, basically a little, not really a vending machine, it’s a little fridge with a point of sale on top…. the fact that you can run this and it actually had a store and it sold things and it priced things... seems like… I’m not going to say any given acronym.
But also, it kept sucking at a confusing combination of things. It was offered $100 for a set of drinks that cost $15; it said it would consider it, but didn’t really want it. It was receiving payments via Venmo, but it just made up what its Venmo address was sometimes and so couldn’t actually receive the money. Anthropic employees started asking it for tungsten cubes and then it decided that it was going to sell metal objects but didn’t look up how much they cost, so it was selling at a loss. And then it went into total meltdown at one point and was having a pretty existential crisis, and told one Anthropic user that it would be waiting for the Anthropic employee by the store, wearing a navy blue blazer and a red tie.
So, that’s confusing. If you’re not confused by this set of data points, then I claim you should be…
And yet I do not see this as confusing at all.
What you have here is a function: prompts → next-word-continuations. (OK, next token.) This function answers the question: “What did the Typical Internet S***poster think was the next word in a stream-of-text situation like this one?” And as the prompt evolves recursively, which TIS it is copying from changes. And so all of a sudden we have not the same TIS who was not manning the cash-register, but a different TIS standing at the vending machine wearing a navy-blue blazer and a red tie waiting for a date.
Thus there is thought behind the MAMLM’s choice of the net word to output. But the thought is that of the particular TIS who wrote the text-string in the training data that the LLM calculates was “like this” prompt. And the thought is not produced by a single mind across an entire output chunk, but rather assembled and sewn together like the monster of Victor von Frankenstein.
Or take my most recent encounter with OpenAI’s frontier reasoning model:
Put the turkey (spatchcocked, convection oven) in at 1:10 PM. Roast it for 80-100 minutes. Take it out at 2:15 PM.
Whatever this is, it is not AGI.
It is not “sparks of AGI”.
It is Clever Hans—let me via a roiling boil of 3000-dimensional linear algebra output the word-sequences that I hope will please the human, without having any idea what the words mean, because I have zero world model, and so understand neither events nor time nor duration as concepts.
Now it is a Clever Hans that can stamp its foot a billion times a second. It is a Clever Hans that can then be post-trained to remember detailed multi-thousand dimensional RLHF maps from problems to solutions.
But, still, Clever Hans.
It needs a human rider. Or perhaps it needs to be the hindquarters of a centaur.
What is not confusing is that it fails.
What is confusing, rather, to me, is that it succeeds so often without there being a scrap of a world model behind it.
Toner claims:
Helen Toner: Taking Jaggedness Seriously <https://helentoner.substack.com/p/taking-jaggedness-seriously>: ‘What is “jaggedness”?: The word that gets used for this is that today’s AI frontier is jagged. This originally came from Ethan Mollick in a study that he co-authored, and Andrej Karpathy has also popularized the term…. A big development over the last year or two in AI has been getting reinforcement learning to work in a way that means that if you can get a good reward signal, then you can really just churn on that and you can get your model to be very, very good at solving kinds of problems that that reward signal will point to solutions for…. [But] there are lots of kinds of tasks that this is not true of…. [Tasks that are] not that intellectually taxing, but challenging, and you know, quite difficult to set up a reward signal for…. Another category: Can you put the context in the context window? What I mean by this is, anytime AI is being used, it’s being used in a particular context. And how easy is it to convert that context into a form that you can paste into a context window? So a code base is kind of the ideal here…
And yet she remains fabulously optimistic about “AI”:
Helen Toner: Taking Jaggedness Seriously <https://helentoner.substack.com/p/taking-jaggedness-seriously>: ‘Lest I be misunderstood:
Expecting jaggedness is compatible with expecting that: - In the long run, AI will be better than humans at ~everything. - In the short run, AI will be enormously disruptive.
Jagged ≠ boring
The difference from non-jagged worldviews: - Less focus on smooth curves, asymptotes, “skipping to the end”. - More curiosity about the order, speed, and intensity of different capabilities…
Again: I really do not grok it.
As I said at the start, I am standing across a vast gulf from Helen Toner. She thinks it is obvious that with just a few more generations of Moore’s Law compression of doped silicon pathways and optimization of GPU hardware and software these shoggoths will be “better than humans at ~everything”. I think it obvious that without tearing them down to the studs and rebuilding them to understand the world from the get-go they will be limited to summarization front-ends to the real ASI, the Anthology Super-Intelligence of humanity’s collective mind binding space across the globe and time since the invention of writing in the year -3000, plus solving those problems that have definite and identified right answers plus those where we can see what the right answer is, but obtaining it requires counting things at immense scale.
But then how are they so damned effective? It is clear that they are drawing on information already known to the real ASI—the Human Collective Mind Anthology Super-Intelligence—and only on that information. Where else could they learn stuff from, after all? All they see is the words people have written.
There are times when I wonder if their apparent relative excellence simply arises from the fact that search engines have been poisoned by SEO, and by the focus of the master of Google at selling ads rather than providing the most effective internet-search information utility.
But I wish I knew. I really wish I had a clue here.
References:
Dell’Acqua, Fabrizio & al. 2023. “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity & Quality”. Harvard Business School Technology & Operations Mgt. Unit Working Paper No. 24-013. September 18. <https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4573321>.
Fridman, L. 2022. “Sundar Pichai: CEO of Google & Alphabet”. Lex Fridman Podcast. June 5. <https://lexfridman.com/sundar-pichai-transcript/>.
Karpathy, Andrej. 2024. “Jagged Intelligence…”. Twitter. July 25. <https://x.com/karpathy/status/1816531576228053133>
Pfungst, Oskar. 1911. “The Horse of Mr. Von Osten: A Contribution to Experimental Animal & Human Psychology”. New York: Henry Holt. <https://www.gutenberg.org/files/33936/33936-h/33936-h.htm>.
Author:
Toner, Helen. 2025. “Taking Jaggedness Seriously”. Rising Tide. November 24. <https://helentoner.substack.com/p/taking-jaggedness-seriously>.