Are We Really on the Path to AGI?

May 27, 2026 Sometimes, a curious amateur’s eye can discern tensions in the work of professionals that the intensity of their focused labor keeps them from seeing clearly themselves. Such amateurs point to questions more than answers, but innocent questions sometimes clear a path toward answers, too. This is the spirit in which what follows is put forward. This year, I’ve found myself a curious amateur working to engage with the enormously impressive conceptual and practical work that is pushing forward the development of artificial intelligence. I am not a practitioner of any of the relevant fields. I’m a recovering political scientist whose work is focused on the health of American political and social institutions. But in the course of exploring the possibility of a short book about the social and ethical challenges inherent in our society’s early engagement with AI — a book that would not pretend to substantial expertise about the technology but would reflect on the risks involved in taking shortcuts around vitally formative work — I have had the opportunity to explore some of the literature being produced by leading thinkers and practitioners in AI, and also to engage with a few such people personally. The experience has taught me a lot, and has shed light on my own limitations of knowledge and ability. Many brilliant and creative people are toiling in this field. But the experience has also left me perplexed by several assumptions in the field that seem, at the very least, not sufficiently articulated and conceptualized, if not in fact ultimately unsupportable. These assumptions pertain in particular to the character of the progress of frontier large language models and the agentic systems built on them, and to this technology’s relation to the various domains of the human experience. They are by no means universally held views, but they are remarkably widespread. My aim here is to describe those assumptions, and to question their alignment with actual developments in the field. I do not imagine that any particular element of the case will be new to anyone involved in AI work, but I hope that formulating this set of reflections into a challenge might be of use. Implicit in much of the thinking of those working to frame the direction of the field, and also in the arguments of those expressing the most intense hopes and fears about its implications for mankind, is the presumption that, as AI grows more capable, its scope of competence will grow more general. This view takes the progress of AI to point in the direction of increasingly generalizable capacities, and ultimately perhaps toward so-called artificial general intelligence, or AGI, which would resemble the broad multi-domain competence of the human mind but at much higher levels of analytical and intellectual ability. In this conception, AGI might be different in kind from what the world has come to understand as artificial intelligence over the past half-decade, but it would plausibly evolve out of today’s AI as a natural extension of the path the technology has followed. The pursuit of AGI has shaped the culture of the field of artificial intelligence from its earliest conceptual origins, and so preceded a lot of the specific technical breakthroughs that have created the technology that has begun to revolutionize the human experience. The way the field now approaches its own work is deeply shaped by that long formation. Worries about the emergence of an unstoppable superintelligence that could swiftly push humanity to extinction — as well as hopes for a complete transformation of the experience of life and an unprecedented explosion of abundance — often assume not only progress in AI but the emergence of a superior general intelligence. Such expectations are sometimes made explicit. But even when they remain unstated, they can be readily discerned beneath much of the rhetoric of people contemplating the future of the field. But is today’s AI really moving in the direction of a generalized intelligence? There are reasons to think so. The extraordinary reach of AI in its present forms is undeniable. Because it has been trained on a body of data that consists of nearly the entire corpus of recorded human knowledge and expression, and because it has ready access to the vast body of knowledge available on the Internet, today’s AI displays an impressive command of a staggeringly broad range of subjects. It is conversant in the parlance of every field of endeavor, and is able not only to call upon existing materials in these fields but also to draw new connections, propose innovations, and articulate complex ideas at many levels of specificity and generality. And perhaps even more remarkably, today’s AI can interact with human beings in our own languages with seemingly total fluency. By absorbing all available articulated human knowledge and discerning patterns in the data that it can extend, it has essentially learned to speak human. Fluency and breadth of knowledge are indicators of general intelligence in the human experience. And it is impossible to interact with today’s large language models without being struck by their capacity to display these in every domain in which we can assess them. They are not yet operating at the highest human levels in every such domain, but they can do impressive work. In every facet of white-collar work, for example, they are already functioning at least at the level of a capable novice, in some cases far higher, and there is every reason to expect them to improve. If they get better and better along so many avenues of human activity, are they not on a path toward a general intelligence that operates at a higher level than ours? One reason we might intuitively resist that conclusion is that simultaneous improvement along so broad a range of domains would not follow in the model of human intelligence. Human ability usually does not grow more general but rather more specific as it improves. Beyond a base level, improvement in most realms requires a narrowing of focus and comes at a cost in terms of other skills. This happens because of scarce time and material resources, but also because of scarce intellectual and physical resources. Greater skill is achieved by greater concentration, which inherently implies a narrowing. But this is not in itself a persuasive case against the potential for progress toward AGI. Artificial intelligence is not human intelligence, and it need not be subject to the same constraints and dynamics. Most significantly, it need not advance in the way that human intelligence does, whether individually or collectively. Precisely that point, however, should cause us to ask a simple question: Just how does artificial intelligence advance? That question — the question of the nature of improvement in contemporary AI — does raise some serious challenges to the case for AGI. Here we should notice a peculiar fact. The means by which today’s AI came to be and the means by which it is now improving are actually quite different from each other. To see that, it might be best to think about large-language-model AI as developing in two broad phases. We can call these a phase of absorptive learning and a phase of recursive learning. Absorptive learning is what it sounds like: it is when a model takes in new information from the world. Recursive learning is when the model’s own outputs, after they have been verified by a separate process, are fed back into it for training. Absorptive learning is likely always to be part of improving the models’ training, while recursive learning has been part of training from the beginning. These phases are not completely separate. But they are distinct and worth understanding separately, because the latter is increasingly displacing the former. The large language models we have come to know began with an extensive process of absorbing a vast mass of learning data. These data consisted of essentially everything that developers could feed their burgeoning models. The rise of the Internet over the previous three decades had created strong incentives to digitize the entire available body of human expression, amassed throughout the history of our species. Almost every product of human culture that survived into the twenty-first century was made legible in digital form, and then every new product of human culture created since the rise of the Internet was produced in digitally legible form. All this was available to serve as training data. For all of the Internet’s transformative power over the human experience, it may ultimately turn out to have been most important for having made the entire available record of human expression accessible to this first phase of training AI models. Crudely speaking, this consisted of feeding all of that data into a statistical model and training the model to spot patterns and predict how they would extend. This first looked like predicting the likeliest next word or item in a sequence. But over time, with vast computing power, an immense body of data, and some critical conceptual breakthroughs in the prediction techniques themselves, projecting the extension of a sequence involved the models internalizing the grammar, forms, tones, turns, and conventions of human expression. And since what was being expressed in all of that data was the substantive work of human civilization, the models effectively became conversant in that substance. They grew increasingly able to match input to output in ways that extended the patterns of human expression, and so could effectively take up complex prompts for information or action and address them with novel complex responses. For human beings, there is an intervening conceptual step between such inputs and outputs. That does not appear to be the case for the large language models. The model effectively predicts what a human response might have been based on the pattern of prior human expression. But it is crucial to see that, because all of this was achieved by a statistical model, it was not scripted or formalized. No programmer wrote precise rules for it. No one specifically decided what the models should learn from the data. And because they discerned underlying patterns in the data they were given, the models learned more than the words or other forms of expression they were fed. They absorbed some dynamics of latent and unstated human expression. They grew able to articulate some judgments they cannot fully explain and that no one had specified. These were inherited implicitly in the absorptive process from the patterns of human expression. This is part of what people mean when they say we can’t fully explain how the models work. The first and foremost achievement of this astonishing absorptive learning technique was, as noted above, the breaking of the language barrier between humanity and its computers. Until that time, communication between people and computers needed to be mediated by various scripts — computer languages, as we called them. So human goals needed to be translated into a highly formalized script before they could be taken up by computers, and the results of computing then needed to be translated from such scripts into forms we would find legible and useful. The first phase of large language models produced unscripted or natural-language computing, which has surely been the most significant invention of the twenty-first century so far. Almost everything that we laymen have found impressive about AI in the last few years has been a function of unscripted computing. The breathtaking fluency and broad scope of the models; their ability to understand plain-language prompts and to respond in our languages; their capacity to write, speak, and produce visual and audio expressions tailor-made to our requests directly in whatever medium we ask for; their ability to translate between languages; and (perhaps most important in the long run) their aptitude in producing computer code that meets parameters we provide in plain language — all these are functions of this breaking of the language barrier. The processes of translation that would have been required to put computing power to work on these goals before would have been insurmountable. In a sense, today’s AI models continue to function as predictive statistical engines. But it has turned out that by digesting as data everything that mankind has had to say to itself and discerning both explicit and implicit patterns in it, such an engine can engage in a kind of emergent learning process that yields more than the sum of its parts. The idea that these models remain “stochastic parrots” grossly understates their transformative potential, especially because it does not reckon with the power of the continuing, fluent interaction of people and computers in the absence of language barriers. The extraordinary, almost magical, process of absorbing all of human expression into a statistical engine has unleashed natural-language computing, and the world will never be the same. The effects of this invention are only beginning to reach into the various domains of the human experience, and to transform them profoundly. We are not wrong to marvel at this power, and to quake before its implications. Both the capacity and the range of what the models can produce are astonishing. Much of the talk in our politics about the need to prepare for immense disruptions in the labor market, education, science, culture, politics, and civic life is warranted. But almost all of that is a function of an innovation that has already occurred but has yet to truly wash over vast swaths of our experience. In terms of the development of the technology itself, the absorptive stage is now largely over. The absorption of learning data is generally not how artificial intelligence will progress further, because there simply isn’t all that much data that remains unabsorbed. Curation of existing data is still making a difference, but that can only continue for so long. And for all its creative potential, there are real limits on the ability of AI to create synthetic new learning data for itself. It does happen, but it requires arduous selection and management, and is meeting diminishing returns. Because it involves extending the patterns of what it has already absorbed, creating artificial learning data threatens to constrain the capabilities of the models rather than extend them further. Its output could still be genuinely novel from our point of view, but it would not contain much information the model did not already possess about how to take its next statistical step, because little or nothing outside the model would have contributed to it. The model can produce options for its next step, but some external information would be needed to choose among them. That is how most progress is likely to happen in this form of AI from here on — by providing the models with some reliable way of choosing among the options they generate and determining the next best step. The patterns found in the data of human experience served this role for the models in the absorptive-learning phase. But in the absence of a lot more external data that can be fed in wholesale, the role will now have to be filled in a more focused and explicit way, by a selection signal or outside verifier. That might be an experiment, a proof-checker, a tester, or some other confrontation with reality. The next phase of AI progress will thus involve the models producing new information and testing it, through recursive trial-and-error processes, against such verifiers. In some respects, recursive learning could be at least as powerful as absorptive learning. The form it has generally taken so far, which is often described as “reinforcement learning,” has produced extraordinary progress. Rather than exposing the model to patterns it can absorb and extend, reinforcement learning involves the model producing new information in response to a query or instruction and then having that information scored in relation to some external standard. The weights of the model are then adjusted accordingly so that attempts that lead to high scores become more likely to be emulated and extended in the future than attempts that lead to low scores. In essence, the model’s own output becomes its training data, once a score is attached to that output. The test is what provides the model with new data, in the form of an answer to the question “is this right?” or “does this work?” For instance, code that the model produces can be run and tested. When it works, the model will learn how to handle such coding circumstances in the future; when it fails, the model will learn it should try a different approach. These are things it didn’t know. In some circumstances, like training the model to deal with sensitive subjects, the model’s output might be scored by a human verifier. In others, like training it to play a game, it might be tested against formal rules. Sometimes it might learn from running its proposed way forward through a simulation, or, on longer timeframes, from the result of a physical or biological experiment it proposes. Some reinforcement was used in the earlier absorptive-learning phase, too, to help align the models with training goals or direct them away from dangerous or undesirable paths. But as new data grows increasingly scarce, reinforcement through recursive experimentation has come to the fore as the mode of learning that will guide this technology further. Combined with the capacities unleashed by unscripted computing, this makes it possible for the models to progress very quickly in readily verifiable domains — essentially learning by trying out their own ideas in the real world. But some domains are much more readily verifiable than others. Above all, computer coding is uniquely verifiable. You can run code in fractions of a second and see if the program compiles and works. If it reports an error, you can try another approach immediately. That process can repeat recursively, and the model can learn a huge amount from it very quickly. That doesn’t just mean that you eventually end up with working code suited to requirements described in natural language; it also means that (when this process happens as a way of training a model) you end up with a model that gets better and better at coding. It learns from its mistakes and improves its performance at staggering speeds. A computer program, which is just a formal model that executes by set rules with no unmodeled intrusions, is a near-ideal environment for reinforcement learning. This is why so much of the action in AI progress is now in coding itself. It’s not just because the AI models are themselves programs created by coders but also because the way they now improve is uniquely well suited to making them better and better at coding. It would not be unfair to say that large-language-model AI is now (and will increasingly be) a coding engine that, on the side, also utterly transforms every realm of human life. Most of the transformation of the rest of the world is a function of the spread of unscripted computing achieved in the absorptive-learning phase of this technology. But in coding, there is a seemingly endless series of revolutionary advances happening right now. This can be hard for us outsiders to grasp. Most of the excitement you see on social media when a new model is released is happening within the coding world. That’s why it’s so full of opaque talk about workflows and compilers. At the same time, it can be difficult for insiders to grasp that their own professional environment is changing much more quickly and dramatically than those of other people. Coding is hugely important beyond its own domain, because it is the substrate of computing, and therefore of much of modern life. Just as the Internet drove the digitization of expression that made our civilization legible to the absorptive-learning phase of AI, so it also drove the encoding of agency that has made much of our world manipulable by an artificial programming genius. Both the greatest promise and the greatest dangers that AI harbors are functions of its ability to code increasingly well and increasingly independently, and this domain above all is where this technology could both vastly exceed human abilities and extend beyond our ability to control. That is not a reason to expect the emergence of a superior generalized intelligence, but it is a cause for serious worry and for careful controls. A dangerous instrument that could do enormous harm in the wrong hands is not the same as an independent agent of chaos and calamity. But our society does impose rules on the uses of dangerous instruments. The fact that coding now plays a part in so many other human endeavors does not, however, mean that the abilities of AI models will be improving simultaneously in every domain. The models will not become better and better lawyers, teachers, doctors, and gardeners at anything like the pace at which they are becoming better coders. It also doesn’t mean the models will become more competent at choosing how to direct their extraordinary capacities for coding. Superhuman programming ability does not by itself amount to, or even point toward, anything that might reasonably be called general intelligence. Beyond coding, the recursive-learning phase will probably achieve the greatest progress in those arenas where ready verification is available to it. These include mathematics and the most formalistic of the natural sciences, and domains where reliable modeling, testing, or simulation is available, like some facets of chemistry, electrical engineering and chip design, aerodynamics, protein folding, or meteorology. There are also some fields that are very formal or where verification is available but where there is more noise in the way, so that the models can learn quickly only with significant human help, like some empirical social sciences, or some forms of finance. And there are other domains where there just aren’t ready objective standards, or where quick experimentation is not easily available — not only art or management, but also public policy and geopolitics, and even many subfields of biology. There are also some realms where much if not most knowledge is tacit and unstated, and so is not available for verification just as it was not available for absorption. This is probably true of a lot of what we human beings do with our hands. Artificial intelligence will matter enormously in all these fields, but it will not make similar progress in all of them. There will be improvements in the speed and accuracy of performing tasks that today’s AI is already capable of doing — that is, improvements in performance that do not expand the range of competence. Some progress will also be achieved by the improvement of the overall computing power of the models, and some will be a function of improvements achieved in related fields. The models’ written work and their video and audio abilities, for instance, are likely to noticeably improve from one model version to another, and that will matter in many domains. There are also ways to design tests and verifiers in fields that are not inherently formalized. But this is slow, uncertain work, and such tests are unavoidably crude and will tend to miss exactly the kind of implicit or practical knowledge that the models are already likely to be lacking, or to optimize only for what can be measured. These are problems that computer scientists have understood since long before the advent of large language models. Ironically, the need to formulate verifiers in different domains amounts to a partial reversion to the formal computer science that preceded modern AI. We have made our languages fully intelligible to computers, but that does not mean that we have made our world fully intelligible to them. Language is only a medium. The parts of our world most amenable to being described by highly formalized language are still the ones most accessible to computers. And the difference is a function of the underlying subject, not our language about it. As Aristotle understood 23 centuries ago, we can only “look for precision in each class of things just so far as the nature of the subject admits; it is evidently equally foolish to accept probable reasoning from a mathematician as to demand from a rhetorician scientific proofs.” The absorptive-learning phase of AI caused us to downplay the significance of this point, but the recursive-learning phase is bringing its importance back into the light, and with it the importance of the fact that AI has mastered the medium of the human experience but not fully its substance. It is again becoming necessary to insert a layer of scripting between human beings and computers — this time not in the interface between the two but in the process of learning and so the means of progress of this new technology. Prior generations of computer scientists also attempted to have experts in different fields articulate their patterns of judgment as explicit rules. The potential of such methods proved very limited, and that may well now happen again, even if the spectacular achievements of the absorptive-learning phase mean that AI starts from a much higher baseline of competence in many fields. None of this means that AI capabilities will not be progressing in less formal practices. But the models will not be gaining competence in fields that lack ready formal verifiers at anything like the pace they can achieve in those that do offer such means of recursive learning. One implication of this difference is especially significant for the white-collar world. As noted above, in many professional fields the absorptive-learning phase enabled AI models to achieve roughly the level of competence of novices but not yet of expert practitioners. This has put those fields in the uncomfortable position of questioning the economic value of younger and less experienced professionals and perhaps moving to replace some of them with AI systems. In effect, AI in this early phase has made younger people appear less useful to older people in some parts of the professional world. But replacing those novices with AI would not only be a failure of intergenerational fidelity (though it would be that), it would also be a failure to grapple with the fuller relationship between novices and experts in professions that traffic in significant implicit and informal knowledge. Simply put, today’s AI models can do much of what novices can do in these fields, but they cannot learn much of what novices can learn. If the ability of novices to apprehend informal knowledge in fact cannot be replaced, then replacing them with AI stands to profoundly damage the professions involved, by breaking the continuity that makes it possible to sustain and advance their most difficult work. The unusual resistance to and resentment of AI among the rising generation in swaths of the white-collar world is thus a signal that our society should take very seriously. It is a warning not only against a gross failure of solicitude toward the rising generation but also against a loss of vital knowledge that is contained in informal modes of judgment and is essential to honing the practical wisdom that cannot be attained by a disembodied statistical engine. That is one potential consequence of the highly and increasingly uneven capacity for progress of today’s artificial intelligence. But for the field’s self-conception, perhaps the most significant implication is simply that, as it makes uneven progress, the cutting edge of AI looks likely to narrow, not broaden. Some in the field have described this unevenness as “jagged” progress. But that probably understates the differences between different domains in relation to the potential of AI models to improve their competence. These differences are enormous and likely to grow. They could well prove to be differences in kind and not just in degree. As its capabilities in some domains increase dramatically, the field is likely to focus more and more on those domains. Widening gaps in capabilities will make the returns to AI researchers’ work in different realms very different, and it is only natural for developers to focus where they can achieve the most. That in turn is likely to draw more time, money, talent, and computing resources toward applications of AI in the most formalized fields of knowledge. Moreover, the gaps are likely to grow in a self-perpetuating cycle, because progress in coding, math, and other highly formalized fields will also improve the AI models in ways that make more such progress in these particular domains more achievable. This dynamic seems likely only to intensify if so-called recursive self-improvement were to be achieved. RSI is a kind of holy grail for some artificial intelligence developers, because it holds out the promise of using the extraordinary capability of the models to further enhance the models themselves, and so of throwing the field into a cycle of improvement that requires little if any human intervention and can proceed at the speed of computing and in directions human beings might not imagine or foresee. This prospect underlies both the highest hopes and the deepest fears of many of the best-informed observers and practitioners of AI. But at the root of both the hopes and the fears is the assumption that the speed and creativity of the progress made possible by recursive self-improvement would propel the models toward a more and more generalized intelligence, and ultimately toward an artificial general intelligence that could be capable of unimagined achievements but could also be impossible for human beings to control. A great deal of the ethics work now pursued around artificial intelligence is focused on the challenges such AGI could produce. As we have begun to see, however, the notion that greater ability will be attached to greater generality is not well supported, and seems increasingly poorly aligned with the actual direction of developments in the field. It sometimes leans upon the implicit assumption that enough progress in one or a few domains will yield a magical moment of emergent generality in the models — a notion not well supported by either the character of human intelligence as we understand it or the direction of development in AI. Progress in highly formalized domains like coding and math certainly yields some gains in unrelated fields, and it is imaginable that, at a sufficient pace of improvement, the volume of these could become transformative in those areas or yield a general advancement toward a new technological paradigm. It is also imaginable that better verifiers could be developed in fields that now seem inhospitable to formal recursive trial-and-error tests. But the public case for thinking that today’s AI is on a path to general intelligence too rarely acknowledges these daunting obstacles to be overcome. More importantly, that case would have to confront the likelihood that the very logic of recursive self-improvement may cut in the opposite direction. To the extent that it would magnify existing modes of progress in the field, RSI would entail leaning heavily into the very process that is now yielding increasingly uneven progress for AI in different human domains. It is likely to accelerate a fraction of what the models can now do but without accelerating what now makes them most broadly useful. The assumption that RSI would independently facilitate the emergence of a greater capacity for generality is invoked as a premise in many arguments about the future of AI, when in fact it is a proposition, and one that so far lacks much evidentiary or logical support. At the very least, RSI would not overcome the obstacles to greater generality of competence just by being independently recursive. This point can be stated more bluntly: The notion that we are moving in the direction of artificial general intelligence looks more like an article of faith than the conclusion of any argument or the implication of any set of observations. It was an assumption present at the birth of modern artificial intelligence, planted there by a variety of brilliant theorists and science fiction writers. Its profound influence on the culture of Silicon Valley has meant that it has played an important role in the conceptual evolution of the field. But it is not obviously justified by the evolution of the technology itself in recent years, and seems at odds with the general direction of that evolution. None of this disproves the possibility of AGI. If the burden of proof were on skeptics to show that recursive self-improvement does not point toward AGI and that the field is not likely to move toward a general superintelligence, that burden could not be met at this point. But if the burden is on the partisans of AGI to show that today’s AI models are on a path toward greater generality, that supercharging the recursive-learning phase of the technology by enabling it to accelerate independently would push them further down that path, and that the dilemmas of living with a generalized superintelligence are therefore the ones that should occupy us most intensely, that burden certainly could not be met now either. Needless to say, the narrative of AI progress sketched here is necessarily condensed and incomplete. And it takes up numerous subjects that are dealt with at great depth, and from many different points of view, in the substantial literature that has developed in the field. Its aim is only to encapsulate what appears from the outside to be a serious tension between the general direction of practical development in the field and the general tenor of its practitioners’ expectations regarding its implications for society. This narrative does not mean to downplay the transformative potential of artificial intelligence. In the years to come, our society will confront both unprecedented opportunities and immense challenges in work, education, culture, politics, and many other arenas as a result of the spread of natural-language computing and its implications. The further unique and accelerating transformation of coding, math, and related domains, meanwhile, will also present us with previously unimaginable benefits and predicaments, and the proper containment of superhuman coding and its offshoots in particular will be a crucial, daunting challenge in the coming years. But this view of where we may be headed does suggest a recognizable process of adaptation, utilization of benefits, and mitigation of risks which, colossal though this process must be, has its analogies in our civilizational experience. It argues, for instance, for an approach to regulation that begins with an extension of existing authorities and expertise and their further evolution in response to felt needs and circumstances. And it suggests that the greatest risks we should prepare for involve cybersecurity, economic displacement, and temptations to self-degradation and dehumanization through shortcuts around essential formative work. All of that would be hard enough to handle. But it threatens to become impossible if the people most directly engaged in developing artificial intelligence persuade themselves and others that it is putting us on a path toward a self-accelerating general superintelligence that harbors the end of the epoch of human dominance on earth, and perhaps our extinction. Those voicing such scenarios sometimes insist they raise them only because it makes sense to be aware of even very low-probability catastrophic potential outcomes, and that this isn’t really where they expect AI to point. But such claims are belied by the evident priorities and choices of the leading-edge AI firms in the United States today. These companies are clearly moved by intense concern about recursive self-improvement leading to an artificial general intelligence that could slip out of human control. This sometimes leads them to call for caution, but at other times it leads them to act recklessly and to undermine the cause of effective cybersecurity — seemingly out of a perverse commitment to race toward AGI to prevent such power from falling into the hands of developers less responsible than themselves. These leaders of the industry need to take account of both the scope and the limits of their obligations to the larger society, overcome the intense pull of science fiction, and consider in an open-minded and public way what the evidence emerging from their own work really suggests about the direction of progress in artificial intelligence. The way they tend to speak about the future now seems dangerously disconnected from such reflection. May 27, 2026 May 29, 2026 Exhausted by science and tech debates that go nowhere?