The Big Con of Agentic AI

Today that advice seems more pertinent than ever. The seductive promise of AI has drawn in individuals, organizations, and governments alike. A student submits an AI-generated essay, bypassing the cognitive struggle through which writing becomes understanding. A company replaces human employees with AI agents, shedding the tacit, institutional knowledge needed to evaluate the AI output. A court supplements sentencing decisions with opaque and potentially biased algorithmic risk scores. Whatever the motivation in each case, ceding cognition, judgment, and accountability to AI may well become the default over time, until our dependency on AI — and the players in its broader ecosystem — gets so deeply embedded and normalized that reversing it seems neither necessary nor feasible. This forfeiture of human agency is concerning enough in itself. What makes it more so is that the polished-looking AI output being deferred to is, on closer inspection, often of only generic value, and sometimes simply wrong. There is an illuminating parallel here with the management consulting industry, which performs a strikingly similar confidence trick, or “con”. Consultants routinely package generic analyses in polished presentations and project far greater certainty in their recommendations than the underlying evidence would warrant. Organizations that defer to external consultants wholesale gradually shed the capacity to think for themselves. Consultants can be wrong, both in the quality of their insights and in their ethical conduct, and AI systems are, at their core, probabilistic pattern matchers with no intrinsic values and no accountability for the outputs they produce. Both derive their authority less from demonstrated correctness than from the asymmetry between the confidence with which their recommendations are delivered and the client’s diminishing ability to challenge them. Taking the consulting industry’s structural dynamic as an instructive analogy, this article synthesizes what we know about our growing dependence on agentic AI, traces the risks of overdependence at the individual, organizational, and societal levels, and proposes ways to reclaim agency at each of those levels before our capacity to do so irretrievably slips away. The Original Con In 2023, the economists Mariana Mazzucato and Rosie Collington published The Big Con, arguing that a confluence of incentive structures, information asymmetries, and institutional pressures had allowed the management consulting industry to extract returns in excess of the value it creates. Surely, sophisticated clients would simply stop hiring expensive firms that did not deliver? Not quite, since a client who outsources a strategic function over many years may no longer possess the internal expertise to evaluate whether the advice it receives is sound, which is a key dynamic that perpetuates the engagement. The parallel with today’s AI vendors is noteworthy: deep integration, high costs, and a customer base that progressively grows less capable of evaluating what it is being sold. Despite its conspiratorial connotation, the central problem The Big Con highlights is structural rather than moral. Many consultants in Mazzucato and Collington’s account were genuinely trying to have a positive impact, and many of the officials who hired them were doing their best to act responsibly under budget and staffing pressures. The arrangement required neither cynicism from the consultants nor negligence from the clients, but simply that repeated delegation be allowed to run its natural course. What The Big Con ultimately offers is less a critique of any industry or individual than a warning about how markets reward short-termism. Outsourcing as a “rationalization” measure can paint a falsely favorable financial picture in the near term, shield decision makers by externalizing accountability, and hollow out the capability to strategize and execute. It is the corporate equivalent of managing fitness through expensive personal trainers rather than developing the habit of exercise yourself: no doubt, trainers can deliver real value, but they can also build (over-)dependence rather than self-sufficiency. The mechanism at work is something akin to the dark side of what management scholars call “unlearning by not doing.” The less an organization performs a function internally, the less it knows how to do it; the less it knows, the more it needs outside help; and the more it pays for outside help, the less it builds the knowledge that would eventually make outside help unnecessary. A reform movement that swept through public administration from the 1980s onward, in the U.S. and beyond, popularized exactly this logic under the slogan “steer more, row less.” But Mazzucato and Collington show why that approach backfired for functions where doing and directing cannot be cleanly separated. As they put it, “The less [an organization] rows, the less it learns, the less productive it becomes: the less it can steer.” The risk is especially acute for strategically central functions, because once those functions migrate outside, rebuilding them internally may take years. The broader pattern is rooted in the theory of comparative advantage. In a complex, specialized world, it can be rational to concentrate on what one does best and pay others to handle everything else. Individuals hire plumbers and electricians. Companies bring in specialists for tax and legal work. Governments contract out where private providers have deeper expertise. Each decision is defensible in isolation. It starts to become a problem, however, when the logic of comparative advantage carves out not just peripheral tasks but the essential, strategic work that defines what an individual, organization, or institution actually is. A student who outsources her thinking, a company that outsources its strategy, a government that outsources its policymaking: in each case the delegating party cedes the very capacity that shapes its identity. With agentic AI, this forfeiture goes further still, as agency passes to a system with no real accountability to the entity using it. The Next Big Con The AI story of the 2020s bears intriguing parallels to the consulting story of past decades: promising early wins, rational incremental delegation, and a structural dependency that only becomes visible once reversing it is too costly. Early AI use cases, from drafting text and writing code to summarizing documents and automating routine processes, are intuitive and deliver results with little friction. The barrier to initial adoption feels lower than for almost any prior technology. The mainstreaming of generative AI since the launch of ChatGPT in late 2022 compressed what might have been a decade or more of gradual institutional adoption into just a few years. AI vendors are desperate to rapidly capture market share and show large-scale adoption to justify growth-stock valuations, so AI services are currently priced well below the true cost of delivery. The short-term economics are thus artificially compelling, much as early cloud and SaaS offerings used subsidized pricing to build the integration depth that later made switching painful. As such, agentic AI is poised to accelerate the displacement of work at a pace and scale that consulting never approached: where enterprise software once provided tools for skilled professionals to operate, AI-driven platforms are increasingly closing the books, drafting contracts, and generating strategy documents directly, removing humans from the loop, and with them, the organizational knowledge that justified keeping the function in-house. Before examining the risks, it is worth acknowledging the significant benefits that the judicious use of AI can deliver. Automating repetitive, rules-based, high-volume tasks frees human attention for work that is more intellectually stimulating and requires careful judgment. Personalized AI tutoring tools show early promise — early studies suggest measurable gains in student engagement and comprehension. A well-used AI sparring partner can sharpen thinking rather than replacing it; e.g., one can ask it to argue the opposite side, find the weakest link in one’s reasoning, or surface arguments that have not yet been considered. This is what Cory Doctorow, in his recent book The Reverse Centaur’s Guide to Life After AI, calls being a “centaur”: a person who chooses how and when to use AI as a tool, retaining agency over the work. Used wisely, agentic AI can make people more capable, not less, so it is obvious to see the appeal. However, the benefit curve bends toward diminishing returns when the delegation to AI becomes so pervasive and thoughtless that the human can no longer critically evaluate the AI output and exercise meaningful judgment to override it. Beyond that point, the human is not augmented but replaced, and with them goes the organizational knowledge needed to properly manage the AI — both in terms of method and output. And where humans remain, they are increasingly required to serve the AI (sometimes to the point of exploitation), in effect becoming what Doctorow calls “reverse centaurs.” An individual who stops wrestling with problems and simply accepts the answers, an organization that invests in AI-led process automation without developing employees who can understand and oversee it, a government agency that stops developing regulatory expertise and merely ratifies what the AI recommends: in each case the use of AI leads to atrophy, and the descent into overdependence can be gradual enough that the shift goes unnoticed until reversing it is difficult. Meanwhile, the commercial narrative driving AI adoption is undergoing a significant shift. Vendors and investors are increasingly positioning AI not only as a way to augment human work but as a way to replace it. Software budgets are often smaller and less strategic than payroll budgets, so reframing AI in terms of labor replacement implies a far greater addressable market. The natural beachhead is outsourced labor: activities with well-defined scope, separable budgets, and organizations already accustomed to external solutions. More ambiguous and strategically central in-house roles may follow, as positions are decomposed into tasks amenable to AI automation; see Sangeet Paul Choudary’s book Reshuffle for a deeper discussion. If not managed carefully, this process risks automating the tasks where human oversight and judgment are most consequential. There is also a potential conflict of interest in who is advising the transition. The consultancies that drove the labor-outsourcing wave are now collecting lucrative fees as advisors on AI adoption strategies. McKinsey has projected $2.6 to $4.4 trillion in annual value from generative AI, a forecast that conveniently underpins the case for large-scale advisory engagements. The moral hazard is structural: any external advisor paid to assist with AI transformation has a clear disincentive to help clients build the internal capability that would eventually make follow-on engagements unnecessary. No individual consultant has to act in bad faith for this dynamic to operate. It is the logic of markets, not the malice of firms, and it is the same logic that sustained the original con of consulting elaborated by Mazzucato and Collington. Let us now examine, in turn, the cost of this growing overdependence on AI at the individual, organizational, and societal level. The Individual Cost The individual cost of cognitive delegation is insidious precisely because AI output can increasingly pass for the conclusions of human analysis. George Orwell identified the key pattern in his 1946 essay Confessions of a Book Reviewer, which lamented how industrial book reviewing had hollowed the craft to a simulacrum of engagement. The reviewer was “constantly inventing reactions towards books about which [he] has no spontaneous feelings whatever,” pouring “his immortal spirit down the drain, half a pint at a time.” AI has the capacity to turn all of us into Orwell’s factory reviewer. This phenomenon is not new — people upvote social media posts they have not read and sign petitions they do not understand — but AI industrializes it, making it easy to attach one’s name to consequential work despite minimal involvement. The value of cognitive work lies not only in what is produced but in what the person becomes through the act of producing it. In offloading that struggle to agentic AI, we risk forfeiting not just the output but the capacity for original thought. In May 2026, MIT professor Micah Nathan wrote in The Guardian about a student who submitted an AI-generated story to a creative writing workshop. What struck him was the nature of the student’s intellectual capitulation. She began by feeding her story to the AI for a grammar check. It suggested line edits and she accepted them. Then structural edits. Then a full rewrite. It was, Prof. Nathan wrote, “a sequence similar to an addict’s descent,” each step feeling small, each one making the next more likely. His student had not set out to surrender authorship so completely, but at each point where she might have reclaimed it, fear of judgment by her peers and her teacher proved stronger, and she let the AI take over instead. As Prof. Nathan observed, “Writing isn’t just the production of sentences — it’s the training of endurance by way of sustained attention. It’s a way of learning what one thinks by attempting to say it.” Prolonged use of AI assistance weakens the capacity to form and articulate thoughts independently, until one day a blank page looks not merely uncomfortable but daunting, and what was once a useful tool has become a crutch one cannot imagine doing without. Academic studies are beginning to provide empirical grounding for this concern. Recently published findings from large-scale randomized controlled trials show that even brief AI assistance can impair subsequent independent performance. Study participants who used AI not only performed worse unaided but also stopped trying sooner. The hypothesized mechanism is hedonic adaptation, whereby once ready-made answers become the norm, working through a problem independently begins to feel disproportionately costly. Another study of nearly two thousand professional adults found that while the majority agreed AI “did most of the thinking” across executive function tasks, they also reported that the resulting ideas did not feel fully their own. A third study found measurably lower neural connectivity among participants who used AI to write essays. The researchers coined the term cognitive debt to describe “a condition in which repeated reliance on [AI] replaces the effortful cognitive processes required for independent thinking.” While these studies measure specific task-level performance, the broader claim that atrophy compounds across complex judgment-intensive work is an extrapolation consistent with hedonic adaptation and merits further research. Taken together, the evidence suggests that sustained use of agentic AI may erode both the cognitive skill and the appetite for the struggle that would rebuild it. The Organizational Cost The organizational risks extend beyond individual deskilling. When many people delegate cognitive work over time, distributed institutional memory thins out, governance structures become hollow without the internal judgment they depend on, and strategic autonomy disappears once no one inside the organization can evaluate or challenge the vendor. If the consulting analogy holds, a troubling descent could unfold. Agentic AI initially augments workers, then hiring slows down as the AI covers enough ground that headcount growth seems hard to justify, and eventually — in the absence of deliberate governance — roles are eliminated and workflows redesigned around AI agents. The vital institutional knowledge that the departing staff carry leaves with them and is not rebuilt. Given how much more rapidly AI is being embedded into organizations than consulting relationships ever were, this endpoint could arrive sooner than many expect. Moreover, AI services are currently subsidized far below their true cost to capture market share. When the focus eventually shifts to profitability, vendors may raise prices (e.g., by switching from subscriptions to usage-based billing, as we are already seeing with GitHub Copilot). Organizations that substantially — and perhaps indiscriminately — slash headcount during the subsidy period will find the resulting dependence on AI vendors costly to unwind. The short-term economics are already surprising early adopters. Microsoft (ironically, also a major purveyor of AI hype and an AI vendor in its own right) cut its internal AI coding licenses in 2026, roughly six months after encouraging company-wide adoption, citing spiraling costs. Uber exhausted its annual AI coding budget in four months. Salesforce expects to pay Anthropic approximately $300 million in 2026 alone. An Nvidia vice president acknowledged publicly that “the cost of compute is far beyond the costs of the employees.” Token prices are falling, but agentic AI consumes tokens at a rate that outpaces the price decline. Goldman Sachs forecasts a 24x increase in consumption by 2030, and at some firms AI token expenditures already equal 10% of total labor costs. A recent survey of nearly 2,500 companies found that for every dollar spent on AI tokens, a pitiful 18 cents generated user-facing value, while 44 cents went toward fixing bugs the AI systems themselves introduced. Worse, when organizations track adoption and productivity based on token spend, employees engage in “tokenmaxxing” (inflating usage metrics by deliberately using AI inefficiently). So not only does token spend fall short of delivering commensurate productivity gains, but resources channeled into AI infrastructure are not being invested in people who would accomplish the same work more cost-effectively while building the institutional knowledge the organization will eventually need. There is also what practitioners call the oversight tax. In high-stakes contexts (e.g., legal, financial, medical), organizations typically keep human reviewers in the loop to satisfy liability requirements. The organization pays for both the system and the reviewer, while the reviewer’s independent capacity steadily weakens in the presence of AI. Controlled studies of AI-assisted medical screening suggest that the apparent skill atrophy may be due to a “safety-net effect,” whereby humans put in less effort when they know the AI is there to catch their mistakes. Fred Brooks, in The Mythical Man-Month, argued that quality depends on conceptual integrity: the design must be held in the mind of someone who understands the whole. A human who reviews AI outputs they did not produce and cannot fully interrogate has surrendered that integrity. The oversight tax therefore means that productivity gains are overstated, and the accountability those gains are supposed to justify is hollower than it appears. The Societal Cost When the organization in question is a school, a court, or a government ministry, the stakes are high. Normalizing the use of agentic AI as a substitute for critical thinking in education risks producing a generation of graduates that undermines the economy and culture, and is more vulnerable to political manipulation. A judge who cannot interrogate the algorithmic risk score informing a sentencing decision may inadvertently (yet systematically) disadvantage defendants of a certain race or ethnicity. A legislature that cedes undue influence over the drafting of bills to AI agents may end up governing against the will of the people it represents. The term “human in the loop” (HITL) was coined to describe humans who are in control, can challenge the AI, and are accountable for the result. In practice, however, this often means blindly approving AI output under time pressure, and serving as what analyst Dan Davies calls an accountability sink — there not primarily to prevent errors but to absorb blame for them. The problem is that the people who care most about the societal outcome typically have no access to the AI agent’s reasoning, while the vendors have no institutional incentive to challenge their own systems. Accountability requires someone both motivated and equipped to act. HITL today often produces neither. There is also a geopolitical dimension to all of this. The United States and China together control approximately 90% of global compute capacity and 70-80% of global AI investment. Anton Leicht has recently described the closing of what he called the “Andy Warhol era of AI access,” the brief period when frontier capabilities were available to rich and poor users alike. The emerging structure is decidedly hierarchical: new frontier capabilities flow first to national security establishments, then to large trusted enterprises, then to selected international partners, and only then to everyone else. European nations that depend on American AI infrastructure, and Asian economies relying on either American or Chinese platforms, are building critical public capacity on a foundation that can be conditioned, priced beyond reach, or withdrawn entirely as per the provider’s strategic interests. Singapore parliamentarian Kenneth Tiong astutely observed of his country’s AI strategy that “we are building an AI hub on an assumption we do not control.” That abstract risk became concrete in June 2026, when the U.S. government ordered Anthropic to suspend access to its most advanced models, Fable 5 and Mythos 5, for all foreign nationals, citing national security concerns. Anthropic, which had publicly called for greater government oversight of AI, found itself unable to defend hundreds of millions of users against a directive it described as poorly justified and based on “verbal evidence of a narrow, non-universal” vulnerability. The company was given no choice and had to abruptly disable the said models for all customers worldwide to ensure compliance. The episode illustrates what dependency on frontier AI from a single provider means in practice. The U.S. government has the legal authority to order the suspension of commercial model access globally, a fact that holds regardless of whether this specific order was justified. And the pricing, access, and deployment decisions of a small number of AI vendors now have the power to constrain the strategic options of entire organizations and governments overnight, without warning, and for reasons those organizations cannot contest. Statistician and economist Ernst Friedrich Schumacher examined a structurally similar pattern in 1973, analyzing what happened when capital-intensive Western technology was transferred to developing economies. It increased aggregate output while creating a “dual society”, with the benefits distributed unevenly across social classes, and building dependency on foreign expertise that local populations could neither maintain nor replicate. In the age of agentic AI, open-weight models partially address this risk by allowing local deployment and adaptation, but the gap between open and frontier capability remains large and is actively maintained by the leading providers. The appropriate technology movement that grew from Schumacher’s work asked: Does the new technology increase local agency, or does it concentrate benefit elsewhere? Can it be operated without permanent external dependency? Those questions deserve to be asked of AI deployment today, at every scale from the individual to the societal. Two Dead Ends and a Third Path Two responses to the risks described above are tempting but most likely wrong. The first is banning agentic AI outright, as some jurisdictions have previously contemplated with facial recognition or autonomous weapons. This would cede the field to the actors least inclined to exercise restraint while forfeiting potential benefits, including advances in medicine, scientific research, and education. The second is uncritical adoption, letting market incentives and vendor roadmaps determine how deeply agentic AI penetrates cognitive and democratic life. As Reich, Sahami, and Weinstein warn in their book System Error, this is not a neutral choice. Allowing the optimization logic of markets — which is indifferent to cognitive ownership, institutional memory, and democratic accountability — to make decisions that affect people is itself a decision, made by default. Even well-intentioned innovators can optimize the wrong things at scale and end up imposing certain values and choices on the rest of us whether we like it or not. The third path lies somewhere between prohibition and sleepwalking and requires active choices at each of the three levels of analysis covered in the preceding sections. At the individual level, agentic AI is well-suited to automating grunt work and serving as a sparring partner. Judgment, however, should remain with the human user who writes the synthesis, makes the call, and signs off the outcome. The productive struggle is worth maintaining deliberately. Research on professional adults suggests that workers who modified AI outputs more often showed significantly higher confidence in their independent reasoning, making override frequency a practical proxy for retained cognitive agency. For each task, we should ask whether AI assistance is building capacity or substituting for it. Where it is the former, use it fully. Where it is the latter, deliberate resistance may be necessary. At the organizational level, institutional memory must be treated as a strategic asset that requires active investment. Build documentation practices, mentorship structures, and rotation of staff through functions that AI assists but does not own. Diversify AI suppliers deliberately: no single-vendor dependency for any critical cognitive function, and hybrid architectures that combine small, locally-operable models for routine tasks with larger models for high-stakes reasoning to preserve strategic options. Ensure that “humans in the loop” do in fact have the knowledge and access to challenge the AI output, the time to do so, and organizational accountability for the decision. And above all, make structural sovereignty (retaining meaningful control over the knowledge, compute, data, and models that underpin critical operations) a board-level concern. At the societal level, public policy can draw on the appropriate technology criteria Schumacher articulated: Does this deployment increase local agency? Can it function without permanent external dependency? Governments investing in AI for public services should also invest in open-weight models, interoperability standards, and shared compute infrastructure to reduce geopolitical vulnerability. HITL in consequential public decisions (e.g., judicial, welfare, healthcare, immigration) should carry a legal standard: the human reviewer must be able to explain and defend the output they ratify, not merely attest that they were present when it was generated. The field of AI ethics deserves more recognition and must be empowered to ensure rigorous governance of agentic AI along the lines that bioethics established for medicine. Individual and organizational discipline are necessary but not sufficient. Structural change requires regulation, procurement standards, and international coordination in addition to individual resolve. That shift will not happen through market forces alone, which have thus far pointed in the opposite direction — citizens and institutions must demand it. What all three levels share is the same underlying principle: the tools should serve the people who use them, which requires a deliberate choice to keep it that way. Overdependence is not inevitable. And beyond a certain adoption threshold, the evidence points to significant risks of harm, even if the overall economic impact remains contested. We can acknowledge this without being anti-technology. It is the same maturity that medicine developed when it established contraindications, dosage limits, and informed consent. It is the recognition that deliberate governance should complement the enthusiasm of a rollout. The Wrap The consulting and AI waves share a similar structural logic. Seemingly rational short-term delegation accumulates into a dependency that erodes the capacity to steer, without the need for bad actors, and without the consequences becoming fully visible until reversing course has become too difficult. The AI version of this process is faster, more deeply embedded, and operates simultaneously at the individual, organizational, and societal level. Ironically, the analytical work, code generation, and software delivery that consulting firms have long sold as high-value services are the very categories that AI vendors are now automating and bundling directly into their platforms. The same disintermediation logic the consulting industry applied to corporate functions is now being applied to the consulting industry itself, as indicated by the sharp declines in share prices of large, people-heavy consulting firms (Accenture, Capgemini, etc.) over the past several months. Whether that proves a corrective is doubtful: substituting a relationship-based dependency for a platform-based one, where the vendor’s leverage is structural rather than personal, may actually lead to deeper lock-in. What seems likely, though, is that the logic of the con will not spare the conmen. Ultimately, it is worth remembering that we are the ones building AI tools, and we can decide how to use them. The advice that stayed with me from that first-year course on programming was not about avoiding computers — it was about doing the thinking first. That principle scales to the design of companies and public institutions. The question is whether we will apply it deliberately, or let the current of short-term convenience carry us somewhere none of us intend to go.