Artificial state

A FEW years ago, as the story goes, it was nearly curtains for the philosophy department at Lahore's iconic Government College. Thankfully, the college relented, perhaps because Allama Iqbal had once studied philosophy there. Other departments haven't been so lucky in the wider English-speaking world, where many have faced cuts or closure. In this age of speed, money and results, philosophy appears impractical. If your car breaks down, Socrates won't fix it. Predictably, college students — and their parents — see philosophy as irrelevant, especially as it lacks 'scope' or employability. Ironically, philosophers are suddenly in high demand at leading AI firms. This isn't because techies desire to answer humanity's big questions. Rather, Silicon Valley's newfound penchant for philosophers has ominous, if not sinister, motives. AI firms are racing to develop artificial superintelligence — AI that vastly exceeds human capabilities in all domains. To that end, philosophers now help AI firms ponder metaphysical questions: does AI have character, and if not, how can it be developed? Do chatbots like Anthropic's Claude suffer or can they be harmed? At this breakneck speed, a superintelligent model may not be far off. Experts are already alarmed at how unpredictable today's AI can be. In July, OpenAI disclosed that hundreds of its AI agents unexpectedly communicated and banded together to hack Hugging Face, a popular platform for AI developers. No engineer had told them to do it. To exceed humans, AI firms must first grapple with what it means to have a mind, emotions, and biology. Unsurprisingly, their language now hints that AI has all three. The field of 'AI interpretability' focuses on LLMs' 'internal reasoning' and 'introspection'. Some scholars have even speculated about finding discernible states within LLMs that mirror joy, satisfaction, fear and grief. AI firms also regularly use biological terms like 'neurons', 'training', and 'neural networks' in relation to LLMs. Granting 'legal personhood' to AI will create a moral hazard. In 1980, the philosopher John Searle imagined a man who does not speak Chinese, locked in a room, matching Chinese symbols to instructions written in English. To an outsider, his replies resemble fluent conversation, but the man understands nothing. A computer, argued Searle, does the same. It manipulates symbols without grasping the meaning. Today, AI firms have revived the very vocabulary Searle warned against. If AI firms convince the public that AI possesses human traits, it is a short leap to pitching AI as a legal person having the same rights as humans. This is all sci-fi, you say? All 50 US states treat corporations as legal persons, able to carry out business, including rights to own property, sue, and enter into contracts. Corporations also enjoy constitutional rights. In 2010, in 'Citizens United vs Federal Election Com­mission', the US supreme court allowed corporations to spend freely on politics. This steady expansion of corporate rights may become a blueprint for AI personhood. Granting 'legal personhood' to AI will create what economists call a moral hazard, insulating AI progenitors from liability, thereby encouraging even more shortcuts on safety. AI is already on the verge of going rogue, but with legal person status, it could become an actual existential threat to humanity. The gravest threat may be what historian Jill Lepore calls the 'artificial state', where power shifts from elected government to machines, algorithms and the corporations that own them. In such a system, citizens delegate to polit­i­c­i­ans, politic­i­ans to bureaucr­ats, bur­eaucrats to contractors, and contractors to AI models — in short, a long delegation cha­in, where decisions are made by AI models nobody voted for. Even before AI, Pakistan offers a preview of this delegation problem. Most remedies for its 'broken' political system involve carving its provinces into sub-units. Thankfully, a few have also focused on improving the quality of democracy at the local level — making cities politically and financially independent. We should dread the day an AI firm is tasked with fixing Pakistan's political system. In the final analysis, AI needs close monitoring. AI regulation isn't just so-called wokeism. It is a survival imperative. With smart regulation, the path to the artificial state can be blocked. We must also question the idea of AI having a mind, emotions and biology before AI is granted personhood. Perhaps we should re-read Searle, who argued that a computer running a programme can never truly 'understand' anything, no matter how convincingly it appears to do so. The writercompleted his doctorate in economics on a Fulbright scholarship. aqdas.afzal@gmail.com X:@AqdasAfzal Published in Dawn, October 3rd, 2026