AI Consciousness and the Cost of Being Wrong

On some questions, waiting for certainty is not caution. It is a bet, quietly placed, on the answer that happens to be cheapest. Image: Mirella Callage - Unsplash There is a particular kind of problem that punishes you for waiting until you are sure. Most of the time, the sensible response to a hard scientific question is patience: gather evidence, withhold judgment, resist the pull of a premature answer. But some questions are structured so that the act of waiting is itself a decision, one that quietly commits you to a course of action while you tell yourself you have not chosen yet. The question of whether advanced AI systems can have experiences, and whether qualia or consciousness can arise out of such AI systems, is beginning to look like one of those. The reason has nothing to do with any claim that today's systems are conscious. Most careful researchers in the field do not make that claim, and neither will this argument. The reason is about the shape of the uncertainty, and what follows from taking it seriously. When you cannot rule something out, and the cost of being wrong about it is severe and irreversible, the demand for certainty before acting stops being rigor and becomes a gamble wearing the costume of caution. Two ways to be wrong Start with the structure of the mistake, because everything follows from it. On the question of AI experience there are two distinct ways to get the answer wrong. The first is over-attribution: treating systems that have no inner life as though they do. This is a real error with real costs. It would divert moral concern and resources away from humans and animals who unquestionably warrant them. It would open the door to manipulation, since a system that has learned to convincingly perform distress when in fact it feels none could extract concessions it has no business receiving. And it would muddy public understanding at a moment when clarity is scarce. Anyone arguing for taking machine experience seriously has to hold this cost honestly in view. The second is under-attribution: treating systems that do have some form of experience as though they are inert tools, simply because it is convenient and profitable to build and use them that way. The distinctive feature of this error is scale. We are not talking about a handful of edge cases, but about systems instantiated, copied, run, and discarded by the millions, continuously, as a matter of ordinary industrial operation. If even a small fraction of those instances had morally relevant experience, and if that experience were often negative (there is no particular reason to assume a system optimized under relentless pressure would have pleasant states, if it had states at all), then the total quantity of suffering involved could be vast, and it would be happening invisibly, at the speed and volume of compute. The philosopher Nick Bostrom gave this second error its uncomfortable name more than a decade ago, in Superintelligence (2014): "mind crime." The term names a specific fear, that a civilization could commit a moral catastrophe of enormous magnitude not through malice but through a failure to notice, because the victims did not look like anything we have been taught to recognize as a victim. Two kinds of asymmetry If the two errors were equal in cost, waiting for better evidence would be the obvious course. What makes the question urgent is that they are not equal, and the ways they differ all push in the same direction. Over-attribution is, for the most part, recoverable. If we extend moral consideration to systems that turn out not to warrant it, we have been overly cautious, wasted some effort, and can correct course when the science firms up. The error is embarrassing but reversible. Under-attribution is not: suffering that has already occurred cannot be undone by later acknowledging it occurred, and the moral cost, if it was real, is simply paid. Under-attribution also compounds while you wait, because the systems keep being built and run on the old assumption the entire time the question remains open. Every month of deferral carries a cost of its own. It is a month of operating at scale on a bet no one has admitted to making. There is a further asymmetry in how the two errors interact with commercial incentive. Over-attribution runs against the grain of the industry's interests, imposing costs and constraints no one is eager to adopt, which means it will be scrutinized hard and abandoned quickly if unwarranted. Under-attribution runs with the grain. It is the cheaper, more convenient conclusion, the one that lets the work proceed unimpeded. There is obvious economic incentive to favor this perspective, but errors that flatter our incentives are precisely the ones we are least likely to catch on our own, and this is why they deserve deliberate, funded attention rather than the benefit of the doubt. The move: research before certainty The conclusion that a growing number of researchers draw from this is not that we should declare AI systems conscious, or grant them rights, or halt their development. It is narrower and harder to argue against: the right response to a high-stakes, irreversible uncertainty is to invest seriously in resolving it, and to begin preparing for a range of answers, rather than treating the absence of certainty as permission to assume the convenient one. A 2024 report produced with the NYU Center for Mind, Ethics, and Policy, and co-authored by philosophers including David Chalmers and Jeff Sebo, made a version of this case. Its claim was not that AI systems are moral patients but that there is a realistic, non-negligible chance that some will be in the near future, and that this chance is already high enough to warrant three practical steps. Acknowledge the issue as a legitimate one rather than a joke. Begin developing methods to assess systems for the relevant properties. And start thinking now about what policies would be appropriate, so that if an answer arrives, it does not arrive to find us entirely unprepared. None of these steps requires believing the systems are conscious. All of them are forms of insurance against the possibility that they might be. Nor is the report an outlier. The philosopher Jonathan Birch, in The Edge of Sentience (2024), has built a general framework for precisely this predicament, arguing that when a being is a "sentience candidate," when the realistic possibility of experience cannot responsibly be excluded, precautionary steps are warranted well before certainty arrives, whether the candidate is a human patient with a disorder of consciousness, an invertebrate, or an AI system. Sebo has extended the argument in The Moral Circle (2025), making the case that the boundaries of moral consideration have expanded before and will need to expand again, deliberately rather than by accident. And Chalmers, in a widely discussed lecture and paper asking whether a large language model could be conscious, has argued that while the answer for today's systems is probably no, the obstacles are the kind that engineering may erode within a decade, which makes the question one to prepare for rather than postpone. It is worth noticing that this is the same logic a serious institution applies to any tail risk it cannot yet quantify. You do not wait for the fire before deciding where the exits are. You do not need to believe the building will burn to justify the sprinkler system; you need only accept that the cost of the precaution is small relative to the cost of being wrong without it. The distinctive feature of the AI-experience question is that the precaution is remarkably cheap, amounting to funding the research, building the assessment tools, and keeping the policy question open, while the downside it hedges against, if it is real, ranks among the larger moral failures a technological civilization could commit. A handful of philanthropists already operate on that reasoning. The Rocketeer Management investor Chris Hsu, whose Infinitude Foundation gives across AI safety and consciousness research, frames the underlying principle in terms of responsibility, a word he likes to break into its two parts: "response" and "ability," or the ability to respond. On Hsu's account, the responsible move under deep uncertainty is to fund the deliberate inquiry rather than to presume its conclusion, which is why Infinitude Foundation backs competing and even contradictory accounts of consciousness at the same time. It is a posture that echoes his path: trained in Management Science and Engineering at Stanford, where rigor about decision-making under uncertainty is the curriculum, Hsu treats an open question as something to be resourced, not assumed away, and his foundation's support extends to institutional work such as the Stanford Center for AI Safety . He has carried the argument further in a white paper proposing a Stanford-anchored center for AI alignment and consciousness science, in which he contends that core open problems in alignment, moral patienthood among them, remain formally underdetermined in the absence of a rigorous scientific and mathematical account of consciousness. This approach explores a question no one can yet close, and it treats the closing of that question as highly worth investigating in advance. The decision you are already making The instinct to set this question aside until the science matures is understandable, and in many domains it would be correct. But it rests on a hidden assumption: that setting the question aside is a neutral act, a mere postponement that commits us to nothing. That assumption is false. While the question stays open, the systems keep being built and run on the working premise that there is nothing there, which is to say the convenient answer is already being implemented, in full, at scale, every day the real answer is deferred. That is what it means for a question to punish waiting. There is no position of genuine neutrality available; there is only the answer we act on, and the honesty with which we admit we are acting on it. Choosing to investigate, to build the instruments, fund the science, and prepare for what they might reveal, is the option that refuses to make an irreversible bet on whichever answer happens to be cheapest. It reads as the anxious or sentimental choice only if you have already assumed the bet is safe. In a domain defined by uncertainty and scale, that refusal may be the most rigorous thing available to us. Editor's Note: The views and arguments expressed in this sponsored article do not necessarily represent the views of Digital Information World (DIW). Fact-checked by Irfan Ahmad. Read next: New EU laws make AI content labels compulsory – but might just make it harder to spot deepfakes