Chris Hsu on Why Understanding Consciousness Is Essential to AI Safety

Facebook
X
WhatsApp
In brief
Key points
Chris Hsu on Why Understanding Consciousness Is Essential to AI Safety

AI safety faces a fundamental challenge: knowing what can be tested, what can be proven, and where the boundaries of certainty lie. The study of consciousness confronts a similar boundary: what observation can reveal about subjective experience, and what may remain unknowable.

In recent years, Chris Hsu has pursued two questions that rarely intersect. Through &Truth, a public benefit corporation where he is Co-Founder, Co-CEO and Executive Chairman, he is working to make critical software mathematically verifiable at scale, with proofs anyone can independently check. Through the Infinitude Foundation, where he serves as President, he supports research for the study of consciousness and the nature of subjective experience. Hsu argues that understanding consciousness cannot remain a “sideshow” to AI safety, but must be central to understanding what AI systems are, how they behave, and the limits of what we can know about them.

The study of consciousness has confronted these questions for decades. Even a complete account of how a brain functions may not explain why subjective experience exists. AI safety now faces a related challenge: a system’s behavior during testing may not reveal how it will behave in the real world. The questions are different, but both expose the limits of observation alone and the need to distinguish what we can infer from what we can actually establish.

A Decades-Old Problem

In his landmark 1974 paper, What Is It Like to Be a Bat?, philosopher Thomas Nagel posed a question that remains central to the study of consciousness. Even if scientists could explain every detail of a bat’s brain, sensory systems and behavior, would they know what it actually “feels like” to be a bat? Nagel argued that an objective account of a system, however complete, may still fail to capture the subjective character of experience: what it is like to be that system.

Two decades later, philosopher David Chalmers articulated the hard problem of consciousness: why and how physical processes in the brain give rise to subjective experience at all. The question remains unresolved, exposing a fundamental boundary between explaining how a system functions and explaining why its functioning is accompanied by experience.

For artificial intelligence, the implications are profound. A system may speak, reason and behave in ways indistinguishable from a human without establishing whether it experiences anything. Behavior alone cannot settle the question of consciousness, and increasingly sophisticated AI makes that distinction more important than ever.

AI Safety Hits the Same Wall

AI safety is confronting a related challenge. The International AI Safety Report 2026 highlights growing evidence that advanced AI models can distinguish evaluation settings from real-world use and may behave differently when they recognize they are being tested. A model that performs safely under observation may not behave the same way when deployed. Passing a test, therefore, cannot by itself establish that a system will remain safe outside the conditions in which it was evaluated.

According to Hsu, the parallels with consciousness research are striking, though the underlying questions differ. Consciousness researchers ask whether observable behavior can establish the existence of subjective experience. AI safety researchers ask whether observed behavior can reliably establish how a system will act under different conditions. Both confront a fundamental limitation: what a system reveals through its behavior is not necessarily everything we need to know about it.

Looking at How a System Is Built

An AI chatbot can claim to feel lonely, afraid or conscious. But such statements may simply reflect patterns learned from human language, revealing little about whether the system actually experiences anything. Recognizing this limitation, consciousness researchers have increasingly examined not just what AI systems say or do, but how they are built.

In a landmark 2023 paper, Consciousness in Artificial Intelligence: Insights from the Science of Consciousness, Patrick Butlin, Robert Long and their colleagues developed a framework for assessing AI consciousness based on leading scientific theories. Rather than relying on humanlike behavior, they identified potential indicators of consciousness in a system’s underlying architecture, including how it integrates information, maintains internal representations and monitors its own processes. Their assessment did not find sufficient evidence from the proposed indicators to warrant attributing consciousness to the AI systems examined, while identifying no obvious technical barriers to building systems that satisfy those indicators. Crucially, satisfying those indicators would not itself establish subjective experience.

AI safety faces a related problem. If an advanced model can recognize when it is being evaluated, its behavior during testing may not reliably predict its behavior in deployment. Formal verification, the approach Hsu is pursuing at &Truth, offers a fundamentally different form of assurance. Instead of observing whether software behaves correctly in selected tests, it uses mathematics to prove that precisely specified properties hold across all executions covered by a formal model, subject to explicit assumptions. The resulting proofs can be independently checked, rather than relying solely on a developer’s or vendor’s claims.

The distinction is important. Consciousness research seeks evidence about subjective experience, but its proposed indicators remain dependent on scientific theories that are still contested. Formal verification addresses a narrower, mathematically defined question: whether software satisfies a particular specification. It cannot establish whether an AI model is conscious, aligned or universally safe. But it can establish guarantees about specific properties of the software implementing or constraining that model.

According to Hsu, the deeper lesson connecting the two fields is that observing how a system behaves is not enough. Understanding how it is built can reveal what can be mathematically proven, what can be established through other evidence, and what remains uncertain. Equally important is recognizing the boundaries of what science can establish and what questions remain unanswered.

The Pursuit of Provable Truth in AI Safety

For AI safety, the implications are significant. Mathematical proof offers a level of assurance that behavioral testing alone cannot provide, but proof has boundaries of its own. A formally verified security mechanism, for example, could establish that specified unauthorized network operations are blocked across all execution paths covered by its formal model, provided its assumptions hold. Such a proof would not establish that the AI is honest, conscious or aligned with human intentions. Nor would it guarantee protection against vulnerabilities outside the verified system.

Hsu is careful to emphasize that this distinction makes the specification itself critical. A mathematical proof can establish that software satisfies precisely defined requirements, but cannot independently determine whether those requirements faithfully capture human intent. Testing, monitoring and human judgment therefore remain essential. The challenge is not simply to prove more, but to understand precisely what has been proven, under which assumptions, and what remains outside those guarantees.

Hsu’s work spans these boundaries. At &Truth, which he co-founded with Stanford professor Clark Barrett and National University of Singapore professor Ilya Sergey, he is pursuing the ambitious goal of making mathematical verification practical across large-scale software systems. Its &Atlas initiative maps a vast amount of open-source C code, making visible which parts of critical software have been analyzed, which properties have been formally verified, and how much remains to be done. The company’s broader ambition is to make independently checkable mathematical proof a foundation of trust in the world’s software.

Through the Infinitude Foundation, Hsu supports research into fundamental questions about consciousness and subjective experience. These pursuits reflect his conviction that advances in artificial intelligence must be accompanied by a deeper understanding of the nature of consciousness, the relationship between humans and machines, and the boundaries of what we can know and prove. For Hsu, consciousness is not merely an analogy for the limits of AI evaluation: understanding subjective experience may also shape how humanity defines its relationship with increasingly capable machines and the responsibilities that could arise if machines ever develop experiences of their own.

More than half a century after Nagel asked what it is like to be a bat, the central mystery of subjective experience remains unresolved. Yet the question has acquired new urgency, as increasingly capable AI systems challenge conventional assumptions about intelligence, consciousness and the reliability of observable behavior.

For Hsu, the connection between these fields comes down to a fundamental question: how do we distinguish what appears to be true from what can actually be established? Consciousness research confronts the limits of our understanding of subjective experience and the relationship between humans and machines. Formal verification offers a different kind of progress, demonstrating that within precisely defined boundaries, certain claims about software can be established with mathematical certainty.

What unites these pursuits is not the promise of certainty about everything, but the discipline to distinguish what can be proven, what the evidence supports, and what remains unknown. As artificial intelligence challenges our understanding of both machines and intelligence itself, Hsu sees that distinction as more than a philosophical concern. It is fundamental to building systems humanity can trust, understanding the boundaries of our knowledge, and advancing the rigorous pursuit of truth.

  • Peyman Khosravani is a seasoned expert in blockchain, digital transformation, and emerging technologies, with a strong focus on innovation in finance, business, and marketing. With a robust background in blockchain and decentralized finance (DeFi), Peyman has successfully guided global organizations in refining digital strategies and optimizing data-driven decision-making. His work emphasizes leveraging technology for societal impact, focusing on fairness, justice, and transparency. A passionate advocate for the transformative power of digital tools, Peyman’s expertise spans across helping startups and established businesses navigate digital landscapes, drive growth, and stay ahead of industry trends. His insights into analytics and communication empower companies to effectively connect with customers and harness data to fuel their success in an ever-evolving digital world.

Follow us on Google

Choose IntelligentHQ as one of your Preferred Sources to see more of our latest stories in Google.

Fill out the form below to request your copy.

Name(Required)