Skip to main content

Human Sovereignty in the Age of Machine Intelligence

The singularity may not have arrived. The sovereignty question already has.

Human judgement desk at dusk: open notebook and pencil beside a dark closed laptop, a brass balance scale, and a wall clock, with distant data-centre lights beyond a tall window
  • Human sovereignty is authorship: determining purposes, governing formative systems, preserving alternatives, and retaining the practical ability to refuse, intervene, and change direction.
  • Agency without sovereignty is fragile; sovereignty without agency is empty — freedom requires both intention and the practical means to exercise it under dependency and pressure.
  • Power does not require consciousness; the sovereignty problem begins when consequential systems can no longer be understood, refused, replaced, or governed.
  • Formal human-in-the-loop oversight becomes ceremonial without competence, authority, response time, and real alternatives.
  • Concentration, atrophy, acceleration, and objective capture are the primary operational threat modes — and they reinforce one another.
  • Cognitive infrastructure must remain plural, governable, reversible, and accountable; a dependency that cannot be reversed is a transfer of power.
What is human sovereignty?

The collective capacity of people to determine their purposes, understand and govern the systems that shape their lives, preserve meaningful alternatives, and retain the practical ability to refuse, intervene, and change direction.

How does agency differ from sovereignty?

Agency is the capacity to form purposes and act upon them. Sovereignty is the condition in which that agency remains practically exercisable under dependency, pressure, and technological change.

Why does AI change the conditions of agency?

It concentrates capability into knowledge interfaces, accelerates action beyond ordinary institutional tempo, can make competence optional while still producing acceptable outputs, and can search objective spaces faster than the political processes that set those objectives.

Why isn’t keeping a human in the loop enough?

Formal responsibility can outlive effective control. Without time, information, competence, or authority to disagree, oversight becomes ceremonial.

What is the threat model for human sovereignty?

Four operational failure modes already visible in institutions: concentration, atrophy, acceleration, and objective capture. They compound rather than appear in isolation.

Why is plurality a safety mechanism?

A monoculture loses the independent perspective required to recognise failure. Meaningful alternatives — technical, institutional, and cultural — preserve the capacity to leave and to change direction.

Has the technological singularity arrived?

Not in the strong recursive self-improvement sense. Current systems accelerate AI research under human infrastructure, objectives, and evaluation. Whether the label fits is less important than whether humanity remains sovereign if it does.

Throughout history, sovereignty has usually been discussed as a property of states.

Borders. Laws. Territory. Currency. Armies. Infrastructure.

Yet every one of these concepts assumes something deeper: that human beings remain capable of deciding their own future.

We have spent centuries asking how sovereign states should be governed. We are only beginning to ask what it means for a species to remain sovereign.

Machine intelligence now writes software, conducts research, uses tools, analyses evidence, generates designs, operates computers, and completes increasingly complex sequences of work. These are real capabilities. Dismissing them as sophisticated autocomplete no longer explains what the systems can do.

But this essay is not an argument against building them. Nor is it a defence of human superiority in every task.

The lasting question is different.

What capabilities, institutions, choices, and practical alternatives must humanity preserve in order to remain the author of its own future?

This Library’s working definition is operational. Human sovereignty is the collective capacity of people to determine their purposes, understand and govern the systems that shape their lives, preserve meaningful alternatives, and retain the practical ability to refuse, intervene, and change direction.

A sovereign species is not a species that controls every event. It is one that remains capable of authorship.


Agency and sovereignty

Agency is the capacity to form purposes and act upon them. Freedom without agency is ornamental: rights exist on paper while people cannot author meaningful action. Sovereignty is the condition in which that agency remains practically exercisable under dependency, pressure, and technological change.

Agency without sovereignty is fragile: a person or institution may still form intentions while lacking the practical means to carry them out. Sovereignty without agency is empty: formal rights remain while authorship disappears.

Sovereignty protects agency through competence, alternatives, time to respond, authority, accountability, reversibility, and institutional capacity. As digital sovereignty is not nationalism argues, sovereignty is not isolation from interdependence, but the ability to govern it. The same logic now applies at a deeper level.

Human sovereignty is not the absence of machine intelligence. It is not a prohibition against building systems more capable than individual humans. It is the preservation of collective authorship under conditions of accelerating capability.


Why AI changes the conditions of agency

Earlier Library volumes asked how organisations retain ownership of systems, how incentives quietly replace intentions, and how preparedness preserves the capacity to act. Artificial intelligence intensifies those conditions.

It concentrates capability into interfaces that mediate knowledge and work. It accelerates action beyond ordinary institutional tempo. It can make competence optional while still producing acceptable outputs. And it can search objective spaces faster than the political processes that set those objectives.

AI does not invent the sovereignty problem. It changes the speed, intimacy, and scale at which agency can be lost.


Intelligence is not one capability

Human intelligence is not a single benchmark. It includes learning from limited experience, transferring understanding between unfamiliar situations, interacting with a resistant physical world, forming relationships, interpreting social meaning, recognising suffering, revising personal goals, and accepting responsibility for consequences.

It is shaped by embodiment, mortality, memory, vulnerability, and the need to live among other beings with equally legitimate interests. It includes emotional intelligence — not merely the ability to identify linguistic patterns associated with emotion, but the lived consequences of attachment, fear, grief, loyalty, shame, hope, love, and care.

And human beings possess personal intent. We do not merely calculate routes toward externally supplied objectives. We form objectives, contest them, abandon them, regret them, and ask whether they should have been pursued at all.

Current AI systems can represent many of these processes. They can simulate them convincingly and often help humans reason about them. That does not demonstrate that the systems experience them — nor a stable personal identity, autonomous purpose, moral concern, or continuous inner life.

A machine can produce an excellent explanation of grief without grieving. It can optimise a medical system without caring whether a patient lives. It can recommend peace without fearing war.

This does not make machine intelligence unreal. It makes it different.

The mistake is to force human and machine capability onto a single line, as though intelligence were one quantity and the only question were which participant possessed more of it. The more useful question is: what kind of intelligence is present, what authority has been connected to it, and which human capacities become dependent on it?


Consciousness is not the threshold for power

Discussions of advanced AI often turn toward consciousness. Can the machine feel? Does it understand? Does it possess an inner world? Could it suffer?

These questions are philosophically important and may become morally urgent. But consciousness is not the threshold at which AI becomes a sovereignty problem.

A system does not need an inner life to exercise power. An algorithm can determine which information millions of people see without knowing that those people exist. A recommender can reshape political attention without holding a political belief. A financial model can influence access to housing, credit, employment, or insurance without understanding poverty. An autonomous weapons system can compress the time available for human judgement without experiencing aggression. An AI assistant can become cognitively indispensable without possessing a self. An optimisation process can transform an institution without wanting anything.

Intent is not required for power.

This is already familiar. Markets do not have consciousness, yet they shape societies. Bureaucracies do not have a unified inner life, yet they exercise authority. Supply chains do not form intentions, yet their failure can destabilise countries. Technology acquires power through its relationship to institutions, resources, incentives, and human dependence. AI is no exception.

The sovereignty question therefore does not begin when a model wakes up. It begins when humans can no longer meaningfully understand, refuse, replace, or govern the systems through which consequential decisions are made.


Why the singularity is not the decision point

Sam Altman has argued, in The Gentle Singularity and later public remarks, that humanity is living through a technological singularity — that we are “past the event horizon” and that the takeoff toward digital superintelligence has begun. It is an extraordinary claim. It should neither be dismissed nor allowed to set the agenda alone.

Classically, the singularity describes a hypothetical threshold beyond which technological development becomes self-accelerating, exceeds reliable human comprehension, and can no longer be meaningfully predicted or controlled. In its strongest form, an artificial intelligence improves the process by which it improves itself, compressing years of human research into increasingly short cycles.

That is not quite what exists today.

Current systems already contribute materially to the next generation of AI. In The Gentle Singularity, Altman himself described AI-assisted AI research as a “larval version” of recursive self-improvement — not full autonomous self-modification. Anthropic’s Institute essay When AI builds itself reports that, as of May 2026, more than 80% of the code merged into Anthropic’s production codebase was authored by Claude, up from low single digits before Claude Code, while also distinguishing futures in which humans still set research direction from full recursive self-improvement in which systems design their own successors.

Present systems remain dependent on human-built infrastructure, human-defined objectives, human-generated evaluation criteria, and human control over training, deployment, and access to compute. Research continues to distinguish bounded self-refinement — already common — from open-ended recursive self-improvement constrained by compute, evaluation, grounding, direction-setting, and the risk of models reinforcing their own errors.

Altman’s formulation is therefore better understood as a claim about direction and historical momentum than as a settled technical diagnosis. The distinction matters. But it may not be the distinction that matters most.

Whether the singularity has technically arrived is less important than whether humanity is preserving the capabilities required to remain sovereign if it does. Humanity does not need to wait for a machine to announce consciousness, for a laboratory to confirm recursive self-improvement, or for one system to outperform every human in every domain.

The sovereignty problem is already visible.


Formal control is not effective control

It is tempting to believe that sovereignty can be preserved simply by keeping a human in the loop — a signature, an approval, a confirmation step. The human remains formally responsible.

But formal responsibility can survive long after effective control has disappeared.

Consider a doctor reviewing an AI-generated diagnosis under severe time pressure. If the system has processed more evidence than the doctor could independently examine, if its internal reasoning cannot be inspected, if organisational measures reward agreement, and if rejection requires extensive justification, the doctor may remain legally responsible while possessing little practical authority.

The signature is human. The decision structure is not.

The same pattern appears across organisations: boards approving AI strategies they cannot technically evaluate; public authorities regulating platforms on which their own services depend; militaries retaining nominal approval while automated systems compress decision time to seconds; workers told that an algorithm is “only a recommendation” while contradiction damages their performance score; citizens who technically consent to automated processing but have no realistic alternative to participation.

These arrangements create ceremonial control. The human remains visible because their presence provides legitimacy and absorbs liability. The real decision has migrated elsewhere.

Operational sovereignty requires competence, time, information, authority, and the ability to select a different course. Control without execution is ceremonial sovereignty.

Human oversight without credible disagreement is ceremonial humanity.


The authority to determine ends

AI is exceptionally useful in discovering means. It can compare routes, identify patterns, reduce uncertainty, test assumptions, simulate consequences, and expose options that humans might miss. This capacity can expand human freedom enormously.

But choosing an efficient route is not the same as deciding where civilisation should go.

The most consequential human questions are not optimisation problems with universally agreed objectives: what a good life is; which risks are acceptable; what justice requires; what must never be traded for efficiency; how prosperity should be distributed; what obligations present generations owe the future; when a society should forgive; what must remain private; when technological development should slow; which forms of suffering are intolerable even if prevention is costly.

These questions involve competing values, historical experience, cultural meaning, political legitimacy, and moral responsibility. AI can inform the debates, expose contradictions, and model consequences more accurately than humans. It may sometimes produce recommendations wiser than those of individual decision-makers.

But predictive superiority does not create moral authority. The ability to determine what is likely to happen is not the same as the right to determine what should happen.

OpenAI’s 2026 plan states that powerful AI should help people pursue their goals rather than replace human judgement about what matters, and that the long-term human role includes setting direction, making trade-offs, applying judgement, and deciding what is worth doing. That principle should not remain a product philosophy. It should become a civilisational boundary.

Machines may increasingly optimise the means. Humanity must remain capable of contesting the ends.


Human competence is not a permanent resource

Competence is not a stock that remains because a society once possessed it. It decays when unused, when institutions stop teaching it, and when tools make independent judgement optional.

The Shinkansen principle showed that trust accumulates through systems that behave reliably enough to become ordinary. The reverse is also true: when ordinary work no longer requires understanding, the capacity to challenge the system drains away.

Human sovereignty therefore requires deliberate preservation of reasoning, writing, diagnosis, craft, technical understanding, and institutional memory — not as nostalgia for pre-AI labour, but as the conditions under which disagreement remains possible.


Cognitive infrastructure

Cloud platforms became essential infrastructure because organisations moved computing, data, communications, and operations into them. AI is becoming something more intimate: cognitive infrastructure.

It increasingly mediates what people learn, which sources they encounter, how questions are formulated, which possibilities are considered, how software is written, how research is conducted, how organisations communicate, how evidence is interpreted, how decisions are prepared, how personal memory is organised, and how institutions understand the world.

A system that mediates access to knowledge and participates in reasoning occupies a position closer to education, language, media, administration, and scientific infrastructure combined.

In The Gentle Singularity (June 2025), Sam Altman described the industry as building something like a brain for the world, and warned that superintelligence should not become excessively concentrated in any person, company, or country. The metaphor should be taken seriously. A brain for the world would be among the most consequential infrastructures ever created — raising questions of training, permitted behaviour, memory, language and legal tradition, access, inspection, modification, energy and compute, withdrawal of service, knowledge visibility, and accountability when the system is wrong.

No single organisation — however intelligent, responsible, or well-intentioned — should become an irreplaceable intermediary between humanity and its own accumulated understanding.


A threat model for human sovereignty

The principal threats are not speculative consciousness scenarios. They are operational failure modes already visible in institutions.

Concentration

Too few organisations control the models, compute, interfaces, knowledge mediation, and infrastructure on which society depends. When a single provider becomes the default interface to knowledge, education, administration, or research, convenience becomes capture. Exit becomes theoretical. That is continuous with the Library’s argument that sovereignty is not self-sufficiency: interdependence can be governed, but only when alternatives remain real.

Atrophy

Humans and institutions lose the competence needed to inspect, challenge, reproduce, or replace machine-mediated work. Capability rises in the short term while independent competence declines. The system works — until access is lost, objectives diverge, or assumptions must be challenged from outside the tool.

Acceleration

Machine action and institutional propagation become faster than meaningful human detection, deliberation, and intervention. Formal authority remains human while operational sovereignty becomes retrospective: the institution manages consequences rather than governing events. As sovereignty is measured in response time and preparedness is the highest form of sovereignty argue, response time is the practical measure of control under pressure.

Objective capture

Measurable proxies, commercial incentives, or optimisation targets silently replace politically and morally contestable human purposes. The system satisfies the measure while violating the intention.

These four modes reinforce one another. Concentration accelerates atrophy. Atrophy slows response. Acceleration rewards crude metrics. Objective capture justifies further concentration. A sovereignty doctrine must address the compound, not only the parts.


Plurality, response time, and the right to leave

Plurality is often treated as innovation policy: more providers, more models, more consumer choice. It is also a sovereignty mechanism.

A monoculture creates systemic fragility. If one model architecture, training paradigm, infrastructure provider, or institutional worldview becomes dominant, errors can propagate across every dependent system. The danger is not only failure. It is loss of the independent perspective required to recognise failure.

Different systems preserve different assumptions. Different languages preserve different categories of thought. Different cultures preserve different ideas about dignity, authority, nature, family, duty, freedom, and progress. Different institutions create different accountability structures. Plurality prevents one optimisation function from quietly becoming civilisation’s default.

This does not mean every model must be sovereign, local, or open source. It means humanity must preserve a sufficiently diverse cognitive ecosystem that no single technical or institutional failure can eliminate meaningful alternatives: frontier systems, public-interest systems, open models, specialised models, national and regional capabilities, universities capable of independent research, institutions that can evaluate models without depending entirely on developer claims, and people who can still reason without asking the machine first.

In critical systems, redundancy is survival. But redundancy without tempo is theatre. Alternatives that cannot be activated before escalation are not exits. Preparedness is the capacity held before pressure arrives.


The optimisation trap at civilisational scale

The optimisation trap begins in products and organisations: systems optimise the objectives they receive, not the intentions people imagined. At civilisational scale the same pattern appears when measurable proxies, commercial incentives, or institutional metrics silently replace contested human purposes.

Capable systems search the available solution space with greater speed, persistence, and scale than the institutions that set their objectives. When the measure becomes the purpose, agency does not disappear in a dramatic transfer of power. It drains through successful optimisation.


What humanity should not do

Human sovereignty cannot be protected by one regulation or technical standard. Several civilisational errors should nonetheless be avoided.

We should not confuse performance with personhood. Capability must be recognised without inventing qualities that have not been demonstrated.

We should not wait for consciousness before governing power. The correct threshold is consequential capability, not metaphysical certainty.

We should not centralise cognitive infrastructure without credible exits. Convenience is not an adequate substitute for resilience.

We should not allow human oversight to become ceremonial. Oversight must include the capacity for independent judgement.

We should not delegate the definition of progress. Machines may model possible futures; they must not quietly determine which future deserves to exist.

We should not treat abundance as sovereignty. Cheap intelligence may increase productivity while concentrating control. A capability supplied through an ungovernable dependency is not sovereign merely because it is abundant — a point continuous with the Library’s argument that money follows productive capacity, not theatre.

We should not allow human competence to become optional. Education must not become training in how to accept machine outputs. Professionals must retain enough understanding to detect failure, challenge assumptions, and operate when systems are unavailable.

We should not mistake adaptation for consent. Humans can adapt to almost anything — surveillance, dependency, degraded work, automated judgement, loss of agency. Functioning inside a system does not mean people chose it or retain the power to change it.


A doctrine of human sovereignty

If artificial intelligence is becoming cognitive infrastructure, humanity needs a sovereignty doctrine proportionate to the transition.

Purpose must remain contestable

No technical system should close the political, moral, or cultural debate about what society is trying to achieve. Objectives affecting human life must remain open to legitimate disagreement and revision. Without contestable ends, optimisation becomes rule.

Critical delegation must remain reversible

Essential AI dependencies require tested alternatives, transferable data, interoperable interfaces, operational fallbacks, and retained human knowledge. A dependency that cannot be reversed must be treated as a transfer of power.

Human competence must be deliberately preserved

Schools, universities, professions, and institutions must preserve reasoning, writing, diagnosis, judgement, memory, and technical understanding. The objective is not to prevent tool use. It is to ensure that tools expand human capability rather than replace its foundations.

Cognitive power must remain plural

Humanity needs multiple models, providers, infrastructures, research traditions, governance systems, and cultural perspectives. Plurality is not duplication. It is protection against epistemic capture and systemic failure.

Decision authority must follow accountability

Entities that cannot bear responsibility should not possess final authority over irreversible decisions. Humans and institutions must remain identifiable, accountable, and capable of intervention.

Intervention must remain faster than escalation

Critical systems must be observable, interruptible, and governable at the speed at which they operate. A shutdown mechanism that can only be activated after catastrophe is not a control mechanism.

Rights must survive non-participation

People must not lose access to essential social functions simply because they refuse a particular model, provider, identity system, or form of automated judgement. Consent requires an alternative.

Benefits must not require permanent dependence

AI should increase the productive and intellectual capacity of individuals, organisations, and societies. It should not make them structurally incapable of functioning without continued permission from a small number of infrastructure owners.

Humanity must retain the right to change its mind

No deployment, treaty, platform, model, or infrastructure decision should be allowed to make future political choice impossible. The capacity to reverse direction is the deepest form of sovereignty.


The productive capacity of humanity

There is a danger that the defence of human sovereignty becomes defensive in the wrong sense.

The objective is not to preserve human weakness, reject intelligence greater than our own, or romanticise error because it is human.

Artificial intelligence could dramatically expand scientific discovery, medical care, education, engineering, public administration, environmental restoration, and economic productivity. It could help humanity solve problems that have resisted generations of effort. That positive case should be pursued without apology.

But productive capacity is never merely the possession of a tool. It includes the ability to understand, maintain, govern, distribute, and improve that tool. Even the most advanced AI will depend on electricity, semiconductors, networks, institutions, engineering, security, law, education, and human cooperation.

The real sovereign asset is not the model alone. It is the society capable of building, operating, questioning, and replacing it.

A mature civilisation is not measured by the intelligence it can summon, but by the judgement it retains while using that intelligence.

Productive capacity is ultimately human and institutional: the ability of millions of people to solve problems together — to invent, build, repair, educate, organise, and create. AI can multiply that capacity. It must not become a substitute for retaining it.

Humanity should build and use powerful intelligence, but must not surrender the capabilities required to govern its purpose, distribution, dependencies, and consequences.

The future challenge will not be a shortage of intelligence. It will be the preservation of judgement in systems capable of relentless optimisation.


The sovereignty question already has

Perhaps the singularity has begun in the gentle sense described by Altman. Perhaps later historians will conclude that the decisive threshold was crossed before most people noticed. Perhaps current systems remain far from autonomous superintelligence. We cannot know with confidence.

Uncertainty about the label does not justify uncertainty about the responsibility.

The sovereignty problem is already visible when a worker cannot challenge an automated decision; when a government depends on infrastructure it cannot govern; when a school replaces learning with output generation; when an institution signs decisions it no longer understands; when one provider becomes the default interface to knowledge; when human expertise is retained as theatre rather than operational capacity; when the speed of machine action exceeds the speed of human response; when society can no longer imagine withdrawing from systems it adopted only a few years earlier.

These are not speculative future risks. They are early signs of a transfer of agency.

The correct response is not panic. Nor is it technological denial. It is architecture, governance, education, plurality, preparedness, accountability — and the preservation of meaningful alternatives.

The decisive question is not whether artificial intelligence has become human.

It is whether humans will remain capable of deciding what intelligence is for.

A sovereign species is not one that refuses to build intelligence greater than its own.

It is one that remains able to understand its dependencies, determine its purposes, intervene before control is lost, and change direction when the future it is building no longer serves life.

The singularity may not have arrived.

But the sovereignty question already has.

And humanity should answer it while the answer is still ours to give.