We Gave AI the Keys

Nine people, twelve questions

On 15 October 2026, nine people sat in a room in Lucerne and answered twelve questions by raising their hands. They had come to a small editorial session called “AI-First. But not blind.” Each of them works with AI in some way. That was the whole sample. No survey company, no industry barometer. Nine people in a room.

Six of them said they already let AI carry out tasks with several steps on its own. Six also said they could limit an AI agent’s rights technically, not just by telling it what not to do. We do not know whether those were the same six. The counts were taken over from the speaker as he called them out, and nobody verified them. And we checked none of those technical limits.

None of this needs hiding. It is where the right question starts. What does control mean when software no longer only answers, but acts?

The title of this volume came from the room. We Gave AI the Keys was confirmed out loud; there was no counted vote. Perhaps because the picture is so ordinary. We hand someone a key so that a job gets done. With an agent, we sometimes hand over the master key at the same time, and hope that a well-phrased instruction will be enough.

From a helpful text to an action

A chatbot that suggests a draft is one thing. An agent that changes files, sends messages, queries a database or prepares changes in a code repository is another. The first mostly produces information. The second can change the state of a system. A slip of language that would be embarrassing in a draft can have real consequences in a privileged workflow.

The uncertainty does not sit in the language model alone. An agent moves through interfaces, tools, tokens, APIs and human approvals. What it can actually do follows from that whole arrangement. A line in the system prompt, “You may not delete anything”, gives a different kind of assurance from an account whose permissions simply contain no delete operation.

OWASP calls this risk in LLM systems Excessive Agency. It arises, for example, when a tool has more functions than the task needs, when permissions reach too far, or when high-impact actions can run without an effective check.¹ OWASP’s newer list for agentic applications names tool misuse and identity and privilege abuse as separate risk categories.² Lists like these map kinds of risk. They do not measure how often real incidents happen.

A stranger’s text is not a new boss

Another break opens where agents read. An email, a web page or a file can hold data the task genuinely needs. It can also hold instructions the real user never gave.

Google DeepMind’s security team describes this as indirect prompt injection, a term Google’s own post credits to Greshake and colleagues: an attacker places manipulative instructions in outside content that the system processes during a legitimate task.³ Greshake and colleagues showed in 2023 that such attacks worked against real applications, and concluded that effective mitigations were lacking.⁴ Google’s published threat model pictures an agent asked to summarise an email, which the email’s own content could push into sending confidential information to an outside address. That example is a security model. It is no evidence that every agent can be compromised this way. Nor is the problem solved: in a later report, Google DeepMind found that adaptive attacks, which change as the defence changes, made baseline defences that had looked adequate against fixed tests much less effective.⁵

It does show where the boundary of trust runs. The text of an email may serve as evidence for its summary. It must not acquire the authority to send more data anywhere. A sturdy design decides outside the language model which operation is allowed, with which data and for which job. Technically limited identities, narrowly cut tools and a second check on critical actions therefore count for more than an assurance that the agent will be careful.

Who holds the key, and who answers for it?

It would be convenient to hand the responsibility to the agent. But handing the work to an agent does not obviously hand over the responsibility for it. Someone has to limit the job, grant the access, check the result and be able to stop it when in doubt.

NIST’s AI Risk Management Framework places risk management in four functions: Govern, Map, Measure and Manage.⁶ It prescribes no single technical design. It does keep the deciding questions in view. Who sets the rules? Which risks are plausible in this particular use? How do we tell whether the safeguards work? Who acts when something goes wrong?

A 2025 paper from Google on secure AI agents sets out three linked principles: well-defined human controllers, limited powers, and actions and planning that can be observed.⁷ What matters is the combination. Logging alone does not make a dangerous permission safe. Tight access alone does not show that the result is right. And a human approval that is only clicked through protects hardly better than none.

From this, a working model of control follows.

Least rights. A research agent normally needs to read. It does not need to overwrite its sources or change production systems. A writing agent does not automatically need the right to publish its own work.

Limits on every run. Each task needs a defined scope, a budget for time and cost, and rules for sensitive data. Rights should be short-lived and tied to the specific purpose.

Results that can be checked. “Task complete” is not acceptance. A workflow should say in advance what must be checked and which failures stop it.

Independent approval. Actions with serious consequences, such as publishing, paying, deleting or deploying to production, must not happen just because the same agent thinks they are right.

A real stop. A system must be able to interrupt actions technically, not merely ask the agent politely.

These five points are our own editorial synthesis from the security and risk-management literature. They quote no standard directly.

The strongest objection: too much control makes AI useless

There is a fair objection. Every extra approval slows the work down. Rights that are too narrow lead to constant questions, and an agent that needs a person at every step may automate very little. Ordinary software, too, runs on service accounts, queues and scheduled jobs without anyone signing off each action.

The objection is right about one thing: not every action deserves the same amount of control. An internal sketch is not a production deploy. A database query is not a database deletion. So the sensible answer is not to make agents unable to act. It is to grade permissions and checks by effect and risk. Low-risk tasks can run largely on their own, while the critical boundaries stay protected, technically and organisationally.

Even this model is not enough on its own. Rights can be misconfigured. Approval chains can be bypassed. Agents can combine tools in ways nobody expected. And a human approval can wear thin. Someone who is asked to confirm one routine action after another starts to confirm by habit, and people tend to trust a system that has been right many times before. The sign-off then becomes a rubber stamp: it is still in the diagram, but it no longer checks anything. That is one more reason to keep approvals for the few actions that really need them. Teams therefore have to test not only the allowed paths but, above all, the forbidden ones. Can the research agent really not write? Can an outside document really not talk it into handing over data? Does the stop work when the budget runs out? A safety rule whose enforcement nobody has tried is, for now, a claim.

Where we keep the responsibility

The show of hands in Lucerne measured nothing about the security of the AI industry. It did bring one question into the open that every team should ask before its agents take on more: when we give them keys, do we know exactly which doors those keys open?

Autonomous work and unchecked work are different things. Good agent systems earn their value not because nobody looks any more, but because people do less routine and still know who decides what is allowed and when to stop.

A prompt asks. A permission limits. A test shows whether the limit holds.

Method and limitations

The counts come from one live session in Lucerne, Switzerland, on 15 October 2026, with nine people present whose work, according to the event description, involves AI. It was not a random sample. Twelve questions were asked in turn and a raised hand counted as yes; anything else could mean no, undecided, not taking part or a hand nobody saw. The speaker counted the hands; the counts are taken over from him and were not independently verified. Answers cannot be linked across questions, so we do not know whether the six on one question are the six on another. All answers are self-reports; no technical control was tested. Nine voices in one room support no generalisation to Lucerne, to companies, to professions or to the AI industry. No photographs of participants were taken for this volume.

Most of the security sources here come from Google describing its own systems and research; OWASP, NIST and the academic study by Greshake and colleagues are the independent counterweight. NIST’s generative AI profile was used as background only.⁸ The security sources describe kinds of risk and ways of managing them. None of them gives a frequency of incidents or of damage, and this volume gives none either.

Sources