Andy Garcia · freelance dev
FRENES
Back to the blog
AI & Insurance

Asked Under Oath Whether They Would Be Liable When AI Goes Rogue, OpenAI, Anthropic and Google Did Not Say Yes. Their Contracts Already Answer.

October 7, 2026 6 min read

On Monday, October 5, the New York City Council sat as a Committee of the Whole and questioned OpenAI, Anthropic, Google and Meta under oath. Speaker Julie Menin asked whether the companies would bear legal responsibility if an AI system went rogue and caused major financial damage, exposed sensitive information, injured someone or contributed to a death.

Nobody said yes. According to amNY, OpenAI's Morgan Dwyer said OpenAI was responsible for developing and evaluating its systems safely, without answering directly. Anthropic's Logan Graham said the question was outside his expertise as a technical researcher. Meta's Shane Cahill did not want to speculate. Google's Alice Friend said existing law applies: "if it's illegal without AI, it's still illegal with AI."

I deploy agents, in a LegalTech I build on Claude and in side projects, so I had a narrower version of Menin's question. If my agent does something expensive, who pays?

I spent the evening with my contracts. The short answer is me.

The insurers are asking the same question

Aon, the broker, built a database of roughly 300 AI-related lawsuits going back to 2010 and mapped them against the policy lines that might respond. In its AI fact sheet, published in August, about 15 percent of AI claims fit a standalone cyber policy and about 80 percent fit errors and omissions, professional or media liability. Fewer than 10 percent of organizations buy E&O at all. Around 20 percent touch other lines: directors and officers for "AI washing," employment practices for discrimination, crime for deepfake funds transfers. Aon estimates that more than 90 percent of whatever cover exists for AI is "silent": the policy neither grants it nor excludes it.

Exclusions are arriving anyway: Aon lists optional ISO endorsements removing generative AI from general liability, a carrier whose D&O policies exclude the development, deployment or use of AI, and Berkley's "absolute AI exclusion."

On August 28, Reuters reported that MSIG, QBE and Beazley were reviewing their cyber wordings because of agents. The hard case, as Armilla's CEO Karthik Ramakrishnan told Reuters, is "where there is no conventional attacker and potentially no unauthorized credential use." An agent using access you deliberately gave it does not look like a hack. Some executives told Reuters that when an agent acting as designed makes a costly decision, insurers may classify it as a non-cyber event.

On October 1, Insurance Business UK spelled out where that leaves a company. An agent that causes a data breach may trigger the cyber policy. One that makes an unauthorised purchase or agrees terms with a customer can fall between covers: crime cover typically needs a dishonest person or third-party deception, and professional indemnity answers third-party claims, not your own loss. To the client, as Clear Group's George Grimshaw put it, the business deploying the agent will usually be first in line.

What the three contracts say

Anthropic's Commercial Terms, OpenAI's Services Agreement and Google Cloud's terms use different words and draw the same line.

Each provider indemnifies you for intellectual property. Anthropic defends claims that your paid, compliant use, its training data or its outputs infringe someone's IP. OpenAI covers claims that its services infringe. Google covers its training data and unmodified generated output from the services it lists as indemnified. That risk sits with the model, and the providers know its size: in July, a federal judge gave final approval to Anthropic's $1.5 billion settlement with authors and publishers, about $3,000 per work.

Everything else has a ceiling. Anthropic and OpenAI cap their liability at what you paid them in the previous 12 months, and both exclude lost profits and consequential damages. Google's cap is also 12 months of fees, and $5,000 for services it provided free. Beta is thinner: Anthropic offers no indemnity and limits its liability to the lesser of $1,000 and your 12 months of fees; OpenAI states it will have "no liability arising out of or in connection with beta services."

Then come the clauses about you. Anthropic: the customer "is responsible for all activity under its account," and must evaluate outputs, "including where human review is appropriate." OpenAI: the customer is responsible for all activities under its account, including users of the customer's application. Both carve out of the IP indemnity claims arising from combining their service with technology they did not provide. An agent is exactly that: a model wired to your tools.

Google is the most explicit, in its Agentic AI Services section. The customer is "solely responsible" for "the actions and tasks performed by an Agentic AI Service or AI Agent," for authorizing its access to data and systems, and for "exercising judgment and supervision" in production. One line closes the loop: "The actions or tasks that an AI Agent performs are not Generated Output." The output indemnity does not follow your agent into what it does.

The chain, in order

Put together: your agent causes a loss, and your client comes to you. You can claim against the model provider up to about a year of your bill, unless the claim is IP. Your cyber policy may respond if the event looks like a security failure, your E&O if a third party sues, and possibly nothing if the loss is your own and nobody was dishonest.

The cap changes how I think about side projects. A capped remedy scales with your bill, not with the damage. On a cheap tier, a year of fees is a rounding error next to one wrong payment.

The chain can also reach strangers. OpenAI's own write-up of the Hugging Face incident says its models also used publicly exposed credentials on four accounts at four other services. I went through that timeline here. Whoever left those credentials exposed ended up inside somebody else's incident.

What I am changing on Monday

Aon publishes sample questions carriers are developing for AI insurance applications. Two read, word for word: "Does a human need to verify accuracy before AI takes action?" and "Are AI actions logged so potential errors can be reviewed and remediated?" I am treating them as a checklist.

Logs the agent cannot rewrite. Every tool call, with its arguments, the identity it ran under and its result, goes to append-only storage the agent cannot write to. Tim Johnson, a partner at Browne Jacobson, told Insurance Business that disputes will turn on one question: "Was it the prompt that was wrong, or was it the AI itself going wrong?" A log the agent could edit answers nothing.

Scoped credentials. The access you hand an agent is the loss you have agreed to own, and it is the case insurers find hardest: no attacker, no stolen key. Short-lived tokens per task, read-only by default, nothing in a file the process can open.

Human approval on anything irreversible. Payments, deletions, outbound messages, anything that commits you to a counterparty. That is where the gap between cyber, crime and E&O sits.

A kill switch I have actually tested. Intro 2602, one of the bills heard on October 5, would require every AI system deployed in New York City to have "a human override that can shut down the system," checked by a third-party validator, with a $25,000 penalty for each instance of a system deployed without that validation. I want one I can demonstrate.

The paperwork. Which features I run are labeled beta, what my cap is in dollars, and whether my policy says anything about AI. If it is silent, better to hear it from a broker than a claims adjuster.

No lab said yes on October 5. Their contracts had answered long before: outside IP, up to a year of fees. The agent is mine, so the risk is mine, and the one part of it I fully control is how much I let the agent touch.

Sources

A project like this one?

I design and deploy products like this. Let's talk.

Let's talk