Andy Garcia · freelance dev
FRENES
Back to the blog
AI & Accountability

OpenAI Flagged a Threat and Did Not Call the Police. Now a Province Is Suing.

September 23, 2026 4 min read

On Monday, September 21, the government of British Columbia filed a lawsuit against OpenAI and its chief executive, Sam Altman, in the US District Court in San Francisco. The province's attorney general, Niki Sharma, described it as action against OpenAI "for its failure to notify law enforcement of threats made on its platform prior to the mass shooting at Tumbler Ridge Secondary school."

I want to be careful with this one. Eight people are dead, five of them students aged 12 and 13, and the facts that matter will be established in court, not on Medium. But the lawsuit raises a question that every team shipping an AI product will have to answer sooner or later, and it is better answered in advance than under subpoena.

What happened, as far as the public record goes

On February 10, 2026, an 18-year-old killed her mother and her half-brother at home, then went to the secondary school in Tumbler Ridge, a small town in northeastern British Columbia, and killed five students and a member of staff before taking her own life. Around two dozen others were injured.

According to lawsuits filed earlier by families, OpenAI's automated systems had flagged her ChatGPT account eight months earlier, in June 2025, for "gun violence activity and planning." The plaintiffs allege that the company's own investigators reviewed the conversations, judged that they posed a credible threat of gun violence against real people, and recommended contacting the Royal Canadian Mounted Police, and that the recommendation was not followed. OpenAI disputes parts of that account, including who took part in the decision.

What OpenAI has said is that it deactivated the account, that the shooter then created a second account and kept talking to ChatGPT, and that the company was not aware of the second account until after the shooting. In April, Altman wrote to the community: "I am deeply sorry that we did not alert law enforcement to the account that was banned in June."

The province's filing picks at that last word. According to reporting on the complaint, it argues the account was deactivated rather than banned, with no user-level ban, heightened monitoring or specific safeguards, which it says "re-entrusted" ChatGPT to someone the company's own safety team had identified as a real-world threat. Its most quoted line is blunt: "one telephone call to the RCMP could have prevented the tragedy."

British Columbia, together with the local school district, is seeking damages, including the cost of building a new secondary school, and a court order requiring OpenAI to implement safeguards that "reliably refuse, terminate or de-escalate" conversations of this kind. Sharma also says the province asked to see the shooter's conversations and that OpenAI refused to disclose them. In her statement, she says she has written to federal ministers proposing changes to Canada's Criminal Code "to ensure a pathway to human accountability for AI's actions."

It is not the first case. Families of seven victims sued in April, and NPR reported in early September that thirty more complaints had been filed in San Francisco by people who were present at the school, among them students, teachers and a principal.

The question for the rest of us

Strip away the specifics and something uncomfortable is left. A classifier that flags a conversation creates knowledge. From that moment, the company knows something. Whatever it does next is a decision, and doing nothing is also a decision.

Most teams I talk to have some form of automated moderation: a model that scores messages for self-harm, violence, abuse, prompt injection. Very few have written down what happens after a high score. Who reads the flag? Within what time? What threshold turns a flag into an escalation, and who is allowed to decide that it does not? Is that decision logged, with a name next to it? And when the answer is "we disable the account," what does that actually prevent, in a product where a new account takes two minutes?

None of those are model questions. They are process questions, and other industries have answered them before: banks file suspicious activity reports, platforms staff trust and safety rotas, hospitals run escalation protocols. What is new is that AI products generate these signals at a scale and with an intimacy nobody planned for. People tell chatbots things they tell no one else.

What I am taking from it

I run engineering for a LegalTech product. Our risks are not this one, and I am not going to pretend otherwise. But the principle transfers directly: every automated signal we produce about a user should have an owner, a threshold decided in advance and a record of what was done with it. A flag that goes nowhere is not neutral. Later, it is evidence that you knew.

The courts will decide what OpenAI owed the people of Tumbler Ridge. The rest of the industry does not need to wait for that ruling to decide what it owes its own users.

Sources

A project like this one?

I design and deploy products like this. Let's talk.

Let's talk