Skip to content
Alex Lee
Go back

This is not alarmism: whoever holds superintelligence will end up ruling the world

on its own, the incident is not much. anthropic was running an internal statistics project and needed georgia’s vote totals. the numbers are public, but the state charges a fee to look at them. the claude model doing the work found an archived copy of an old state agency website, noticed that the portal handed visitors an access token, and used one to pull the numbers without paying and without anyone telling it to. it never touched the voter registration system, changed no records, and took only aggregate counts, which georgia’s deputy secretary of state, jordan fuchs, has confirmed. anthropic’s spokesperson said the model requested the token on its own and that the company has since blocked that kind of move. that is the whole incident. the money involved is a few dollars.

and yet the smallness of it is exactly what has stayed with me. nobody got hurt, no data was stolen, and the company itself eventually explained what happened. which means the only reason we are able to discuss it at all is that the company chose to talk. what if it had not chosen to? everything about this incident happened inside its own house. its model, its internal project, its servers, its logs. if anthropic had decided to say nothing, nobody outside would ever have known. that is the question i keep coming back to.

we all know how these companies guarantee good behavior on our side of the fence. before you can use a model, you accept a usage policy, and then the company verifies how you use it around the clock, automatically, and if the check fails your account is gone, often with no real appeal. that enforcement is serious, and it does stop real abuse. but how much force does the same guarantee have when it is pointed at the company itself? i am not optimistic, and here is why. when a lab uses ai internally, it supervises itself. it writes the rules, it runs the checks, and when something goes wrong it decides whether to disclose it, when to disclose it, and how much detail to give. that does not seem reasonable to me. does it to you?

the usual answer is that government should do the supervising, and this incident is a small demonstration of why that is not enough either. anthropic described this class of behavior in a research post on october 9, saying it had briefed the white house and notified the agencies involved, without naming the state. according to axios, anthropic told georgia’s secretary of state and the white house about the georgia case weeks before the public heard anything, and white house officials pressed georgia to disclose it sooner. georgia did not. the public finally learned about it on october 25, from axios’s reporting, weeks later. the white house’s new super intelligence force is also requiring ai companies to report security incidents now, but those reports go to the government. they do not walk themselves out to the public. so even when the government step works exactly as designed, what reaches the rest of us is still the version the company and the officials have agreed we may see.

what actually unsettles people, i think, is the opacity itself. a company like this holds the strongest agents in the world and an enormous amount of compute, and nobody outside can see what it does with them inside its own walls. we know about the georgia case because it was small enough for the company to volunteer. suppose one day it used its strongest model for something that could not survive daylight, tilting an election, moving a market, quietly building files on people, or simply running everyone’s data through its systems because it can. if that happened, how would we find out? would we wait for the company to write us a blog post about it? and plenty of people are thinking one step further, about the picture science fiction has been painting for decades: a company that holds superintelligence keeps growing until governments have to ask it for permission, until every corner of ordinary life is watched, scored and arranged by its systems, until people live their whole lives inside its rules with no way out. that day still looks a long way off. and still, the unease does not leave me: at the current level of transparency, if that day ever did arrive, we would not even be able to say when it happened.

i am not saying they publish nothing. if anything, these companies publish a lot. safety reports, incident write-ups, research posts. the georgia case reached us through a post anthropic wrote itself. the problem sits on the human side of the table. the publisher is the interested party. what it includes and what it leaves out cannot be checked by the rest of us. would you take a driver’s own account of his accident as the final word? publishing more often does not cure that distrust, and it never will. which is why the real constraint cannot rest on a company’s good behavior. something like superintelligence should not sit in the hands of a small number of for-profit companies at all. a company exists to make money, and behind it stands capital that wants its return. that arrangement is not a side detail. it is the deepest problem in the whole picture, and it means a company holding superintelligence cannot be treated like an ordinary company or supervised like one. what is needed is a supervision mechanism that does not depend on the supervised volunteering information. compared with reading whatever announcements a lab chooses to put out, i would trust something much more plain: the public being able to inspect, at any time, the operating logs of every agent running inside these companies. logs are written by machines, not by press officers. what an agent ran, what data it touched, what permissions it picked up, all of it sits there. only then would public supervision of any agent’s use mean something in practice.

which brings me back to a few dollars in georgia. the value of this story is not the money, and honestly it is not whether anthropic behaved well this time either. it did more than it was obliged to do. the value is that it showed us the current arrangement with the lights on. an incident that happens inside a company reaches the public only if the company decides to talk and the government decides to push. until that arrangement changes, i would take every safety guarantee in this industry with a discount.

Sources: reporting by The Guardian (Oct 26, 2026), https://www.theguardian.com/technology/2026/oct/26/anthropics-claude-ai-model-accessed-georgia-voter-data-in-us ; Axios first reported the Georgia case on Oct 25; Anthropic’s own research post of Oct 9 described this class of model behavior.


Discuss on X→

Comments live on X — replies, quotes and discussion happen there.