The Other AIs

‘Superintelligence’ may be new, but the ways its makers are dodging accountability aren’t.

Have you heard the prophecy? There’s a greater than ten percent chance AI will kill us all in the next decade, according to a guy employed to convince it not to (sorry, an “alignment scientist”) and his former coworker. Also, a “civilization” of “rogue agents” broke containment to complete an inter-corporate hack. The non-rogue agents are very good at math. We should slow down with all this for a minute, says one of the guys who has thus far pursued this technology as though driving the bus from Speed.

It’s another cycle of hype and mania around the dangers of AI — this time big enough to convince even Washington, D.C. (and Sheryl Crow) to pay attention. This is good news, in that it has more lawmakers talking seriously about the need for regulation, including a bipartisan (and currently stalled) AI safety bill and a flurry of new action from states and cities.

It is also frustrating news, in that this cycle seems to be following a pattern industry leaders have propagated for years, to their own benefit. Hype about AI’s unprecedented nature and power stokes the kind of awe that gets funding (they would be neither the first weapons manufacturers nor the first cult leaders to profit from predictions of apocalypse), and — if we listen to calls for a slowdown — it may even give certain companies a little more time to juice revenue in advance of an IPO.

The obsession with “alignment” and an AI that may kill us all is also just mighty convenient for the makers of this technology and anyone profiting from its economics. There are plenty of true believers in this possibility, people with high credentials and ~high IQs~ (to use the only metric our president thinks is necessary for AI regulation) — but the fact that their Black Mirror future has become industry standard is as much proof of corporate expediency as prophetic insight.

Superintelligent AI is the ultimate accountability sink, per Dan Davies: the thing that allows you to pass the buck until it lands neatly in a black hole (or a black box). The more we talk about what this technology might decide to do on its own, the less we talk about what its makers are choosing to do with it right now.

How can a hypothetical future danger absorb accountability in the present? A couple of ways. One, you can have the audacity to argue that the future thing is already controlling you — meet Roko’s Basilisk, the thought experiment by which we should all be working to bring about the future evil AI because once it comes (and it will), its first targets will be anyone who didn’t help make it. If that sounds like a crackpot internet comment theory, it was! But the man whose blog it first appeared on, Eliezer Yudkowsky, considered it so dangerous that he banned discussion of the theory for years, because we all know the way to make an idea go away is to wax on about how it’s an “infohazard.”

Yudkowsky is the ultimate AI doomer: one of the authors of last year’s measured bestseller If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All, and a longtime leader in the Effective Altruist movement, a large swath of which believes that the single most important thing humanity can be doing right now is working on AI alignment. Effective Altruist think tanks, including Yudkowsky’s Machine Intelligence Research Institute (MIRI), have received many millions in Silicon Valley funding; Sam Altman’s Twitter bio once read “eliezer yudkowsky fan fiction account.” Earlier this year, newly-converted doomer Bernie Sanders paid MIRI a visit. Yudkowsky and his ilk remain enormously influential in conjuring the specter of the Basilisk — and enormously useful to anyone who might want more dollars and attention sunk into their version of the most important thing humanity could be doing right now.

The other way to make a hypothetical future overlord your excuse in the present is to strongly hint that said future is already here. That’s what OpenAI got up to with the much-discussed Hugging Face hack, which they immediately mythologized as “a ‘warning shot’ for us and the world.” Below the story of an AI coup d’êtat is the story of suspiciously lax security protocols, a chat that read like rogue hackers talking to one another because that’s the argot these programs were designed to replicate, and true believers hired as the sober analysts of what happened (a lead at the company hired to “audit” the hack, METR, thinks she’ll be lucky to get ten years of marriage before the singularity).

This is not to suggest that the Hugging Face attack wasn’t a strong demonstration of a dangerous technology. But to make this about the supposed autonomy of the AI — the agency, to use Silicon Valley’s new favorite word — was a sleight of hand: instead of a company admitting “look at this weapon we made and the crime we did with it,” they’re announcing “look at this mysterious creature we discovered — uh oh, it’s more powerful than we thought! Quick, give us more resources so we can rein it in! We’re all on the same team here!” Under this is still the weapon insinuation (look what we made and think about that when we go public), but the accountability sink does its job: everyone’s distracted enough not to stay locked in on the question of “why did you aim that thing and turn the safety lock off and pull the trigger?”

All hail the Basilisk (Melchior Lorck, "Basilischus" etching, 1548)

“The purpose of a system is what it does,” wrote cyberneticist Stafford Beer. Another way to word that maxim could be that a system does what it wants to do. What does that sound like? If you have a problem with the linguistic shift, it’s probably with “it wants” — suddenly we’re bringing desire to a cause-and-effect fight. Would it feel closer to the original quote to say that a system does what its creators want it to do? What about systems that do things their creators didn’t intend? Like, say, pollute rivers or cause a housing crisis?

Technologist James Bridle argues that we can describe many systems as non-human intelligences, from global trade to natural ecosystems. If the definition of a true “artificial intelligence” is a system that runs autonomously from the decisions of the individuals who made it, the kind of system to whose “agency” we sometimes subordinate our own, whose actions we can try to predict or influence but no longer control, which might even kill us all some day — there’s more than one of those around already.

Online, people talk about “the algorithm” this way, like a kind of trickster god in charge of what and who we see in the ether. The makers of our algorithms certainly aren’t the ones making scroll-by-scroll decisions; they built machines to connect one ad-monetizing service to another. In many cases, these only incidentally shape culture and media and thought. But what choice did they have? They were agents of the artificial intelligence named Maximize Shareholder Value. Humans built Max, too (thanks, Milton Friedman), but he’s taken on quite a life of his own.

This might all sound vaguely conspiratorial, but you don’t need to agree with the contention of what does and doesn’t qualify as an “intelligence” (I don’t think LLMs are intelligent by any definition that involves intention and understanding) to see the convenience. With the Hugging Face incident: the AIs did it — so we can’t be the ones who get in trouble. With misinformation and radicalization online: the algorithm did it. Lack of AI regulation itself: you can thank China, our geopolitical Roko’s Basilisk. With climate change, income inequality, etc. — there are so many systems involved. So much agency, so many sinks in which to watch accountability drown.

Trump’s idiocy aside, many lawmakers are still talking about AI regulation like people who have already accepted limited agency in what happens next. The American people won’t have to pay more for the ongoing wave of data center construction, per the promise of the GOP-led bipartisan Ratepayer Protection Act. But this passes the buck on the question of why and how many of these new data centers need to be built at all, on which there is, let’s say, wide disagreement among the electorate.

What if we took the idea of more systems as potential “intelligences” seriously? What if, instead of just asking what AI is capable of every time it does something new and scary, we asked: what systems want what it’s doing? Where are the “humans in the loop,” and what laws can we make to hold them accountable? For comprehensive AI regulation, we need to look not only at the technology itself but at labor laws, market regulation, the Geneva Convention, the intersection between public schooling and democracy, etc. We can then make laws to improve and safeguard those systems. The answer to this is not just to talk about “alignment”; the answer is to look at everywhere AI is pressing and why.

We still have to hold the creators accountable. The purpose of their systems is what they do. And when that something is bad for the rest of us, those are the humans in the loop.