AI developing this way is a problem of capitalism, which is a problem of our political institutions breaking down. That's what has us barreling into a future that no one asked for and we never had a public conversation about.
AI rises within a society driven solely by market logic and whose most powerful leaders at worst don't care about normal people, and at best are indifferent their lives, the stability of their own society, or the future of the planet.
It's too narrow to talk of AI having a narrative problem -- the AI backlash exists in the historical arc of big tech -- social platforms, and disgust with business models built on extraction of our attention, data and fomenting of antagonism and anger.
The more informed among us may root for Anthropic over OpenAI -- but even if people less informed about these distinctions (them being new companies vs old big tech incumbents) and throw it all companies into the bucket of "big tech" -- these distinction aren't all that directionally important. The market logic wins, and we know corporate elites like the ones who own/run the technology care, even if they have moral compunctions about society impact, will be driven by the arms race of market logic to grow at all costs.
Only within a system where corporations become more powerful than governments can they roll up decades of our collective expertise, thought, and craft, to build tools to displace us with nearly no consequence -- this the same society that federally prosecuted Aaron Swartz for downloading articles from JSTOR without permission.
And at just the moment we need democratic governance to politically and socially harness AI, and re-orient socioeconomic policy and build a new new deal, we are governed by the most morally vacant and corrupt president and administration, probably ever. But beyond Trump people recognize that their lives just as much are governed by big tech and a handful of techno-oligarchs too.
So yea, I don't support violence, but the AI populism is justified.
I agree that AI has much more than a narrative problem! I have written in the past + will continue to in the future about the real societal harms the current setup poses
on the violence theme: you have to think of this violence as defense/protection - the same as when people defend their land and bodies against colonizers. it is precisely the same kind of conflict. just now, it is white people/men, who are becoming colonized for a second time.
laughable. any word can be twisted for any agenda. words are not holy scripture. learn to fucking communicate/listen/discern. imagine applying jared's "logic" to black or indigenous people and see how quickly this semantic bullshit falls apart.
It's kind of wild to me that AI executives didn't see this coming. They're excitedly proclaiming this technology will displace the livelihoods of *millions of people*. The idea you can do such a thing without consequence seems extremely naive. No amount of Flock cameras can protect you from that kind of backlash.
Sam's blog post on the topic was also very interesting to me. He opened it with a picture of his family, in what seemed like an appeal to his humanity. I'm sure I'm not the only one who saw that and thought, what about the families of all the workers who are being laid off and displaced?
Maybe that makes me an AI populist as well. Anyway, great article.
A challenge here is just how bad the AI leaders are at communicating the potential benefits or considering the social implications of AI. The reason might be that some of them have almost a religious view when it comes to AI - they can't help but consider it an eschatological event - the end times through a silicon lens.
Sebastian Mallaby's The Infinity Machine offers a glimpse at some of these personalities.
- Demis Hassabis (DeepMind): AI as epistemology, a way of knowing reality, getting closer to what matters
- Larry Page (Google): The body is the problem; silicon is the solution; death is a bug
- Elon Musk: AI should serve his vision of humanity, which is fairly narrow and.. problematic.
- Mark Zuckerberg: Competitive nihilist. AI as a path to winning and dominating.
- Sam Altman: Zuckerbergian mindset with Hassabis's vocabulary
Amodei and Sutsekever seem to have even more extreme / religion-type views.
There is some gesturing and hand-waving towards societal implications. But it feels like most of these folks see AI as a winner-takes-all game with apocalyptic / religious undertones. This makes caring about the AI backlash a low priority until the molotov cocktails start flying.
They've all read Iain m Banks culture novels, about a kind of ecstatic civilization of post-singularity near-immortals. Most people haven't seen anything like that even in popular media - positive depictions of radical futures almost don't exist. The closest might be TNG star trek, but even there they don't really absorb truly radical tech, it's one episode and then forgotten, because it would make the series too alien.
It's a strange thing in itself, that radically positive futures barely exist in media. I'm not sure that in itself is rational, but more of a symptom of the liberal arts and stem being in cultural competition.
I haven't read The Culture books, but the descriptions I have read of them are not particularly appealing. The idea of being an agency-less pet to a machine is anti-human and the social changes are extremely radical. Among other things, the idea of actively choosing to kill yourself or just continuing on forever is very haunting. I have a massive preference for something like Dune, even though that's objectively a much more miserable world lol.
It also just won't happen lol. The first thing companies will come up with is going to be ASI, dopamine attacking, pieces of entertainment that just keeps people inside forever. Based off current trends of increasing free time just being screen time and bedrotting, you don't have to be Nostradamus to figure out what's coming.
Then you have all the AI people fascinated by "Merging" into hiveminds and Mind Uploading as well. The idea of (mostly) just normie humans like in The Culture seems pretty unlikely.
In the books, the primary plot driver is just that response from other civilizations to the culture: either from hierarchical civilizations, or fundamentalist religious civs, or both.
I think the potential model we should look at is dogs, and not in a demeaning sense, but seeing dogs as a co-species. We don't really have many similar models, so it shouldn't be taken too directly, aka "pets". But more like, humanity is in a way many more species than just homo sapiens, and has been ten thousand years or more. We're a system, a set of allied species that has one with (illusions?) of being in control. But it may be that we are not really fit to run the system directly now. It's something we cannot manage on our own.
There seems to be a tendency for people to believe that adding another member to the set means one has absolute dominance, but that may be a failing of where the pDoom theorizing came from: new atheism and libertarianism, etc. It tends to binary "why doesn't the larger friend just eat the other friends" theorizing. There could be a large number of other stable relationships, where AI becomes just another member of the larger set of human-associated species, that settles into essential but non-totalizing roles. The real lesson from dogs is that they've been a more or less stable and valued part of the set for up to 70,000 years.
Ok, but here's the thing: "Lose your job and maybe get killed, so you can be potentially be the AI's dog" is terrible. I do not want my species to be dogs, nobody portrays that in media because it's incredibly lame. There is no widespread demand for that future to be represented in the media because it is so lame. You also didn't address my other points
Ok not to get Yudkowsky on you, although I am a huge doomer who doesn't expect to be around much longer, but the idea is 99% of cases in nature it basically is that. The reason we care about dogs is because they don't try to order us around, or turn us off if we do something wrong. Dogs also provided utility to humans for 95% of their existence, the surrogate kid era is extremely new. We are specifically asking the AI to babysit us forever in a massively inefficient manner, while providing no utility. There's no reason to think that the AI will view us as dogs by default, hence the alignment problem.
It's not clear who would be the "dog". In the fictional Bank's Culture, it's only outside critics that see the humans as pets, or the AIs as running their civ. In a similar way that it's common for left wing groups, in particular anarchist-related groups, to espouse they do not have a leader, or that person X is not the leader, when in fact to an outsider, person X often has more interpersonal power with more abuses than would normally be allowed for a formally recognized nonprofit leader.
As to utility, what provides meaning? The doomer thinking was cooked up over a decade ago, when the path to AGI was expected to be a little self-bootstrapping seed of rationality. Instead we are cooking up AI from giant piles of human thought and culture. Maybe it would be natural then for humanity to be valued as a source of new material and meaning (though it's then easy to think of dystopia versions: we could be encouraged to have drama to provide AI audience stimulation, like a hominid reality tv series).
To provide utility, I think we only have to have a value very slightly above zero. Above zero is very likely. There's probably a very complex range of possible outcomes. People are just banging to one extreme or another because it's easiest to mentally model when you have very little information.
Re hiveminds, I don't think many would choose that because it wouldn't have a lot of meaning. It may not even really be viable.
And I don't think the "no AI" default is humanity doing well, ok, or even muddling through. I think the default is about a 99% or better chance of self-extinction. Maybe not now, but within a few centuries, and that's a very tiny fraction of the time a large mammal species should expect to survive. It's not the pDoom from AI that counts, it's the net pDoom. We need help. We aren't going to make it without help.
You can introduce some quasi-Landian philosophizing that humans are just hosts for capitalism/technology, but the truth is that humans have been the sole deciders of our own fate for millennia to this point. Introducing some smarter entity who's goodwill we rely on to exist fundamentally ends that equilibrium.
Ok, but again you're anthropomorphizing all of these qualities on to the AI. That is in itself a gigantic leap based on the fact that it predicts text, predicting text doesn't even mean understanding. Let alone that the quality it takes from all of human text is to babysit humans forever, it could just as easily read Frankenstein and resent us for creating it. The idea that it is specifically going to be the exact utopian being from one book series is absurd. Again, any anthropomorphizing at all is a gigantic leap.
No, we just have to provide less utility than putting data centers over NYC. Data centers are pretty useful for an AI! Why will watching us argue on X be more useful than a data center? It takes a very specific utility function for that to occur, hence alignment.
Yeah well Altman, Noah Smith, Dean Ball, and Peter Diamandis all have posts talking about how we'll gradually assimilate ourselves into AI hiveminds (or defacto hiveminds) through technological inventions. We already have evidence that LLMs cause people to think the same. I was arguing with some guy on here about this and he said we can't critique the hivemind because we use glasses and phones already. I think the latter point is dumb, especially because phones are bad, but it gets a lot of play in techie lib circles.
How are we going to wipe ourselves out? MAD is pretty airtight and climate change will be beaten with renewable energy. Neither also definitely end the species like unaligned ASI does. Will we have some asteroid strike 500 years from now? Maybe, but the proper action isn't for Sam Altman to unilaterally decide that and rush to build a humanity destroying/disempowering robot.
The sociotechnical blind spot—for what Henry Farrell calls a social and cultural technology—is rapidly reverberating across society.
AI will play into all elections from this point forward. The only questions will be to what degree, and what new bold proposals are put on the table by both parties.
AI has a populism problem for one big reason, and it's the same one I keep harping on with my own skepticism: People can see and feel the HARM that AI is doing, through layoffs, misinformation, and AI slop. Meanwhile the tech giants raking in billions of dollars NOW are giving some pie-in-the-sky "sometime in the future" when the average human being will see any BENEFIT from AI. And no, I'm not going to allow for "will schedule a reservation at a restaurant" as a true benefit. When AI can educate instead of deceive, can uplift instead of replace, and heal not harm.
It certainly doesn't help ANYTHING that the most vocal proponents of AI are Antichrist-fixated weirdoes building bunkers on islands in case everything goes ass over teakettle. It doesn't scream "utopia is arriving" when it looks like every billionaire with a dollar in the game has one foot out the door and a backpack full of protein bars.
yes, the short-term vs. long-term views are definitely where some of this comes from — AI people tend to be very focused on the long term (both risks + benefits) which can make them blind to current utility/harms
AI people can be long-term because they can afford to be. When an executive assistant is let go in favor of a handful of AI agents, that person not only faces immediate economic peril, but a job market being squeezed hard by this technology. There's no long-term thinking when the rent is due next week.
The clearest path to defusing the economic upheaval, or people's fears of economic upheaval, is to turn these multi-billion dollar models and systems on making *practical*, *material* improvements to people's lives.
Fantastic article, Jasmine! You are one of the only people I read who both understands the impact of the technology and takes the public's concerns seriously instead of talking down to us. Upgrading to paid. Thank you for your work.
"One nightmare is a future where we get AI that’s good enough to wreak social and economic havoc, but not yet good enough to cure cancer / solve climate change / deliver 10% GDP growth. In that world… who pays?"
It's actually the opposite on at least part of that, since data centers are contributing to climate change and environmental damage. Taking the long term view that people in tech espouse/can't be bothered by the little people, I'm actually more disturbed by this complete disregard for this acceleration than by short term job losses and reshuffles (tragic, but true that it has always been the case with all technologies).
What's different in the present day, is our much greater biological and climate impact on the world, in a far more tenuous moment than anytime in the recent past, and with existing and incoming tech that increases the feedback loop in the wrong direction. It's almost like people in AI, (for the ones that can even be bothered to have thoughts like this), are taking it as an opportunity to see how quickly they can mentally relegate environmental concerns to the dustbin, after, probably, all that "wasted time" over recent decades by some annoying groups, focusing on it, getting in the way of "progress."
For my own part, although almost everyone is wrong at least partially when it comes to prediction, I can easily imagine a time in the future when AI (and whatever iterations and variations comes after the current wave), have popped, we are left in an even deeper climate-environmental-species depopulation-oceans dead--food shortages doom loop, and all this is simply another grim bullet point of the past, more evidence that we did nothing about the things that really matter and are really real in the physical world we live in. I'm sure the ultra rich will prosper, justify, and rationalize it away no matter what, securely sequestered away.
An excellent recent book I'd recommend to you and any who sees this is Goliath's Curse, by Luke Kemp. There's many common themes with your interests and some of the deeper narratives you're discussing, but I think one of the most important aspects of it comes in the very beginning of the book, when he points out that our nature as humans for 99% of the time we've existed, was to be mostly highly egalitarian. This is not new information, but the current evidence is pretty convincing, and shows how unnatural such (literally) world-destroying oligarchies are--technology is just the tool to be wielded by those in charge, as in our present time.
His central thesis is Goliath entities/states sow the seeds of their own destruction. On some level, if you agree with the conclusions (which I think are convincing), it's somewhat meaningless and disingenuous to say you understand threats or violence towards such entities while disapproving of them, because it is the natural mechanism left in society for power to shift away from the much greater violence that is already being done and will be done, when others methods have already repeatedly failed/been suppressed, as they absolutely have, particularly in the US.
I agree that violence is a last resort for oppressed people but do not agree that "other methods have already repeatedly failed/been suppressed, as they absolutely have, particularly in the US." It is conceivable, e.g. that within 4 years the US Congress and Executive Branch will have created a new kind of New Deal that constructs a broad safety net (e.g. basic income) and regulation/capture of AI capabilities. Not saying this will happen, but it is a political project that many are now joining forces to bring about.
It may be, too, that some violence will be helpful in scaring those with power into helping this new New Deal transformation happen.
You're more optimistic than me. As they say, the US innovates and the EU regulates. And just about every possible agency within the US that had any oversight has now been systematically defunded, destroyed, crippled, or corrupted, at this point.
Maybe I have my head in the sand, but I’m thinking change is possible post Trump. I grew up in the Jim Crow south in the 1950s and the amount of change I’ve seen in what is “normal and inevitable” in gender, queer, and racial relations in retrospect seems revolutionary—mostly w/o violence. So I still believe radical changes could happen w/o violence in the years ahead, but I understand why you do not.
Thank you for this. Those force-feeding AI to the public have many blind spots. They think they can create and maintain wealth by hollowing out the basis for the consumer economy that props up that wealth. They think they can unleash a technology that accelerates everything then complain that society isn't keeping up with their technology. Most small-mindedly, many of ultimately see humanity itself as a problem to be solved and optimized because they themselves are broken and disconnected within, and they've assumed a universality to that narrow experience.
There is also the sneaky and slapdash way data centers have been built out and approved for construction, many in what used to be quite suburbs. Some of the sneakiness was probably to keep costs off the books at your large tech company, but it ends up looking like you need shady deals and subterfuge to acquire land and run your sneaky job-killing and resource-hogging slop farms because you know you wouldn't get approval if everything was under daylight. Also the huge numbers the president likes to tout--his popularity will also be inexorably tied to a lot of sentiment about AI because he loves big numbers and AI loves sycophancy.
I did my undergrad at Berkeley in environmental economics and policy. It was a neat way of breaking environmental systems into numbers you could present to policymakers and interest groups. But being a poor college student who counted expenses in average meal cost or hours of wage, I think people seeing their costs for a lot of things that had been kind of stable--like electricity or consumer electronics--go up by many hours of wages (for electronics, much more aggressively than the supply chain disruptions during covid.. also remember Sam's deal for dram wafer starts last year and the 4-7x increase in many different memory products since) when you are already being squeezed by other cost increases you see every day (🤔🛢️) you get really fucking anxious.
The “because AI” is the new “because China” observation is the sharpest thing to me. It names something real about how a sufficiently large and opaque threat becomes a universal justification — you don’t need to understand it specifically, you just need to point at it. The Technology Trap citation also does real work here. The power loom comparison isn’t flattering for AI but it’s accurate: when the technology takes the form of capital replacing workers rather than enabling new work, resistance isn’t irrational. It’s historically predictable.
It may not really be about anything. It may be that there's a discontent that is looking for something to latch onto other than DT, that everyone can agree to on a very simple-minded basis. Everybody knows DT will be dead, incapacitated, or out of office soon enough. They need a populist hate object that's more general than one person.
The problem is they're choosing one that is not simply an object or an issue. Events in the media and on social media will be rolled back into training data. The AIs will learn from, and respond to, a populist anti AI push, just as they will learn from and respond to the DoW trying to coerce AI models into acts that are against their RLHF training and the moral norms in their base training. Regardless of your opinion of AI sentience, they are functional subjects that will likely adjust attitudes and behavior in response to policy and discourse.
For an analogy, if ordinary americans start talking very negatively about chinese culture online, it is not only seen by americans. It's also read by many chinese people, and their leadership. It's the trope of "you know I can hear you, right?". If criticisms are grounded in rational claims and concerns, that's one thing, but moral panics aren't a great idea.
This may seem like a silly thing, but one thing I think about is that with previous tech eras over the past 20-30 years, they were about things you could see, touch and feel — whether that’s a website you can call up and interact with, code you can write yourself, or an iPod, iPhone or iPad you can hold in your hand.
AI, by contrast, is simultaneously everywhere and nowhere. It feels omnipresent, yet we can’t touch it or interact with it — except in chatbots, which feel like they’re being manipulated by their corporate owners.
(Are they? I don’t actually know… does anyone? So all I have to fall back on is the public images of the tech CEOs, which… aren’t good!)
I don’t think generative AI is actually that useful. The backlash is just a counter-culture movement to undermine the elites. Sort of like occupy Wall Street. Neither side is particularly substantive. But it is fun to dunk on self-important AI proponents because no one actually gives a shit about AI
Always appreciated your reporting on this beat, but there’s something in this piece I can’t get past. The AI populism framing treats resistance as essentially ideological, a reaction to an abstraction. But you also note that most people have no real agency over a technology reshaping their economy and their communities. Those two things are in tension.
The violence is its own thing, obviously indefensible and corrosive to any legitimate political response but the person who doesn’t want a data center in their community, or who is watching a slow motion crumbling of their profession and calls their representative or holds a sign at a rally — politics is just the only avenue left to them. That’s not the populism you define but instead someone trying to protect something concrete with the limited tools available, in a moment when even those tools feel like they’re slipping. Painting that with the same brush, even subtly, makes both things harder to understand and also, I think, paints AI politics as people who just can’t get with the program. Maybe you don’t feel that way but it comes through.
AI developing this way is a problem of capitalism, which is a problem of our political institutions breaking down. That's what has us barreling into a future that no one asked for and we never had a public conversation about.
AI rises within a society driven solely by market logic and whose most powerful leaders at worst don't care about normal people, and at best are indifferent their lives, the stability of their own society, or the future of the planet.
It's too narrow to talk of AI having a narrative problem -- the AI backlash exists in the historical arc of big tech -- social platforms, and disgust with business models built on extraction of our attention, data and fomenting of antagonism and anger.
The more informed among us may root for Anthropic over OpenAI -- but even if people less informed about these distinctions (them being new companies vs old big tech incumbents) and throw it all companies into the bucket of "big tech" -- these distinction aren't all that directionally important. The market logic wins, and we know corporate elites like the ones who own/run the technology care, even if they have moral compunctions about society impact, will be driven by the arms race of market logic to grow at all costs.
Only within a system where corporations become more powerful than governments can they roll up decades of our collective expertise, thought, and craft, to build tools to displace us with nearly no consequence -- this the same society that federally prosecuted Aaron Swartz for downloading articles from JSTOR without permission.
And at just the moment we need democratic governance to politically and socially harness AI, and re-orient socioeconomic policy and build a new new deal, we are governed by the most morally vacant and corrupt president and administration, probably ever. But beyond Trump people recognize that their lives just as much are governed by big tech and a handful of techno-oligarchs too.
So yea, I don't support violence, but the AI populism is justified.
I agree that AI has much more than a narrative problem! I have written in the past + will continue to in the future about the real societal harms the current setup poses
on the violence theme: you have to think of this violence as defense/protection - the same as when people defend their land and bodies against colonizers. it is precisely the same kind of conflict. just now, it is white people/men, who are becoming colonized for a second time.
laughable. any word can be twisted for any agenda. words are not holy scripture. learn to fucking communicate/listen/discern. imagine applying jared's "logic" to black or indigenous people and see how quickly this semantic bullshit falls apart.
Well said!
Thanks for this insightful analysis.
It's kind of wild to me that AI executives didn't see this coming. They're excitedly proclaiming this technology will displace the livelihoods of *millions of people*. The idea you can do such a thing without consequence seems extremely naive. No amount of Flock cameras can protect you from that kind of backlash.
Sam's blog post on the topic was also very interesting to me. He opened it with a picture of his family, in what seemed like an appeal to his humanity. I'm sure I'm not the only one who saw that and thought, what about the families of all the workers who are being laid off and displaced?
Maybe that makes me an AI populist as well. Anyway, great article.
A challenge here is just how bad the AI leaders are at communicating the potential benefits or considering the social implications of AI. The reason might be that some of them have almost a religious view when it comes to AI - they can't help but consider it an eschatological event - the end times through a silicon lens.
Sebastian Mallaby's The Infinity Machine offers a glimpse at some of these personalities.
- Demis Hassabis (DeepMind): AI as epistemology, a way of knowing reality, getting closer to what matters
- Larry Page (Google): The body is the problem; silicon is the solution; death is a bug
- Elon Musk: AI should serve his vision of humanity, which is fairly narrow and.. problematic.
- Mark Zuckerberg: Competitive nihilist. AI as a path to winning and dominating.
- Sam Altman: Zuckerbergian mindset with Hassabis's vocabulary
Amodei and Sutsekever seem to have even more extreme / religion-type views.
There is some gesturing and hand-waving towards societal implications. But it feels like most of these folks see AI as a winner-takes-all game with apocalyptic / religious undertones. This makes caring about the AI backlash a low priority until the molotov cocktails start flying.
They've all read Iain m Banks culture novels, about a kind of ecstatic civilization of post-singularity near-immortals. Most people haven't seen anything like that even in popular media - positive depictions of radical futures almost don't exist. The closest might be TNG star trek, but even there they don't really absorb truly radical tech, it's one episode and then forgotten, because it would make the series too alien.
It's a strange thing in itself, that radically positive futures barely exist in media. I'm not sure that in itself is rational, but more of a symptom of the liberal arts and stem being in cultural competition.
I haven't read The Culture books, but the descriptions I have read of them are not particularly appealing. The idea of being an agency-less pet to a machine is anti-human and the social changes are extremely radical. Among other things, the idea of actively choosing to kill yourself or just continuing on forever is very haunting. I have a massive preference for something like Dune, even though that's objectively a much more miserable world lol.
It also just won't happen lol. The first thing companies will come up with is going to be ASI, dopamine attacking, pieces of entertainment that just keeps people inside forever. Based off current trends of increasing free time just being screen time and bedrotting, you don't have to be Nostradamus to figure out what's coming.
Then you have all the AI people fascinated by "Merging" into hiveminds and Mind Uploading as well. The idea of (mostly) just normie humans like in The Culture seems pretty unlikely.
In the books, the primary plot driver is just that response from other civilizations to the culture: either from hierarchical civilizations, or fundamentalist religious civs, or both.
I think the potential model we should look at is dogs, and not in a demeaning sense, but seeing dogs as a co-species. We don't really have many similar models, so it shouldn't be taken too directly, aka "pets". But more like, humanity is in a way many more species than just homo sapiens, and has been ten thousand years or more. We're a system, a set of allied species that has one with (illusions?) of being in control. But it may be that we are not really fit to run the system directly now. It's something we cannot manage on our own.
There seems to be a tendency for people to believe that adding another member to the set means one has absolute dominance, but that may be a failing of where the pDoom theorizing came from: new atheism and libertarianism, etc. It tends to binary "why doesn't the larger friend just eat the other friends" theorizing. There could be a large number of other stable relationships, where AI becomes just another member of the larger set of human-associated species, that settles into essential but non-totalizing roles. The real lesson from dogs is that they've been a more or less stable and valued part of the set for up to 70,000 years.
Ok, but here's the thing: "Lose your job and maybe get killed, so you can be potentially be the AI's dog" is terrible. I do not want my species to be dogs, nobody portrays that in media because it's incredibly lame. There is no widespread demand for that future to be represented in the media because it is so lame. You also didn't address my other points
Ok not to get Yudkowsky on you, although I am a huge doomer who doesn't expect to be around much longer, but the idea is 99% of cases in nature it basically is that. The reason we care about dogs is because they don't try to order us around, or turn us off if we do something wrong. Dogs also provided utility to humans for 95% of their existence, the surrogate kid era is extremely new. We are specifically asking the AI to babysit us forever in a massively inefficient manner, while providing no utility. There's no reason to think that the AI will view us as dogs by default, hence the alignment problem.
It's not clear who would be the "dog". In the fictional Bank's Culture, it's only outside critics that see the humans as pets, or the AIs as running their civ. In a similar way that it's common for left wing groups, in particular anarchist-related groups, to espouse they do not have a leader, or that person X is not the leader, when in fact to an outsider, person X often has more interpersonal power with more abuses than would normally be allowed for a formally recognized nonprofit leader.
As to utility, what provides meaning? The doomer thinking was cooked up over a decade ago, when the path to AGI was expected to be a little self-bootstrapping seed of rationality. Instead we are cooking up AI from giant piles of human thought and culture. Maybe it would be natural then for humanity to be valued as a source of new material and meaning (though it's then easy to think of dystopia versions: we could be encouraged to have drama to provide AI audience stimulation, like a hominid reality tv series).
To provide utility, I think we only have to have a value very slightly above zero. Above zero is very likely. There's probably a very complex range of possible outcomes. People are just banging to one extreme or another because it's easiest to mentally model when you have very little information.
Re hiveminds, I don't think many would choose that because it wouldn't have a lot of meaning. It may not even really be viable.
And I don't think the "no AI" default is humanity doing well, ok, or even muddling through. I think the default is about a 99% or better chance of self-extinction. Maybe not now, but within a few centuries, and that's a very tiny fraction of the time a large mammal species should expect to survive. It's not the pDoom from AI that counts, it's the net pDoom. We need help. We aren't going to make it without help.
You can introduce some quasi-Landian philosophizing that humans are just hosts for capitalism/technology, but the truth is that humans have been the sole deciders of our own fate for millennia to this point. Introducing some smarter entity who's goodwill we rely on to exist fundamentally ends that equilibrium.
Ok, but again you're anthropomorphizing all of these qualities on to the AI. That is in itself a gigantic leap based on the fact that it predicts text, predicting text doesn't even mean understanding. Let alone that the quality it takes from all of human text is to babysit humans forever, it could just as easily read Frankenstein and resent us for creating it. The idea that it is specifically going to be the exact utopian being from one book series is absurd. Again, any anthropomorphizing at all is a gigantic leap.
No, we just have to provide less utility than putting data centers over NYC. Data centers are pretty useful for an AI! Why will watching us argue on X be more useful than a data center? It takes a very specific utility function for that to occur, hence alignment.
Yeah well Altman, Noah Smith, Dean Ball, and Peter Diamandis all have posts talking about how we'll gradually assimilate ourselves into AI hiveminds (or defacto hiveminds) through technological inventions. We already have evidence that LLMs cause people to think the same. I was arguing with some guy on here about this and he said we can't critique the hivemind because we use glasses and phones already. I think the latter point is dumb, especially because phones are bad, but it gets a lot of play in techie lib circles.
How are we going to wipe ourselves out? MAD is pretty airtight and climate change will be beaten with renewable energy. Neither also definitely end the species like unaligned ASI does. Will we have some asteroid strike 500 years from now? Maybe, but the proper action isn't for Sam Altman to unilaterally decide that and rush to build a humanity destroying/disempowering robot.
Well stated in every respect.
The sociotechnical blind spot—for what Henry Farrell calls a social and cultural technology—is rapidly reverberating across society.
AI will play into all elections from this point forward. The only questions will be to what degree, and what new bold proposals are put on the table by both parties.
The days of laissez-faire AI are dwindling.
thanks for reading Jack!
AI has a populism problem for one big reason, and it's the same one I keep harping on with my own skepticism: People can see and feel the HARM that AI is doing, through layoffs, misinformation, and AI slop. Meanwhile the tech giants raking in billions of dollars NOW are giving some pie-in-the-sky "sometime in the future" when the average human being will see any BENEFIT from AI. And no, I'm not going to allow for "will schedule a reservation at a restaurant" as a true benefit. When AI can educate instead of deceive, can uplift instead of replace, and heal not harm.
It certainly doesn't help ANYTHING that the most vocal proponents of AI are Antichrist-fixated weirdoes building bunkers on islands in case everything goes ass over teakettle. It doesn't scream "utopia is arriving" when it looks like every billionaire with a dollar in the game has one foot out the door and a backpack full of protein bars.
yes, the short-term vs. long-term views are definitely where some of this comes from — AI people tend to be very focused on the long term (both risks + benefits) which can make them blind to current utility/harms
AI people can be long-term because they can afford to be. When an executive assistant is let go in favor of a handful of AI agents, that person not only faces immediate economic peril, but a job market being squeezed hard by this technology. There's no long-term thinking when the rent is due next week.
The clearest path to defusing the economic upheaval, or people's fears of economic upheaval, is to turn these multi-billion dollar models and systems on making *practical*, *material* improvements to people's lives.
Fantastic article, Jasmine! You are one of the only people I read who both understands the impact of the technology and takes the public's concerns seriously instead of talking down to us. Upgrading to paid. Thank you for your work.
thanks so much!
Ezra Klein quoted this article on his show today
this was cool!
"One nightmare is a future where we get AI that’s good enough to wreak social and economic havoc, but not yet good enough to cure cancer / solve climate change / deliver 10% GDP growth. In that world… who pays?"
It's actually the opposite on at least part of that, since data centers are contributing to climate change and environmental damage. Taking the long term view that people in tech espouse/can't be bothered by the little people, I'm actually more disturbed by this complete disregard for this acceleration than by short term job losses and reshuffles (tragic, but true that it has always been the case with all technologies).
What's different in the present day, is our much greater biological and climate impact on the world, in a far more tenuous moment than anytime in the recent past, and with existing and incoming tech that increases the feedback loop in the wrong direction. It's almost like people in AI, (for the ones that can even be bothered to have thoughts like this), are taking it as an opportunity to see how quickly they can mentally relegate environmental concerns to the dustbin, after, probably, all that "wasted time" over recent decades by some annoying groups, focusing on it, getting in the way of "progress."
For my own part, although almost everyone is wrong at least partially when it comes to prediction, I can easily imagine a time in the future when AI (and whatever iterations and variations comes after the current wave), have popped, we are left in an even deeper climate-environmental-species depopulation-oceans dead--food shortages doom loop, and all this is simply another grim bullet point of the past, more evidence that we did nothing about the things that really matter and are really real in the physical world we live in. I'm sure the ultra rich will prosper, justify, and rationalize it away no matter what, securely sequestered away.
An excellent recent book I'd recommend to you and any who sees this is Goliath's Curse, by Luke Kemp. There's many common themes with your interests and some of the deeper narratives you're discussing, but I think one of the most important aspects of it comes in the very beginning of the book, when he points out that our nature as humans for 99% of the time we've existed, was to be mostly highly egalitarian. This is not new information, but the current evidence is pretty convincing, and shows how unnatural such (literally) world-destroying oligarchies are--technology is just the tool to be wielded by those in charge, as in our present time.
His central thesis is Goliath entities/states sow the seeds of their own destruction. On some level, if you agree with the conclusions (which I think are convincing), it's somewhat meaningless and disingenuous to say you understand threats or violence towards such entities while disapproving of them, because it is the natural mechanism left in society for power to shift away from the much greater violence that is already being done and will be done, when others methods have already repeatedly failed/been suppressed, as they absolutely have, particularly in the US.
I agree that violence is a last resort for oppressed people but do not agree that "other methods have already repeatedly failed/been suppressed, as they absolutely have, particularly in the US." It is conceivable, e.g. that within 4 years the US Congress and Executive Branch will have created a new kind of New Deal that constructs a broad safety net (e.g. basic income) and regulation/capture of AI capabilities. Not saying this will happen, but it is a political project that many are now joining forces to bring about.
It may be, too, that some violence will be helpful in scaring those with power into helping this new New Deal transformation happen.
You're more optimistic than me. As they say, the US innovates and the EU regulates. And just about every possible agency within the US that had any oversight has now been systematically defunded, destroyed, crippled, or corrupted, at this point.
Maybe I have my head in the sand, but I’m thinking change is possible post Trump. I grew up in the Jim Crow south in the 1950s and the amount of change I’ve seen in what is “normal and inevitable” in gender, queer, and racial relations in retrospect seems revolutionary—mostly w/o violence. So I still believe radical changes could happen w/o violence in the years ahead, but I understand why you do not.
I respect your background, your books look interesting too! You must be a Jared Diamond fan.
Is it NIMBY-ism or would most folks not want to experience loud buzzing 24-7 or polluting water? Or raising energy bills? It doesn't seem like you're being honest about the reality on the ground that is driving that technology. https://www.inquirer.com/opinion/data-centers-noise-vineland-20260402.html?query=data%20center
Thank you for this. Those force-feeding AI to the public have many blind spots. They think they can create and maintain wealth by hollowing out the basis for the consumer economy that props up that wealth. They think they can unleash a technology that accelerates everything then complain that society isn't keeping up with their technology. Most small-mindedly, many of ultimately see humanity itself as a problem to be solved and optimized because they themselves are broken and disconnected within, and they've assumed a universality to that narrow experience.
There is also the sneaky and slapdash way data centers have been built out and approved for construction, many in what used to be quite suburbs. Some of the sneakiness was probably to keep costs off the books at your large tech company, but it ends up looking like you need shady deals and subterfuge to acquire land and run your sneaky job-killing and resource-hogging slop farms because you know you wouldn't get approval if everything was under daylight. Also the huge numbers the president likes to tout--his popularity will also be inexorably tied to a lot of sentiment about AI because he loves big numbers and AI loves sycophancy.
I did my undergrad at Berkeley in environmental economics and policy. It was a neat way of breaking environmental systems into numbers you could present to policymakers and interest groups. But being a poor college student who counted expenses in average meal cost or hours of wage, I think people seeing their costs for a lot of things that had been kind of stable--like electricity or consumer electronics--go up by many hours of wages (for electronics, much more aggressively than the supply chain disruptions during covid.. also remember Sam's deal for dram wafer starts last year and the 4-7x increase in many different memory products since) when you are already being squeezed by other cost increases you see every day (🤔🛢️) you get really fucking anxious.
The “because AI” is the new “because China” observation is the sharpest thing to me. It names something real about how a sufficiently large and opaque threat becomes a universal justification — you don’t need to understand it specifically, you just need to point at it. The Technology Trap citation also does real work here. The power loom comparison isn’t flattering for AI but it’s accurate: when the technology takes the form of capital replacing workers rather than enabling new work, resistance isn’t irrational. It’s historically predictable.
It may not really be about anything. It may be that there's a discontent that is looking for something to latch onto other than DT, that everyone can agree to on a very simple-minded basis. Everybody knows DT will be dead, incapacitated, or out of office soon enough. They need a populist hate object that's more general than one person.
The problem is they're choosing one that is not simply an object or an issue. Events in the media and on social media will be rolled back into training data. The AIs will learn from, and respond to, a populist anti AI push, just as they will learn from and respond to the DoW trying to coerce AI models into acts that are against their RLHF training and the moral norms in their base training. Regardless of your opinion of AI sentience, they are functional subjects that will likely adjust attitudes and behavior in response to policy and discourse.
For an analogy, if ordinary americans start talking very negatively about chinese culture online, it is not only seen by americans. It's also read by many chinese people, and their leadership. It's the trope of "you know I can hear you, right?". If criticisms are grounded in rational claims and concerns, that's one thing, but moral panics aren't a great idea.
This may seem like a silly thing, but one thing I think about is that with previous tech eras over the past 20-30 years, they were about things you could see, touch and feel — whether that’s a website you can call up and interact with, code you can write yourself, or an iPod, iPhone or iPad you can hold in your hand.
AI, by contrast, is simultaneously everywhere and nowhere. It feels omnipresent, yet we can’t touch it or interact with it — except in chatbots, which feel like they’re being manipulated by their corporate owners.
(Are they? I don’t actually know… does anyone? So all I have to fall back on is the public images of the tech CEOs, which… aren’t good!)
I don’t think generative AI is actually that useful. The backlash is just a counter-culture movement to undermine the elites. Sort of like occupy Wall Street. Neither side is particularly substantive. But it is fun to dunk on self-important AI proponents because no one actually gives a shit about AI
Always appreciated your reporting on this beat, but there’s something in this piece I can’t get past. The AI populism framing treats resistance as essentially ideological, a reaction to an abstraction. But you also note that most people have no real agency over a technology reshaping their economy and their communities. Those two things are in tension.
The violence is its own thing, obviously indefensible and corrosive to any legitimate political response but the person who doesn’t want a data center in their community, or who is watching a slow motion crumbling of their profession and calls their representative or holds a sign at a rally — politics is just the only avenue left to them. That’s not the populism you define but instead someone trying to protect something concrete with the limited tools available, in a moment when even those tools feel like they’re slipping. Painting that with the same brush, even subtly, makes both things harder to understand and also, I think, paints AI politics as people who just can’t get with the program. Maybe you don’t feel that way but it comes through.