[Based on the first half of https://slatestarcodex.com/2014/07/30/meditations-on-moloch/]
Nearly everyone who reads that poem wants to know: what is Moloch.
You can almost see him, with his fingers of armies and his skyscraper-window eyes.
A lot of the commentators say Moloch represents capitalism. This is definitely a piece of it, even a big piece. But it doesn’t quite fit. Capitalism, whose fate is a cloud of sexless hydrogen?
There’s a passage in the Principia Discordia where Malaclypse complains to the Goddess about the evils of human society. “Everyone is hurting each other, the planet is rampant with injustices, whole societies plunder groups of their own people, mothers imprison sons, children perish while brothers war.”
The Goddess answers: “What is the matter with that, if it’s what you want to do?”
“But nobody wants it! Everybody hates it!”
“Oh. Well, then stop.”
The implicit question is – if everyone hates the current system, who perpetuates it? And Ginsberg answers: “Moloch”. It’s powerful not because it’s correct – nobody literally thinks an ancient Carthaginian demon causes everything – but because thinking of the system as an agent throws into sharp relief the degree to which it isn’t.
Bostrom proposes a “dictatorless dystopia”, one that every single citizen including the leadership hates but which nevertheless endures unconquered. It’s easy enough to imagine. Two rules: first, every person must spend eight hours a day giving themselves strong electric shocks. Second, if anyone fails to follow a rule (including this one), or speaks out against it, or fails to enforce it, all citizens must unite to kill that person.
So you shock yourself for eight hours a day, because you know if you don’t everyone else will kill you, because if they don’t, everyone else will kill them, and so on. Every single citizen hates the system, but for lack of a good coordination mechanism it endures.
And okay, this example is kind of contrived. And yet…
Pollution and over-fishing: bad for the people who do it, but worse for the first to stop.
Capitalism: In which every practitioner was born with kindly impulses, but those who remain in the game are those who learned to control them
Positional Goods: Including the notoriously expensive “house in a good neighborhood”
Arms Races: Even if the weapons are never used for evil, their construction was still food from the hungry, shelter from the homeless
Cancer: Which inevitably kills itself when it kills its host
A basic principle unites all of the multipolar traps above. In some competition optimizing for X, the opportunity arises to throw some other value under the bus for improved X. Those who take it prosper. Those who don’t take it die out. Eventually, everyone’s relative status is about the same as before, but everyone’s absolute status is worse than before.
“The opportunity arises to throw some other value under the bus…”
That’s a rather ominous phrase. Technology is all about creating new opportunities.
Multipolar traps are currently restrained by physical limitations, excess resources, utility maximization, and coordination.
The ability of office workers to sacrifice joy and beauty for an edge in office politics is bounded by the need to remain sane – less so with modafinil and adderall.
The sheer bounty of hyper-exponential productivity outpaces population growth and allows for many things – so long as the population is limited to naturally-evolved human bodies.
The ability of companies to cut all corners in their products is bounded by customers’ desire for a good product – or at least the companies’ ability to convince them that a product is good.
Coordination is what’s left. And technology has the potential to seriously improve coordination efforts. People can use the Internet to get in touch with one another, launch political movements, and fracture off into subcommunities.
Or they can come up with some brilliant trick to make coordination impossible.
When watching developments in cryptocurrency, one waivers between wanting to praise these inventors as bold libertarian heroes to wanting to drag them in front of a blackboard and making them write a hundred times “I WILL NOT CALL UP THAT WHICH I CANNOT PUT DOWN”
Absent an extraordinary effort, these opportunities end one of two places.
It can end with a superintelligence optimizing for some random thing (classically paper clips) because we weren’t smart enough to channel its optimization efforts the right way. This is the ultimate trap, the trap that catches the universe. Everything except the one thing being maximized is destroyed utterly in pursuit of the single goal, including all the human values.
Or it can end with malthusian competition between emulated humans that can copy themselves and edit their own source code as desired. Their total self-control can wipe out even the desire for human values in their all-consuming contest.
But even after we have thrown away science, art, love, and philosophy, there’s still one thing left to lose, one final sacrifice Moloch might demand of us. Bostrom again:
We could thus imagine, as an extreme case, a technologically highly advanced society, containing many complex structures, some of them far more intricate and intelligent than anything that exists on the planet today – a society which nevertheless lacks any type of being that is conscious or whose welfare has moral significance. In a sense, this would be an uninhabited society. It would be a society of economic miracles and technological awesomeness, with nobody there to benefit. A Disneyland with no children.
Nearly everyone who reads that poem wants to know: what is Moloch.
Moloch is exactly what the history books say he is. He is the Carthaginian god of child sacrifice, the fiery furnace into which you can toss your babies in exchange for victory in war.
Carthago Delenda Est
edit