[A version with links is available on Taymon’s blog]
Our community is ambitious. We aim to do big things, to solve adult problems.
Four years ago, in New York City, in a ceremony much like this one, Ray Arnold, the creator of Secular Solstice, spoke about this. He said:
We have people in this room, right now, who are working on fixing big problems in the medical industry. We have people in this room who are trying to understand and help fix the criminal justice system. We have people in this room who are dedicating their lives to eradicating global poverty. We have people in this room who are literally working to set in motion plans to optimize everything ever. We have people in this room who are working to make sure that the human race doesn’t destroy itself before we have a chance to become the people we really want to be.
And while they aren’t in this room, there are people we know who would be here if they could, who are doing their part to try and solve this whole death problem once and for all.
And I don’t know whether and how well any of us are going to succeed at any of these things, but…
God damn, people. You people are amazing, and even if only one of you made a dent in some of the problems you’re working on, that… that would just be incredible.
Indeed it would. It would be incredible. And I believe that the same is true of the people in this room today, four years later. We recognize that these things matter, that they matter more than most of society recognizes, more even than any of us can really visualize. The utilitarian significance of even the least of those problems is easily in the millions of lives.
But we are also a community of truth seekers. It’s not enough for the stories we tell ourselves about what we’re going to accomplish to be motivating; they have to be accurate. So, what should we expect when we set out to change the world?
Large scale technological and other change in society tends to obey power laws; a small minority of individual cases wind up having the vast majority of the impact. Y Combinator, the most prestigious startup accelerator in the world, has funded about a thousand startups; about three-quarters of their returns have come from just two of these. Making a big profit is a crude proxy for impact through a startup, but if we run with it, that gives us a probability of 0.3%. And that’s one of the higher-risk-higher-reward life paths available to someone today.
I didn’t have time to find more numbers, but as far as I know, they basically all look roughly like this. According to a study by the Financial Times [PDF], 40% of millennials think they’ll have a global impact. In other words, almost 40% of millennials are extremely miscalibrated.
So we see that truly changing the world is a rare event. But that, too, is part of the standard story; because it’s rare, you’ll have to work very hard and be very resourceful to pull it off. But you can still do it if you try, right?
This is the kind of thing that our brains tell us because of availability bias. As Bruce Schneier put it: “We tend to exaggerate [the probability of] spectacular, strange and rare events, and downplay ordinary, familiar and common ones.” He was talking about things like terrorist attacks, but it’s just as true of things like making a major scientific breakthrough. “Stories engage us at a much more visceral level [than data], especially stories that are vivid, exciting or personally involving.”
In Terry Pratchett’s Discworld, “million-to-one chances crop up nine times out of ten”. It’s not any different from other fiction in this regard; it’s just more honest. Nonfiction isn’t much better because of selection bias; you hear about the guy who won the Nobel, and not about the hundreds of thousands of others who set out to win it and failed.
Eliezer Yudkowsky summed up the problem in “A Technical Explanation of Technical Explanations”:
Remember Spock from Star Trek? Spock often says something along the lines of, “Captain, if you steer the Enterprise directly into a black hole, our probability of survival is only 2.837%.” Yet nine times out of ten the Enterprise is not destroyed. The people who write this stuff have no idea what scientists mean by “probability”. They suppose that a probability of 99.9% is something like feeling really sure. They suppose that Spock’s statement expresses the challenge of successfully steering the Enterprise through a black hole, like a video game rated five stars for difficulty. What we mean by “probability” is that if you utter the words “two percent probability” on fifty independent occasions, it better not happen more than once.
(At least, not in the limit.)
There are fewer than fifty of us in this room, and the probability of an individual achieving the kinds of ambitions we’re talking about here is probably lower than two percent.
So that’s the problem: if you want to change the world, the outside view says it’s probably going to be all for nothing. Any attempt to do so in a way that avoids that problem is unlikely to have a large individual impact.
This problem is so omnipresent in any attempt to do something big that it’s common to just ignore it as background noise. We shouldn’t. We seek the truth because avoiding it inevitably leads to failure when dealing with hard problems. If we’re serious about solving them, we have to face this harsh reality head-on.
So what do we do about it?
One option is to give up, and stick to easy things that are likely to work. You can optimize for your own happiness pretty well that way. But this is no solution at all; we have to do the hard things because, as the Comet King said, “Somebody has to and no one else will.” At least, not on the margin.
Another option is unjustified optimism. This seems to be unfortunately common in our community. Some higher-level rationalists have implemented an advanced form of this where they make their monkey brains believe one thing even as they know the truth to be something else. This might work, if you can pull it off.
A third option is to forcibly suppress your monkey brain’s risk-aversion and submit yourself in obedience to the power laws: spend your life doing the risky thing, knowing that it will probably all be for naught, because it’s the right thing to do. If you’re working on more incremental things like malaria nets, this can work out okay. You can be like the man who throws the starfish back into the sea every day; others may say it doesn’t matter because more will always wash up, but it mattered to that one. If you’re aiming for something more all-or-nothing, like getting AGI right, then this may be psychologically harder.
But there’s one other option, and I think this is the one I like best: Embrace the community.
This works because of two reasons. The first is that, when you’re in a community of people with similar goals, you don’t have to do everything yourself. You can clerk the office that handles the funding, or host the convention that summons the family. You can be one of the people who makes the future turn out okay, even if you’re not the one-in-a-million who makes the scientific breakthrough that fixes everything.
The second reason is that our community might actually be awesome enough to beat the outside-view odds.
Four years ago, when Ray spoke those words, it didn’t necessarily look that way. That was around the time I found the community online, though I did not meet most of you until later. To be sure, some interesting things had already happened by then, and maybe the future will look back and see some of them as the beginning of the Great Change (or maybe not). But in those past four years, I’ve been lucky enough to watch us grow up, and start accumulating a track record of changing the world in real ways.
We brought AI risk from a topic argued about by nerds on internet forums to an issue taken seriously by the world’s foremost experts in the relevant fields, by billionaire influencers like Bill Gates and Elon Musk, and, as of last October, by the sitting President of the United States.
We solved the problem of figuring out which charities actually work for rescuing people from global poverty and moved hundreds of millions of dollars to them, and that number’s only going to grow.
With help from allied coalitions, we got giant corporations to dump the worst offenders of factory farming, banned confinement cages right here in Massachusetts, and brought funding and attention to an industry that has invented a hamburger that doesn’t come from a cow, not as some futuristic fantasy but as something that you can eat today if you live in the right city.
Not everything we’ve attempted has yet borne such undeniable fruit, and we’ve had our share of failures. But I think we’ve earned the right to call ourselves a community that’s capable of getting results, and I think there’ll be more soon, including in areas we haven’t even thought of yet.
The law of the universe is that you can’t beat the odds. But conditional on the right things? Yeah, I think you can.
edit