The original blog post this speech is based on can be found here.
The future is inherently unpredictable. The laws of physics may be absolute, but there’s no way to simulate a chaotic system, not when you live inside of it. All we can do is let the future come to pass—though, perhaps, not entirely passively.
You might think I’m talking about AI, and I am, but even outside of AI, even the most near-term progress is terrifyingly difficult to predict.
In the San Francisco Exploratorium, I’m told there’s a gearbox with an astronomically low gear ratio. Worm gears drive standard spur gears which in turn connect to more worm gears, each connection dropping the speed by a factor of 60. A motor spins the first gear at 200 RPM, and the final gear is set to make its first revolution in 13.7 billion years.
This system is trivially easy to predict.
But imagine it worked backwards. In practice, there’d be far too much resistance to turn the twelfth gear in the sequence, and even if you could, it would cause the first gear to exceed the speed of light. But imagine if you could give that twelfth gear just a quarter turn. Can we predict how many rotations the first gear makes?
We’re already imagining something that’s impossible, but even in our imagination, the manufacturing of the gears isn’t perfectly precise. We can’t just take one quarter and multiply it by 60 a bunch of times. Even if we could, if you’re off by a tenth of an arcsecond when you made your initial quarter-turn, you’re now hopelessly lost in your calculations. This is closer to how our world works.
AI may mean that we can’t predict ten years out. But even if we never invent AI, a thousand years? We have no idea what’s coming. The best technologist, the best historian, the best sociologist—even if they were all fused into one mega-expert, they just can’t forecast out that far.
But shouldn’t we make up scenarios anyway? Won’t that help?
I say no. Absolutely not. You’ll still be confused, but now you’ll also be meta-confused. You’ll have a fictional example to generalize from, which might make you forget that you’re still confused.
There are certain broad categories of technology—AI, synthetic biology—that we can reason about. But the specifics? Quantum computers versus holograms versus Ozempic versus flying cars? Prediction errors cascade rapidly.
So… declare bankruptcy. Stop trying. Much easier to not worry about what wild things, good or bad, the future will bring, when you accept that none of the details will be anything like what you imagine. No matter how intense or bizarre your worries are about the unpredictably far future, whether that means a thousand years of normal life or ten of AI super-progress, you can say to yourself, confidently: it will certainly not be like that.
And you’ll be right.
Which doesn’t mean we can’t say anything about the future. The future will be wild. It will not look like the present. Either we will be dead, or we will have the power to ever-more-radically alter the universe, or both.
Probably not in that order.
Our successors, whether they’re human or not, will have that power. They’ll be capable of climbing the Kardashev Scale, using the Sun, and then many suns, and then eventually they’ll hit the limits of the physical universe—and, unless it turns out we’re in a simulation, we can confidently guess that physics will be solved.
But details? A boot stamping on a human face forever? Planets tiled with factory farms? The Matrix?
No. Those aren’t going to happen. It might be worse, and it might be better, but it will be weirder. Inconceivably weird, almost certainly.
But this doesn’t mean we have to lie down and wait to see what happens. We can remain optimistic, and we can still attempt to steer. We shouldn’t steer toward specific detailed futures; that will only mislead us. But we can steer toward properties of futures.
It is good to steer toward averting apocalypses.
It is good to steer toward humanity being able to recover from apocalypses.
It is good to steer toward humans maintaining control of AGI.
And finally, it’s good to steer towards the meta-value of retaining those values that we believe will continue to serve us.
If you see a way to steer the world in those directions, that’s a reason to be optimistic. Like driving though the fog, you can’t know exactly what route you need to take. Advance carefully, reassess, and then keep going. On larger time scales, each small correction you make will echo through history. That’s the best anyone can do.
edit