Singularity Summit 2012

Why Now? A Quest in Metaphysics

[Video]

For more transcripts, videos and audio of Singularity Summit talks visit intelligence.org/singularitysummit

Speaker: Jaan Tallinn

Transcriber(s): Ethan Dickinson and Jeremy Miller

Moderator: One more speaker before lunch, that is Jaan Tallinn. He describes himself as a hacker, entrepreneur, investor, and physicist. He grew up in communist Estonia, but his start-ups have repeatedly made international news. In 2000, he helped create the infamous file-sharing platform Kazaa, which according to some sources accounted for more than half of all Internet traffic at its peak. In 2003, he launched Skype, a free online calling system, that logged 115 billion minutes of talk time in just the second quarter of this year.

Today he's a partner at the seed-stage venture capital firm Ambient Sound Investments, a member of the Lifeboat Foundation advisory board, and a member of the Estonian president's academic advisory board. Recently, Jaan has developed a keen interest in the possibility of a technological singularity, has become a major supporter of the Singularity Institute, and has begun to explain the singularity concept in various public fora. Please welcome back to the Singularity Summit, Jaan Tallinn.

[applause]

Jaan Tallinn: Good morning. I'm glad that Robin [Hanson] gently led you to the section of this summit where we're actually talking about the real far out stuff. I'm going to take the baton now and run with it. What Robin was talking about was scenarios that required soft takeoff. I'm going to focus right on hard takeoff and see what might happen.

Anna Solomon, who now is the Executive Director of the Center for Applied Rationality once told me a story about a lottery. She told me that if you win a lottery and the odds of winning were 10 percent or so, the correct reaction is to be glad. However, if the odds of winning were 10 to the power of minus 60, then the correct reaction is to assume that you have misunderstood the rules of the lottery.

In the last few years, I had my mind completely blown to pieces. Not once, but twice. The first time was when I heard about the singularity hypothesis, and understood that we are in seemingly incredible luck to be born in an era where the entire universe might transform as the result of a technological singularity. The second time, much like in the lottery story that Anna told me, was when I actually realized that, wait a minute, the singularity may not be that special moment after all. Not because I misunderstood the nature of the singularity, but because I might have misunderstood the nature and the rules of the universe.

This presentation is my attempt to convey this second mind-blowing event. Trying to cram the necessary ideas into 30 minutes is a tough challenge, so I partnered up with a young and talented Estonian artist, and chose cartoon as our medium. Because this way, we could visualize everything, and on the other hand, perhaps for people who find the ideas a bit too abstract, they will at least get to enjoy the nice artwork.

[laughter]

Jaan: As you can see, the presentation is divided into nine episodes. In the first episode, we meet our hero, Frank – sorry, Fred. It's interesting –

[laughter]

Jaan: – I kept mixing up the names, so I might do that again. It's Fred. We meet him, and let's observe him being hit with the first mind-blowing event, that he's hearing the news about the upcoming singularity.

Fred is a computer programmer in a research institute. He lives in a quiet, boring neighborhood of an average town in a rather unremarkable country, really. The only remarkable thing about the story is that it is fictional. Although Fred has an interest in philosophy, he's drawn to it in the same way he's drawn to crossword puzzles or programming problems. In no way does he believe that philosophy can actually be relevant to his life, or indeed to the real world in general.

One lazy afternoon, we catch Fred watching his news on his ancient but trusty TV set. The news is about a massive data center being built nearby. Since the advent of cloud computing, these babies have been mushrooming all over the place, and they say the percentage of the global electricity budget consumed by computers is soon approaching double digits. The news story is exploring the seemingly unstoppable hunger for computation, and they have asked a famous math professor to join them in the studio in order to present his views on the matter.

According to the professor, the world has seen nothing yet. His so-called "intelligence explosion" theory predicts that the trend of pouring ever-increasing amounts of matter and energy into computers will continue, and eventually morph from an economical phenomenon into an astronomical one. He says that these data centers are harbingers of an entire universe turning into computronium, that is, matter arranged in the most efficient form for computations.

To back his claim up, he charts the exponential growth of computers and points out that the speed at which computers have been evolving is limited by the speed at which humans can think and communicate. The moment we create the first computer that is able to design other computers, humans drop out of the loop and the process explodes. The professor calls such a computer our last invention.

Of course, assuming that the laws of physics don't allow for infinitely powerful computers, such an intelligence explosion has to stop, to level out at some point. Once every free quark has been captured and put to work, the universe will stabilize again around the new equilibrium. This equilibrium will no longer be dominated by stars, planets, and free energy, but by computronium. Therefore, we seem to be witnessing the most important moment since the Big Bang, the moment that the intelligence content of the universe goes from near zero, to 100 percent.

As if this wasn't shocking enough, the professor goes on to explain that there isn't just one outcome of the intelligence explosion, but a great number of different scenarios. Some of them can be wonderful, from our perspective, whereas in others we die, as our bodies are devoured by the expanding cloud of computronium, or something even worse happens. Furthermore, the destiny is ultimately determined by programmers as they shape our last invention.

"You heard it folks. Not only eyewitnessing the last years of the universe as we know it, but if you are a programmer you might be responsible for its ultimate fate."

Because Fred's company is developing the next generation of computers, these words strike home. It has never occurred to Fred that his work might blow up the entire universe.

[laughter]

Jaan: He decides to find that professor and debug his theory.

Before heading out though, Fred stops to pack. There are at least three important things every aspiring philosopher must bring to metaphysical explorations. First, empiricism. You should always keep an eye on the world, and not invent theories that don't match what you see. Second, logic. Hypotheses whose internals are twisted and don't align should be discarded. And finally, skepticism. No hypothesis should consume you to the point where you stop considering the alternatives.

Having equipped himself in this manner, Fred is ready to hit the road.

Next episode, Anthropics. In this episode, I'll be using so-called "anthropic principle" to explain what it means to be in a surprising situation, such as Fred suddenly finding himself responsible for the fate of the universe.

Since Fred doesn't own a car, and the university is too far to bike, he decides to hitchhike. Luckily, it doesn't take long before a kindly driver picks him up. Somewhat less fortunately though, they end up in an awful traffic jam moving slower than walking pace. Fred decides against walking though, because the driver turns out to be an interesting fellow who's well-versed in philosophy.

For example, he tells Fred about a paper by Oxford philosopher Nick Bostrom, entitled "The Cars in the Next Lane Really Do Go Faster." "You see," the driver explains, "slow lanes contain more people, so it's more probable that you'll end up in one. Even worse, since travel in slow lanes takes longer, they dominate our memories. Therefore, if you speed up your life into series of moments, a personal photo album of experiences, there will be many more pictures of driving in a slower lane than in a faster one. This illustrates the so-called 'anthropic principle,' which really states the obvious. On average, you have a greater number of experiences that are more abundant."

Having heard that, Fred's thoughts drift back to the news he just received. What are the odds of finding yourself responsible for the fate of the universe? Isn't that like driving in one of the least populated lanes of the highway named "Life, the Universe, and Everything?" But is it? Could it be that Fred's experience is not that unique after all, because in the grand library of human experiences, there's a large section called "Shaping the Singularity?"

Algorithm. In this episode, my goal is to explain how everything in the universe seems to be an algorithm, that is, step-by-step procedure for calculations.

By the time traffic clears, it's already dark, so Fred decides to find a place to stay for the night. He hops out near a pub to grab some dinner and ask around about cheap hotels in the area. In the pub, Fred gets challenged to a game of pool by a friendly looking fellow who happens to know a thing or two about physics. The pool player quips that the most important contribution that pool has made to humanity is not the countless hours filled with near-misses and beer, but the role it has played in physics.

"About 200 years ago," he explains, "an English scientist named John Dalton proposed the billiard ball model of atoms. Even though modern physics has replaced the model with quantum mechanics, it's still sufficient in many scenarios. According to the billiard ball model, atoms are tiny indivisible particles, some of which vibrate in place and some of which bounce around like the balls on a pool table. Very importantly, once you know the initial positions and velocities of the atoms, you can calculate the exact positions they will end up in.

"Since macroscopic objects such as the actual billiard balls are composed of atoms, you can determine their paths by adding up the movements of their atoms. You can keep going on with this, with bigger and bigger objects. Of course, since the vast majority of atom movements cancel out, you can use much simpler statistical models, such as Kepler's laws for planetary movement, and Newton's laws for the movement of billiard balls."

Although people's heads look somewhat similar to billiard balls, they clearly move along much more complex trajectories. The trajectories of these heads are mostly determined by the brains inside them. Still, it's unlikely that individual atoms are important in determining people's thoughts, experiences, and behavior, so there must be some way of aggregating the atomic movements inside the brains.

Regardless of what the actual structure is, as long as it emerges from deterministic interactions of underlying atoms, it can be viewed as an algorithm. Indeed, one way to interpret what goes on in people's heads is to say that it is a process of translating the electrical input signal from the senses to the electrical output signals to the muscles. For all practical purposes, we are the algorithm that our brains implement. If we were implemented on top of some other hardware, we probably would not feel any difference, just like Robin was explaining in the last presentation.

Now is a good moment to apply the first tool that Fred brought along with him, the empiricism. Is our world deterministic and computable? What about quantum uncertainty? Well, we have yet to witness a physical process that we can't, at least in principle, model in a computer. Also, quantum effects only introduce randomness that is pure, and doesn't depend on anything. Such randomness can easily be included in computer models. Given the evidence we have today, it seems likely that we do live in a world that is indeed fully computable.

Copies. How does it feel to exist in several places at once?

When Fred asks the pool-player about places to stay for the night, he gets a surprising suggestion. The pool guy mentions that he works at an institute investigating sleep cycles, and Fred is welcome to volunteer as a test subject. There's a catch though. A scientist at the institute explains that they monitor people for two consecutive nights, and Fred wouldn't be able to leave until the day after tomorrow. Hmm. Although Fred didn't plan on staying in town for an entire day, he knows that he could use the time to read and think. Plus, being short on cash as usual, he finds the price just right, it's free.

Fred agrees, and because it's late, he soon falls asleep. The next day, nothing unusual happens. Fred catches up on some reading, snacks on some food he was given, and ponders about the intelligence explosion, anthropics, and being an algorithm. As the night draws in, he's ready for bed again.

Next morning. While debriefing Fred, the scientist reveals what the experiment was really about. "You see," he says, "we built a matter replicator. As you went to sleep the first night, we used it to make an identical copy of you, of your room, and everything in it, including you. After you went to sleep the second night, the same machine was used in reverse to merge the two rooms again.

"So we're not testing sleeping patterns after all. Instead, we're interested in whether people can tell they spent the day as two copies. So far, nobody's noticed."

[laughter]

Jaan: After the initial shock of such an unauthorized copy event wears off, Fred decides to simply leave.

[laughter]

Jaan: Since indeed, even combing his memory fails to reveal anything special about his day in the room. He's not even sure if the scientist was telling the truth. Anyway, the weather's nice, and Fred is ready to hit the road again.

While Fred continues his journey, let's go back to the experiment for a second. Imagine Fred suddenly asking himself, "Am I in the left room or the right one?" As you can see, this question is absurd, because Fred is simultaneously in both rooms while simply being unaware of it. Similarly, if there were many copies of the Earth scattered across the universe, we wouldn't feel it either, yet it would be absurd to ask which particular one are we seeing at any given moment. If you can understand this at a gut level, congratulations, you're ready for the next episode.

Multiverse.

[laughter]

Jaan: Despite the nice weather, Fred finds it difficult to get a ride. At least he finds a fellow hitchhiker to keep him company. This guy happens to be a cosmologist, so as they stride along, he explains to Fred the fascinating physical phenomenon called cosmological background radiation. It turns out that if you use radio telescopes to look far enough, our gaze finds this background wallpaper that dates back to a time when the universe was not yet transparent. The most interesting thing about this wallpaper is that it is in just the last couple of decades physicists have analyzed the patterns on it, and discovered that they’re hallmarks of a very large, if not infinite universe.

What we used to think was the entire universe, we now know is just one tiny cell of it, called the Hubble volume. Furthermore, the physical characteristics of our cell appear extremely finely tuned to support the emergence of stars, planets, and ultimately life, which means that there was either some process that turned those metaphorical knobs, or the universe is sufficiently large and diverse to contain cells with all possible configurations. If you go with the latter hypothesis, this is what we get. A huge multiverse with individual universes sprinkled around it.

Let's pause the story for a moment, and review what we have learned alongside Fred. If the cosmologists are right, and we are indeed inhabiting a cell in a vast multiverse, what can we infer from it? Well, as I hinted at the end of the last episode, if the multiverse is big enough, then without intuitively realizing it, we don't just inhabit one cell in it, but a whole lot of them. Just like Fred unknowingly spent the day spread between parallel rooms, we might be spread between parallel universes. Moreover, just as it was meaningless to ask whether Fred was in the left room or in the right one, it would be meaningless to ask which universe we are in.

One way to try to appreciate this would be to look at your hand, and imagine that you're not really looking at one hand, but at a multitude of them layered on top of each other. Obviously, the same applies to all experiences. They are produced in many places in the multiverse. Some of them, like the experience of travelling in a slower lane, are more frequent than the others.

What if we took all the collective human experience, that is, pictures in the photo albums of life, of all the people in the past, present, and future, and sorted them neatly on an imaginary highway, like this. Then according to the anthropic principle, you would almost always find your experiences in a traffic jam there, as it was the case with the real highway. In other words, you should expect to live a life that is typical, a life that the multiverse produces over and over and over again, in abundant quantities.

There's a weak point in the logic of this argument, so let's examine it in detail. You see, if the multiverse is infinite, then we can say that the density of the experiences in those lanes doesn't really matter, because you can pair up the pictures in adjacent lanes, one by one, and thus conclude that there is an equal number of all of them in all lanes.

Of course, there are counterarguments, and the most powerful one is just empirical. For example, since we don't experience winning the lottery as often as we experience not winning it, then either the multiverse is not infinite, or the frequency must make a difference, even in an infinite one.

If the frequency matters, then one way of addressing the surprise of finding yourself in a seemingly exotic moment in the history of the universe, that is, the moments leading up to the singularity, is to come up with a hypothesis in which these moments are not exotic after all, because there are processes in the multiverse that produce such experiences in abundant quantities.

One place to look for such processes are simulations. So let's go back to our story and introduce simulations, and not just any simulations, but detailed and historically accurate simulations of entire universes.

It took awhile for Fred, and his friend the cosmologist, to reach their respective rides, so when Fred arrives in the next town, it's already afternoon and time for lunch. Fred walks into a restaurant, and he's given the lunch menu. After chatting with the waiter for a moment, however, it turns out that the people running the restaurant are also deeply interested in philosophy, and have prepared a special menu for customers like Fred.

[laughter]

Jaan: The waiter explains that this menu is inspired by another famous paper by Nick Bostrom, in which he laid out three propositions in the so-called "simulation argument." Much like in a restaurant with a set menu, you have to pick one option because, or so the paper argues, at least one of them must be true. The options are, first the human species is very likely to become extinct before reaching the posthuman stage. Second, any posthuman civilization is extremely unlikely to run a significant number of historical simulations. Or three, we are almost certainly living in a computer simulation.

To illustrate the argument, it is logically inconsistent to believe that you are living in a physical universe capable of eventually producing a great number of detailed historical simulations, simulations that you can't distinguish from reality. Instead, because simulated experiences greatly outnumber the real ones, you should bet on living in a simulation.

Let's examine the options in detail, given our multiverse assumption. The first option means that practically all humans all over the multiverse will become extinct before they gain the ability to run detailed simulations. This seems unlikely, not to mention overly pessimistic, to me.

The second option means that, even though we, or our biological or artificial descendants, will gain the ability to run detailed simulations, we will suddenly and universally lose all interest in the history of the universe. Again, I find this hard to believe, since the current trend in universe simulations is definitely up. For example, this beautiful simulation was just recently released by Harvard scientists, whereas in other recent news, they talked about the third fastest supercomputer on the planet being set up to run universe simulations. Not to mention that in a computable universe, even superintelligence might have no choice but to run simulations in order to make predictions and carry out planning.

Let's see how the third option would play out in the multiverse. If you assume that there are supercomputers out there producing historical simulations, some of which contain humans, then whatever we encounter or experience in the multiverse, for anthropic purposes, that is, when we are trying to figure out how typical our experiences are, we should count both the physical experiences and virtual ones when we reconstruct the lanes on the anthropic highway, like this. Furthermore, the greater the number of singularity simulations out there compared to other historical simulations, the more packed the pre-singularity lanes become, as compared to others, and therefore the less surprised we should be to find ourselves experiencing this seemingly exotic and critical moment in the 21st century.

In the final episode, I'm going to go ahead and present one such possible scenario, where the virtual, that is, simulated 21st centuries are really abundant in the multiverse. To do this, let's peek into another cell in the multiverse that contains a post-singularity world. It's a world where the intelligence explosion has run its course, and the universe has stabilized near the ceiling dictated by the laws of physics. Importantly, according to our model, this should be a very typical moment in the corners of the multiverse where intelligent species can evolve in the first place. Even according to extremely conservative estimates, the post-singularity era should be at least a trillion, trillion times longer than the pre-singularity period.

In this place exists one or more superintelligences. What could they be thinking about? While it's impossible for us to know exactly, of course, we can certainly speculate about the types of challenges they might be interested in solving. For example, there are good arguments for assuming that the superintelligences are interested in finding each other and communicating. There's an interesting problem that a fast-thinking superintelligence must have in a physical universe. From its perspective, the speed of light is really slow. Not to mention that the light signals are not able to reach other the superintelligences in distant parts of the multiverse. On top of all that, travelling around in order to find someone to talk to must be an atrocious waste of energy. So what's a poor superintelligence to do?

Remember that in a computable universe, everything, including the superintelligences themselves, are produced by a natural computation process of atoms bouncing around like billiard balls on a table. If you look at it that way, when one superintelligence is trying to communicate with another, all it is doing is examining the results of, and interacting with, a naturally occurring computation. Therefore, a pragmatic alternative to physical travel and communication would be to create interactive simulated copies of other superintelligences. If two superintelligences interact with accurate simulations of each other, then the situation could be indistinguishable from communication between the actual physical copies.

However, we now have another problem. How would a superintelligence know which process to simulate? One way to solve this problem might be to perform a search of the so-called "mind space." That is, you simulate many different physically possible superintelligences, one after another, and then pick the ones that seem most interesting and useful as communication partners.

Of course, assuming that the computational resources are limited, you don't want to compute the same thing twice. If multiple superintelligences have a shared past, you want to simulate that part only once. In other words, you'd perform what programmers call a tree search. That is, you first simulate one superintelligence, then back up one step, compute a slightly different superintelligence, back up two steps, and so on.

If you remember the Harvard simulation, the equivalent there would be to first create one simulation video, then erase one last frame and render it slightly differently. Then, erase two last frames and render them differently, et cetera. A very interesting property of such a tree search is that if you compare two lines corresponding to different times in the simulated evolution, the number of simulated moments rises exponentially, as the simulation nears the singularity, that is, gets closer to the creation of a superintelligence.

In other words, in this particular type of simulation there’s a huge traffic jam in the lane of the anthropic highway that contains pre-singularity moments. Therefore, given this world model, our hero Fred should no longer be surprised to find himself in such special times. Just like the cars in our lane really do go slower, there could well be lots and lots of 21st centuries out there in the multiverse.  

Epilogue. To wrap things up let’s review the hypothesis that Fred’s metaphysical journey helped us to build. We started by observing that living and playing a role in the 21st century seems to be a mind-boggling privilege, because the coming singularity might be the biggest event in the past and future history of the universe. Then we combined the computable multiverse hypothesis with the simulation argument, to arrive at the conclusion that in order to determine how special our century really is, we need to count both the physical and virtual instantiations of it.

We further talked about the motivations of post-singularity superintelligences, speculating that they might want to use simulations as a way to get in touch with each other. Finally we analyzed a particular simulation scenario in which superintelligences are searching for one another in the so called mind space, and found that, indeed, this search should generate a large number of virtual moments near the singularity, thus reducing our surprise in finding ourselves in one.  

There's an important point still to be made. Even if we find it likely that we are witnessing a simulation, that is not a license to go crazy and start treating the world as if it didn't exist. Even in the world model I just presented, there are two good reasons to keep behaving as if we were in an actual, physical universe.

The first is that our actions still have consequences, because we control the results of the simulation. Assuming that the simulators are interested in the results, and why would you run a simulation if you didn't care about the results, we can influence their world this way. The second reason for behaving as if we were in a physical universe was once really put well by Michael Vassar, he said that, "If you think you are Napoleon, yet you know that most people who believe that are in a mental institution, you should still behave as if you were the real Napoleon, because on the off chance that you are, your actions matter a lot."

[laughter]

Jaan: Respectively, some of our copies really are in the real universe, and their actions in their respective Hubble volumes have real and unsupervised cosmic consequences. It would be crazy to abandon them by starting to behave as if we were in a simulation that was about to end. After all, in a deterministic world, our physical copies have no added choice than to mirror everything we do.

Finally, and most importantly, as we say goodbye to Fred and wish him good luck in figuring all this out, we should observe that he has always worn his third tool for metaphysics, the skepticism. Everything I talked about was just a hypothesis, and I'm sure there will be more convincing ones in the future. For example, while this talk explored ways in which we could be spread out in space, it would be interesting to think about ways how we could also be spread out in time.

Still, I hope I managed to give you a taste of how fascinating metaphysics can be if you are truly curious about the world around you. It exposes you to completely new ways of pouring fundamental and new meaning into everyday experiences, experiences that seem routine and uninteresting at first. I'm sure that, just as Fred's metaphysical journey will continue in his fictional world, so will mine in the real world. Well, both the real and simulated worlds, I should say perhaps.

Finally, I'd like to thank the amazing people – most of them are in this room – for whose work and personal guidance this talk was based on, and here are just a few of them [slide states: "Acknowledgements: Michael Vassar, Eliezer Yudkowsky, Nick Bostrom, Max Tegmark, Carl Shulman, Justin Shovelain."]. Thank you.

[applause]

Moderator: Jaan Tallinn. Incredible presentation.