(2024-03-24) Michael Nielsen On Collaboration Quantum Computing And Civilizations Fragility

Michael Nielsen on Collaboration, Quantum Computing, and Civilization's Fragility (Ep. 213) (with Tyler Cowen).

COWEN: Why is the universe beautiful to human eyes? Is it natural selection?

NIELSEN: I have no idea. Selection is a very attractive kind of idea. I’m inclined to think not, just instinctively. I don’t know. Why are there simple rules? Why do we have simple rules governing the universe? In fact, why is simplicity an arguable truth, somehow associated to beauty?

COWEN: What’s the most beautiful image of the universe?

NIELSEN: We have a sequence of improved images of the 3-degree microwave background. I don’t know, is it the most beautiful? It’s maybe the most extraordinary. It really is a photograph of the universe as a whole. You can look at that, and it says something about structure out in creation.

COWEN: Why do the sounds of the universe not appeal to us so much?

COWEN: Now, you’ve written that in the first half of your life, you typically were the youngest person in your circle and that in the second half of your life, which is probably now, you’re typically the eldest person in your circle. How would you model that as a claim about you?

The good reason in the first half was, so much of the work I was doing was kind of new fields of science, and those tend to be dominated essentially, for almost sunk-cost reasons — people who don’t have any sunk costs tend to be younger. They go into these fields.

Now, maybe it’s just because I found San Francisco, and it’s such an interesting cultural institution or achievement of civilization. We’ve got this amplifier for 25-year-olds that lets them make dreams in the world. That’s, for me, anyway, for a person with my personality, very attractive for many of the same reasons. (scene)

What’s your theory of your own collaborators?

They’re all extremely open to experience. They’re all extremely curious. They’re all extremely parasocial. They’re all extremely ambitious. They’re all extremely imaginative.

I will select for somebody who has at least one very strong skill which I do not have. That’s enough diversity from my point of view.

what’s the influence of Simone Weil on you?

She’s one of the maybe best examples of sincerity that I know of. The fact that she wrote what she wrote for herself.

Everything that she did, she did at 500 miles an hour. She’s remarkable. She’s a very extreme type of a human being in a way that I find very interesting.

COWEN: You and I — we’re both fans of Olaf Stapledon, who wrote the dual classics Last and First Men and Star Maker. What’s the biggest analytical mistake he made in those narratives?

NIELSEN: He was both, certainly to some degree, a socialist and certainly a pacifist.

Though in World War II he switched out of pacifism.

he has this very long view of history, much longer than most people who say they have a long view of history. I think he sees some of his pacifism in that light.

I worry that he too quickly assumes collective-action problems are solved, which is close to your answer. He thinks the League of Nations can be effective for a long period of time, which I suspect was not really contingently possible

And he has this Hegelian sense. What Hegel would call a national spirit, for him is a civilizational or certain stage of man’s spirit that so shapes how people think.

your comment about collective-action problems seems much more to the heart of it. I think he didn’t really believe in them, or actually understand just how difficult they are to solve, how difficult it is to supply public goods and these kinds of things

NIELSEN: I’m just saying there’s a gap between aspiration and what actually happened with the League, and then later, with the United Nations. I think you had the hopes, and then you had what actually happened, and there’s a very large gap. Although, of course, as prototypes over the next few centuries, maybe these things are terrific.

COWEN: I have a very concrete question for you, and this is to clear up a confusion of mine. I’ve asked experts in quantum computing, “What’s the status of quantum computing right now?”

I stopped. I worked on it from 1992 to 2007

It’s very impressive progress each year. It is an extremely difficult problem. It’s not solved. There’s no way. It’s definitely not solved

Will there be quantum money?

idea of Stephen Wiesner, which he called quantum money. It’s meant to be uncounterfeitable. I don’t know.

Do you think that leads to a mass privatization of a lot of social activity?

work on nuclear weapons was actually nationalized in, I think, 1948,

certainly, it’s a very Neil Postman point of view. You have this, basically, almost Larry Lessig “Code is Law.” You just keep building more and more governance infrastructure into the technology, and you’re moving it out of the hands of the population and into the technology.

COWEN: Is the status of linear algebra rising?

NIELSEN: [laughs] That’s a great question. It probably has, yes.

COWEN: It’s prominent in quantum, right? It’s prominent in AI. ([GenAI])

NIELSEN: Google is built on matrix multiplication. It’s prominent for a lot of reasons.

NIELSEN: I never quite know what your status questions mean, Tyler. I don’t know what it means for something to rise in status.

NIELSEN: Do you mean it’s going to have more money go to it, more power go to it, more glamor go to it? Are people going to regard this as sexy or what?

COWEN: All of those, but you would revise your ideas about the fundamental nature of the universe. Just like our current understanding of quantum mechanics, it might be incorrect, but at least in the short run, it seems like probability theory is somehow more important than Einstein might have thought. As you know, he famously asserted God is not playing dice with the universe, perhaps incorrectly.

NIELSEN: The people who remake this understanding are very good at ignoring status.

*If you’re searching for comparative advantage in doing creative work, you want to know where status is, but mostly so you can avoid it.

COWEN: Yes, absolutely. “Be short status,” as Peter Thiel likes to say*

COWEN: Is there any chance Roger Penrose is right and the human brain is some kind of quantum computer?

NIELSEN: I would love it if he was right. I think the answer, unfortunately, is not really, no.

COWEN: How are we going to make progress toward a theory of quantum gravity, a general understanding of everything? We seem to be stuck. Many people hate string theory. Many people hate Hugh Everett’s many worlds (multiverse).

The problem in some ways in physics has been that the fundamental theories have been just too successful for the last 50 years. Yes, you’re right again, it’s very attractive for a few years, but over 50 or 60 years, it’s terrible. I think that’s certainly part of the issue with quantum gravity.

*COWEN: Does it bother you that so many people hate string theory, think it’s now low status, think it’s not aesthetic, think it’s unintuitive? Does that carry any weight with you? Or do you want to be short status again on this one?

NIELSEN: There’s the question of inside and outside the profession. There’s also the question of inside and outside the group of people who know something*

I think I’m more interested in the question of just how much diversity of opinion is there? Are people pursuing lots of different ideas?

I find mathematicians — when I talk to them, it’s such a healthy culture because each mathematician is really, well, a lot of them are very unique. They’ve got their own particular path and their set of beliefs.

Theoretical physics often seems just a little bit more monotone. (monoculture)

COWEN: What makes for physicists who age well?

NIELSEN: [laughs] I spent quite a bit of time thinking about this, actually, in my late 20s

As far as I could tell, having younger mentors was really the key thing.

What I think is probably the case — it’s almost a network effect. Basically, if there is some slight downhill slide, and most of your friends are not quite at the edge anymore, that’s going to infect you. If you still have mentors who are 25, 28, extremely active, and they’re active in the latest ways, you get to partake of the positive network effects.

I think that’s why it’s very important not to have people who work for you — lots of 70-year-old physicists have 23-year-old students — but actually to have 23-year-olds, 28-year-olds who you really learn from and you regard as your mentors.

COWEN: You have a lot of younger mentors.

NIELSEN: Yes.

COWEN: You’re known famously for being very nice, right?

NIELSEN: I am extremely disagreeable, but in a polite way, I hope, and a kind way, hopefully.

COWEN: [laughs] So you are very nice, then.

NIELSEN: People often find people who are disagreeable actually quite difficult, but if you look at all of the younger mentors I’ve had in the last, say, seven or eight years, they’re all people who enjoy disagreement.

On belief in God

COWEN: As the years pass, do you think your probability for God existing is going up or down?

NIELSEN: Oh. That hasn’t changed since I was seven years old.

COWEN: That’s weird that it hasn’t changed, right? You’ve learned a lot. Why shouldn’t it change in whichever direction?

I’ve learned a lot. Gosh, it’s really annoying that it hasn’t impacted that question more.

NIELSEN: I think my appreciation for God has gone way up. I appreciate the construction of the religions far more than I did. What notions of God do for people — I’m vastly more appreciative. My probability, I don’t think, has really changed.

COWEN: What about evolutionary frameworks where there’s some Darwinian process, some kinds of universes within a broader metaverse? They reproduce at greater frequencies. That shapes the properties of what we live in. Isn’t that a substitute for a good explanation, and that rises in probability just a bit?

NIELSEN: No.

COWEN: Why not?

NIELSEN: You’re relabeling what you mean by universe. If you just use a term that means everything that is, then that hasn’t changed

On open science

why do some fields have preprint platforms and others not? Is there an actual regularity, or is that random and path-dependent?

NIELSEN: I think a lot of that probably comes down to individuals

NIELSEN: It just seems like it’s just a cultural problem. It turns out it’s a little bit more like fashion, or something like that.

Some of their ideas were done by Paul Ginsparg when he was starting up the preprint server. He went very narrow. He didn’t try and solve the problem all across all the fields. He went and twisted the arms, at some level, of some very high-status, high-profile physicists to say, “I would like you to use this server, so just send me your best paper.”

COWEN: Why do so many crummy journals survive? They can be quite expensive

NIELSEN: There’s a complicated set of things going on. One is that libraries pay, not individuals, usually, for subscriptions. The person getting the utility is not the same as the person making the buying decision. That’s always bad.

There’s also the fact that since the 1990s and the rise of the internet, we get economies of scale. Libraries don’t subscribe to individual journals for the most part. They subscribe to all these giant bundles. (bundling)

COWEN: Right now, how high are the marginal returns to greater openness?

NIELSEN: Openness per se — that’s a very weak word. You need to be much more specific. If you look at, say, the culture around Jupyter Notebooks in machine learning, I think having those very openly available and widely available really has driven a lot of progress.

It’s not the same as making your journal article openly available. It’s a much more active kind of a material.

Openness is always with respect to what platform, with respect to what set of institutions, with respect to what set of norms. With the current sets of norms and institutions that we have, it buys you a little bit. I don’t think it buys you that much, but the norms and institutions — they’re going to change in response. The way in which people work will change in response — the Jupyter Notebook example I gave is, I think, a good example of that.

On progress in science

COWEN: You have a well-known article with Patrick Collison on progress in science slowing down, and it’s published at a point, say, right before mRNA vaccines, right before GPT-4, other developments

NIELSEN: Yes, it’s amusing to think about different points of time at which you could try and write the same article.

think AI — it’s not yet 100 percent clear, but I think it’s very likely to drive a lot of scientific progress over the next few years. That’s just a case of, we’re moving so much of our cognition, and eventually, also the actuators, the way we operate in the world, out into these devices, where all of a sudden it becomes much more mutable and hopefully improvable.

On civilizational fragility

COWEN: Now, you’re working on what I think you call the vulnerable world hypothesis. Yes? (existential threat)

NIELSEN: Nick Bostrom — that’s his term.

nuclear weapons are terrible, but they’re not civilization threatening directly.

COWEN: Sure, but if enough of these go off, life as we know it is over.

COWEN: It’d be like the fall of the Roman Empire, maybe worse.

COWEN: In what year do you think that the cost will be low enough that that happens?

NIELSEN: At this point, I don’t have a good sense. I suppose I’m actually more concerned about other threats. Biosafety is the obvious thing. Computer security.

NIELSEN: I think getting off planet Earth and establishing a civilization elsewhere is very, very important. (space migration)

NIELSEN: Oh, we’re talking about a million people

*NIELSEN: Sure. I think it’s not going to be self — what’s the right term? It’s not going to be an autarky, or whatever the right term is. It’s not going to be completely self —

COWEN: Sustaining, yes.*

but at a million people, it has a lot of the civilizational infrastructure.

COWEN: Economics aside, what’s the main scientific constraint that has to be overcome? Is it gravity? Is it effects of radiation on the human body? Is it water?

NIELSEN: To some extent, we’re not going to know until we go.

COWEN: You’re making me think civilization as we know it won’t last a thousand years.

NIELSEN: No. I also have a lot of, I guess, faith in long-run economic growth. Basically, at the moment, for us to go to Mars is very, very, very expensive given the return, or to establish a permanent human presence in space. If we continue to have economic growth, the relative cost is just going to keep going down.

*COWEN: Does a vulnerable world mean near-universal surveillance?

NIELSEN: Unfortunately, I think probably yes.*

COWEN: Doesn’t that then become the great point of vulnerability?

NIELSEN: Oh, absolutely. Yes.

COWEN: If you could ban universal surveillance from here on out forever, would you press that button?

NIELSEN: No.

COWEN: Maybe it needs opaqueness as well. It’s this optimal mix of surveillance and opaqueness, that you actually have some latitude to break certain laws, to misbehave, that keeps the system stable, limits the abuses of power

NIELSEN: There has to be some Madisonian point of view where you’re bringing the powerful institutions into conflict with each other

but it seems like the organizations which do the surveillance are too powerful. They don’t have strong enough checks on them. (BigTech)

COWEN: In a strong AI future, where do the economies of scale lie? Say, within your lifetime, not 500 years from now. There’ll be one company, there’ll be one chip maker.

Where do you see the monopoly power evolving? Because it’s essential, I think, to predictions of the model.

NIELSEN: When I talk to people who know much more than I do, they all point at ASML as having been surprisingly hard to duplicate.

COWEN: So we should be long Netherlands?

NIELSEN: Probably, yes.

NIELSEN: It would be an amazing conclusion. A return to the Dutch Renaissance.

COWEN: Yes. What do you think of the Netherlands as a country?

NIELSEN: Oh, I love it.

People view it as being a little orderly. I’ve heard people say it’s dull, but I think some of the most interesting experiences of my life were there. I went to a — it was like a jamboree in a field. What was it? Five days long, called Hacking at Random, in 2011,

It was really hacker culture. It was just intellectually wild in the most interesting way. It grew out of Dutch hacker culture. That spirit of the Dutch Renaissance is still visible

COWEN: How would you describe the quality of those conversations? What were they like? Different than what’s in San Francisco?

NIELSEN: Oh, yes.

COWEN: How are they different?

NIELSEN: They’re not captured by capital to the same extent. Conversations in San Francisco, particularly with younger people, tend to be extremely idealistic and often very pro-social. But then later, there’s this negotiation that goes on, where they need access to capital to make their dreams come true. A certain amount of compromise is made, although they also often keep a lot of their original pro-social and idealistic character. In the Netherlands, in those particular events, there have been less of that. They also have less access to capital.

On non-legibility

COWEN: Let’s say we all had better memories. How big is the social gain there? Is there any social gain at all?

There’s a long sequence of papers trying to elucidate the connection between deep practical expertise and the role of memory. I suppose, most famously, people like Herbert Simon and Anders Ericsson and people like this have tried to understand what relationship, if any. It’s a little bit murky.

the initial impact somehow is created or defined by the later act of memorization. People who take trips and until they photograph something, they don’t feel they’ve seen it.

NIELSEN: Yes. They very likely didn’t see it.

COWEN: Perhaps, yes.

NIELSEN: Certainly, it’s part of the reason why I take photos. I will look more closely. It’s part of the reason I will take notes (note-taking). It is part of the reason why I do spaced repetition (SRS). It provides me with another way of paying attention to the world.

Any general-purpose strategy you have which will cause you to pay attention to the world is incredibly valuable, so I collect things like that.

COWEN: In some other ways, you’re a fan of non-legibility, as am I at other margins. There’s some tension because when you take the photo, when you remember something, when you write it down, there’s less legibility.

NIELSEN: There’s no tension at all there. You’re constantly expanding the legible, and when you do that, there’s this penumbra of illegibility that surrounds it

somebody once said to me that travel is the only education, and it’s really stayed with me as expressing some deep truth. I think mostly just the world is so incredibly deep.

the Midway Aircraft Carrier... Why is that so interesting?

For about 10 years, it was probably the most dangerous object in the world. It carried nuclear weapons, I believe, having 5,000 people on it.

you have 4, 4 ½ thousand people, all completely dedicated to a single purpose. They all care an enormous amount about this purpose. They don’t suffer like large organizations that have all kinds of bloat and all kinds of problems. So many of those problems are gone away there.

You have incredible belief in this purpose. It actually is a high purpose in their case. Talking to some of the sailors — they felt very strongly they were protecting civilization that they cared about a great deal

COWEN: Here’s something you once wrote, and I quote, “The great talent identifiers I know or know of all seem very idiosyncratic. They’re rather like Michelin chefs.”

NIELSEN: Actually, it’s just because the boundaries of knowledge at any given time tend to be idiosyncratic almost by definition

*Thomas Schelling, I think, was your PhD supervisor?

COWEN: Yes.

NIELSEN: When you read Schelling, you realize that some of the things he did, he did very well. It must have been remarkable to talk to him, but he’s a little bit illegible —*

COWEN: That’s right.

COWEN: Here’s a question you wanted me to ask you. “You initially were skeptical of Emergent Ventures, but you’ve changed your mind and become enthusiastic about it.

NIELSEN: The biggest single thing, it’s just empirical. I’ve met a bunch of EV grantees. I’ve encouraged a bunch of people to apply.

I was talking to somebody — I’ve actually forgotten who it is — they clearly had some socialist, quite anti-libertarian ideas, and you’ve given them a grant.

COWEN: Here’s something else you wrote, and I quote. “I internalized a lot of Ivan Illich, John Holt, A.S. Neill, and Paulo Freire as a kid.” What did you mean? You were talking, I think, in the context of agency, but how did that shape you?

NIELSEN: At the time, as a 12- or 13-year-old, it mostly probably made me insufferable to my parents because I hated school already. (schooling)

I think over the long term, the most important of those was, for many years I would have said Illich. I’d maybe still say Illich. Basically, his point — it’s about the question of what’s the relationship between human beings and institutions, and how paternalistic are those institutions towards the humans. In Deschooling Society, he really makes the point that, in fact, schools do not treat children as human to some extent at all.

Just thinking about that relationship, what relationship should our institutions have to individuals, was very, very important to me as a teenager and through my entire life.

why you didn’t join OpenAI in 2015?

NIELSEN: [laughs] I suppose I did consider going as they were getting started

If you wake up in the morning, and then it turns out that some institution is mad keen to pay you to do whatever it is you’re doing, you should actually think about whether or not you’re in the right line of business. I tend to think, unfortunately for creative work, there’s an anticorrelation.

COWEN: Your Twitter biography says, “Searching for the numinous.” What does that mean with respect to you?

NIELSEN: Just trying to find the deepest possible experiences in the world, in people and things and ideas and places.