Why Science Slowed Down

I'm here again with Samo Burja, who is going to be talking about science. You were saying that you think science is slowing down. Can you say more about that?

In the nineteenth and twentieth centuries there were numerous fields of science advancing at a very rapid pace. Theory of evolution, theory of electromagnetism, eventually things like relativity. There were technological advances as well, but I am differentiating here between science and technology. Purely the human body of knowledge, and the rigorous theories built on top of that, were advancing in a multitude of domains.

Today I think we mostly have a scientific stagnation in biology, in chemistry, arguably in physics. The one area I would highlight where we have had real scientific advances is AI. But I note that most of the theory of AI — the theory of why it works — was worked out in the nineties and the early 2000s. Even if AI models are becoming much more capable, if that is because we are building much more compute rather than some fundamental insight into the nature of artificial intelligence or information processing, that is a technological advance, not a scientific one. I am trying to be precise with what we are saying here.

I have heard some people claim that science is slowing down because we figured out most of the science, and there are just a few smaller areas for us to fill in, but we basically understand the world. I don't think that's what you're saying.

No. The idea that we have picked all the low-hanging fruit in science — you could have said this idea in 500 AD and it would have seemed true for about a thousand years. In 500 AD we kind of know the world is round. Sure, they are going to forget this in about two hundred years. We know all of the formal and sufficient causes, and we have differentiated action, and we have agreed through philosophical progress that there is just one God, but maybe there are more gods who are minor aspects of the one God, and so on. We have of course explored the known world. Alexander went as far as we could go.

You can tell this story at 500 BC, 500 AD, 1500 AD, and of course in 2026 AD. Merely because we cannot immediately spot areas where we should be having completely conceptual breakthroughs does not mean those don't exist. I will even go further. We even know there are certain things we don't know. Therefore we actually see a bunch of fruit. We just can't reach it.

Why we sleep

The one that messes me up the most is sleep. We have very little idea what's actually going on there, other than: you will get very tired, and it will be pretty bad for you if you don't do it.

There was of course a book, Why We Sleep, that was debunked. Merely because people say we have good answers does not mean we actually do. If we go beyond something very fundamental like why we sleep — when my kids ask me that question, I have to give them the honest answer: we don't know. But we sure are tired, and it's time to go to bed.

I heard once from my dad, who's a physician, that before you go to medical school, or when you're at medical school, your friends will ask you to solve their medical problems and you'll have to say I don't know. Then you finish medical school and you get to tell them we don't know.

I think many things are so. Even look at physics. Theoretical physicists, when you talk to them, will of course assure you that we are making rapid and immense progress in physics. If you ask them whether we have exhausted all the physics there is to do, they will tend to say no, and they are going to rightly point at gravity at very small scales, and also at just gravity in general. I don't think dark matter is real. But if it is real, that's a big physics puzzle — this missing mass in the galaxy so that they don't spin apart.

Most of the physicists I have talked to — there are enough of these weird things where we are kludging together the different models and the different systems — their expectation is that at some point someone will get some sort of Copernican breakthrough, a way of reconceptualizing the system, and we will all be like: oh, I see. A lot of the building of bigger and bigger colliders was hoping that that data would take them closer to this next big theoretical paradigm.

That hasn't happened, or not yet.

Not so far. They're still working on it.

I raised dark matter, which was a hypothesis for: gravity seems to be stronger than it should be at the very large scales. That suggests that there's something screwy with our cosmology, because the galaxy would literally fly apart if it was just the visible matter that we can detect.

There are many, many mysteries in the world. We can keep going over them. The basic point is we don't know everything. If science has slowed down, it is not because we figured everything out. It is something else.

What is gating science

I can give my provisional answer, but I have a few hypotheses. First I want to propose that we exhaust some of the answers that don't make sense.

The first answer that I think is not quite correct is the idea that the experiments needed to differentiate between scientific theories are too expensive. We built the Large Hadron Collider. It turned out not to be very clarifying. We could always build bigger telescopes, and so on. Sleep studies are very clearly not gated on expense.

Maybe someone could make the argument that they are gated by medical ethics. Would not be the first time. There was a certain point where people thought an autopsy was profoundly immoral, because it is a desecration of the body, and that held back anatomy. One could imagine a situation where our animal or human-rights ethics prevent us from looking into a sleeping brain, and you just have to do that to figure out what it is. I don't think that's the case, though, especially since we don't treat the animals very well.

In fact we do a fair number of sleep studies. Not too long ago, ten years ago, I remember being on public transit in Boston and they were advertising: come in, we'll give you some money and mess with your sleep and observe you. These are happening. They get published. People try to draw conclusions from them. They have figured out some things — this type of wave correlates with this type of activity — but the core of it: they are in fact doing those experiments.

They are in fact doing a lot of those experiments. While I do believe instruments can notably advance scientific progress — we all know the Galileo story, looks through the telescope at the moons of Jupiter, and lo and behold Jupiter has moons, and they are circling around Jupiter rather than Earth or the Sun or whatever — there are some discoveries that await us with better instrumentation. I am not denying that.

What is my best guess for what's happening? I actually think that science happens, to a surprising degree, in the interrelation between theory and empirical observation. I suspect it has to be the same person. The scientific division between theoreticians and experimentalists works less well than we would like it to, because we would like to professionalize it and break down these components. Unless you are pretty good at theory, you are not going to know how to adjust your experiment, whatever that might be — how to tweak the sleep study, what kind of things to measure about the collision of near-light-speed particles, and so on.

The theoretical side is the one that is actually missing. I am arguing we have gotten bad at theory. We have gotten bad at creating new scientific theory as an activity in itself: making these abstract good hypotheses that make sense of what we know, both quantitatively and qualitatively. That already gets us somewhere interesting.

The experiments are still happening. In a certain narrow sense many of them are bunk, but at least many of them are well designed. Then the people who would want to come in and synthesize that are, for whatever reason, not being trained up — or training themselves up — to the standards of the very rapid pace that we were seeing, let's say, a hundred years ago.

Generators

Whenever I evaluate progress in any kind of knowledge, I think one of the biggest challenges is transferring the cutting edge of knowledge from one person to the other. Not in the "I can answer all the questions from the back of the textbook" sense, but in the sense of the spirit of the thing, the generators, the underlying principles that allow you both to answer the questions at the back of the textbook and to generate them in the first place.

Which is why you see so often that the greats in physics or math, or pretty much any field, are so often the people who are personally trained by the greatest of the last generation. You can't get it all through the textbook. You can get a lot more of it through detailed one-on-one instruction.

There's a ton of that. Was there a sharp break in the chain? Did it gradually decay? Where did it go?

With traditions of knowledge there are thousands of them, and you can in fact empirically study an organization and figure out whether it is plausible that there is this kind of chain of direct transmission at play. I have to admit, every now and then people will reinvent or rediscover new traditions of knowledge. Sometimes you can just find a math textbook in India, like Ramanujan did, and you re-derive your own approach to math. Unsurprisingly, when you do that — when you reinvent a new generator for yourself — you are going to go back to the Western mathematicians, and they are going to be amazed at the kind of stuff you have been deriving. That means you discovered some sort of new generator.

All of this stuff is very tacit, hard to formalize. It underlies it. It is this kind of felt sense. I think we are systematically biased against that in the modern world. Even though there are thousands or tens of thousands of traditions of knowledge, other than people who understand the value of deep mentorship — and there are enough of those that we are not completely hopeless — I don't even think we consider this a KPI. Has anyone endowed, say, at the Institute for Advanced Study, a chair of Einsteinian imagination, where you go and run thought experiments in your head all day as your research program, and you are supposed to do it exactly as Einstein describes for the problems he was dealing with, and then try to just make up mental images in your mind for new problems?

One thought I've had. There was this famous interview with the great mathematician Terence Tao, where he talks about solving a math problem while staying at his aunt's house. The problem has him rotating a whole bunch of different spheres in his mind. The way he best rotates all of these spheres is he gets on the ground and starts rolling around. His aunt walks in and asks him what he is doing. He says, I'm working on this math problem. She's like: what?

This has me wondering. There are so many people, especially here in San Francisco, who do all of this really deep introspection, looking at exactly why the mind does what it does. Can you get a couple of those people to try to figure out what he is doing with his mind with the sphere rotation while he's rolling on the floor, and try to proceduralize that and teach it to other people? There are so many things like this, because that's going to be one little corner of his method.

At the end of the day people know about things like Feynman recommending literally taking a shower, because the tactile sensation of the shower actually resets your mind in a certain way and helps you think about certain things. It is not just a metaphor for "go refresh yourself." Maybe we don't have great political economists because men don't luxuriate in baths anymore. There is this Twitter meme which was funny about how the modern paper has a thousand caveats. Meanwhile the eighteenth-century Enlightenment philosopher's treatise is: some thoughts that I had in the bath that constitute self-evident truths of the world.

You're doing a different activity with your mind. One is you're trying to defend this fence. The other is you're trying to proclaim truths. Maybe there's an important difference here.

You brought up Feynman. I remember reading these very concrete descriptions of how he will be counter-exampling things. Someone will be giving an abstract description and he'll take some property and imagine it as being fuzzy. He gives these very precise, very step-by-step instructions for how you visualize things, which remind me of some of the more detailed Christian prayer manuals. They are just giving very detailed, step-by-step instructions: here is what you do with your mind, and then you do this, and then you do this.

Some of this can be codified. It is not just a matter of apprentice relationships, though you could certainly get higher bandwidth with those. I do suspect that more of it can be written down and proceduralized and codified than often has been.

Or than people have tried to do recently. We should remember that so much of our mathematical notation is partially an explication of things that occurred in mathematics first as phenomena that people observed or intuited. In mathematics we see new notation implemented frequently to talk about these things. There was certainly a time early in the history of mathematics, in the Pythagorean era, when these were considered sacred mysteries and truths — things that you could maybe draw, but you couldn't write down really. Today we have gotten much better in some ways at writing some of them down. The Greeks would do mathematics geometrically. They would have geometric approximations of computational results, and we have computational approximations of geometric results.

One of the big advances we have is Cartesian coordinates, named after René Descartes, who made the big advance that turned into our coordinate graphs in the appendix to the work where he is doing I think, therefore I am, and imagining the thought experiment of the demon deceiving you and all that stuff. That seems to have produced something pretty interesting. It always kind of blows my mind when I think of how late in the game the idea of coordinate geometry as a notation was invented.

This is fairly late in our history, even though we had been using geometry to measure 2D planes for a long time, for taxation purposes, probably for five thousand or more years. It's odd we didn't think of 3D. And he was lying in bed, I think, imagining how to best mathematically describe the flight of a fly. The fly was buzzing above the bed. He often had headaches or something that caused him to stay in bed.

He was a very sickly and bedridden man, and seems to have advanced mathematical notation as well as the philosophy of the mind.

If you are bedridden, you will perhaps come to the conclusion that the mind is the only thing that really matters.

It does have me wondering what new mathematical notation will exist in a thousand years. I feel kind of sad that I won't get to see it.

Science as a faculty of the mind

When we are talking about mathematical notation or the scientific process in general, we have arrived at a place where we have decided it is perhaps more similar to the technical instructions followed by, let's say, California Buddhists when they are trying to reach third jhanas or whatever. I honestly think it's slightly overrated. Or rather: I am glad that there is attention paid to meditation. I wish there was more attention paid to the other faculties of the mind that can be as phenomenologically rich or developed in their own way. Buddhist meditation, Christian prayer, scientific inquiry: maybe three instances of a class. I think that already is pretty heretical. If we were looking at the modern university system, you would think science is a bureaucratic process.

In the modern university system there definitely is a bureaucratic process that goes by the name of science. Its relation to the historical science that produced the bulk of our knowledge of the world is an exercise left to the viewer.

I don't want to leave it to the viewer. I think we can say a bunch about it.

These ideas of peer review as this bureaucratized, centralized thing that sends it out and gets comments and puts it in publications — these are very recent things. For most of the history of what we would call scientific modernity, until basically the middle of the twentieth century, it was much more informal, distributed, much more like posting a blog post and getting a bunch of people in the comments telling you that you're a moron.

Sometimes it was just your friends reviewing your article before publication and giving you their opinion. Einstein's original paper describing special relativity — he mailed it to his friend, and the friend was like, okay, this checks out, and then he published it. There was no almost double-blind type process. There was no scientific publishing business as we understand it today.

There are people whose reputations are being tracked. We're paying attention: Rutherford's a sharp guy, we care a lot what he thinks. To some degree it did track the degrees. One of the important things is that the meaning of these degrees has largely been inflated. Between 1900 and 2000 in the USA, per capita, the increase in PhDs was a factor of thirty. In absolute terms, a factor of a hundred, because the population tripled. I don't think it's because we got thirty times more people who are as good at doing that thing.

There was more of a coupling between who the field-leading people are — who, just by virtue of their opinion, you should in fact be paying attention to as to whether the new thing is legit or not — and the degrees. The degrees were kind of secondary to this system of people tracking the arguments and tracking which individuals had good judgment.

Career versus reputation

Tracking who has good judgment is remarkably resistant to proceduralization, be it in military matters, scientific matters, or honestly even economic matters. A peacetime military will have many generals, and until the war happens you have no idea which of them is any good at a modern war. All of them, of course, will be very good at paper-pushing. Economics has a similar analogy.

When it comes to science, for the mere gathering of data points, the gathering of known facts about the world, I think this process can work okay. I'm sure we could collect all the world's beetles. Coming up with the theory of evolution, natural selection — I think the academic process would not be good at that.

This difference between staking your personal reputation versus hurting or advancing your career — the difference between career and social reputation is where this key tension happens, and why something is lost once you made science a bureaucratic career.

What is the equivalent? In the general example the forcing function is the war. Is there an equivalent for science — the thing that forces you to get sharp and get serious? Was there something forcing in the earlier days, or is that not really the way to think about it? Is it just that there was this later bureaucratic craft that was able to use this as a justification to will itself into existence?

Vannevar Bush

The scientific revolution after the war was partially an attempt to figure out how to have a very centralized state fund science and how to make these investments accountable to the public. It kind of began with the Vannevarian turn in science, the Vannevar Bush era. These papers were actually pretty good — Science, the Endless Frontier. I will only recommend them to the viewers to read. As far as individuals patronizing science, Vannevar Bush was pretty good actually. His scientific judgment was not bad. His taste in people who have it and don't have it was pretty decent.

He was generally in charge of American technological development during the Second World War, which produced a lot of technology. He was clearly doing some things very right when he was doing that big centralizing push.

I think he was doing good things with the power he had. Whether or not the centralizing push was a good structure to build, the jury remains out on that. They were advancing the technology much more. The war was too short to be making big scientific pushes.

Possibly the most technological war in human history — a war where the technology race mattered to its outcome. Perhaps we can repurpose these to endlessly push forward technology, and implicitly to endlessly push forward science. But the thing with having a centralized philosopher-king is that you need the philosopher-king to be very good at their job. The people who came after him don't seem to have had Bush's taste. Even if there are some people in the national scientific bureaucracy who have Bush's taste, power has fragmented so much that they couldn't really unilaterally give this kind of funding or research.

There are a few exceptions. I think DARPA works technologically quite well because the program directors have a lot of discretion. It is not a decision by committee. A program director with a limited term decides: actually, we're going to fund this approach to self-driving cars. And they seem so right.

That was the origin of — I think a bunch of those ended up being critical Waymo people, if I recall correctly. This was from your work in the Bismarck Brief, which you guys write. DARPA does seem to do a pretty good job of getting a bunch of people with good taste into those frontline roles.

We still have to be careful here with science and technology. In theory NASA has a mandate to foster scientific discovery, but they are an agency that's accidentally oriented around specific spaceships and vehicles. They are de facto an exploration agency. To the degree that they do science, it is much more the science that is load-bearing for missions, rather than something we can point to as a novel physics breakthrough that came out of JPL.

There is also a problem in the Vannevarian approach to science, which is that the Manhattan Project was secret. What happens if the greatest scientific advances that are made with government money are immediately classified, because they were made in the course of developing a better thermonuclear bomb, or decoding — maybe some stuff in quantum computing where you are decoding encryption of a nation such as China, or the Soviet Union back in the day? That knowledge is immediately rendered inert to a great extent, unless it is load-bearing to a particular piece of still-classified technology.

There is an aspect of science where I am not going to say it has to be completely open. But I do think there has to be a scientific community that is ideologically driven to discover the truth and exposes these truths to each other. There might still be a barrier between them and the general public.

Esoteric cliques

I don't think it's an accident that the scientific revolutions from the sixteenth to, let's say, the twentieth century happened when these groups of correspondents — who started in alchemy — began writing to the public in more plain language. The Republic of Letters was half internet, half esoteric conspiracy: anyone could write in, and not all truths were for everyone. There's more than one conspiracy embedded in the internet.

For people using the internet or using AI, they would be very much surprised at the worldview of the people that built the internet, or that built artificial intelligence. They also are adherents of an esoteric philosophy as strange as alchemy. Alchemy proposed to turn lead into gold. This singularitarian worldview proposes to turn man into machine — to turn brain into computer, and then of course live forever and discover the secrets of the universe and all of that usual stuff that we all desire.

The earlier internet stuff was these very politically radical people: we're going to free man from the shackles of the state. These very cypherpunk — we now look back at them and see them as very idealistic, with all this "information wants to be free." Imagine what's going to happen when all of humanity is able to just talk freely to each other. That was the vision that motivated them to do that. It's not just science. It's not just technology. Great things tend to be done by these cliques of very ideological people with these esoteric and often utopian worldviews of what it is that they are doing.

The thing is, you're very good at science only if this esoteric ideology happens to have an almost irrational commitment to truth. That doesn't mean you have to share all truths with all people. The philosophers had the whole semi-obscuring, to protect yourself from the public, to protect yourself from political persecution in the palace. I wouldn't go as far as Leo Strauss, but I would say that philosophers were sometimes speaking obliquely, they were sometimes speaking indirectly. But they did have a commitment to truth.

The Greeks were scientists

I do think that the philosophers of ancient Greece legitimately were not just the predecessors of science. I think some of them just were scientists. Let me justify this, because some of them did advance all the way to the point of pairing theoretical concepts of how the world works not just with claims about how the world works, but even in quantitative, measurable ways. Archimedes, who comes later in history, about a century or so later than Plato or Aristotle, is calculating the water displacement of a ship with methods that approach calculus.

And Eratosthenes is calculating the circumference of the Earth with trigonometry.

This was also the conclusion that Joseph Needham had when he was looking at what was different in the Western tradition from the Chinese tradition. If you're looking in 800 AD or 1300 AD, then technologically China is just way ahead of the West. Then there's this weird thing that happens where, when you come back and look around 1700, it's totally different. He was looking at exactly this combination of quantified empiricism with a fully descriptive theory as a thing to be reached for. As far as I know, the only times you have seen that in human history are modern science and the places it spread, and then those ancient Greek philosophers, up to a few of them still around in the earlier Roman period. They don't seem to have lasted throughout the whole empire.

Probably not a coincidence, given that Archimedes was killed by a Roman legionnaire.

When the city of Syracuse was taken, the weapon systems were already valuable. He was a weapons engineer. I don't quite believe the story of the ancient lasers, but weaponized use of mirrors — oh yeah, I would believe that. Let alone claws and levers and various siege engines that made it very, very difficult to carry out a naval siege of the highly fortified city. This is a sort of wartime science. It's not just World War II that's a science-and-technology story. I think the conquest of the Mediterranean was almost just Roman persistence overcoming Greek siege engineering.

That part of the story was forgotten, and we only have these vague recollections: oh, I guess the library did burn down multiple times when the Romans invaded. The Library of Alexandria — that's part of the Mouseion, which is a weapons-research institute as much as it is a philosophical institute. Also they were doing their data collection. It was a data center. Ptolemy II, the Greek king of Egypt, passed a law that every ship that went into Alexandria, which was a massive production and commercial center, be inspected for books. If a book was found, it was copied. You were given a copy, and the original was kept.

Essentially this is like if you go through China and they copy your hard drive whether you want it or not.

Or if you're training your new artificial intelligence and you're buying all of the books so that you can scan them in and put them into the soup that becomes the weights.

Exactly. The library wasn't built just through donations, but sometimes just directly through state power. Eratosthenes was the chief cartographer of the Ptolemies. He was kind of like the head of the national geological survey today, and is attributed to be the author of a full map of all the territories owned by the Ptolemies — an effort a single individual couldn't undertake. For taxation purposes they compiled these massive records and sent out expeditions and all of that.

You seem to have this fusion of quantitative empiricism with the sort of general theory, which emerges, sticks around for centuries, and then goes away, comes back much later, at least substantially through people trying to revive the ancient texts and seeming to come to an even more functional version of it than what the Greeks had. Depending on how much credit you give to the Renaissance exploration of the ancient texts for the revival of science, this may have only been invented once in the history of the human species.

And that's kind of incredible, isn't it?

The Forgotten Revolution

This argument is also advanced by Professor Lucio Russo in his book The Forgotten Revolution: how science was discovered around 300 BC, forgotten, and had to be rediscovered. He makes the argument not only that we discovered Greek texts and so on. He goes further. He says people in the fifteenth, sixteenth, and seventeenth centuries had access to scientific works from the Greek world that we have since lost.

When I first read this claim and was looking at the sources, I was like: this has to be false. Surely we have all the ancient Greek books. No. We on record have multiple copies of books that people in their correspondences in the Republic of Letters claim to have. Now — claim to have. People sometimes make stuff up to be cool, obviously, especially people who want to show off to their super-smart friends. In the Bay Area you see this all the time. "We found some troubling things with our latest AI model. I tell you, but it's under NDA. Too bad your AI company is so far behind. You don't understand the weight I'm bearing."

There was a period where, a month before every new AI model would come out, all of your friends at one of the labs would be like: oh yes, there's some very interesting things. I can tell you a little bit.

Exactly. And then sometimes they just blanket make up stuff, or overestimate what they have, and honestly boast. But to say one more thing: there are, again, dozens, possibly hundreds, if an enterprising historian who knows both Greek and medieval Latin were to go through all the letters — I bet they would come up with hundreds of titles of scientific and naturalistic works, both ancient Greek and Roman, that were known in the fifteenth, sixteenth, and seventeenth centuries, never saw a reprint, and then were destroyed or lost. Because this was stuff in private collections. But hey, libraries were still burning down.

Libraries still burn down today.

I know that there was a Brazilian library. Do you remember which one it was? National Library or something?

I don't remember the name. This was the one with those language recordings?

Exactly. A few years ago an important part of the National Library and Archives of Brazil burned down, and we lost some of the last recordings and last written descriptions of now-extinct Native American languages and groups from the Amazon. So knowledge is still lost in library fires. Unfortunately that stuff was not digitized. And even if something is digitized, it might still be lost.

I don't know if you heard about the legal decision that made it so that the AI labs are now destroying books.

Yes, because it's illegal for them to copy them, but they can't transfer them, is my rough understanding of why that ends up being the only thing they're allowed to do if they want the information.

Honestly, potentially they could just keep the copy of the book they bought and run a library, and I would respect them all the more for it. I feel like Anthropic can afford it. They should just sue people back and actually fight for the books.

In fact, huge quantities of books — the amount that we're just getting rid of because we have too many books. Libraries will very, very frequently just be like: all right, we just can't store all of these, so we're just going to get rid of 80,000 old—

Totally. Look, books are mass-produced, but the labs are making an extra effort to have the largest data set and a unique data set no one has. So I'm sure they're getting a lot of unique books. If they had a process where they destroy a book after they verifiably established that there are like ten copies in existence other than that one book, I'd be okay. But I think they probably destroy books, sometimes knowingly, sometimes unknowingly, where it's the last copy, or one of five copies, or something. I'm not that worried if it's a thousand copies. Once we get to ten, I'm like: oh, actually, you are destroying the knowledge of mankind and destroying the record of our civilization, as a side effect of a legal battle.

I would attribute much, much more of the blame to the copyright system. But you can at least imagine things they could do. It'd be interesting to look into exactly where the lines of how legally permissible that would be.

If they even spent ten million dollars to relitigate the legal battle, I'd be like: okay, you guys tried.

Or publicly petition the president to pardon them. "We are keeping the books. We call on the President of the United States to pardon us for preserving these extremely rare books. Here is the list of books we have." That would be pretty cool.

I think that'll be a live-player move. Or just make them actually get a judgment against you, and just be like: we're not destroying the books.

Yeah, we're just going to pay the fines. This is what happens with some of the European data-privacy laws. The big companies in the US just pay the fines, and it's fine.

Kind of. I think those laws are mostly just an irregular taxation mechanism, and it doesn't really matter what the American companies do.

I'm not saying they're fair laws. But even if, say, Anthropic or OpenAI or Google had to pay a million dollars a year in fines, I think it'll be fine. I think it'll be fine, and I think we'd all like them better for it.

The same person has to do both

Lucio Russo, by the way — I just want to name him again as someone who is very influential in my own thinking. I have recommended his book. I recommend it to the viewers here. My colleagues at Bismarck have read it.

You ran that reading group on it a little while ago.

I did actually. I'll probably run it again. Maybe we will do a live lecture on the topic someday with The Students. I know you guys are organizing live lectures now. Very much looking forward to all of them.

He is a rare example of a physicist-mathematician. He unfortunately passed away about two or three years ago. I was still lucky enough to speak with him before his passing. He was both a mathematician and a physicist that spoke Latin and Greek. Because of this he could read the original texts. Rather than, as a historian, you translate it, and then you go to a physicist and you kind of talk to them about physics or math — there are going to be some deep misunderstandings happening there. It has to be the same person. To evaluate ancient science you must be both a scientist and a historian.

It's a little bit like what you were saying about the separation of the theorists and the experimentalists. Once you're taking these activities out of the same mind, they just kind of lose touch with each other.

This is another mechanism of how things are lost, and why knowledge is difficult to scale. Knowledge lives in heads, not in books alone. If you have a bunch of Renaissance humanists who are into alchemy, and they think the ancient writers probably discovered the secret of alchemy because they were the real ones — until you could read them, you had to read them, and the motive in a way was the alchemical motive, and then a little bit slowly it also came.

These books were close enough that the languages were still mutually intelligible. But even the Romans themselves were not that interested in Greek science to begin with. They kind of wanted the pop-science version of it. Imagine if all that's left of our civilization, of our idea of physics, would be some Neil deGrasse Tyson videos. I think that would be about the equivalent of what happened with Greeks and Romans. You had all of these Greeks trying to understand the fundamentals of communication and rhetoric, and then the Romans would bring them over and be like: let's do a public-speaking course for our kids. They were a very practical-minded people who didn't tend to get into the real esoteric depths of what some of the more ideologically inclined Greeks often got up to.

And of course it was a minority taste among the Greeks to begin with, something constrained to a few city-states, a few classes of people. It still is.

Proof of possibility

Technology can tinker without theory. Almost usually in technology, when there's a proof of possibility — when someone builds a machine — suddenly lots of other people build into this. That proof of possibility somehow unlocks all these people. What scientific theory does very well is to say: okay, theoretically this kind of rocket engine is possible, or theoretically a space elevator is possible, even if the materials are missing. It supplies a proof of possibility for your machine.

When you do have an explosion of scientific progress, you undeniably have an explosion of material technology. Today we actually have very few material technological advances that you can attribute to science rather than to tinkering and scaling.

When it comes to the physical technology of the last several decades — the big advances in the physical as opposed to digital — a lot of it is in biology and a lot of it is in materials science, and those both seem like extremely tinkering-heavy domains.

We don't even have great theories of why materials behave the way they do, why certain chemical elements are catalysts versus not. We certainly don't have an exhaustive theory of aging. We have this list of various types of damage we accumulate, and so on. It's very much: what happens if we get a slightly different molecule and give that to people and see what happens.

A calling

In recent years there is perhaps a belief that human science is obsolete because AI will figure it all out. We have seen some evidence that AI can do versions of self-play on basically mathematics. It can sort of self-game mathematics. I actually believe it will be quite a while before we have genuinely novel scientific breakthroughs, because the LLMs after all are trained on human literature of how the world works. I think they will have a harder time than we think exceeding that literature.

It's certainly something that most humans aren't able to do, and it's certainly something that they haven't done yet.

My totally baseless intuition is that when it comes to purely abstract things like mathematics, that can be done just with a simple manipulation, they will be making breakthroughs like that in the not super-distant future. They will have a harder time when it comes to stuff that impacts the material world, where things get a lot messier, outside of everything functioning according to internally specified rules. We'll have to see.

I hope that for now people are not too discouraged from science, even though allegedly it is a white-collar profession. I would deny that. I would say that it is first and foremost a calling, and it remains a calling even if there are machines who can do it. Therefore I hope that no one gives up their dreams of understanding the world because of AI.

Pythagoras certainly would not have.

Well, thank you very much for coming. A great conversation as always.

A pleasure, Ben.