Justin Solomon: How AI is upending the world of mathematics

Why mathematics, and why now

Hello and welcome to another episode of the Odd Lots podcast. I'm Joe Weisenthal.

And I'm Tracy Alloway.

So Tracy, with AI, obviously one of the big concerns is job loss. In the future, what are humans going to be better at than the machines? What are we going to do all day?

What are we going to do all day, et cetera.

We really don't know the answer to any of these questions yet, I think. But I am aware, obviously, that the models are getting very good at math, and at least some mathematicians are in a state of existential angst about this. Certainly some perhaps feel a crisis: well, what are we doing here, these problems that I devoted my life to solving, and the model solves it in a weekend? Is this what's potentially coming for many of us?

Yeah, math is an interesting one, because you can tell that a lot of the AI companies themselves have an interest in math. They see math as a sort of holy grail for, I guess, proving that the models are able to reason. It's the thing they need to do to be able to say that they've achieved AGI, I think, is the way it's couched. And it also makes good headlines. So even though I have no idea what the Navier–Stokes problem actually is, and I don't even know if I'm pronouncing that correctly, is it Navier or Navier? Okay, Navier–Stokes. I see the headline and I'm like, ooh, a model has solved the Navier–Stokes problem, as if I know what that actually means and its significance for the mathematics profession.

It's also funny because I downloaded the paper, and it's like one paragraph, and I can't even begin to parse the output. Another fascinating aspect of this is that it would not have been intuitive to me, say two or three years ago, that the language models would be so good at mathematics. In fact, at one point they weren't very good at math at all. Go back to, I don't know, GPT-3 or GPT-3.5.

Oh, yeah.

If you asked them to count the number of letters in a word or something, they would get it wrong.

Yeah. And weirdly, they still struggle with certain things like that.

But it's not obvious to me. Okay, you scan all of the text, language, et cetera, and yet somehow that generalizes into this extraordinary math capability. That too is very surprising.

Can I just say, before we start, I would also just like to learn what advanced mathematics actually is. Because everything I know about it, which is nothing, comes from watching Good Will Hunting: a bunch of formulas on the chalkboard, and people get really excited when they start crossing off certain parentheses and stuff like that. I have no idea what it is. I have no idea how it differs from the pure maths of two plus two equals four. And so I would just like to come away from this conversation with a better understanding of what it is overall and how AI actually fits into it.

Absolutely. I was thinking along the same lines. Okay, for someone who devotes their life to advanced theoretical math, and also things like advanced theoretical physics that don't necessarily have obvious industrial applications, who is the type of person, what are they hoping to achieve, why is there still interest in some of these things where perhaps the applicability is not so obvious? And then, obviously, what is the role of the mathematician at this point?

And then, specifically, this is actually the thing I'm very curious about. My son studies math, right? Both my kids study math.

Hopefully not theoretical math just yet.

Not yet.

How old are they?

My daughter is 10, my son is seven. They take math, et cetera. But setting aside what the future holds, I'm very interested in just the present. You have all these math departments at universities all around the country, and maybe some are excited about AI, et cetera. But what do you do today in terms of rethinking pedagogies? Because I would assume it's pretty different in 2026 than those same departments in 2021 or something like that.

Absolutely.

How are professors, teachers, et cetera thinking about the process of teaching math right now? Just very straightforwardly, how is it affecting their jobs?

I used to be really, really good at mental math, like times tables. This is a question I have too. If you are an advanced mathematician doing theoretical maths, how often do you actually use a calculator? I'm curious. That's going to be my first question.

Actually, Jensen Huang, I don't know if you read the transcript of his interview a couple of weeks ago. He has a little bit of a weird interview. He said something like, well, maybe knowing your times tables isn't going to be an important skill in the future.

Or your address.

Yeah, which is kind of weird. I still want my kids to know their times tables, but it raises a very interesting question.

No, it raises a very important question, which is that you can go to a model and have it verify a proof or a theorem or something like that. But if you don't understand how you actually got there, and how the model is getting to its verification, then what are we actually learning here?

Right, and then there's the question that people learn a lot in the process, right? Presumably people work on some of these problems for years and years, and maybe they never arrive at any proof, but intuitively you would think there is something, maybe it's exercising a muscle, some other benefit to just the process of learning. How does that change? Anyway, I have a lot of questions about this.

And we really do have the perfect guest. We're going to be speaking with Justin Solomon. He's associate dean for engineering education at MIT, and very interested and involved in math education and in thinking about the intersection of math and AI. Justin, thank you so much for coming on Odd Lots.

Thanks for having me.

Who becomes a mathematician

Tracy sort of already suggested it, but set aside AI for a second. The people who end up going into advanced math, or advanced theoretical math, who are these people, and what are they hoping to do?

Yeah, it's true. I feel like the typical character that you see is maybe the character in Good Will Hunting. Incidentally, they're always writing math on windows, which is very practical.

Good visuals, though.

Yeah, that's true. It's great for a camera. But that aside, I think it's an intensely personal thing. To be clear, my personal career, I'm a professor of engineering, is in an area of math called applied math, which sometimes is applied and sometimes not. That's a discussion for another day. But essentially, some people go into math because they want to see it translated into applications, or to bring fundamental insight into the kinds of things we do every day, whether it's predicting markets or physics, what have you. Other people, in areas of math you sometimes call pure math (I don't love that terminology, pure versus applied), do it almost just for the joy of the game, or for understanding the universe around them. All of these are totally legitimate reasons.

And true intellectual pursuit for the sake of intellectual pursuit.

That's right. But along the way, I think you develop a lot of really important reasoning skills, and skills that support work in all kinds of different areas.

And how often do advanced mathematicians, whether they're in applied maths or purely theoretical maths, use a calculator?

Very often.

Okay.

My impression again is that everyone is doing this by hand on chalkboards and windows.

I mean, there are lots of Greek symbols and letters and variables and things, but no, I'm terrible at arithmetic. I've got a calculator out all the time.

For real?

Oh, yeah.

This gives me hope, actually, that maybe I could understand some of this, even though I'm pretty bad at it. Okay, so what's the difference between a math problem, then, and a math proof or a theorem? These big-picture things that we're talking about AI models actually doing now.

That's a great question, and it's really important to dig into exactly what the models are doing and not doing, and who deserves credit, by the way. But essentially, in mathematics, verifying true facts is one part of the puzzle. It's also all about figuring out what the interesting problems to study are, and what insight they bring to the broader world, right? In some areas of mathematics there are all kinds of different questions you could ask, and some of them are super uninteresting, but you could ask a computer to prove them. Others maybe tell you something about a physical system, or about a system of equations, something that brings broader understanding. And so this exercise of proof writing is all about writing a mathematically convincing argument that tells mathematicians (so there's a human aspect, which is an important one) that a certain fact is actually reality, right? It aligns with logic. There's not some flaw hiding in there somewhere.

And where do ideas for a particular mathematical proof or theory come from? Because again, I'm going to come at this from the movie angle, but I think about A Beautiful Mind, right? And the famous bar scene. It was John Nash, right? That was his name. And he has this big realization while he's at the bar with these women, and it ends up turning into a famous mathematical theory. Where do the ideas actually come from, for the rules that you are trying to prove in maths, or these big-picture ideas?

This is a great question. Sometimes in mathematics there are some really natural questions out there in the universe that people have been asking for a long time. For example, these days in the news we hear about some of these really high-profile math problems that have been around forever. There are other types of mathematicians that are more theory builders, where they're the ones asking the questions rather than answering someone else's questions. And there are definitely different approaches out there.

Now, when it comes to where these questions are coming from, some of them are inspired purely mathematically, and others maybe come from a very practical setting. Maybe somebody observed some phenomenon out there in the real world, and they're trying to explain it with the smallest model possible.

Now, one thing I should clarify is that math is a really social exercise. I think that's something people don't necessarily understand, especially because these movies really play into this almost great-man theory of mathematics, but that's just not how it works. There's this huge community of people working on important problems, communicating with one another, collaborating. And I would say, for example, from my own research, almost everything that our team has worked on comes as a team effort. We're all sitting in the same room, at the blackboard, writing computer models, what have you, and that's where the insight is coming from. So it's not just sitting in your ivory tower. It's being out there in the world, observing things happening and talking to your colleagues.

Applied and pure

Before we get more into the AI question, I've actually been curious: what's applied math? What is the difference between applied math and pure math?

Yeah, it's a great question. There's sort of a divide in the math world, but I think it's fuzzier than maybe it sounds when I describe it here. Roughly, there's an area of mathematics you might call applied math, where the goal is either to derive theoretical results or even to come up with computer algorithms that explain phenomena out there in the real world. For example, in applied math, maybe you work on algorithms that simulate the weather. Behind the scenes is a really complex mathematical model, and it's all about making sure that your model is correct and efficient and well behaved, all that good stuff. In pure mathematics, sometimes the motivation is a little less attached to a particular application and more just attached to the math problem itself.

So something like Fermat's Last Theorem, or something like that, where he's like... okay, I forget what it is exactly. Well, that would not be applied math.

Right, so Fermat's Last Theorem, and many things in number theory, I think, fall into that category. But many times these things also bleed into one another. For example, a lot of modern developments in cryptography came from areas of mathematics that we used to think of as extremely pure and abstract.

Oh, interesting.

This is also Good Will Hunting, by the way. It all goes back to that.

That's right, that's right.

You worked at Pixar before.

Many years ago.

Was that applied maths, working at Pixar?

Oh, I love that question. The answer is absolutely yes.

Okay.

Yeah, the movie studios are a great example of a place where, I would say, state-of-the-art work in certain types of mathematics is taking place. When you watch a typical movie with different effects, there's cloth, there's smoke, there are all kinds of fluids, rigid bodies flying around. The dynamics of all of those are governed by things called partial differential equations, and coming up with the right algorithms for simulating them, and, by the way, making them so well behaved that an artist doesn't have to think about math, is a really challenging mathematical problem. So I typically tell my students that if you're looking for really challenging math problems, you don't have to make them up. They're out there in the real world. And that's a nice example.

Navier–Stokes, or proving that water doesn't explode

What's Navier–Stokes?

Okay, so Navier–Stokes has been in the news a lot lately, and it's been great for those of us with a little bit of background in differential equations.

Getting lots of media inquiries.

That's exactly right. Yeah, we have our 15 minutes of fame here. It'll die fast, but we'll enjoy it for now.

Right. So the Navier–Stokes problem comes from the world of fluid dynamics. The Navier–Stokes equations are this model, and they go back quite a long time, that explains the motion of, for example, liquid in a cup. Both of you have coffee cups here on the table. If you shake that coffee cup up, and you know the initial velocity, you know what's inside the cup, then the Navier–Stokes equations are trying to predict where the fluid moves over time, right? Where's the velocity, the vorticity, all these different things.

Now, the Navier–Stokes equations are really complicated. When you look at your latte in the morning and you see the milk making these cool shapes and so on, there's all kinds of complex behavior that can happen. And so the Navier–Stokes equations, although they're easy to write down, are very hard to say a whole lot about. And so there was this big open math problem, which I think is what you're referring to because it's been in the news, which involves existence and uniqueness of solutions.

My undergrad differential equations professor, years ago, explained the basic idea to me in kind of a funny way. He said, you know, all you've got to do to make a million dollars is prove that water doesn't explode. And that's roughly right, with a lot of asterisks. Essentially, what we didn't know is this: if you know the initial configuration of your fluid, and you let it go under its dynamics over time, does it exist for all time and not have some really crazy turbulent behavior, or, for example, does it maybe blow up, or develop a singularity or something like that? So this was kind of a yes-or-no question, prove or disprove. And I guess the recent news was actually a counterexample. In other words, it showed that in some finite amount of time, with an asterisk, which is with a very crazy spoon, if you stir your fluid just the right way, you really can create this totally singular and, frankly, non-physical behavior.

Interesting.

What a proof is

Proving in math is one of these things where I think the formal proof that two plus two equals four is a nine-step thing, isn't it? Or sometimes eight. Can you explain what a proof is? Sure, I know two plus two equals four. It seems obvious. But explain what a proof is, such that proving two plus two equals four is something that requires eight steps.

Right. This idea of proof writing, you can think of it as almost the result of an identity crisis among mathematicians maybe 100, 200 years ago. It used to be that a lot of math was drawing pictures, and working in the kind of intuitive fashion you're talking about. But at some point people realized that they wanted to make sure the field had really strong foundations. And so mathematicians agree on a certain set of logical principles or rules. Precisely what they should be, I would say, is more the domain of philosophy. And then the idea of proof writing is to build up from these basic facts, like, we agree that these five facts are true, to a much more complicated fact, by combining them in different and creative ways, right? And so theoretical mathematics oftentimes is all about, I don't want to just say mixing and matching different facts, but coming up with smart ways to justify mathematical ideas, built on very simple principles and steps from logic.

Lean, and who checks the machine

So when it comes to AI currently, is the role more in proof checking, I guess, versus proof generation or idea generation? What are the models actually doing at the moment?

Well, they're sort of doing it all and they're sort of doing nothing.

Okay.

In terms of the types of mathematics the current AI models are doing, I don't think it's quite right to say that it's just proof checking. In fact, they're actually quite bad at proof checking. And one of the revolutions of the last year or so, I would say, is the realization that what you actually have to do is provide your model, your large language model, with a second module that can check the AI's proof.

For example, there's a programming language called Lean, which is basically just a sort of codified version of these mathematical rules. These days what people will do is put in their conjecture, the thing they want to prove or disprove, and then ask the AI system to generate a proof. But then, rather than just having it write in human-readable language, which is the substrate mathematicians use to communicate with each other, maybe in addition to that they'll have it produce Lean code, so that there's a separate piece of software out there that can check that this proof truly is correct. So the correctness part of it is actually usually deferred to another piece of software, which isn't AI. But somehow, because of the interaction between the AI tool and this proof checker, that's what has created a lot of progress that we can believe in a little bit more than we did in the past, if that makes sense.

I have heard of it only because I've been reading the news, and I've heard of this thing called Lean.

Yes.

And it seems extraordinary to me. You explained a little bit. It seems extraordinary that there is this, I don't know, is it software, or this framework, such that these extraordinarily complex problems, or solutions, I guess, can be dropped into it, and it's capable of saying, yes, this is valid. But it's also kind of crazy that you basically need a translator for it, right?

Yeah, can you explain a little bit how Lean works, perhaps?

I can explain a little bit. To be clear, I'm not an expert in this particular corner. But essentially, as we all know, things that come out of AI are unreliable.

Yeah.

All of your listeners should know that.

We all know this, yeah.

And that goes for mathematics as well, and math is an area we think of as extremely reliable, right? Everything should be true or false. Now, there are gray areas, but certainly we try to avoid them. So what this programming language Lean does is it has a few simple axioms built in: math things that we agree are the starting points for writing proofs, like the steps of basic logic. And then you write a theorem in this Lean language. So rather than in English, you write it in this programming language. And it's cleverly designed so that once the software checks that the theorem is true, it can get added to the library of true facts, and then you can build up a more and more complex edifice out of these simple steps.

And it's been really impressive what they can do. Some of the modern proofs that you're seeing, especially the AI-generated ones, are hundreds of thousands of lines of Lean code. I don't think a human could check it. It also means, by the way, that we're trusting that someone wrote that software correctly, which is a philosophical question. But if you believe that Lean is good at checking proofs, then it provides a lot of certainty that the things the AIs are writing down are actually correct.

Why understanding still matters

A devil's advocate question: why is it important that we understand the actual problem and the proof? Because if I'm an engineer and, I don't know, some model somewhere comes up with this really new, interesting mathematical theory, and I can apply it to my day-to-day work, is it not enough that I can use it? It seems to work for the most part. Maybe it's been rigorously tested in a physical environment or whatever. Why do I need to understand everything that's gone into it?

Oh, that's a fantastic question. Let's see. My research group works a lot on simulation algorithms. So I have some physical system, and I develop a technique for predicting what it will look like in the future. And we spend a lot of time writing theoretical results to verify that our algorithms are correct, that they behave the way we expect, and so on. And that's absolutely critical, because without that you really can't trust what your software is producing. For example, if I'm trying to say, yes or no, is this building going to collapse during an earthquake?

That's right. And all of that is built on mathematical models, techniques like finite element modeling and so on. They're not too exciting outside of my universe, but you're all trusting these things to do really important engineering calculations. Sure, one way to justify your software is that your building hasn't collapsed yet, but I hope we can do a little bit better than that.

You know, my dad was a physics professor, and I have memories of watching the video of the famous Tacoma Narrows Bridge collapse, that really horrifying video. For him that was a big thing. I've watched it several times in my household.

Our engineering students have to take a lot of ethics courses that are pretty fraught that way.

Oh, interesting.

A discipline in an identity crisis

There are a couple of avenues I want to explore, including just the process of teaching, and how it's different in 2026 versus 2016 given the new technology. But maybe give us a lay of the land for the math community. Because I know there are some professors, some people, in despair, I believe.

Oh, yeah.

Very anxious about what this means for the future of math as an intellectual discipline, et cetera. And then others seem to be very enthusiastic about the opportunity of humans and AI exploring new ground. But what are the conversations that you're in these days with fellow mathematicians?

Oh my gosh, things are changing by the day. And really, it's a huge collection of individuals. Every single person has their own opinion here. So I can tell you my view of the world, and if you talk to somebody else, I'm sure they'll have a very different one. Math is an academic discipline that also shows up in a lot of applications, as we mentioned. But every aspect of the work is changing, from how people produce mathematics and choose problems to work on, to how mathematics is measured in terms of productivity, right? And when we talk about hiring faculty, for example, what are the standards to evaluate a person's body of work, and what did the human contribute? Those, I think, are really critical questions, along with why do this exercise at all if maybe it leads to a computer. There are all kinds of identity crises happening.

Now, among mathematicians, there are people who are maybe theory builders and there are people who are theorem provers. The theorem provers, I think, are especially under threat at the current moment. Maybe you've been working on one problem, and it's well defined, and you've known about it for 50 years, and it's this crowning question that you want to answer about the universe.

Yeah.

And these AI tools are at least good at cherry-picking the ones that are somehow close to being proved, and taking that last mile in some cases. Other people disagree with that characterization.

What do you mean when you say they disagree with it?

Exactly how interesting a mathematical proof is, for example, is a very human judgment, as is what problems you should or should not actually go about working on in the real world. There are a lot of true facts. It doesn't mean we have to write them all down, essentially. And because of that, this is definitely a human exercise, right? To a large extent, mathematical research is about humans communicating with other humans. It's not just about verifying true things about the universe, but bringing insight and helping people understand systems and so on.

You mentioned an identity crisis. I think what's happened, and this is reflected in a really nice write-up by Terence Tao, the famous mathematician, is that there were a lot of different signs of mathematical progress that used to be correlated with one another, right? For example, if I wanted to judge whether a mathematician has quality research or output, maybe I look at where they're publishing, the number of things they're writing, the interest of their results, whether they're earning these prizes, and so on. And now these things are a bit at odds. For example, the AI companies have figured out, as you pointed out, that proving some big theoretical result is maybe an interesting press release, right? And now there's another economic component to it.

Knowing how to write the right prompt, right?

That's right. And then there's a big question. At least in the cartoon version of a mathematician, you sit in a room, you think about a really hard problem, you write down a proof that's subtle and challenging. Now, as you say, maybe you're the guy who wrote down the prompt. If the prompt, for example, were "prove or disprove the Navier–Stokes conjecture," did that guy really bring a whole lot of insight to that problem? That's an interesting question, and one that I think we're really grappling with today.

Sycophancy and proof abundance

I want to ask more questions about credit, but just on the models themselves: does sycophancy exist when it comes to math prompts, the same way it exists for other, I guess, language-based queries? Does the model go, oh, what an interesting problem, you're on the right track, maybe look into this?

Judging from my email inbox, I think the answer is yes. It's a funny anecdote. All the time I get emails from people looking for an expert to verify something. And what I've noticed lately is that I get a lot of emails from outside folks who have been playing with their AI system, they've proven a new result, and the AI told them it was really important and that they needed an MIT professor to validate it, which is absolutely fascinating.

In fact, for fun, I played with some of these tools and put in a few open problems from my own domain. Interestingly, they did not get solved, so apparently my corner of math is safe at the moment. But the first thing it did was prove something that was, from my perspective, pretty superficial. And then it had a little footnote at the bottom saying, as far as I can tell, this doesn't appear in the literature; would you like me to convert this into the template for a journal? I think it explains a lot of what's going on in my email.

So you can imagine a future where you're overwhelmed with new proofs, and the bottleneck becomes the ability to actually verify them with humans.

That is a problem right now. And in fact, it's not just in mathematics, it's across academia. Interestingly, I would say the academic field that's been most affected by the phenomenon you're talking about is actually AI and machine learning itself. In that particular corner of academia, research is driven by conferences. You write your research paper, and there's an annual cycle where you submit and it undergoes peer review, and the whole community spends a month or two reviewing each other's papers. Sort of like passing your paper to the left in class and grading it; it's a very complicated version of that.

If you look at the number of submissions to the machine learning conferences, it is wild. About 10 years ago, I think, it was maybe one or two thousand. At the latest deadline for ICLR, the International Conference on Learning Representations, one of the big ones, I think it had 60,000 submissions. And if you think about it, it completely breaks the academic system the way we're used to thinking about it. We don't have the bandwidth to, never mind check all of these proofs, even open up the PDF. And so you're then relying on an extremely superficial review process. There are many anecdotes of high school students who have somehow gotten involved in peer review because they had a project once and the system decided they were okay, which is frustrating when you're trying to build an academic career. Essentially, some of the stamps of quality that we're used to in the field, you really can't rely on, for exactly the reason you identified.

The way Terence Tao put it is that we used to be in an era of proof scarcity. It was really hard, and a proof was this very bespoke object that took a lot of craft work and training to produce. Now we're in an era of proof abundance, so we have to think about different ways to do our job, and to do mathematics in a productive and interesting way.

Yeah, I think a lot of fields are dealing with very similar issues. Even take writing, for example. We used to have certain heuristics, shortcuts that we used. It's like, okay, if this person can construct a series of well-written sentences...

No spelling errors.

Yeah. I never make spelling errors.

...we might start from there: okay, this person might be worth reading, they might have something. We all have to operate with these heuristics and shortcuts.

That's right.

And now, obviously, producing text with no spelling errors or anything is available to literally everyone. So we have to go a step further, et cetera.

I think that's right.

Speaking of everyone now thinking they can come up with a proof: do you remember last year, Travis Kalanick, the Uber founder, was on the All-In podcast, and he was talking about how he was doing vibe physics with Grok?

Oh boy.

And he felt that he was, quote, "getting pretty damn close to some interesting breakthroughs."

Vibe physics. The crazy thing is, he could just hire a bunch of physicists to verify the research.

And then you'd have a real sycophancy problem.

Yeah. And I think beyond that, I work in education. One of the things we really rely on is somehow the friction in the creative process and the technical process. You mentioned writing; it's a fantastic example. How do you learn how to write? Well, you do a lot of writing. And now it's very difficult to convince our students to sit down and write an essay, because they have these AI tools that can do it. But writing a five-paragraph essay was never the goal. It was to teach you to do other stuff. Mathematics is experiencing a very similar challenge.

The students

So let's talk about the field of education itself. Are the students very anxious right now?

Absolutely.

Let's start there. What's the vibe among students, especially maybe grad students who started their careers before the idea of AI-generated proofs, and suddenly they're in grad school? What's going through their minds these days?

Yeah, the landscape of both graduate and undergraduate education is changing a lot. In the graduate school world, which for me would mean PhDs (in other worlds that's maybe master's degrees and so on), this divides into maybe professional education, preparing you for a job, and then academic-style PhD work. The PhD world, of course, is strongly affected by the larger trends and changes in the research universe. And when it comes to professional degrees, the main question we have to ask is, what is the right thing to teach our students in the current moment?

For example, I'm in a department that teaches computer science. The field of computer science is changing by the day, right? It used to be, for example, that we talked about programming as this core skill, a thing that everybody should learn from day one. I would argue that's probably still the case, the same way that learning to write and be literate is a key part of being a person in society, even if you're not doing it every day. But jobs as an actual programmer are becoming more rare, right? We see a decline in enrollment in computer science, for example, for the first time in like 20 years.

P versus NP, and the rest of the Navier–Stokes story

Since you mentioned computer science and unsolved questions, what is P equals NP all about?

I see, I can answer all your math questions today.

Yeah, absolutely. There are just all these questions I've stored up in my head.

Right, and we should come back to that. I don't think I actually told you the end of the Navier–Stokes story, so I can come back to that. But yeah, P versus NP and Navier–Stokes are examples of very famous mathematical problems that have been identified by the Clay Mathematics Institute as a benchmark for progress in modern mathematics, right? They're open questions we don't know how to answer.

P versus NP is a computer science problem?

Yeah, in the sense that it's basically asking whether all problems are solvable in polynomial time, meaning can your computer solve them... "all" is a little strong, but there are certain classes of problems that we know are solvable pretty efficiently. For example, your computer can sort a list of numbers really fast. There are other problems that we don't know how to solve efficiently, but that's different from saying we can't solve them efficiently.

Okay.

And so that's the fundamental question in that universe: do there exist problems that fundamentally you can't solve efficiently?

Is this like the known unknowns and unknown unknowns equivalent of the computer world?

Yeah, yeah. It's hard to communicate about it that way.

That's absolutely right. In fact, years ago there was a really terrible movie that came out that involved P versus NP: secretly some mathematician had proved that they were equal, but there was a government conspiracy because the markets were all going to collapse. I think it was called Travelling Salesman. Anyway, the reviews were really funny. They were like, it turns out it's really hard to make drama out of people in a room at a blackboard.

They should have used windows instead.

That's right, yeah. It would have made it a little more visually compelling at least.

Right. So a lot of these questions are yes or no. We don't know if a fact is true. Navier–Stokes is another one: either you need to show that these equations have a solution for all time, like fluid moving around, or give a counterexample where they don't. So in this AI case, what the AI gave was a counterexample. It said this conjecture is not true; there exists a particular configuration of your fluid that becomes infinitely turbulent in a finite amount of time.

With a special spoon.

That's right, with a very special spoon. And the reason I'm highlighting that is because I think it's a really great example of the really complicated relationship between humans and AI. The construction of that spoon arguably was done in large part by a team of human mathematicians in Spain, on pen and paper. And then essentially what happened was, there was also this interpersonal fight between two different AI teams, which is a whole hot mess. But that aside, these two teams were in some sense leveraging those mathematicians' progress over a very long period of time, and then doing that last mile. So it's true that the AI made some really critical insights in that particular counterexample, but it isn't true that it's a totally surprising result that came out of left field.

Who gets the credit, and the million dollars

Well, this seems like a scenario academia is going to have to deal with more and more. You could have a theoretical situation where one person or a team comes up with the problem, one person or a team comes up with the solution, and then you have a model that generates the verification, or the final leg of the problem, or whatever. Who deserves credit in that scenario? How do you actually handle that?

How do we handle that currently? Poorly. And it's a really interesting and challenging discussion. I don't think we have a good answer right now. It used to be, for example, that these Clay math prizes, which are at the very top of the universe there (I'm not going to get one, not going to get anywhere close), were signifiers of two things. One is that the person who proved the theoretical result is a really smart person. The other is that there's been a big achievement in mathematics itself: we now know something to be true, or false, that we didn't know before. Now it's not clear that those two things are correlated, as you say, right? If a computer generates the interesting part of a proof, who deserves credit? Should that person be hired if they're the one who wrote the prompt? It really is an interesting question for a math or an engineering department, and it's not so clear.

I think it really depends on your value systems. What do we get out of this exercise of proof writing as humans? For example, is a counterexample to Navier–Stokes going to revolutionize the markets or change the way we write software? Almost certainly not, right? So this is more a demonstration of a technology than any particular practical thing. And so I think we have to be really careful in understanding the incentive system behind who's claiming credit, and also in making sure that the humans who are doing all of the labor of composing the data that goes into these systems, which is getting remixed into a proof somehow, get some attribution. The technology is currently really bad at that, right? You talk to an AI, and it's not so clear where it's getting its answers from, even though we know that it's echoing data that may be scraped off the internet, or someone else's research paper, or what have you.

Right, and this might be the time when we still know that for sure. Five years from now, who knows where it's getting its data from? It could be scraping it from itself, or other models could be coming up with their own stuff.

I have a slight aside, which is the prizes you mentioned, which have also come up in movies like Good Will Hunting. The prize money that people get for that, is that for them personally, or is it supposed to go into their research?

That's a good question. I have no idea. The Clay award specifically, I'm not sure.

Okay. Because some of them are big. And so now I'm envisioning the scenario where maybe one day a model ends up winning some big mathematics prize, and the money goes to the lab.

I don't know. I think one day I need the one million dollars.

Well, yeah, yeah. I'm just wondering.

It's actually a good question, and we now have an example of it. With this Navier–Stokes problem, I should be clear, the proof has not been officially verified and checked. In particular, there's a rule that it has to exist in the community for some number of years and not be refuted. But we have to ask that question right now. OpenAI, arguably because they rushed at the last second, were maybe the first to release something that satisfies the Clay criteria. It's kind of a choose-your-own-adventure situation: there are a few different conjectures, and you could prove any one of them and get this particular prize. OpenAI was quick to say that they don't need the million dollars, because of course they don't. In about two minutes they're burning through a million dollars' worth of computing anyway, which I think raises the question: well, why are they doing this? It's not for the money.

Yeah.

Now, I don't think it was for the money for the mathematicians either, but the million dollars made a big difference to an individual.

There was some story about a Russian mathematician who solved one of those big prizes, right?

Yeah.

And then declined the award altogether.

The money. That's right.

I'm not going to decline the money, for the record.

No, I would happily take it, when I finally solve some unprovable mathematical theorem. But actually, this leads to another, more serious question. It costs a lot of money for these models to be run, and they consume vast amounts of tokens for very advanced mathematics. Do you end up in a situation where whoever has the most tokens wins the mathematics prize?

Arguably, I think, that is one of the major concerns in our community right now. And by the way, you mentioned education; it's also a concern there. Does the student who has a $200 account get to have a better homework solution?

Yeah.

It's a real question we have to grapple with right now. On the advanced end, I think we saw that. For example, I don't think it's controversial to say that when the news came out that people were getting close to this Navier–Stokes counterexample, it appears that OpenAI suddenly poured some ridiculous amount of tokens or money or whatever into writing their proof first. The typical academic can't do that.

And I think there's an important element: I believe it was with an unreleased model. And so I'm curious. Let's say you're a mathematician and you're an enthusiast. You say, you know what, I'm going to embrace this new world of AI augmenting our understanding of mathematics. If you're a mathematician at a university, it's certain that there's a model inside the labs that's better than the model you have access to.

That's right.

And so you start to work on some problem. You start to demonstrate that using AI you can actually make progress towards it. And then suddenly it's like, well, they have a model two generations ahead of you, and they knock it out in a weekend. And by the way, for all you know, their model was trained partially on your work. It's really pernicious if you think about it.

Yeah, that's right. At least in the current moment, the AI system is very centralized, right? There are two or three tools that almost everybody is using, and they're owned by these companies. When you write your query, it goes to their servers, and they're basically the guys controlling the faucet. That's a big concern.

Another issue related to that is research funding. As your listeners may or may not know, even independently of AI, research funding is currently in a bit of a crisis. It's in a much more challenging moment than it was in the past. I'm so fortunate to work at MIT, and even for me it's a challenge to pay for my graduate students and so on, and the costs change every year. Now we have to add on to that, for example, AI accounts. That sounds theoretical, but it's not. My PhD students, for example, want the most expensive possible Claude account, of course, which is great, but at, what, $200 per person per month, it almost becomes comparable with the cost of another human.

Elegant or grindable?

Now, I read that some people had surmised that, of the outstanding Clay prizes, Navier–Stokes might be the first one solved. And I think it's interesting that it was solved via a counterexample, and it raises the question: did the model find an elegant solution, or did it just grind? Is this a grindable problem, where it just ground through a lot of states and then hit on the magic spoon that proves it? Finding that counterexample seems like a process of grinding through hundreds of thousands of different states. Is that elegant math, or is that just some sort of sheer mechanical process?

That's a great question. And indeed, you mentioned at the beginning of this talk that you opened that paper and it's totally unreadable.

Yeah.

And that's not just a question of whether you have a math degree or not. I can't read past the first couple of pages, right? In that particular result, I think there was some beautiful insight, again arguably generated by humans, that went into the overall construction. And then indeed the proof is really horrifying. There's a lot going on. It's technical; it's got inequalities and bounds and indices and all kinds of crazy stuff. And so, as you say, in this particular case, is it elegant? Probably not. Is it correct? Yes, probably.

So again, the basic question: why do we do this exercise? Is it just so we can now say with confidence that this conjecture is false? Or is it that we learn something along the way that'll bring insight to some other problem? Those two things are now divorced, right? If you just have this frictionless way to prove a theorem, then all the beautiful insight, and maybe the fun garden paths along the way that might have inspired some new research field, have been pruned out, which is a big challenge, right? In some sense, that's really the key human aspect of all of this.

Can the machine ask a new question?

So this was going to be my next question, and I guess I'm really asking you to opine on whether or not you think AGI is going to be a thing, or superintelligence, or whatever we're calling it now. But okay, AI right now can sort of brute-force a solution, and it's not particularly elegant. Can it come up with ideas that are separate from what we already know, or from questions we are already positing? Can it actually inform us that, no, the way you think the world works is actually different? Can it unveil mathematical rules, or rules about the world, that we are not even thinking about or asking about?

New conjectures.

Yeah, exactly. New conjectures.

It's a great question. Actually, my colleague Nestor Guillen at NYU has a nice blog post that talks about this a bit. When you look at a lot of the proofs and new results coming out of these AI systems, they're arguably in what mathematicians might call the convex hull of existing knowledge. Meaning there's a bunch of stuff we already know, and a lot of these proofs are really intricate and interesting ways to mix and match those things, kind of inside the span of knowledge we already have. And you've asked the key question, which is: can a mathematical AI system actually come up with a truly surprising new result? Not just mixing and matching facts we already know, or maybe finishing off a calculation that somebody got stuck on, but making an entirely new theory, for example, that just wasn't present before. I don't think we've seen a whole lot of examples of that so far. This technology changes by the day, so who knows. But at least so far, I think, it's for the most part in that first category: maybe making really interesting and surprising connections between things we already knew, but not necessarily generating something entirely new out of left field, right? That would really be the next step for a lot of these tools.

I know one of the questions, too, with some of the, I think, maybe lesser, less prestigious proofs that have been established, is, okay, maybe they just ground their way to an answer, right? And I know another question is, well, was this already proven somewhere? Maybe there's some obscure Russian math journal where most of the work had been done, and most mathematicians weren't aware of it, but it was in the training data. And so...

Hard to know.

Yeah. Data attribution in AI generally, mathematics or otherwise, is a really critical problem. It's exactly the same problem as, my AI generated a cool new cartoon; did it borrow from Disney, and is that okay? In some sense it's a very similar problem, and the current tools are very bad at answering it.

Teaching in 2026

Why don't you just tell us a little bit about being a professor in 2026, and what you're doing differently in education?

Yes. The education field is having its own, I don't know about identity crisis, but certainly a lot of navel-gazing and thinking very hard about the fundamentals. MIT actually released a really nice little report on AI and education. I encourage everybody to take a look. My colleagues Eric Klopfer and Sam Madden were in charge of writing it.

But every aspect of education has changed. I think one thing that's maybe undervalued is how much of education is social, even if the primary artifacts you're producing are things like problem sets. One thing we've seen among our youngest students, for example, the early undergraduates, is that it's really hard to draw them out of their dorm rooms. There's an interesting perception challenge here. You now have, again, this frictionless robot that can help you with your problem sets, or, frankly, do your problem set for you. Is that learning? Probably not.

Now, the interesting thing is that there's a difference between perception and reality. A lot of our students may perceive that they're learning when they use these different AI tools, but then they go and take the exam, and the score says otherwise. So what do you do to maximize the chance that your students are learning?

At MIT we have a slogan. I think it's roughly "mind and hand," and then they added "and heart" on the end, which is good. And I think we're really leaning into that second one. We've known since long before AI that putting a bunch of students in a classroom, talking at them, and then having them work on a problem set alone is not a very effective way to teach. So now I think we're really focusing on experiential learning, on getting the students to show up physically, and on answering the question of why we take all these students and put them on a college campus instead of having them work from a laptop at home.

Now, I would argue that's still a really valuable exercise. MIT, for example, has the luxury of having the world expert on just about anything physically within a mile of your dorm.

Yeah.

And so we're designing all kinds of programs, for example in the School of Engineering, that try to bring the students in for research experience, for hands-on learning, for going into industry and seeing what things look like there, and trying to build the undergraduate experience more out of that and a little bit less out of exams and problem sets.

Now, there remain a lot of questions. For example, assessment is a big challenge for us. What we've observed in the last year or two is that our students are getting nearly 100% on all their homework. Maybe it's because MIT students are super bright.

It's a great grade for a professor.

Can I just say? I take the train from New York to New Haven quite a lot, and there are a lot of Yale students, and I watch them on their laptops. They're all on, mostly, Claude actually, writing essays, or at least doing the basic research via chatbot.

So this semester, for example, I'm teaching a course called Shape Analysis that covers discrete differential geometry, if you're into that kind of thing.

Of course.

Who isn't, right? It's a pretty mathy class. In the past, most of what the students did was two things: written assignments, and then an open-ended project where they come up with a new result and write it down. And both of those had to change.

So in our course now, the way we structure it, there's still homework. But the analogy I'm using, as are many of my colleagues at MIT, is closer to weightlifting. You don't do it because it's fun, necessarily. I do sometimes, but not everybody's like that. You do it more because you're trying to achieve some other goal, right? You're preparing, or you're becoming a stronger person. And then maybe you have to test understanding in some other fashion. So we have a short quiz after the homework, basically a copy-pasted homework problem, but it makes sure that the students at least comprehend what they did. We also had to restructure the project a bit, add an oral presentation component to it, and bring in some more interaction, to make sure that essentially we're educating the human and not just testing the capabilities of Claude, right? Those are very different things. And I would argue the real goal of an institution like MIT is to prepare brilliant students for the next set of problems.

Brewing beer before yeast

We were talking about how difficult it is to just look at the output of the Navier–Stokes solution, evidently even for mathematicians. And I was thinking, okay, so what's the future of math, then? And I had this analogy in my head: we knew how to brew beer before we understood the science of yeast, right? Alcohol has been around a long time. People knew the process, but they didn't understand why the process worked. And then eventually they discovered yeast and why this sequence works. And so I thought, well, could it be that the future of science is, yes, the AI models are going to find all these answers, or be able to do all these things, but there will still be a role for humans to, I don't know, translate it or explain it?

I think that's right.

And then I also wonder, well, probably the AI is also going to be better at translating and explaining things too. But could there be a general reorientation of math such that, rather than discovering new things per se, it's about understanding these things that have been proven?

That's right. I think the right way to think about these AI systems is that they're tools. People's jobs evolve all the time, and apparently mine is no different. I want to be clear: I don't think we need to be subservient to the AI and just check what it makes.

Sure.

I guess there are a few different aspects of that. One is, of course, that my job is both education and research, and I think for the education part, even if what it looks like day to day has changed a bit, the mission is still there. We still want to educate the world's best humans at MIT, and we do an okay job of that.

In terms of the mathematical labor, I think one thing that's really undervalued in this field is that the typical mathematician is not working on the Clay Mathematics Institute problems. Those are really hard. They're for really brilliant people, and I wish I could do it. I can't compete there. And similarly, math contest problems were another thing the AI companies were quick to pounce on.

The Math Olympiad.

Exactly. That was the previous generation. All of these are great in the sense that they're problems you can write down. They're well contained. They're in a little box. But a lot of mathematics is more about taste than it is about just proof writing and verifying some conjecture that somebody else wrote down.

For example, in my particular area (and again, I want to be clear, I'm an applied guy; a lot of pure mathematicians would absolutely look down on what I'm doing and say, ah, that's not real math; I would say otherwise, but we'll have that debate another day), by the time we get to writing a proof, we probably already know it's true. It's more of a double-check situation. The really interesting part is the modeling, and figuring out which questions to ask and what insight they bring us about the universe. And the AI tools are not doing that, right? And so you're right: maybe proof writing becomes more of an exercise in writing prompts. But writing the prompts is very much what we've been teaching our students to do for centuries now, and that skill, I think, persists.

Yeah. Broadly, it feels like the emphasis is shifting to being able to ask the right questions and come up with new ideas, versus just coming up with a basic answer.

That's right. And the typical math student probably doesn't end up in theoretical math. They maybe end up in finance, they end up in engineering, and for those folks that skill is the really critical one, right? It's exactly what you need to be a good manager, to be a good project designer, to lead and innovate in all kinds of different fields. And that skill is somehow becoming more important than ever, while all that boring proof-writing stuff that everybody's allergic to is actually being pushed to the side a bit.

So we should still teach kids math?

I think so.

Justin Solomon, thank you so much for coming on Odd Lots.

Of course. It's a pleasure.

Out of the chip thinking

Well, I think, to say the least, the future of math and math education, it's like, we have no idea where this is going, right? Because the models are going to keep getting better. And even just the prosaic point: okay, here you have some conference that used to have a thousand submissions, and now it has 60,000 submissions. Some of them might even be very good, but that's the type of thing that, as he said, pretty straightforwardly breaks existing systems.

Yeah, it feels like there's a tautology here with AI, where because of AI a lot of things are getting easier and more difficult at the same time, right? We have the ability to generate tons and tons of content, but the ability to verify it doesn't necessarily go up by the same amount at the same time. Maybe it will one day.

I also think about the emphasis on questions rather than simple answers, and the idea of taste even in something like mathematics, which is kind of alien to me. You hear about it in code all the time, the ability to write elegant code. And I guess the same applies to mathematics: the ability to elegantly prove a formula, or elegantly come up with a theory. That seems to be repeated throughout everything now. If you're able to generate ideas, think outside the box, to use a terrible cliché, and curate with some taste, or write or perform with a particular amount of taste, that seems to be the edge here.

I would also say this is actually a pretty high-stakes question even outside the academic setting, this research taste. Because, of course, one of the big questions for AI, period, is this idea of recursive self-improvement. Can a model build a successor model?

Right.

And to do that means generating new ideas to then experiment on. And I think it's still a pretty big open question whether the models are good at novel questions, right? So, okay, there's a bunch of outstanding questions, there's reason to think they're solvable, and maybe the models get very good at that. Can the model come up with the next Navier–Stokes? Can it come up with the next P equals NP? Can it come up with the next Fermat's Last Theorem? That's in math, but whatever the equivalent of that is in areas like machine learning and so forth, it's a pretty important question for the future of AI: are the models good at proposing the next research experiment, and then, of course, carrying it out, conducting the test?

Right, and they might be one day, but...

But right now there's clearly still a pretty significant role for a human in the loop, I guess for intuition or whatever, to say, okay, let's try this out, et cetera. And my impression is that the models still aren't great at that particular aspect of, I guess, the scientific method.

Yeah, I think that's what Justin said as well. So that gets me back to this idea that the human edge remains idea generation, out-of-the-box thinking, whatever you want to call it. Out of the AI box.

Out-of-the-chip thinking.

Out-of-the-chip thinking, that's good. Shall we leave it there?

Let's leave it there.