DeepMind’s AlphaGeometry AI: 100,000,000 Examples!

My goodness, AlphaGeometry, an  amazing Google DeepMind paper fresh out of the oven. And I think this  paper might be history in the making. So, how is an AI that learned on a 100,000,000  mathematical problems able to compete in the International Mathematical Olympiad?

That is  perhaps the most prestigious math competition in the world. So, did they achieve a  breakthrough? Well, let’s see together.

Dear Fellow Scholars, this is Two Minute  Papers with Dr Károly Zsolnai-Fehér. Now, competing on such problems is trouble. This  requires not just a huge glorified calculator for adding numbers, but it requires planning your  solution, logic and reasoning.

And more. But, of course, GPT-4 can do all of that, so just give  the problems to it and and off you go. End of the video, right?

Just as a demonstration, GPT-4  can ace the bar exam, would be hired to Amazon for its coding skills, and it is better than  almost all of humans on the biological olympiad. So, it can sweep all of these, easy, right?  Well, let’s see, if you try to use GPT-4 and give it 30 of these tasks, it can solve  exactly…wow.

It can solve zero of them. I hope this gives you a feel of how difficult  these problems are. Devilishly difficult.

And scientists at DeepMind just proposed a system that  is about a 100 times smaller? I can’t believe it. But to start, how do you even solve such a  problem?

Let’s see an easy example. For instance, if we ask a human mathematician to prove  that there is an infinite number of primes, we could prove it by listing them. But no  one can list an infinite amount of numbers, that is impossible.

No. So what do we do  instead? How do we do the impossible?

Well, instead we start with the assumption  that there is a finite number of primes, and then find a contradiction that says  this assumption cannot possibly be true. This part of the proof requires thinking  out of the box. It requires a moment of brilliance.

And without it, the problem is  intractable. It is akin to pulling a rabbit out of your hat. And when you have the rabbit,  mechanically finishing the problem is relatively easy.

And the strategy could be that if the  rabbit didn’t work, try to pick a new one. We can do that. But, can an AI, can a machine  pull a rabbit out of a hat?

And the initial answer is no. Mostly not. But let’s try  anyway.

This is how a human would do it, and here is their proposed AI  that could hopefully try to do it. So, does it work? Well, let’s have a look. 

When given a problem, with blue, it first creates the key ideas, the rabbit, and then, the  green part is the remainder of the calculation that leads to the solution. Okay, but this was  easy peasy. Now give me some proper problems.

Oh my, now we’re talking! So, little AI,  can you solve this? Whoa, it pulls the blue rabbit out of the hat, and then, runs  the green calculations until it solves it.

And make no mistake, this is just an excerpt of  the solution. Now hold on to your papers Fellow Scholars, because the full solution looks  more like this. My goodness, look at that.

More than a hundred steps have been concealed  here. And it had done all of this correctly. And that is not even the longest proof it  is capable of writing.

Not even close. Wow. So, how good is it?

Well, a previous technique  was this good, and the new technique without the rabbit is this good. So it is almost as good  as the average mathematical olympiad contestant. Note that these are really smart people, so the  average of those is also really smart.

And it can compete with that. But, wait a second.  This can run the mechanical calculations, but it only takes you so far.

You also need the  brilliance to pull the rabbit out of the hat. And as soon as we add the rabbit part of the  solution, what happens? What?

Are you seeing what I am seeing? It is nearly as good as  the smartest of these super smart people. And, Fellow Scholars, if you think that is  impressive, hold on to your papers because we are just getting started.

Here are  2 mind-blowing facts about the paper: One, it learned from scratch by itself, without  any human demonstrations. Yes, the proposed system does all this without human intervention. This  is essentially an AI implementation of the two modes of human thinking, and that is thinking  fast and slow.

Thinking fast is about quick, instinctive responses, like reading something,  while thinking slow involves deliberate, logical, and calculated decision-making. This can do both  as well as some of the smartest humans can do. So when we are worried that this  cannot possibly get any better, because there are no humans good enough to  teach it anymore, now we know that it only needs synthetic training data, so it can learn  by itself.

And it has already found more general, more elegant solutions for some  of the tasks than humans did. Two, this project is open source from day 1.  Every piece of the solution is out there for you, for free.

Yes, you Fellow Scholars  can run your own experiments with it. And all this with a model that is about a  100 times smaller than GPT-4. An absolute slam dunk of a paper.

We are still early, but  I think it might be fair to say that this is a breakthrough. Now, as incredible as this AI is,  note that it is still relatively narrow. It can do geometry, but it cannot play StarCraft or do  anything else.

However, the ideas and concepts described in the paper are general enough to make  sure that this can be applied to other problem domains as well. And that, Fellow Scholars, is  going to be a series of incredible breakthroughs. By the way, it is a possibility that I will visit  San Francisco around mid April.

For the first time ever. If this is the case, if you are a local lab  like DeepMind and OpenAI and you would like me to visit, or if there is someone who I should  really meet, please let me know on Twitter/X.