a lot of computer scientists believe that if we just make AI models big enough they'll eventually learn pretty much everything including reason and I certainly hope they will I mean wouldn't you want an autonomous vehicle to follow basic laws of logic officer if I didn't hit anyone I wouldn't be here but I'm here so clearly I didn't hit anyone but I like my dystopia to have scientific support so when I heard that a group of Searchers has demonstrated that the current AI can't reason I of course had a look what is reason I'll leave this
to your favorite 3-hour philosophy podcaster to discuss for the purposes of this video reasoning will just mean getting basic maths and logic right you know arguments of the sort if pizza then no pineapple I hope that's clear the AIS that we use today were not born knowing logic they're almost all of the same type deep neuron Nets trained on large amounts of data sometimes with reinforcement learning these AIS learn to recognize patterns and to interpolate and extrapolate from them no one knows how much they can learn if you feed them more and more data some
computer scientists believe that as the models are trained on more and more data they acquire emergent abilities indeed they say this has already happened these emergent abilities are unexpected new features that have suddenly appeared as models got bigger such as for example the ability to unscramble words or to understand relative locations in space including Geographic Maps they've also acquired mathematical abilities if you look at how these models perform on some Benchmark tests then their performance on standard math tests has dramatically improved as the models have gotten bigger this naturally raises the question will they just
go on to learn everything I strongly doubt that these emergent abilities will ever give rise to sound reasoning skills the models can learn patterns that are in that training data but language is not good for encoding logic or maths in the first place it's just too vague and people make logical mistakes all the time like I've heard that some of them put pineapple on pizza but the question of whether deep neuronet can reason is more relevant today than ever because many companies might use them for cases that require solid reasoning abilities it's not just autonomous
vehicles imagine an AI advising governments or the military or heaven forbid pizzerias I don't know about you but I want to know whether I can reason and the easiest way to test this is with math questions so far I've not been impressed by the math abilities of today's large language models here is me asking GPT for the smallest integer whose square is larger than five but smaller than 17 it gave me two garbled answers According to which the answer is three the correct answer is minus 4 you can see what goes wrong there GPT neither
knows what integers are nor know what order relations are it just throws together text and math symbols that look more or less plausible the output it provides is close to the correct answer in the sense that the text is semantically similar but it's very far in terms of mathematical accuracy in the new paper now a group of researchers from Deep Mind and apple systematically studied this very question they looked at the reasoning capabilities of large language Which models including the news GPT and llama with a common maths test called GSM 8K that's great school maths
with about 8K questions it's not a complicated test it consists of questions like James writes a three page letter to two different friends twice a week how much pineapple does he put on pizza just checking if you're listening the question is how many pages does he write each year and in case the police come to ask the correct answer answer is 624 for the new study they altered the Math's questions by changing names or numbers or adding unnecessary text they found that the math performance significantly dropped for all models this strongly suggests that the models
don't actually reason they just draw language relations and output something similar to what they've learned though the job is just a few percentage points so it's not as bad as it sounds then again this was a super simple maths test so it may be worse than it sounds one can also argue that it it have been only fair to compare the results with that of humans instead of asking for a perfect score if you ask me how many pages James writes per year I might well answer pineapple just to get rid of you indeed just
a few days after this study came out a parody paper paper appeared with a comprehensive analysis of human reasoning from the Chicken Little Center for stopping the sky from falling the fictional authors who include cognos biasm claim among other things that the fact that some humans still Advocate communism even after having been presented with the undeniable evidence that communism has never worked anywhere in the entire history of the world is proof that homo sapiens are basically crazy monkeys who excel at creative storytelling rather than genuine logical analysis I believe they're joking but why humans learned
logical reasoning to begin with is a good question I'd guess we learned it more or less because nature obeys the rules of logic this makes me think that the more AIS learn about physical reality the more likely they are to understand reasoning basically I'm saying teach them physics what else would you expect a physicist to say I used to struggle to keep up with news especially during election campaigns when you think you kind of need to know what's going on if you know the problem I recommend you check out ground news which has really helped
me ground news is a news platform that collects and summarizes news which have been published all over the world not only do they collect all articles on the same story in one place and give you a quick summary but they also give you a lot of extra information that you don't find in the standard media I found this to be so useful to keep track of what's going on in the US election they have a special page for this for each item they'll tell you how much it's been covered by right center and left leaning
Outlets you also get a factuality rating for each news item and it tells you whom the media Outlets are owned by it's really saving me a lot of time and it's not like they only cover US News you can set your preferred location for example to Europe if you want to give it a try yourself use our link ground. news/ Saina this will get you a big discount on the Vantage plan with access to all the features so go and check this out thanks for watching see you tomorrow
