Ex-Google CEO's BANNED Interview | Eric Schmidt

Eric Schmidt gave a facety talk at Stanford just a few days ago about AI he was of course the former Google CEO and perceived by people as a bit of a futurist he's able to kind of often very accurately predict where technology is going the interview was over an hour long and it was fascinating covering topics like ai ai agents where this whole thing is going Etc and had a lot of pretty brilliant insights by Eric into where he believes this is going but unfortunately at our plus long interview there was just one comment that

people didn't like the media machine went into overdrive mode and the video has been basically taken down it's no longer available to see it was deemed too hot for the internet and no longer available but just between you and me and keep this on the DL please I might have a copy of it maybe allegedly I feel like I need to say something here that reduces my liability some sort of a legal statement like I take no resp responsibility for my actions yeah that should do subscribe for more sweet AI content let's take a look

at a few clips from this Stanford lecture by Eric Schmidt about AI I think you will like it if you do hit that thumbs up please because this is somebody that has a bit of a Clairvoyance about the future of tech he tends to see how things will unfold with startling accuracy so before he starts talking he asks the class to basically Define what a context window is and what AI agents are context Windows of course you can think of it as sort of a short-term memory it's basically the amount of words that a large

language model can read and also write and output and kind of interact with right so if it has a large context window you can give it a whole book and then ask it questions about the book if it has a very short context window we might start answering a question and then halfway through forget what I was talking about it would be kind of a whoops I run out of memory type of thing and AI agents are things they can sort of complete tasks on your behalf you can tell them to go order something off

of Amazon they'll go they'll search they'll do reviews they'll check the sizes the colors whatever and then order it and then report that they've done it back to you so you can think of it as kind of a smart intern that's online that's Ai and Eric is talking about potentially a seeing a 10 million token context window sometime soon potentially in next two years or so he doesn't specify a timeline but soon so the first thing he talks about is what are we going to expect to happen in the next 2 to 5 years so

in the short term specifically at the intersection of something like a 10 million token context window and improving autonomous AI agents take a listen so an agent is something that does that does some kind of a task another definition would be that it's an LM State and memory okay can anybody again scientist can can any of you define text action taking text and turning into an action right here go ahead yes instead of taking text and turning it into more text more text taking text and have the AI trigger actions based so another definition would

be language to python a pro programming language I never wanted to see survive and and everything in AI is being done in Python there's a new language called Mojo that has just come out which looks like they finally have addressed AI programming but we'll see if that actually survives over the dominance of python um one more technical question why is NVIDIA worth2 trillion doll and the other companies are struggling technical answer I mean I think it just boils down to like most of code needs to run with C optimizations that currently only support so other

companies can make whatever they want to but unless they have the 10 years of software there you don't have the machine learning optimization I like to think of Cuda as the C programming language for yeah right that's the way I like to think of it was founded in 2008 I always thought it was a terrible language and yet it's become dominant there's another Insight there's a set of Open Source libraries which are highly optimized to Cuda and not anything else and everybody who builds all these Stacks right this is completely missed in any of the

discussions right the com it's technically called VM and a whole bunch of libraries like that highly optimized Cuda very hard to replicate that if you're a competitor so what does all this mean in the next year you're going to see very large context Windows agents and text action when they are delivered at scale it's going to have an impact on world at a scale that no one understands yet much bigger than the horrific impact we've had on by social media right in my view so here's why in a context window you can basically use that

as short-term memory and I was shocked that context Windows get this long the technical reasons have to do with the fact that it's hard to serve hard to calculate and so forth the interesting thing about short-term memory when you feed the you ask a question read 20 books you give it the text of the books is the query and you say tell me what they say it forgets the middle which is exactly how human brains work too right that's where we are with respect to agents there are people who are now building essentially llm agents

and the way they do it is they read something like chemistry they discover the principles of chemistry and then they test it and then they add that back into their understanding right that's extremely powerful and then the third thing as I mentioned is text action so I'll give you an example the government is in the process of trying to ban Tik to we'll see if that actually happens if Tik Tok is banned here's what I propose each and every one of you do say to your L the foll make me a copy of Tik Tok

steal all the users steal all the music put my preferences in it produce this program in the next 30 seconds release it and in one hour if it's not viral do something different along the same lines that's the command boom boom boom boom right you understand how powerful that is if you could go from arbitrary language to arbitrary digital command which is essentially what python in this scenario is imagine that each and every human on the planet has their own programmer that actually does what they want as opposed to the programmers that work for me

who don't do what I ask or potentially a whole Army of AI influencers each with its own little tweak that are completely managed by these AI agents trying to fill a certain Niche a certain Gap a certain personality type you can have an infinite amount of them boasting 24 hours a day the programmers here know what I'm talking about so imagine a non- arrogant programmer that actually does what you want and you don't have to pay all that money to and there's infinite supply of these program this is all within the next year or two

very soon those three things and I'm quite convinced it's the union of those three things that will happen in the next wave so you asked about what else is going to happen um every six months I oscillate so we're on a it's an even odd oscillation the moment the gap between the frontier models which they're now only three I'll ref few who they are and everybody else appears to me to be getting larger six months ago I was convinced that the Gap was getting smaller so I invested lots of money in the little companies now

I'm not so sure and I'm talking to the big companies and the big companies are telling me that they need 10 billion 20 billion 50 billion 100 billion is 100 billion right they very very hard I talked Sam Stargate is Microsoft and open AI sort of co- project where they plan to invest a 100 billion into a Data Center and by the way that's not the only project where they're kind of working together the other one is the power plant the fusion power plant somewhere near Seattle Washington in that area that they hope could potentially

provide a lot of you know maybe not unlimited energy but clean cheap inexpensive energy to power a lot of the stuff next he's talking about what he and Sam Alpin talked about Sam Alman is a close friend he believes it's going to take about 300 billion maybe more I pointed out to him that I done the calculation on the amount of energy required and I and I then in the spirit of full disclosure went to the White House on Friday and told them that we need to become best friends with Canada because Canada has really

nice people helped invent Ai and lots of hydrop power because we as a country do not have enough power to do this keep in mind too that there's a lot of sort of heat generated in these processes so the need to cool these buildings these infrastructures you know if you're running it somewhere where it's hot you also have to consider your AC Bill running it somewhere where it's naturally cold old could be very you know profitable you would greatly slash your costs because you don't have to worry about cooling or at least not as much

the alternative is to have the Arabs fund it and I like the Arabs personally I spent lots of time there right but they're not going to adhere to our national security rules whereas Canada and the US are part of a trit where that we all so the1 billion $300 billion data centers electricity starts becoming the scarce resource well but and by the way if you follow this line of reasoning why did I discuss Cuda and Nvidia if $300 billion is all going to go to Nvidia you know what to do in the stock market okay

that's sounds like he's recommending people buy Nvidia not a stock recommendation I'm not a licens well part of it so we're need a lot more chips but Intel is getting a lot of money from the US government AMD they're trying to build you know Fabs raise your hand if you have an Intel computer in your Intel chip in any of your Computing devices okay so much for the Monopoly well that that's that's the point though they once did have a monopoly absolutely and Nvidia has a monopoly now so are those barriers T like like Cuda

is that is there something that other so I was talking to Percy Percy land the other day he's switching between tpus and Nvidia depending on what you can get access so for those who are not aware one big race that's happening right now is kind of the race to produce these chips that power AI growth and Nvidia has been kind of the I mean you can say the Undisputed leader in the field producing gpus graphical processing units and those have been very effective they kind of started with you know computer games acceleration because instead of

running each process kind of independently finishing and running the next one which could create bottlenecks it runs them in parallel so for example if you're rendering and gain them on screen right you have some some crazy 3D game with solar rays and whatever that's running in you know 30 frames per second 60 frames per second you know what I mean you need to be able to Rend each pixel on the screen in real time you can't wait for it to get rendered you have to kind of render it all in parallel and gpus were really

good at that that's part of the power of why Nvidia is so powerful as well as Cuda the programming language that they run on but we do have other alternatives for these AI chips the TPU the tensor processing unit is Google's creation recently we've also seen Gro which is a something that they refer to as a language processing unit and lpu there's been some talk of neuromorphic chips which are more designed like the human brain they're not digital they're more analog but I don't know if any progress has been made in that field and it

sounds like Sam Alman is cooking up a brand new chip that might eventually go on to compete with Nvidia that's still in the very early stages but that's the big question is NVIDIA continue to dominate or is somebody going to come out with something that's brand new and much much better for AI specifically cuz gpus aren't necessarily the best possible sort of architecture for AI they happen to be very good they happen to be the best thing we have right now but nothing's to say that some different architecture can't be a million times better just

because it's built from scratch for AI inference and training absolutely and Nvidia has a monopoly now so are those barriers like like Cuda is that is there something that other so I was talking to Percy Percy the other day he's switching between tpus and Nvidia chips depending on what he can get access to because he doesn't have a choice if he had infinite money he would today he would pick the b200 architecture out of Nvidia because it would be faster and I'm not suggest I mean it's great to have competition i' AMD and Lisa Su

at create length they have built a a thing which will translate from um this Cuda architecture that you were describing to their own which is called rock it doesn't quite work yet they're working on it so Lisa Sue that's of AMD so one of the Nidia competitors AMD and Lisa Su great length they have built a a thing which will translate from um this Cuda architecture that you were describing to their own which is called roam it doesn't quite work yet that get transcribed as cud architecture but that's not what it is he's saying Cuda

architecture cud da so the Nidia sort of architecture versus Rockham I'm not familiar with that one I guess that's amd's version of Cuda U you were at Google for a long time and uh they invented the Transformer architecture it's all Peter's fault thanks to uh to brilliant people over there like Peter and Jeff and everyone um but now it doesn't seem like they're they kind of lost ini so I guess this is the part that caused all the hubub I'm just skipping it I'm not touching it it's not really related to AI necessarily unfortunately I

think there's a lot of overlap with politics so I'm just trying to avoid that but it seems like he's saying that it might be like a cultural issue but let's continue and see what else they've talked about yes sir in terms of national security or interest have play a role or competition with China as well so I was the chairman of an AI commission that sort of looked at this very carefully and um you can read it it's about 752 pages and I'll just summarize it by saying we're ahead we need to stay ahead and

we need lots of money to do so our customers were the senate in the house um and out of that came the chips act and a lot of other stuff like that um a rough scenario is that the chips act invested billions into building out the US infrastructure to develop AI chips and also I believe that's what limited a lot of the nvidia's export to China some of the best chips the most powerful chips it limited the exports to China how much they could ship and sell in China how it seems like that's creating a

bit of a black market of people trying to import those into China through other means but if you assume the frontier models drive forward and a few of the open source models it's likely that a very small number of companies can play this game countries excuse me what are those countries or who are they countries with a lot of money and a lot of talent strong Educational Systems and a willingness to win the US is one of them China is another one how many others are there are there any others I don't know maybe but

certainly the in your lifetimes the battle between the US and China for knowledge Supremacy is going to be the big fight right so the US government banned essentially the Invidia chips although they weren't allowed to say that was what they were doing but they actually did that into China um they have about a 10year chip advant we have a roughly 10e chip advantage in terms of subdv is sub five roughly 10 years wow um and so you're gonna have so an example would be today we're a couple of years ahead of China my guess is

we'll get a few more years ahead of China and the Chinese are whopping mad about this it's like hugely upset about it so that's a big deal that was a decision made by the Trump Administration and furthered by the Biden Administration do you think that it's going to make that scale of investment I mean OB the chips act but beyond that build building a massive AI system so so as you know I I lead a an informal ad hoc non-legal group that's that's different from illegal exactly justar which includes all the usual which includes all

The Usual Suspects yes and The Usual Suspects over the last year came up with basis of the reasoning that became the um uh uh the B administration's uh AI act which is the longest Presidential Directive in history you're talking the special competitive studies Pro no this is the actual the actual act from the executive office and there implementing the details so far they've got it right and so for example one of the debates that we had for last year has been how do you detect danger in a system which has learned it but you don't

know what to ask it okay so in other words it's a core it's a sort of a core problem it's learned something bad but it can't tell you what it learned and you don't know what to ask it and there's so many threats right like it learned how to mix chemistry in some new way but you don't know how to ask it and so people are working hard on that but we ultimately wrote in our memos to them that there was a threshold which we arbitrarily named as 10 to the 26 flops which technically is

a measure of computation that that threshold you had to report to the government that you were doing this and that's part of the rule the EU to just make sure they were different did 10 to 25 yeah but it's all kind of close enough I think all of these distinctions go away because the technology will now the technical term is called Federated training where basically you can take pieces and Union them together so we may not be able to keep keep people safe from these new things well rumors are that that's how open ey has

had to train partly because of the power uh consumption there's no one place where so this is interesting I haven't heard that specific terminology before so he's saying that you can kind of build different pie pieces of LMS or sort of AI systems you train those pieces separately and then you Union them together you glue them together we've seen this with the recent paper published where you can take sort of the large smart model and a very like a cheaper faster model and have sort of another sort of gatekeeper another yet another a model that

kind of chooses where it routes the questions to so certain questions need to be answered by the bigger model but there's a lot that can be answered just as effectively by the smaller model and so this model just routes them to the cheaper model when needed and the results have been astonishing because it's if I remember my numbers correctly it has shown in some cases to reduce the cost of running these things by 80% while maintaining something like 96% accuracy so for just and I don't recall those the exact numbers but it was some big

big numbers like that so it's a tiny reduction in accuracy for a potentially massive reduction in cost so I'm not sure if that's an example of specifically what he's talking about but this idea of Federated training sounds like it's training piece in individual sort of batches right and taking those pieces and then putting them together to create the final model next they're talking about war well let's talk to about a real war that's going on I know that uh something you've been very involved in is uh the Ukraine war and in particular uh I know

that uh something you've been very involved in is uh the Ukraine war and in particular uh I how you talk about white stor and your your goal of having a 500,000 $500 drones destroy $5 million tanks so Howes that changing Warfare I work for the Secretary of Defense for seven years and and tried to change the way we run our military I'm I'm not a particularly big fan of the military but it's very expensive and I wanted to see if I could be helpful and I think in my view I largely failed they gave me

a medal so they must give medals to failure or you know whatever but my self-criticism was nothing has really changed and the system in America is not going to lead to real Innovation so watching the Russians use tanks to destroy apartment buildings with little old ladies and kids just drove me crazy so I decided to work on a company with your friend Sebastian thrun and as a former faculty member here and a bunch of Stanford people and the idea basically is to do two things use AI in complicated powerful ways for these essentially robotic War

and the second one is to lower the cost of the robots now you sit there and you go why would a good liberal like me do that and the answer is that the whole theory of armies is tanks artilleries and Mort and we can eliminate all of them and we can make the penalty for invading a country at least by lamb essentially being impossible it should eliminate the kind of land battles well this this is Rel question is that does it give more of advantage to defense versus offense can you can you even make that

distinction because I've been doing this for the last year I've learned a lot about war that I really did not want to know and one of the things to know about war is that the offense always has the advantage because you can always overwhelm the defensive systems and so you're better off as a strategy of National Defense to have a very strong offense that you can use if you need to and the systems that I and others are building will do that um because of the way the system works I now a licensed I think

he might have misspoken there let me know in the comments if you think so too or I'm missing something I think he meant to say that he will he and others like him will build incredible defensive systems to reduce the effectiveness of invading and using offensive systems that's how I'm interpreting I think he kind of to me it sounds like he misspoke a little bit if if I understand correctly because of the way the system works I am now a licensed arms deer others are building we do that um because of the way the system

works I am now a licensed arms dealer so computer scientist businessman arms dealer and I'm sorry to agression I I don't know I do not recommend this in your Carew I stick with they are um and because of the way the laws work um we're doing this privately and then it's this is all legal with the support of the governments it goes straight into Ukraine and then they fight the war and and and without going into all the details things are pretty bad I think if in May or June if the Russians build up as

they expecting to Ukraine will lose a whole chunk of its territory and will begin the process of losing the whole country so I believe this was recorded in August 2024 just a few days ago or at least that's when it was posted if you recall the news headlines about it all of them are from like a few hours ago so I think he's talking about 2025 I'm assuming so he's talking about May or June of 2025 for most of History humans sort of had a mystical understanding of the universe and then there's the Scientific Revolution

and the enlightenment um and in your article you argue that now these models are becoming so complicated and uh difficult to understand that we don't really know what's going on in them I'll take a quote from Richard fean he says what I cannot create I do not understand I saw this quote the other day but now people are creating things they do not that that they can create but they don't really understand what's inside of them is the nature of knowledge changing in a way are we gonna have to start just taking the word for

these models without them able being able to explain it to us the analogy I would offer is to teenagers if you have a teenager you know that they're human but you can't quite figure out what they're thinking um but somehow we've managed in society to adapt to the presence of teenagers right and they eventually grow out of that and this serious so it's probably the case we're going to have knowledge systems that we cannot fully characterize but we understand their boundaries right we understand the limits of what they can do and that's probably the best

outcome we can get do you think we'll understand the limits we we'll get pretty good at it the consensus of my group that meets on every week is that eventually the way you'll do this it's called SOC called adversarial AI is that there will there will actually be companies that you will hire and pay money to to break your AI system so it'll be the r instead of human red teams which is what they do today you'll have whole companies and a whole industry of AI systems whose jobs are to break the existing AI systems

and find their vulnerabilities especially the knowledge that they have that we can't figure out that makes sense to me it's also a great project for you here at Stanford because if you have a graduate student who has to figure out how to attack one of these large models and understand what it does that is a great skill to build the Next Generation so it makes sense to me that the two will travel together if that's something that interests you there's a Twitter SLX account Pini The prompter Who uh basically jailbreaks a lot of these models

usually within a few days of their release so he's just of course one of uh many people that do sort of things like that he does it kind of simly for entertainment and for online media Clouts but if you think about it third party testing third party red teaming of AI models third party quality assurance right QA of these models as they roll out and have contact with consumers with businesses with organizations Etc as well as just with various infrastructure systems and factories automations Etc war of course that's going to be a Big Industry that

doesn't really exist yet it it just is beginning to grow and yet that might be something interesting and profitable to get into if that's uh that's your cup of tea next he comments just a bit on adver serial AI you just mentioned adversarial I'm wonder if elaborate on that more it seems to be besides obviously increase get more perform mod getting them to do what you want isue well you have to assume that the current hallucination problems become less right in as the technology gets better and so forth I'm not suggesting it goes away and

then you also have to assume that there are tests for efficacy so there has to be a way of knowing that the thing succeeded so in the example that I gave of the Tik Tok competitor and by the way I was not arguing that you should illegally steal everybody's music what you would do if you're a silicon value entrepreneur which hopefully all of you will be is if it took off then you'd hire a whole bunch of lawyers to go clean the mess up right but if if nobody uses your product it doesn't matter that

you stole all the content and do not quote me right right you're on camera that's right but you see my point in other words Silicon Valley will run these tests and clean up the mess and that's typically how those things are done so so my own View is that you'll see more and more um performative systems with even better test and eventually adversarial tests and that'll keep it within a box the technical term is called Chain of Thought reasoning and people believe that in the next few years you'll be able to generate a thousand steps

of Chain of Thought reasoning right do this do this it's like building recipes right that the recipes you can run the recipe and you can actually test that It produced the correct outcome and that's how the system will work next they ask what's going to be driving AI progress is it more data more compute is it just like chips and Hardware the amounts of money being thrown around are mindboggling and um I've chose I essentially invest in everything because I can't figure out who's going to win and the amounts of money that are following me

are so large I think some of it is because the early money has been made and the big money people who don't know what they're doing have to have an AI component and everything is now an AI investment so they can't tell the difference I Define ai as Learning Systems systems that actually learn so I think that's one of them the second is that there are very sophisticated new algorithms that are sort of post Transformers my friend my collaborator for a long time has invented a new non- Transformer architecture there's a group that I'm funding

in Paris that has claims to done the same thing so there's enormous invention there a lot of things at Stanford and the final thing is that there is a belief in the market that the invention of intelligence has infinite return so let's say you have you put $50 billion of capital into a company you have to make an awful lot of money from intelligence to pay that back I mean when you phrase it like that it seems to make sense if you invent intelligence certainly that would have an infinite return so it's probably the case

that we'll go through some huge investment bubble and then it'll sort itself out that's always been true in the past and it's likely to be true here and what you said earlier was you think that the leaders are pulling away from right now right now and and this is a really the question is roughly the following there's a company called mrr in France they done a really good job um and I'm obviously an investor um they have produced their second version their third model is likely to be closed because it's so expensive they need revenue

and they can't give their model away so this open source versus closed Source debate in our industry is huge and um my entire career was based on people being willing to share software in open source everything about me is open source much of Google's underpinnings were open source everything I've done technically and yet it may be that the capital costs which are so immense fundamentally changes how software is built you and I were talking um my own view of software programmers is that software programmers productivity will at least double there are three or four software

companies that are trying to do that I've invested in all of them in and they're all trying to make software programmers more productive the most interesting one that I just met with is called augment and I always think of an individual programmer and they said that's not our Target our Target are these 100 person software programming teams on millions of lines of code where nobody knows what's going on that's a really good AI thing will they make money I hope so you mentioned that there's the combination of the context exension the agents and the text

to action is going to unimaginable impacts first of all why is the combination important and second of all I know that you know you're not like a crystal ball and you can't necessarily tell the future but why do you think it's beyond anything that we could imagine I think largely because the context window allows you to solve the problem of recency the current models take a year to train roughly six six there 18 months six months of preparation six months of training six months of fine tuning so they're always out of date context window you

can feed what happened like you can ask it questions about the uh the Hamas Israel war right in a context that's very powerful it becomes current like Google um in the case of Agents I'll give you an example I set up a foundation which is funding a nonprofit which starts there's a I don't know if there's Chemists in the room that I don't really understand chemistry there's a a tool called chro CW which was an LM based system that learned chemistry and what they do is they run it to generate chemistry hypotheses about proteins and

they have a lab which runs the tests overnight and then it learns that's a huge acceleration accelerant in chemistry Material Science and so forth so that's that's an agent model and I think the text to action can be understood by just having a lot of sheet programmers right um and I don't think we understand what happens and this is again your area of expertise what happen happens when everyone has their own programmer I'm not talking about turning on and off the lights you know I imagine another example um for some reason you don't like Google

so you say build me a Google competitor yeah you personally you don't build me a Google competitor uh search the web build a UI make a good copy um add generative AI in an interesting way do it in 30 seconds and see if it works right so a lot of people believe that the incumbents including Google are vulnerable to this kind of an attack now we'll see there were a bunch of questions who were sent over by slider I want to give some of them upload so um here's one um we talked a little about

this last year um how can we stop AI from influencing public opinion misinformation especially during the upcoming he's asked a bit about misinformation how can we prevent AI from causing misinformation he talks about how during his time working with YouTube that was a problem people would upload a video for profit for their own self-gain and as a result of misinformation he says well people could be hurt he even says people die so certainly that is a problem if you saw my yesterday's video about grock 2 and the images it generates they're very re looking and

grock 2 does not seem to have any sort of safety rails in place it just generates whatever you ask for so heading into the selection certainly that could be an issue he talks a little bit about some sort of a digital authentication or a public key authentication basically if somebody like the president speaks there's some key that everybody else can verify saying okay this is him this is his original audio he mentions briefly that it's it seems like he's saying it's a shame that the American universities aren't getting as much resources for computes for Google

cloud and other places like that and this explosion where all the big companies are basically gobbling up the available resources and so he's saying the right thing for us to do is to get them the resources they need to give the universities the compute the resources that they need to pursue research and development and all that stuff next they talk about automation who's getting replaced who's getting to be obsolete in the labor force so let's take a look at that really fast I'll defer to the real expert here uh as your amateur Economist taught by

Eric um I I fundamentally believe that the the sort of college education High skills task will be fine because people will work with these systems I think the systems is no different from any other technology wave the dangerous jobs and the jobs which require very little human judgment will get replaced the next question is kind of like the global balance of AI power so the person asking the question specifically mentions us and China but as you see here Eric specifically focuses on India take a listen the most interesting country is India because the top AI

people come from India to the us and we should let India keep some of its top talent not all of them but some of them um and they don't have the kind of training facilities and programs that we so richly have here to me India is the big swing state in that regard China's Lost it's not going to not going to come back they're not going to change the regime as much as people wish them to do Japan and Korea are clearly in our camp Taiwan is a fantastic country whose software is terrible so that's

not gonna going to work um amazing hard and in the rest of the world there are not a lot of other good choices that are big German the Europe is screwed up because of Brussels it's not a new fact I spent 10 years fighting them and I work really hard to get them to fix the the EU act and they still have all the restrictions that make it very difficult to do our kind of research in Europe my French friends have spent all their time battling Brussels and mcon who's a personal friend is fighting hard

for this and so France I think has a chance I don't see I don't see Germany coming and the rest is not big enough yes ma'am so I know you're an engineer by trainer um given the capabilities that you Enis these mod should we still spend Hing to yeah because because ultimately it's it's the old thing of why do you study English if you can speak English you get better at it right you really do need to understand how these systems work and I feel very strong yes sir yeah I'm curious if you explore good

answer distributed setting and I'm asking because sure like making a large cluster but Ma are powerful there's a lot of small maches across the world so like you think like folding at home are a similar idea works for training yeah we look very hard this so the way the algorithms work is you have a very large Matrix and you have essentially a multiplication function so think of it as going back and forth and back and forth and these systems are completely limited by the speed of memory to CPU or GPU and in fact the next

iteration of Nvidia chips has combined all those functions into one chip the chips are now so big that they glue them all together and in fact the package is so sensitive the package is put together in a clean room as well as the chip itself so the looks like supercomputers and speed of light especially memory interconnect really dominated so I think unlikely for a while is there a way to segment the element like so Jeff de last year when he spoke here talked about having these different parts of it you train separately and then kind

of Federate them each you know in order to do that you'd have to have 10 million such things and then you're the way you ask the questions would be too slow he's talking about eight or 10 12 not at Lev the back way back I know like after releas new yor time open using work for training do you think that's I used to do a lot of work on the music licensing stuff what I learned was that in the 60s there was a series of lawsuits that resulted in an agreement where you get a a

stipulated royalty whenever your song is played even even they don't even know who you are so paid into a bank and my guess is it'll be the same thing there be lots of lawsuits and there'll be some kind of stipulated agreement which will just say you have to pay x per of whatever Revenue you have in order to use pass cap BMI cap BMI look them up it's long it will seem very old to you but I think that's how it will yes sir interesting like there's a few players that are dominating AI right they'll

continue to dominate and they seem to overlap with the large companies that all the antitrust regulation is kind of focus on how do you see those two Trends kind of yeah like do you see Regulators breaking up these companies and how will that affect the yeah so in my career I helped Microsoft get broken up and it wasn't broken up and I fought fought for Google to not be broken up and it's not been broken up so it sure looks to me like the trend is not to be broken up um as long as the

companies avoid being John D Rockefeller the senior and I studied this looked it up it's how antitrust law came I don't think the government is will act the re the reason you're seeing these large companies dominate is who has the capital to build these data centers right right so my friend Reed and my friend com two have we talked to you about the decision that they made to take inflection so I believe Mustafa he's talking about Mustafa siman who is the co-founder of Google Deep Mind friend Reed and my friend M coming next we two

week have Reed talked to you about the decision that they made to take inflection and essentially PE part it into Microsoft basically they decided they couldn't raise the tens of billions of dollars is that number public that you mentioned earlier have have got we want this I was wondering where all this is going to leave countries where in b ipants in phun models and access to compu the rich get richer and the poor do the best they can um they'll have to the fact of the matter is this is a rich country's game right huge

Capital lots of technically strong people strong government support right there are two examples there lots of other countries that have all sorts of problems they don't have those resources they'll have to find a partner they'll have to join with somebody else something like that I last we met you he's fairly blunt he's fairly straightforward he I think kind of calls it how he sees it and it really seems like he's not talking one side or the other he doesn't have he's not pumping his back so to speak so he's not saying the things that benefit

his portfolio or whatever he's just like yeah this is how it's going to be this is how it works by default big companies are not going to broken up the rich get richer Etc and finally he's asked a little bit more about kind of the business and entrepreneur side of things if you're starting your own company you have a product idea what does that look like how do you develop it how do you create a lot of wealth with your own product here's his answer I want I think the last we met you you at

hackathon at AGI house and I know you spent a lot of time helping like young people as they create a lot of wealth and you spoke very passionately about about wanting to to do that do you have any advice for folks here as they're building their they're writing their business PL for this class or policy proposals or research proposals um you know at this stage of the careers going forward well um I teach a class in the business school on this so you should come to my class um the I am struck by the speed

which with which you can build demonstrations of new ideas so in that in one of the hack I did the winning team the command was fly the Drone between two towers and it was given a virtual drone space and it figured out how to fly the Drone what the word between them generated the code in Python and flew the Drone in the simulator through the tower I just it would have taken a week or two you know good professional programmers to do that um I'm telling you that the ability to prototype quickly really you know

part part of the problem being an entrepreneur is everything happens faster well now if you can't get your prototype built in a day using these various tools you need to think about that right because that's who your competitor is doing so I guess my biggest advice is when you start thinking about a company is find the right a business CL plan in fact you should ask the computer to write your business plan for you um talk about that after leave it and and but I think it's very important to prototype your idea using these tools

as quickly as you can because you can be sure there's another person doing exactly that same thing in another company in another University in a place that You' all right thank very cool so this piece was uh taken down so apparently it's not even online right now but hopefully those few little Clips give you glips about what he thinks of AI you know I've kind of cut out all the pieces that I think caused the controversy or whatever that situation was I don't think it really had anything to do with AI or anything of of

substance I mean maybe touched on some political issues but whatever this is a person with a lot of really technical background and even though he's more of like a business guy he is spoken of with a lot of respect by all the engineers and all the very highly technical people in the Bay Area in the Silicon Valley he's somebody that really understands the tech space looks like Forbes has him listed as having a netw worth of almost $24 billion he's good friends he said with Sam Alman president of France France sounds like he knows all

the co-founders of various AI companies big and small I mean this is somebody that's very very on the inside this is somebody that knows where things are going so certainly when he says something I tend to listen I put a great deal of weight into what he's saying and what he's saying is this AI wave is just starting we have two massive countries two big countries with the capital and the talent and everything else that's needed that are vying for AI Supremacy the issues with the chips and the power and the data while those are

bottlenecks it seems like they will slowly be perhaps even maybe not that slowly slowly but but be improved you mentioned building data centers and various other AI infrastructure in Canada potentially which makes a lot of sense if you think about it lots of open land tends to be colder which again you know Heating and the generation of heat and having to cool these facilities is a concern tons land very friendly government to the US you know in factorio how do you just like find a big chunk of the map where there's a lot of open

land and you'd build all your nuclear power plants you'd build out your entire like electricity infrastructure there just have a cable going back to where you need electricity to be it begs the question is this the future of Canada I'm only kidding of course folks but the more and more I think about it there is a lot of land it's close by and it's cold and certainly that would be phenomenal for the revenue of Canada to generate some sort of tax some sort of Revenue based on right one of the kind of superpowers building out

the infrastructure it needs to stay dominant in the AI race right I mean they're talking about creating 100 billion doll data centers potentially even the estimates a size 300 billion so that's that's one company that's not even the US government supported projects that's just a publicly traded company doing that sounds like he's really believing in Nvidia and the power of Nvidia and it's Cuda sort of software that the chips run on and if I'm understanding him correctly he's not really seeing any competition to that I mean there's some things in the Horizon maybe possibly but

nothing yet so it sounds like a lot of this money like a lot of it will be flowing into Nvidia and of course this idea that as the context window expands it'll become more and more sort of similar to training in a sense that right if it takes a while to train up a model but it can do in context learning meaning that you're able to just upload books and stuff to the finished model it's able to learn from them which we've seen that with Gemini they found a language that doesn't have an a lot

of online presence only something like 24 people in the world speak it but they have some notebooks and manuscripts and stuff like that they uploaded to gini the actual finished trained Gemini and that model was able to learn how to speak it how to create sentences in it it wasn't trained on it it would learned in context right so as the context window ramps up to 10 million tokens that certainly seems like it would change how we think about pre- and posttraining or in other words once the model is trained it doesn't stop learning if

you wanted to learn about a specific thing that you wanted to learn about maybe it's you know you have a company or that that have millions of documents and procedures and stuff like that just kind of like everywhere but no one person understands all of it well in the past that was kind of useless it doesn't really help you but if you can upload it to a model like that all of a sudden it knows all those things about your company it can train people on it it can answer your questions it can kind of

manage it update see if there's any conflicts anywhere in it so instead of training a brand new model on that you just use an existing model and upload that stuff into the 10 million token context window combine that with text to action which is another word I've heard it before this phrase but it's not that often used I feel like but yeah I mean that's what agents are basically right it's text to action you type in impr prompt and it goes and does that action it does the thing you're asking it to do as they

get better at long-term plan long-term reasoning and actually Deep Mind CEO de saabi is also just had an interview talking about something very very similar he's talking about this idea that as agents become better long-term reasoners and planners they will become better and better at carrying out those long Horizon tasks and he's saying we're not ready for it when everybody can have these qualified capable agents right he's saying it's going to have an impact on the world at a scale no one understands yet and that certainly I think is true that's certainly something that I

fully agree with a lot of our social structures and stuff we're kind of used to will have to change and of course he's saying if you're worried about losing your job having a degree that's going to become worthless you know should you still learn to code he's saying business as usual there yes some easy simple jobs or maybe very dangerous jobs jobs where there's not a lot of hard decisions to be made yes those might go away and yes these AIS might be pretty good at coding creating projects but learning all those skills getting better

at those skills learning how everything works that's that's not going out of fashion that's never going to just disappear his example was you know if you can speak English why would you learn English right why would you study English well it's to get better at it to improve your abilities with it so I hope you enjoyed that make sure you subscribed please give me a thumbs up if you enjoyed it my name is Wes rth and I'll see you next time thank you for watching