Eric Schmidt Full Controversial Interview on AI Revolution (Former Google CEO)

where do you see AI going in the short term which I think you defined as the next year or two um things have changed so fast I feel like every six months I need to sort of give a new speech on what's going to happen um can anybody hear the computer budget computer scientist in here can anybody explain what a million token context window is for the rest of the class here say your name tell us what it does um basically allows you to prompt with like a million tokens or a million words or whatever

you decide to so you can ask a million word question yes I know this is a very large Direction in Gemini right now um no no no they're going to 10 yes 10 million yes yeah anthropic is at 200,000 going to a million and so forth you can imagine open AI has a similar goal anybody can anybody here give a technical definition of an an AI agent again a computer SST yes sir agent is something some kind of way so that might be calling pages on the web buying things on your behalf a number along

lines yeah so an agent is something that does does some kind of a task another definition would be that it's an llm State and memory okay can anybody again computer scientist can can any of you define text to action taking text and turning it into an action right right here go ahead yes instead of taking text and turning it into more text more text taking text and have the AI trigger actions based on so another definition would be language to python a pro programming language I never wanted to see survive and and everything in AI

is being done in Python there's a new language called Mojo that has just come out which looks like they finally have addressed AI programming but we'll see if that actually survives over the dominance of python um one more technical question why is NVIDIA worth two trillion dollars and the other companies are struggling technical answer I mean I think it just boils down to like most of code needs to run with cud optimizations that currently only Nvidia GPU support so other companies can make whatever they want to but unless they have the 10 years of software

there you don't have the machine learning optimization Str I like to think of Cuda as the C programming language for gpus yeah right that's the way I like to think of it it was founded in 2008 I always thought it was a terrible language and yet it's become dominant there's another Insight there's a set of Open Source libraries which are highly optimized to Cuda and not anything else and everybody who builds all these Stacks right this is completely missed in any of the discussions right the common it's technically called VM and a whole bunch of

libraries like that highly optimized Kula Cuda very hard to replicate that if you're a competitor so what is all this mean in the next year you're going to see very large context windows agents and text action when they are delivered at scale it's going to have an impact on the world at a scale that no one understands yet much bigger than the horrific impact we've had on by social media right in my view so here's why in a context window you can basically use that as short-term memory and I was shocked that context Windows get

this long the technical reasons have to do with the fact that it's hard to serve hard to calculate so forth the interesting thing about short-term memory is when you feed the the you ask a question read 20 books you give it the text of the books is the query and you say tell me what they say it forgets the middle which is exactly how human brains work to right that's where we are with respect to agents there are people who are now building essentially llm agents and the way they do it is they read something

like chemistry they discover the principles of chemistry and then they test it and then they add that back into their understanding right that's extremely powerful and then the third thing as I mentioned is text action so I'll give you an example the government is in the process of trying to ban Tik Tok we'll see if that actually happens if Tik Tok is banned here's what I propose each and every one of you do say to your llm the following make me a copy of Tik Tok steal all the users steal all the music put my

preferences in it produce this program in the next 30 seconds release it and in one hour if it's not viral do something different along the same lines that's the command boom boom boom boom right you understand how powerful that is if you can go from arbitrary language to arbitrary digital command which is essentially what python in this scenario is Imagine that each and every human on the planet has their own programmer that actually does what they want as opposed to the programmers that work for me who don't do what I ask right the programmers here

know what I'm talking about so imagine a non arrogant programmer that actually does what you want and you don't have to pay all that money to and there's infinite supply of these programs and this is all within the next year or two very soon those three things and I'm quite convinced it's the of those three things that will happen in the next wave so you asked about what else is going to happen um every six months I oscillate so we're on a it's an even odd oscillation so at the moment the gap between the frontier

models which they're now only three a few who they are and everybody else appears to me to be getting larger 6 months ago I was convinced that the Gap was getting smaller so I invested lots of money in the little companies now I'm not so sure and I'm talking to the big companies and the big companies are telling me that they need 10 billion 20 billion 50 billion 100 billion Stargate is a what 100 billion right they're very very hard I talked Sam Alman is a close friend he believes that it's going to take about

300 billion maybe more I pointed out to him that I done the calculation on the amount of energy Acquired and I and I then in the spirit of full disclosure went to the White House on Friday and told them that we need to become best friends with Canada because Canada has really nice people helped invent Ai and lots of Hydra power because we as a country do not have enough power to do this the alternative is to have the Arabs fund it and I like the Arabs personally uh spent lots of time there right but

they're not going to adhere to our national security rules whereas Canada and the US are part of a Triumph it where we all agree so these hundred billion $300 billion data centers electricity starts becoming the scarce resource well well or and and by the way if you follow this line of reasoning why did I discuss Cuda and Nvidia if $300 billion is all going to go to Nvidia you know what to do in the stock market okay that's not a stock recommendation I'm not a licensed well well part of it so we're going to need

a lot more chips but Intel is getting a lot of money from the US government AMD and and they're trying to build you know Fabs and raise your hand if you have an Intel computer in your Intel chip in any of your Computing devices okay so much for the Monopoly well that well that's that's the point though they once did have a monopoly absolutely and Nvidia has a monopoly now so are those barriers to entry like like Cuda is that is there something that so I was talking to Percy Percy land the other day he's

switching between tpus and Nvidia chips depending on what he can get access to because he doesn't have a choice if he had infinite money he would today he would pick the b200 architecture out of Nvidia because it would be faster and and I'm not sugges I mean it's great to have competition I've talked to to AMD and Lisa Sue at Great length yeah they have built a a thing which will translate from um this Cuda architecture that you were describing to to their own which is called Rockham it doesn't quite work yet they're working on

it um you were at Google for a long time and uh they invented the Transformer architecture Peter um it's all Peter's fault thanks to uh to brilliant people over there like Peter and Jeff Dean and everyone um but now it doesn't seem like they're they they've kind of lost the initiative to open Ai and even the last leaderboard I saw anthropics Claud was at the top of the list um asked Sundar this he didn't really give me a very sharp answer maybe maybe you have a a sharper or more objective explanation for what's going on

there I'm no longer a Google employee yes um in this beautiful disclosure um Google decided that work life balance and going home early and working from home was more important than winning and the startups the reason startups work is because the people work like hell and I'm sorry to be so blunt but the fact of the matter is if you all leave the university and go found a company you're not going to let people work from home and only come in one day a week if you want to compete against the other startups when when

in the early days of Google Microsoft was like that exactly but now it seems to be there's there's a long history of in my industry our industry I guess of companies winning in a genuinely creative way and really dominant a space and not making this the next transition it's very well documented and I think that the truth is Founders are special the founders need to be in charge the founders are difficult to work with they push people hard um as much as we can dislike elon's personal Behavior look at what he gets out of people

I had dinner with him and he was flying he I was in Montana He was flying that night at 10 p.m. to have a meeting at midnight with x. right think about I was in Taiwan different country different culture and they said that this is tsmc who I'm very impressed with and they have a rule that the starting phds coming out of the they're good good physicists work in the factory on the basement floor now can you imagine getting American physicists to do that with phds highly unlikely different work ethic and the problem here the

the reason I'm being so harsh about work is that these are systems which have Network effects so time matters a lot and in most businesses time doesn't matter that much right you have lots of time you know Coke and Pepsi will still be around and the fight between Coke and Pepsi will continue to go along and it's all glacial right when I dealt with Telos the typical Telco deal would take 18 months to sign right there's no reason to take 18 months to do anything get it done it's just we're in a period of Maximum

growth maximum gain so and also it takes crazy ideas like when Microsoft did the open a deal I thought that was the stupidest idea I'd ever heard Outsourcing essentially your AI leadership to open Ai and Sam and his team I mean that's insane nobody would do that at Microsoft or anywhere else and yet today they're on their way to being the most valuable company they're certainly head-to-head in apple apple does not have a good AI solution and it looks they made it work M yes sir in terms of naal security interest how do you think

AI is going to play a role or competition with China as well so I was the chairman of an AI commission that sort of looked at this very carefully and um you can read it it's about 752 pages and I'll just summarize it by saying we're ahead we need to stay ahead and we need lots of money to do so our customers were the Senate and the house um and out of that came the chips act and a lot of other stuff like that um the a rough scenario is that if you assume the frontier

models drive forward and a few of the open source models it's likely that a very small number of companies can play this game countries excuse me what are those countries or who are they countries with a lot of money and a lot of talent strong Educational Systems and a willingness to win the US is one of them China is another one how many others are there are there any others I don't know maybe but certainly the the in your lifetimes the battle between the US and China for knowledge Supremacy is going to be the big

fight right so the US government banned uh essentially the Nvidia chips although they weren't allowed to say that was what they were doing but they actually did that into China um they have about a 10-year chip advant we have a a roughly 10year chip advantage in terms of subdv that is sub five n years roughly 10 years wow um and so you're going to have so an example would be today we're a couple of years ahead of China my guess is we'll get a few more years ahead of China and the Chinese are whopping mad

about this it's like hugely upset about it so that's a big deal that was a decision made by the Trump Administration and furthered by the Biden Administration do you find that the administration today in Congress is listening to your advice do you think that it's it's going to make that scale of investment I mean obviously the chips act but beyond that building building a massive AI system so so as you know I I lead a an informal ad hoc non-legal group that's not that's different from illegal exactly just to be clear which includes all the

usual which includes all The Usual Suspects yes and The Usual Suspects over the last year came up with the basis of the reasoning that became the um uh uh the V administration's uh AI act which is the longest Presidential Directive in history you're talking about the special competitive studies project no no this is the actual the actual act for from the executive office okay and they're busy implementing the details so far they've got it right and so for example one of the debates that we had for the last year has been how do you detect

danger in the system which has learned it but you don't know what to ask it okay so in other words it's a core it's a sort of a core problem it's learn something bad but it it can't tell you what it learned and you don't know what to ask it and there's so many threats right like it learned how to mix chemistry in some new way but you don't know how to ask it and so people are working hard on that but we ultimately wrote in our memos to them that there was a threshold which

we arbitrarily named as 10 to the 26 flops which technically is a measure of computation that above that threshold you had to report to the government that you were doing this and that's part of the rule the EU to just make sure they were different did it at 10 to the 25 yeah um but it's all kind of close enough I think all of these distinctions go away because the technology will now the technical term is called Federated training where basically you can take pieces and Union them together so we may not be able to

keep keep people safe from these new things well rumors are that that's how open I has had to train partly because of the power uh consumption there's no one place where they did well let's talk to about a real war that's going on I know that uh something you've been very involved in is uh the Ukraine war and in particular uh I don't know how much you can talk about white stor and your your goal of having a $500,000 $500 drones destroy $5 million tanks so so that changing Warfare so I worked for the Secretary

of Defense for seven years and and tried to change the way we run our military I'm I'm not a particularly big fan of the military but it's very expensive and I wanted to see if I could be helpful and I think in my view I largely failed they gave me a medal so they must give medalist to failure or you know whatever but my self-criticism was nothing has really changed and the system in America is not going to lead to real Innovation so watching the Russians use tanks to destroy apartment buildings with little old ladies

and kids just drove me crazy so I decided to work on a company with your friend Sebastian thrun and a as a former faculty member here and a whole bunch of Stanford people and the idea basically is to do two things use Ai and complicated powerful ways for these essentially robotic War and the second one is to lower the cost of the robots now you sit there and you go why would a good liberal like me do that and the answer is that the whole theory of armies is tanks artilleries and mortar and we can

eliminate all of them and we can make the penalty for invading a country at least by land essentially being impossible it should eliminate the kind of land battles well this this is Rel question is that does it give more of an advantage to defense versus offense can you can you even make that distinction I've been doing this for the last year I've learned a lot about war that I really did not want to know and one of the things to know about war is that the offense always has the advantage because you can always overwhelm

the defensive systems and so you're better off as a strategy of National Defense to have a very strong offense that you can use if you need to and the systems that I and others are building will do that um because of the way the system works I am now a licensed arms dealer a so computer scientist businessman arms dealer and and I'm sorry to is that a progression I I don't know I do not recommend this in your career path I'd stick with AI um and because of the way the laws work um we're doing

this privately and then it's this is all legal with the support of the governments it goes straight into the Ukraine and then they fight the war and and and without going into all the details things are pretty bad I think if in May or June if the Russians build up as they are expeced to Ukraine will lose a whole chunk of its territory and will begin the process of losing the whole country so the situation is quite dire and if anyone knows Marjorie Taylor green I would encourage you to delete her from your contact list

because she's the one a single individual is blocking the provision of some number of billions of dollars to save an important democracy I want to switch to a little bit of a philosophical question so there was an article that you and Henry Kissinger and Dan hleer wrote last year about the nature of knowledge and how it's evolving I had a discussion the other night about this as well so for most of History humans sort of had a mystical understanding of the universe and then there's the Scientific Revolution and the enlightenment um and in your article

you argue that now these models are becoming so complicated and uh uh difficult to understand that you don't really know what's going on in them I'll take a quote from Richard feeman he says what I cannot create I do not understand I saw this quote the other day but now people are creating things they do not that that they can create but they don't really understand what's inside of them is the nature of knowledge changing in a way are we going to have to start just taking the word for these models without them able being

able to explain it to us the analogy I would offer is to teenagers if you have a teenager you know that they're human but you can't quite figure out what they're thinking um um but somehow we've managed in society to adapt to the presence of teenagers right and they eventually grow out of it and this serious so it's probably the case that we're going to have knowledge systems that we cannot fully characterize but we understand their boundaries right we understand the limits of what they can do and that's probably the best outcome we can get

do you think we'll understand the limits we we'll get pretty good at it the consensus of my group that meets on uh every week is that eventually the way you'll do this uh it's called so-called adversarial AI is that there will there will actually be companies that you will hire and pay money to to break your AI system team so it'll be the red instead of human red teams which is what they do today you'll have whole companies and a whole industry of AI systems whose jobs are to break the existing AI systems and find

their vulnerabilities especially the knowledge that they have that we can't figure out that makes sense to me it's also a great project for you here at Stanford because if you have a graduate student who has to figure out how to attack one of these large models and understand what it does that is a great skill to build the Next Generation so it makes sense to me that the two will travel together all right let's take some questions from the student there's one right there in the back you say your name earlier you mentioned and this

is related to comment right now I'm getting AI that actually does what you want you just mentioned adversarial a I'm wondering you could elaborate on that more so there seems to be besides obviously compute will increase and can get more performant models but it's getting them to do what we want to issue seems larly unanswered well you have to assume that the current hallucination problems become less right in as the technology gets better and so forth I'm not suggesting it goes away and then you also have to assume that there are tests for efficacy so

there has to be a way of knowing that the thing succeeded so in the example that I gave of the Tik Tok competitor and by the way I was not arguing that you should illegally steal everybody's music what you would do if you're a Silicon Valley entrepreneur which hopefully all of you will be is if it took off then you'd hire a whole bunch of lawyers to go clean the mess up right but if if nobody uses your product it doesn't matter that you stole all the content and do not quote me right right you're

you're on camera yeah that's right but but but you see my point in other words Silicon Valley will run the tests and clean up the mess and that's typically how those things are done so so my own view is that you'll see more and more um performative systems with even better tests and eventually adversarial tests and that'll keep it within a box the technical term is called Chain of Thought reasoning and people believe that in the next few years you'll be able to generate a thousand steps of Chain of Thought reasoning right do this do

this it's like building recipes right that the recipes you can run the recipe and you can actually test that It produced the correct outcome and that's how the system will work yes sir in general you seem super positive about the potential for AI problems I'm curious like what do you think is going to drive that is it just more compute is it more data is it Aral shifts uh yes great the amounts of money being thrown around are mindboggling and and um I've chose I I essentially invest in everything because I can't figure out who's

going to win and the amounts of money that are following me are so large I think some of it is because the early money has been made and the big money people who don't know what they're doing have to have an AI component and everything is now an AI investment so they can't tell the difference I Define ai as Learning Systems systems that actually learn so I think that's one of them the second is that there are very sophisticated new algorithms that are sort of post Transformers my friend my collaborator for a long time has

invented a new non- Transformer architecture there's a group that I'm funding in Paris that has claims to have done the same thing so there there's enormous uh invention there a lot of things at Stanford and the final thing is that there is a belief in the market that the invention of intelligence has infinite return so let's say you have you put $50 billion of capital into a company you have to make a awful lot of money from intelligence to pay that back so it's probably the case that we'll go through some huge investment bubble and

then it'll sort itself out that's always been true in the past and it's likely to be true here and what you said earlier was you think that the leaders are pulling away from right now right now and and this is a really the question is um roughly the following there's a company called mstr in France they've done a really good job um and I'm I'm obviously an investor um they have produced their second version their third model is likely to be closed because it's so expensive they need revenue and they can't give their model away

so this open source versus closed Source debate in our industry is huge and um my entire career was based on people being willing to share software in open source everything about me is open source much of Google's underpinnings were open source everything I've done Tech technically and yet it may be that the capital costs which are so immense fundamentally changes how software is built you and I we're talking um my own view of software programmers is that software programmers productivity will at least double there are three or four software companies that are trying to do

that I've invested in all of them in the spirit and they're all trying to make software programmers more productive the most interesting one that I just met with is called augment and I I always think of an individual program and they said that's not our Target our Target are these 100 person software programming teams on millions of lines of code where nobody knows what's going on well that's a really good AI thing will they make money I hope so so a lot of questions here hi um so at the very beginning um at the very

beginning you mentioned that there's the combination of the context window expansion the agents and the text to action is going to have unimaginable impacts first of all why is the combination important and second of all I know that you know you're not like a crystal ball and you can't necessarily tell the future but why do you think it's beyond anything that we could imagine I think largely because the context window allows you to solve the problem of recency the current models take a year to train roughly six six there's 18 months six months of preparation

six months of training six months of fine-tuning so they're always out of date contact window you can feed what happened like you can ask it questions about the um the Hamas Israel war right in a context that's very powerful it becomes current like Google um in the case of Agents I'll give you an example I set up a foundation which is funding a nonprofit which starts there's a u i don't know if there's Chemists in the room that I don't really understand chemistry there's a a tool called chro C which was an llm based system

that learned chemistry and what they do is they run it to generate chemist chemistry hypothesis about proteins and they have a lab which runs the tests overnight and then it learns that's a huge acceleration accelerant in chemistry Material Science and so forth so that's that's an agent model and I think the text to action can be understood by just having a lot of cheap programmers right um and I don't think we understand what happens and this is again your area of expertise what happens when everyone has their own programmer and I'm not talking about turning

on off the lights you know I imagine another example um for some reason you don't like Google so you say build me a Google competitor yeah you personally you don't build me a Google competitor uh search the web build a UI make a good copy um add generative AI in an interesting way do it in 30 seconds and see if it works right so a lot of people believe that the incumbents including Google are vulnerable to this kind of an attack now we'll see there were a bunch of questions who were sent over by slider

I want to give some of them were upvoted so um here's one um we talked a little bit about this last year um how can we stop AI from influencing public opinion misinformation especially during the upcoming election what are the short and long-term solutions from most of the misinformation in this upcoming election and globally will be on social media and the social media companies are not organized well enough to police it if you look at Tik Tok for example there are lots of accusations that Tik Tok is favoring one kind of misinformation over another and

there many people who claim without proof that I'm aware of that the Chinese are forcing them to do it I think we just we have a mess here and um the country is going to have to learn critical thinking that may be an impossible challenge for the us but but the fact that somebody told you something does not mean that it's true well could it go too far the other way that that there's things that really are true and nobody believes anymore you get some people call epistemological crisis that that now you know Elon says

no I never did that prove it well let's use Donald Trump um okay look I I think we've got we have a trust problem in our society democracies can fail and I think that the the greatest threat to democracy is misinformation because we're going to get really good at it um when I man managed you YouTube the biggest problems we had on YouTube were that people would upload false videos and people would die as a result and we had a no death policy shocking and uh we just went it was just horrendous to try to

address this and this is before generative AI well so I I don't have a good answer one technical is not an answer but one thing that seems like it could mitigate that I don't understand why it's more widely used is uh public key authentication that when when Joe Biden speaks why isn't it digitally signed like SSL is or when uh you know that that that celebrities or public figures or others couldn't they have a public key yeah it's a it's a form of public key and then some some form of certainty of knowing how the

system when I send my credit card to Amazon I know it's Amazon so I wrote a paper and published it with Jonathan ha who's the the one working on the anxiety Generation stuff um it had exactly zero impact in my so and he's a very good communicator I probably am not so my conclusion was that the system is not organized to do what you said you had a paper advocating what what we that advocating your proposal okay my proposal no what you said yeah right and my conclusion is the CEOs in general are maximizing Revenue

to maximize Revenue you maximize engagement to maximize engagement you maximize outrage the algorithms choose outrage because that generates more Revenue right therefore there's a bias to favor crazy stuff and on all sides I'm not making a partisan statement here that's a problem that's got to get addressed in a democracy and my solution to Tik Tok we talked about this earlier privately is there was when I was a a boy there was something called the equal time rule because Tik Tok is really not social media it's really Television right there's a programmer making you the numbers

by the way are um 90 minutes a day 200 uh Tik Tok videos per Tik Tok user in the United States it's a lot right so and the government is not going to do the equal time rule but it's the right thing to do some form of balance that is required all right let's take some more questions um two quick questions one um economic impact of LMS um slower like Market impacts slower than you originally anticipated Che and and customer service people in them too uh do you think Academia uh deserves uh or should get

um AI subsidies or do you think they should just part with big players out there I pushed really really hard on getting data Centered for universities if I were a faculty member in the computer science department here I would be Beyond upset that I can't build the algorithms with my graduate students that will do the kind of PHT research and that I'm for forced to work with these and the companies have not in my view been generous enough with respect to that the faculty member that members that I talk with many of whom you know

spend lots of time waiting for their credits from Google cloud and the peace that's terrible this is an explosion we want America to win we want American universities America you know there's lots of reasons to think that the right thing to do is to get it to them so I'm working hard on that and your first question was Labor Market impact um I'll defer to the real expert here uh as your amateur Economist taught by Eric um I I fundamentally believe that the the sort of college education High skills task will be fine because people

will work with these systems I think the systems is no different from any other technology wave the dangerous jobs and the jobs which require very little human judgment will get replaced we have about five minutes left so let's go really quick with some question I'll let you pick them uh Eric yes ma'am hi um I'm really curious about the taex to action and its impact on for example Computer Science Education wondering what you have thoughts on like how cus education should transform kind of Meet the age well I'm assuming that computer scientists as a group

in undergraduate school will always have a programmer buddy with them so when you when you learn learn your first for Loop and so forth and so on you'll have a tool that will be your natural partner and then that's how the teaching will go on that the professor you know he or she will talk about the concepts but you'll engage with it that way and that's my guess yes ma'am behind you yeah can you talk more about the non- Transformer architectures that you're excited about I think one that's talk about like uh the state models

but then now long cont moris I was curious what you're seeing in space um I I don't understand the maap well enough this is the I'm I'm really pleased that we have produced jobs for mathematicians because the math here is so complicated but basically they are different ways of doing gradient descent Matrix multiply fast faster and better um and Transformers as you know is a is a sort of systematic way of multiplying at the same time that's the way to think I think about it and it's similar to that but different math let's see over

here yes sir go ahead Yeah you mentioned your in your paper on naal security that you have China the US and water capabilities today the next 10 that next cost are all other us allies or te up nicely to the US allies curious what your take is on on those 10 are sort of like the middle that aren't formally allies um what is stuff How likely are they to get on board with securing our superiority lot and what would hold them back from to get get on the most interesting country is India because the top

AI people come from India to the us and we should let India keep some of its top talent not all of them but some of them um they don't have the kind of training facilities and programs that we so richly have here to me India is the big swing state in that regard China's Lost it's not going to not going to come back they're not going to change the regime as much as people wish them to do Japan and Korea are clearly in our camp Taiwan is a fantastic country whose software is terrible so that's

not going to going to work um amazing hardware and in the rest of the world there are not a lot of other good choices that are big German uh the Eur Europe is screwed up because of Brussels it's not a new fact I spent 10 years fighting them and I worked really hard to get them to fix the a the EU act and they still have all the restrictions that make it very difficult to do our kind of research in Europe my French friends have spent all their time battling Brussels and macron who's a personal

friend is fighting hard for this and so France I think has a chance I don't see I don't see Germany coming and the rest is not big enough go ahead yes ma'am um so I know you're an engineer by training like you call a compiler um given the capabilities that you envisioned these models having should we still spend time learning to code yeah because because ultimately it's it's the old thing of why do you study English if you can speak English you get better at it right you really do need to understand how these systems

work and I feel very strong yes sir yeah I'm curious if youve explored the distributed setting and I'm asking because sure like making a large cluster with but MacBooks are powerful there's a lot of small machines across the world so like do you think like folding at home or a similar idea works for training system yeah we've looked very hard at this so the way the algorithms work is you have a very large Matrix and you have essentially a multiplication function so think of it as going back and forth and back and forth and these

systems are completely limited by the speed of memory to CPU or GPU and in fact the next iteration of of Nvidia chips has combined all those functions into one chip the chips are now so big that they glue them all together and in fact the package is so sensitive that the package is put together in a clean room as well as the chip itself so the answer looks like supercomputers and speed of light especially memory interconnect really dominated so I think unlikely for a while is there a way to segment the LM like so Jeff

Dean last year when he spoke here talked about having these different parts of it you train separately and then kind of Federate them each you know uh in order to do that you'd have to have 10 million such things and then your the way you would ask the questions would be too slow he's talking about eight or 10 or 12 level not at his level see in the back yes way back I know like after CH was released in New York Times Su open it for using their works for training where do you think that's

going to go what that means for I used to do a lot of work on the music licensing stuff and what I learned was that in the 60s there was a series of lawsuits that resulted in an agreement where you get a a stipulated royalty whenever your song is played even even they don't even know who you are it's just paid into a bank and my guess is it'll be the same thing there'll be lots of lawsuits and there'll be some kind of stipulated agreement which will just say you have to pay x% of whatever

Revenue you have in order to use ASCAP bemi ASAP bemi look them up it's a long it will seem very old to you but I think that's how it will alter yes sir yeah it seems like there's a few players that are dominating AI right and they'll continue to dominate um and they seem to overlap with the large companies that all the antitrust regulation is kind of focused on how do you see those two Trends kind of yeah like do you see Regulators breaking up these companies and how will that affect the yeah so in

my career I helped Microsoft get broken up and it wasn't broken up and I fought fought for Google to not be broken up and it's not been broken up so it sure looks to me like the trend is not to be broken up um as long as the companies avoid being John D Rockefeller the senior and I studied this look it up it's how antitrust law came I don't think the governments will act act the re the reason you're seeing these large companies dominate is who has the capital to build these data centers right right

so my friend Reed and my friend Mustafa coming next week two two weeks from now have Reed talked to you about the decision that they made to take inflection and essentially peace part it into Microsoft basically they decided they couldn't raise the tens of billions of dollars is that number public that you mentioned earlier no okay have have Reed give may say I know you're you got to go I I don't want hold you I want to leave we should we do one this gent have a question for you yeah go ahead than you so

thank you so much I'll make it quick I was wondering where all of this is going to lead countries who are nonparticipants in of development of Frontier models and access to compete for example the rich get richer and the poor do the best they can um they'll have to the fact of the matter is this is a rich country's game right huge Capital lots of technically strong people strong government support right there are two examples there are lots of other countries that have all sorts of problems they don't have those resources they'll have to find

a partner they'll have to join with somebody else something like that I want to leave I think the last time we met you we you were at a hackathon at AGI house and I know you spent a lot of time helping like young people as they create a lot of wealth and you spoke very passionately about about wanting to to do that do you have any advice for folks here as they're building their they're writing their business plans for this class or policy proposals or research proposals um you know at this stage of the careers

going forward well um I teach a class in the business school on this so you should come to my class um the I am struck by the speed which with which you can build demonstrations of new ideas so in that in one of the hackathons I did the winning team the command was fly the Drone between two towers and it was given a virtual drone space and it figured out how to fly the Drone what the word between meant generated the code in Python and flew the Drone in the simulator through the tower I just

it would have taken a week or two you know good professional programmers to do that um I'm telling you that the ability to prototype quickly really you know part part of the problem with being an entrepreneur is everything happens faster well now if you can't get your prototype built in a day using these various tools you need to think about that right because that's who your competitor is doing so I guess my biggest advice is when you start thinking about a company it's fine to write a business CL plan in fact you should ask the

computer to write your business plan for you um as long as I should talk about that after you leave this and and but but but I think it's very important to prototype your idea using these tools as quickly as you can because you can be sure there's another person doing exactly that same thing in another company in another University in a place that you've never been all right well thanks