Andrew Ng: Opportunities in AI - 2023

[Music] it is my pleasure to welcome Dr Andrew right Andrew is the managing General partner of AI fund founder of deep learning AI and learn and Landing AI chairman and co-founder of Coursera in an Adjunct professor of computer science here at Stanford previously he had started and led the Google brain team which had helped Google adopt modern Ai and he was also director of the Stanford AI lab about 8 million people one in one thousand persons on the planet have taken an AI class from him and through both his education and his AI work he

has changed numerous lives please welcome Dr Andrew [Music] foreign Thank you Lisa it's good to see everyone so what I want to do today is um chat you about some opportunities in AI so I've been seeing AI is a new electricity one of the difficult things to understand about AI is that it is a general purpose technology meaning that it's not useful only for one thing but it's useful for lots of different applications kind of like electricity if I was to ask you what is electricity good for you know it's not any one thing it's

a lot of things so what I'd like to do is start off sharing with you how I view the technology landscape and this will lead into the set of opportunities so a lot of hype lots of excitement about Ai and I think a good way to think about AI is as a collection of tools so this includes a technique called supervised learning which is very good at recognizing things or labeling things and gents of AI which is relatively new exciting development if you're familiar with AI you may have heard of other tools but I'm going

to talk less about these additional tools and I'll focus on a on what I think are currently the two most important tools which are supervised learning and generative AI so supervised learning is very good at labeling things or very good at Computing input to outputs or a to B mappings given input a give me an output B for example given an email we can use supervised learning to label it as spam or not spam the most lucrative application of this that I've ever worked on is probably online advertising we're given an ad we can label

our users likely to click on it and therefore show more relevant ads for self-driving calls given the sensitive readings of a car we can label it with where the other cars one project that my team AI fan worked on was ship route optimization where given a route that the ship is taking or considering taking we can label that with how much fuel we think to consume and use this uh make ships more fuel efficient still a lot of work in automated visual inspection in factories so you can take a picture of a smartphone that was

just manufactured and label is just scratched right near the defect in it or if you want to build a restaurant review reputation monitoring system you can have little piece of software that looks at online restaurant reviews and labels that as positive or negative sentiment so one nice thing one cool thing about supervised learning is that it's not useful for one thing it's useful for all of these different applications and many more besides let me just walk through you know concretely the workflow of one example of a supervised learning labeling things kind of project if you

want to build a system to label restaurant reviews you then collect a few data points or collect the data set where say you know the pastrami sandwich is great to say that is positive civil is slow that's negative my favorite chicken curry that's positive and um here I've shown three data points whether you're building this you may get thousands of data points like this or thousands of trading examples we call it and the workflow of a machine learning project or an AI project is you get labeled data maybe thousands of you know data points then

you have an AI engineering team trained an AI model to learn from this data and then finally you would find maybe a cloud service to run the trained AI model and then you can feed it you know best bubble tea I've ever had and that's positive sense of it and so I think the last decade was maybe the decade of large-scale supervised learning what we found starting about 10 15 years ago was if you were to train a small AI model to train a small neural network with small deep learning algorithm Basia small AI model

maybe not on a very powerful computer then as you fed more data is performance would get better for a little bit but then it would flatten out it would plateau and it would stop being able to use the data to get better and better but if you were to train a very large AI model lots of compute on maybe powerful gpus then as we scaled up the amount of data we gave the machine learning model this performance would kind of keep on getting better and better so this is why when I started let the Google

brain team the primary mission that I directed the team to solve at the time was let's just build really really large neural networks that we then fed a lot of data to and that recipe fortunately worked and I think the idea of driving large compute and large scale of data that recipes really helped us driven a lot of AI progress over the last decade so if that was the last decade of AI I think this decade is turning out to be also doing everything we had to supervised learning but adding to it um the exciting

tool of genus of AI so many of you maybe all of you will play their chai GPT and Bob and so on but just you know given a piece of text which you call a prompt like I love eating if you run this multiple times maybe you get bagels cream cheese or my Mother's Meatloaf or um Ultra friends and the AI system can generate output like that um given the amounts of Buzz and excitement about genes of AI I thought I'd take just half a slide to you know say a little bit about how this

works so it turns out that germs of AI at least this type of text generation the core of it is using supervised learning that inputs output mappings to repeatedly predict the next word and so if your system reads on the Internet it's a sentence like my favorite food is a bagel with cream cheese and locks then this is translated into a few data points where if it sees my favorite food is a in this case try to guess that the right next word was Bagel or my favorite food is a bagel try to guess next

word is with um and similarly if it sees that you know in this case the right guess for the next word would have been cream so by taking text that you find on the internet or other sources and by using this input output supervised learning to try to repeatedly predict the next word if you train a very large AI system on hundreds of billions of words so in the case of the largest models now more than a trillion words then you get a large language model like chai GP and you know there are additional other

important technical details I talked about predicting the next word technically these systems predict the next sub word or part of work called the token and then there are other techniques like rohf for further tuning the AI output to be more helpful honest and harmless but at the heart of it is this using supervised learning to repeatedly predict the next word that that's really what's enabling um the exciting you know really fantastic progress on large language models so um while many people have seen large English models as a fantastic consumer tool you can go to a

website like track GPS website or Bots or other large language models and use it I think it's fantastic too there's one of the trends I think is still underappreciated which is the power of large language models not only not just as consumer two but as a developer to so it turns out that um there are applications that used to take me months to build that a lot of people can now build much faster by using a large language model so specifically the workflow for supervised learning building the restaurant review system say would be that you

need to get a bunch of label data and you know maybe that takes a month to get a few thousand data points and then have an AI team train and tune and really get you know optimized performance on your AI model maybe that'll take three months then find a cloud service to run it make sure it's running robustly make sure it's recognized maybe that'll take another three months so pretty realistic timeline for building a commercial grade machine Learning System is like six to 12 months right so teams I've LED will often took you know roughly

six to 12 months to build and deploy these systems and and some of them turned out to be really valuable uh but this is a realistic timeline for building and deploying a commercial grade AI system in contrast with prompt base AI we write a prompt this is what the workflow looks like you can specify a prompt that takes maybe minutes or hours and then you can deploy it to the cloud and that takes maybe hours or days so there are certain AI applications that used to take me you know literally six months maybe a year

to build that many teams around the world can now build in maybe a week and I think this is already starting but the best is still yet to come this is starting to open up a flood of a lot more AI applications that can be built by a lot of people so I think many people still underestimate the magnitude of the floods of custom AI applications that I think is going to come down the pipe now I know you probably were not expecting me to write code in this presentation but that's what I'm going to

do um so it turns out this is all the code I need um in order to write a sentiment classifier so I'm gonna you know some of you will know python I guess important tools from open Ai and then I have this prompt that says classify the text to below delimited by three dashes as having either a positive or negative sentiment I don't know I I don't know what I rather fantastic time back at Stanford GSB on we learned a lot and also made great new friends all right so that's my prompt and now I'm

just gonna run it and I've never run it before so I really hope oh thank goodness we got the right answer and this is literally all the code it takes to build a sentiment classifier um and so today you know developers around the world can take literally maybe like 10 minutes to build a system like this um and that's a very exciting development so one of the things I've been working on was um trying to teach you know online classes about how to use prompting not just as a consumer too but as a developer too

so stop the technology landscape let me now share my thoughts on what are some of the AI opportunities I see this shows what I think is the value of different AI Technologies today um and I'll talk about three years from now but the vast majority of financial value from AI today is I think supervised learning where for a single company like Google can be worth more than 100 billion US dollars a year and also there are millions of developers building supervised learning applications so it's already massively valuable and also with tremendous momentum behind it just

because of the sheer efforts in you know Finding applications and building applications and then generative AI is the really exciting new entrance which is much smaller right now and then there are the other tools I'm including for completeness we can you know the size of these circles represent the value today this is what I think it might grow to in three years so supervised learning already really massive May double say in the next three years from truly massive to even more massive and gents of AI which is much smaller today I think we're much more

than double in the next three years because of the number of amounts of developer interests the amount of venture capital Investments number of large corporates exploring applications and I also just point out three years is a very short time Horizon if it continues to compound anything near this rate then in six years you know it'll be it'll be even vastly larger but um this light shaded region in green or orange that light shaded region is where the options is for either new startups or for large companies incumbents to create and to enjoy value capture but

one thing I hope you take away from this slide is that all of these Technologies are general purpose Technologies so in the case of supervised learning a lot of the work that had to be done over the last decade but it's continuing for the next decade is identified and execute on the concrete use cases and that process is also kicking off for um generative AI so for this part of the presentation I hope you take away from it that general purpose Technologies are useful for many different tasks a lot of value remains we created using

civilized learning and and even though we're nowhere near finishing figuring out the exciting use cases of supervised learning we have this other you know fantastic two of Genesis AI which further extends the set of things we can now do using AI um but one caveat which is that there will be short-term fans along the way so I don't know if um some of you might remember the app called lenser This is the app that will let you upload pictures of yourself and they'll render a cool picture of you as an astronaut or a scientist or

something um and it was a good idea and people liked it and it's really just took off like crazy like that through last December and then it did that and that's because lenser was it was a good idea people liked it but it was a relatively thin software layer on top of someone else's really powerful apis um and so even though it was a it was a useful product it was in a defensible business um and when I when I one thing about you know apps like lenser um I'm actually reminded that when Steve Jobs

gave us the iPhone shortly after someone wrote an app that I paid 199 for um to do this to turn on the LED to turn the phone into the flashlight and that was also a good idea to write an app to turn on the LED light but it also wasn't a defensible long-term it also didn't create very long-term value because it was a easily replicated and underpriced and eventually incorporated into iOS but with the rise of iOS with the rise of the iPhone someone also figured out how to build things like uber and Airbnb and

Tinder the very long-term very defensible businesses that created you know sustaining value and I think with um the rise of gents of AI or the rise of new AI tools I think what really what excites me is the opportunity to create those really deep really hard applications that hopefully can create very long-term value so the first Trend I want to share is ai's general purpose technology and a lot of the work that lies ahead of us is to find the very diverse use cases and to build them there's a second Trend I want to share

with you which relates to why AI isn't more widely adopted yet it feels like a bunch of us have been talking about AI for like 15 years or something but if you look at where the value of AI is today a lot of it is still very concentrated in consumer software internet right once you go outside you know Tech or consumer software internet there's some air adoption but a lot of views very early so why is that it turns out if you were to take all current and potential AI projects and sort them in decreasing

all their value then to the left of this curve the head of this curve are the multi-billion dollar projects like advertising or web search or um for e-commerce your product recommendations your company Amazon and it turns out that about 10 15 years ago you know variously my friends and I we figured out a recipe for how to hire say 100 Engineers to write one piece of software to serve more relevant ads and apply that one piece of software to billion users and generate massive Financial value so that works um but once you go outside consumer

software internet hardly anyone has a hundred million or a billion users that you can write and apply one piece of software to so once you go to other Industries as we go from the head of this curve on the left over to the long tail these are some of the projects I see and I'm excited about I was working with a pizza maker that was taking pictures of the Pisa they were making because they needed to do things like make sure that the cheese is spread evenly so this is about a five million dollar project

um but that recipe of hiring 100 Engineers or dozens of Engineers to work on a five million dollar project that doesn't make sense um or there's another very example working with an agriculture company that with them we figured out that we use cameras to find out how tall is the wheat and wheat is often bent over because of wind or rain or something and we can chop off the weeds at the very height then that results in more food for the farmer to sell and it's also better for the environment but this is another you

know five million dollar project that that old recipe of having a large group of Highly School Engineers to work on this one project that doesn't make sense um and some of the materials grading cloth grading sheet metal grading many projects like this so whereas to the left in the head of this curve there's a small number of let's say multi-billion dollar projects and we know how to execute those you know delivering value in other Industries I'm seeing a very long tale of tens of thousands of let's call them five million dollar projects that until now

have been very difficult to excuse on because of the high cost of customization the trend that I think is exciting is that the AI Community has been building better tools that lets us aggregate these use cases and make it easy for the end user to do the customization so specifically I'm seeing a lot of exciting low code and no code tools that enable the user to customize the AI system what this means is instead of me needing to worry that much about pictures of Pisa we have tools because we're starting to see tools that can

enable the I.T Departments of the pizza making factory to train an AI system on their own pages of Pisa to realize there's five million dollars worth of value and by the way the pictures of pizza you know they don't exist on the internet so Google and Bing don't have access to these pictures we need tools that can be used by really The Pizza Factory themselves to build and deploy and maintain their own custom AI system that works on their own pictures of pizza and broadly the technology for enabling this um some of it is prompting

with text prompting visual prompting but really large language models and and similar tools like that or a technology called Data Centric AI whereby instead of asking the Pizza Factory to write a lot of code yeah which is which which is challenging we can ask them to provide data which turns out to be more feasible and I think the second trend is important because I think this is a key part of the recipe for taking the value of AI which so far still feels very concentrated in the tech world and the consumer software into their world

and pushing this out to you know all Industries really to the rest of the economy which which you know sometimes is easy to forget the rest of the economy is much bigger than the tech world so there are two Trends I shared it has a general purpose technology lots of concrete use cases to be realized as well as local no code easily used tools enabling AI to be deployed in more Industries how do we go after these opportunities so about five years ago there was a puzzle I want to solve which is I felt that

many valuable AI projects are now possible I was thinking how do we get them done and having LED AI teams in you know Google and Baidu in big tech companies I had a hard time figuring out how I could operate a team in a big tech company to go after a very diverse set of opportunities and everything from Maritime shipping to education to financial services and Healthcare and on and on it's just very diverse use cases very diverse go to markets and very diverse really you know customer bases and applications um and I felt that

the most efficient way to do this would be if we can start a lot of different companies to pursue these very diverse opportunities so that's why I ended up starting AI fun which is Adventure Studio that builds startups to pursue a diverse surveyor opportunities and of course in addition to lots of startups in company companies also have a lot of opportunities to integrate AI into existing businesses in fact one pattern I'm seeing for incumbent businesses is um distribution uh is is often one of the significant advantages of incoming companies is they play the cards right

can allow them to integrate AI into into their products quite efficiently but just be Concrete where are the options so I think of this as a this is what I think of as the AI stack right at the bottom level is the hardware semiconductor layer fantastic opportunities there but very Capital intensive very concentrated so there's a lot of resources relatively few winners so some people can and should play there I personally don't like to play them myself there's also the infrastructure layer also fantastic opportunities but very couple intensive very concentrated so I tend not to

play that myself either and then does it develop a two layer what I showed you just now was I was actually using open ai's API as a developer tool and then I think the developer 2 sector is hyper competitive look at all the startups chasing open AI right now but there will be some Mega winners and so I sometimes play here but primarily when I think of a meaningful technology Advantage because I think that earns you the right or earns you a better shot at being one of the mega winners and then lastly even though

a lot of the media attention in the buzz is in the infrastructure and developer tooling layer it turns out that that layer can be successful only if the application layer is even more successful and we saw this at the rise of SAS as well a lot of the buzz excitement is on the technology the tooling layer which is fine nothing wrong with that but the only way for that to be successful is that the application layer is even more successful so that frankly they can generate enough Revenue to pay the infrastructure and the tooling layer

so actually let me mention one example um I'm all right I was actually just texting the CEO yesterday but armor raw is a completely rebuilt that uses AI for romantic relationship coaching right um and Jesus points out I'm an AI guy and I feel like I know nothing really about Romance um and if you don't believe me you can ask my wife if she will confirm that I know nothing about Romance but what we want to build this we wanted getting together with the former CEO of Tinder or Renata and eyeball and with my team's

expertise in Ai and her expertise in relationships we actually ran Tinder she knows more about you know relationships I think anyone I know we're able to build something pretty unique using AI for you know kind of romantic relationship mentoring um and the interesting thing about applications like these is when we look around um you know how many teams in the world are simultaneously expert in Ai and in relationships and so at the application layer I'm seeing a lot of exciting opportunities that seem to have a very large Market but where the competition sets is very

light relative to the magnitude of the opportunity it's not that they're no competitors but it's just much less intense compared to the developer 2 or the infrastructure layer and so because I've spent a lot of time iterating on the process of building startups what I'm going to do is just you know very transparently tell you the recipe we've developed for building startups and so after many years of iteration and Improvement this is how you know we now build startups my team's always had access to a lot of different ideas you know internally generated ideas from

partners and I want to walk through this with one example or something we did which is a company bearing AI which uses AI to make ships more fuel efficient so this idea came to me when a few years ago a large Japanese conglomerate called mitsui that is a major shareholder in the Opera is a major Shipping Lines they came to me and they said hey Andrew you should build a business to use AI to make ships more fuel efficient and the specific idea was you know think of it as a Google Maps for ships where

you can suggest a ship or towership how to steer so you still get your destination on time by using it turns out about 10 less fuel um and so what we now do is we spend about a month validating the idea so double check is this idea even technically feasible and then talk to prospective customers to make sure this is marketing so you spend up to about a month doing that and if it passes this stage then we will go and recruit the CEO to work with us on the project when I was starting out

I used to spend a long time working on the project myself before bringing on the CEO but after iterating we realized that bringing on the leader at the very beginning to work with us it reduces a lot of the burden of having to transfer knowledge or having a seal come in and rebound having to revalidate whether we discover it so the process is we've learned much more efficiently which is bringing the leader at the very start um and so in the case of bearing AI we found a fantastic CEO Dylan Kyle who's a repeat entrepreneur

a one successful exhibit before and then we spent three months um six two weeks Sprints to work with them to build a prototype as well as do do deep customer validation if it survives the stage and we have about two-thirds 66 survival rate whenever the first check-in which then gives the company resources to hire an executive team you know build the key team get the MVP working minimum viable product working and get some real customers and then after that you know hopefully then successfully raises additional external rounds of funding they can keep on growing and

scaling so I'm really proud of the work that my team was able to do um to support mitsui's idea and telling how as CEO and today there are hundreds of ships on the high seas right now that are steering themselves differently because of bearing Ai and um 10 fuel savings translates the rough order amount to maybe 450 000 in savings and Fuel per ship per year and of course it's also frankly quite a bit better for the environment um and I think the startup I think would not have to existed if not for Dylan's fantastic

work and then also you know mitsui brings this idea to me and I like this example because this is another one it's like you know this is a startup idea that I just point out I would never have come up with myself right because you know I've been on a boat but what do I know about Maritime shipping um but it's the deep subject matter expertise of mitsui um that had this inside together with Dylan and then my team's expertise in AI that that made this possible and so as I operate in AI one thing

I've learned is my swim Lane is AI and that's it because I've done a time it was very difficult for me to be expert in Maritime shipping and romantic relationships and Healthcare and financial services on and on and on and so I've learned that if I can just help get a accurate technical validation and then use you know AI resources to make sure that AI Tech has been quickly and well and I think with all always managed to hope the companies build a strong technical team then partnering with subject matter experts often results in exciting

New Opportunities and I want to share with you one other weird aspect of another one of the weird listing I've learned about you know building startups which is um I like to engage only when there's a concrete idea and this runs counter to a lot of the advice you hear from the design thinking methodology which often says don't rush to solutioning right explore a lot of alternatives for either a solution honestly we tried that it was very slow um but what we've learned is that at the ideation stage is someone comes to me and says

hey Andrew you should apply AI to financial services because I'm not a subject matter expert in financial services is very slow for me to go and learn enough about financial services so you can figure out what to do I mean eventually you could get a good outcome but it's a very labor intensive very slow very expensive process for me to try to learn industry after industry in contrast one of my partners wrote this idea as a tongue-in sheet not not really seriously but you know let's say the country is buy gbt let's eliminate commercials by

automatically buying every product advertising it's changed or not haven't seen any ads it's not a good idea but it is a concrete idea and it turns out concrete ideas can be validated or falsified efficiently um they also give a team a clear direction to execute and I've learned that in today's world especially with you know the excitement Buzz the exposure the AI of a lot of people it turns out that there are a lot of subject matter experts in today's world that have deeply thought about the problem for months sometimes even you know one or

two years but they've not yet had a bill partner and when we get together with them and hear and they share the idea of us it allows us to work with them to very quickly go into validation and building and I find that this works because there are a lot of people that have already done the you know design thinking thing of exploring a lot of ideas and running down to really good ideas and there I find that there's so many good ideas sitting out there that no one is working on on that finding those

good ideas that someone has already had and wants to share of us that wants to build partner for that turns out to be much more efficient engine so um before I wrap up we'll go to question a second just a few slices talk about risks and social impact um so yeah it's very powerful technology to State something you probably guess my teams and I we only work on projects that move Humanity forward and you know we have multiple times cool projects that we assess to be financially sound uh based on ethical grounds it turns out

I've been surprised and sometimes dismayed at the creativity of people to come up with good ideas so to come up with really bad ideas that that seem profitable but really should not be built um we'll kill a few projects on those on those drones and then I think has to be acknowledge the AI today does have problems with bias fairness accuracy um but also you know the technology is improving quickly so I see that AI systems today are less biased than six months ago and more fair than six months ago which is not to dismiss

the importance of these problems they are problems and we should continue to work on them but I'm also gratified at the number of aits working hard on these issues to make them much better when I think of the biggest risk of AI I think that because there is one of the biggest there is is the disruption to jobs um this is a diagram from a paper by our friends at the University of Pennsylvania and some folks at open AI analyzing the exposure of different jobs to AI Automation and it turns out that whereas the previous

wave of automation mainly the most exposed jobs were often the lower wage jobs such as when you know we put robots into factories with this current wave of automation is actually the highway shops further to the right of this apps is that seems to have more of their tasks expose the AI automation so um even as we create tremendous value using AI I feel like as Citizens um and and our corporations and the governments and really our society I feel a strong obligation to make sure that um people especially people whose livelihoods are disrupted are

still well taken care of are still treated well and then lastly um there's also been it feels like every time there's a big wave of progress in AI you know there's a there's a big wave of hype about artificial gender intelligence as well when deep learning start to work really well 10 years ago there was a lot of hype about AGI and now the genify is working really well there's another wave of hyper by AGI um but I think that artificial gender intelligence AI didn't do anything a human can do it's still decades away you

know maybe 30 to 50 years maybe even longer I hope we'll see it in a lifetimes but I don't think as anytime soon one of the challenges is that the biological path to intelligence like humans and the digital path to intelligence you know AI they've taken very different paths and the funny thing about the definition of AGI is yeah benchmarking there's very different digital Parts intelligence with really the biological part of intelligence so I think you know large English models are smarter than any of us in certain key Dimensions but much dumber than any of

us in other dimensions and so forcing it to do everything a human can do is like a funny comparison um but I hope we'll get there maybe hopefully within our lifetimes and then there's also a lot of I think overblown hype about AI creating Extinction risk for Humanity um candidly I don't see it I just don't see how AI creates any meaningful Extinction rest of humanity um I think that people worry we can't control AI but we have lots of AI will be more powerful than any person but with lots of experience steering very powerful

entities such as corporations or nation states that are far more powerful than any single person and making sure they for the most part benefit humanity and also technology develops gradually the so-called hard takeoff scenario where it's not really working today and then suddenly one day overnight it works brilliantly in which is super intelligent takes over world that's just not realistic and I think AI technology would develop slowly like all that you know like and then it gives us plenty of time to make sure that we provide oversight and can manage it to be safe and

lastly if you look at the real Extinction rest of humanity um such as fingers crossed the next pandemic or climate change leading to a massive depopulation of some parts of the planets or much lower odds but maybe someday and as there are doing to us what it had done to the dinosaurs I think if you look the actual real Extinction risk to humanity AI having more intelligence even artificial intelligence in the world would be a key part of the solution so I feel like if you want Humanity to survive and thrive for the next Thousand

Years rather than slowing AI down which some people propose I would mother I would rather maybe ai go as fast as possible um some of that just to summarize this is my last line I think that AI as a general purpose technology creates a lot of new opportunities for everyone and a lot of the excisely and important work that lies ahead of us all is to go and build those concrete use cases and hopefully in the future hopefully I have options to maybe engage with more of you on those opportunities as well so that let

me just say thank you all very much [Applause]