Andrew Ng: Building Faster with AI

It's really great to see all of you. What I want to do today since this is build as startup school is share with you some lessons I've learned about building startups at AI fund. AI funds a venture studio and we build an average of about one startup per month. And because we co-founded startups, we're in there writing code, talking about customers, design on features, detering pricing. And so we've done a lot of reps of not just watching others build startups, but actually being in the weeds, building startups with entrepreneurs. And what I want to do

today is um share with you some of the lessons I've learned building startups, especially around this changing AI technology and what it enables. And it'll be focused on the theme of speed. So it turns out that for those of you that want to build a startup, I think a strong predictor for startup's odds of success is execution speed. And I actually have a lot of respect for the entrepreneurs and executives that can just do things really quickly and new AI technology is enabling startups to go much faster. So what I hope to do is share

with you some of those best practices which are frankly changing every two to three months still to let you get that speed that hopefully lets you have a higher odds of success. Before diving to speed, you know, a lot of people ask me, hey Andrew, where are the opportunities for startup? So this is what I think of as the AI stack where at the lowest level are the semiconductor companies then the clouds are hyperscalers built on top of that. A lot of the AI foundation model companies built on top of that. And even though a

lot of the PR excitement and hype has been on these uh technology layers, it turns out that almost by definition, the biggest opportunities have to be at the application layer because we actually need the applications to generate even more revenue so that they can afford to pay the foundation cloud and semiconductor technology layers. So for whatever reason media and social media tends not to talk about the application layer as much but for those of you think you're building startups almost by definition the biggest opportunities have to be there although of course the opportunities at all

layers of the stack. One of the things that's changed a lot over the last year um and in terms of AI tech trends I if you ask me what's the most important tech trend in AI I would say is the rise of agentic AI and about a year and a half ago when I started to go around and give talks to try to convince people that AI agents might be a thing I did not realize that around last summer a bunch of marketers would get a hold of this term and use it as a sticker

and slap it on everything in site which made it almost lose some of meaning but I want to share with you from a technical perspective why I think agentic AI is exciting and important and also opens up a lot more startup opportunities. So, it turns out that the way a lot of us use LMS is to prompt it to have it gener output. And the way we have an LM output something is as if you're going to a human or in this case an AI and asking it to please type out an essay for you

by writing from the first word to the last word all in one go without ever using backspace. And humans, we don't do our best writing, being forced to type in this linear order. And it turns out neither does AI. But despite the difficulty of being forced to write in this linear way, um our LMS do surprisingly well. With agentic workflows, we can go to AI system and ask it to please first write an essay outline, then do some webs research if it needs to and fetch uh some web pages to put in their own context,

then write a first draft, then read the first draft and critique it and revise it and so on. And so we end up with this iterative workflow where your model does some thinking and some research does some revision goes back to do more thinking and by going around this loop many times. Uh it is slower but it delivers a much better work product. So for a lot of t lot of projects AI fund has worked on everything from um pulling out complex compliance documents to uh medical diagnosis to reasoning about complex legal documents. We found

that these agentic workflows are really a huge difference between it working versus not working. But a lot of the work that needs to be done, a lot of the valuable businesses to be built still will be taking workflows existing or new workflows and figuring out how to implement them into these size of agentic workflows. So just to update the picture for the AI stack, um what has emerged over the last year is a new agentic orchestration layer that helps application builders orchestrate or coordinate a lot of calls to the technology layers underneath. And the good

news is uh the orchestration layer has made it even easier to build applications. But I think the basic conclusion that the application layer has to be the most valuable layer of the stack still holds true with a bias or focus on the application layer. Let me now dive into some of the best practices I've learned for how startups can move faster. It turns out that um at AI fun we only focus on working on concrete ideas. So to me a concrete idea a concrete product idea is one that's specified in enough detail that an engineer

can go and build it. So for example if you say let's use AI to optimize healthcare assets you know that's actually not a concrete idea. It's too vague. If you tell me it's a very software to use AI to optimize healthcare assets different engineers would do totally different things and because it's not concrete you can't build it quickly and you don't have speed. In contrast, if you had a concrete idea like let's write software to let hospitals, let patients book MR machine slots online to optimize usage. I don't know if this is a good or

a bad concrete idea. Actually business is already, you know, doing this. But it is concrete and that means engineers can build it quickly. If it's a good idea, you find out it's not a good idea, you will find out. But having concrete ideas buys you speed. Or someone to say, let's use AI for email personal productivity. Too many interpretations of that. That's not concrete. But if someone says could you build an app Gmail integrate the automation that use let's use the right prom source right filter entire emails that is concrete I could you know I

could go build that this afternoon. So concretness buys you speed and the deceptive thing for a lot of entrepreneurs is the vague ideas tend to get a lot of kudos. If you go and tell all your friends we should use AI to optimize the use of healthcare assets. Everyone will say that's a great idea. But it's actually not a great idea at least in the sense of being something you can build. Uh it turns out when you're vague, you're almost always right. Uh but when you're concrete, you may be right or wrong. Either way is

fine. We can discover that much more fast, which is what's important for SA. In terms of executing on concrete ideas, I I I find that at AI fun, I ask my team to focus on concrete ideas because um a concrete idea gives clear direction and the team can run really fast to build it and either validate it, prove it out or falsify and conclude it doesn't work. Either way is fine. So we can do that quickly and it turns out that finding good concrete ideas usually requires someone could be you could be a subject matter

expert thinking about a problem for a long time. Uh so for example actually before before you know starting Corsera um I spent years right thinking about online education talking to users holding my own intuitions about what would make a good edtech platform and then after that long process I think YC sometimes calls it wondering the idea maze but after thinking about it for a long time you find that the guts of people that have thought about this for a long time can be very good about rapidly making decisions as in after you've thought about this

talked to customers and so on for a long time if you ask this expert, should I build this feature or that feature? You know, the gut, which is an instantaneous decision, uh, can be actually a surprisingly good proxy. It can be surprisingly good mechanism for making decisions. And I know I work on AI, you might think I'll say, oh, we need data. And of course, I love data, but it turns out getting data for a lot of startups is actually slow mechanism for making decisions. And a subject matter expert with a good gut is often

a much better mechanism for making a speedy decision. And then one other thing for many successful startups at any moment in time you're pursuing one very clear hypothesis they building out and trying to sell China value of hotspot. Um and a startup doesn't have resources to hedge and try 10 things at the same time. So pick one go for it and if data tells you to lose faith in that idea that's actually totally fine. Just pivot on a dime to pursue a totally different concrete idea. So that's what often feels like an AI fund. We're

pursuing one thing doggedly with determination until the world tells us we were wrong then change and pursue a totally different thing with equal determination and equal doggedness. And one other pattern I've seen, if every piece of new data causes you to pivot, it probably means you're starting off from two weaker a base of knowledge, right? If every time you talk to a customer, you totally change your mind, probably means you don't know enough about that sector yet to have a really high quality concrete idea and uh finding someone that's thought about a subject for longer

may get you on to better path in order to go faster. The other thing I often think about is the built feedback loop which is rapidly changing when it comes to how we build with AI coding assistance. So when you're building a lot of applications, one of the biggest risks is custom acceptance, right? A lot of startups struggle not because we can't build whatever we want to build, but because we build something and it turns out nobody cares. And so for a lot of the way uh you know I build startups especially applications less so

deep tech less so technology startups but definitely application startups is we often build software so this is an engineuring toss and then we will get feedback from users and this is a product management toss and then we'll go back you know then based on the user feedback we'll tweak our views on what to build go back to write more software and we go around this loop many many times iterate toward product market fit and it turns out that with AI coding assistance which Andre talked about as well um rapid engineering is becoming possible in a

way that just was not posing much more feasible. So the speed of engineing is going up rapidly and the cost of engineing is also going down rapidly. This changes the mechanisms by which we drive startups around this loop. When I think about the software that I do, I maybe put into two major buckets. Sometimes I'll build quick and dirty prototypes to test an idea. You say build a new customer service chatbot. Let's build AI to process legal documents whatever build a quick and dirty prototype to see if we think it works. The other type of

software where I do is write maintain production software maintain legacy software but these massive production ready code bases depending on which analysts report you trust. It's been hard to find very rigorous data on this. You know when writing production quality code maybe we're 30 to 50% faster with AI systems. Hard to find a rigorous number. maybe he's plausible to be but in terms of building quick and dirty prototypes we're not 50% faster I think we're easily 10 times faster maybe much more than 10 times faster and there are a few reasons for this uh when

you're building standalone prototypes there's less integration with legacy software infrastructure legacy data needed um also the requirements reliability even scalability even security are much lower and I know I'm not supposed to tell people to write insecure code right feels like the wrong thing to say, but I routinely go to my team and say, "Go ahead, write insecure code." Because if this software is only going to run on your laptop and you don't plan to maliciously hack your own laptop, it's fine to have insecure code, right? But of course, after it seems to be working, please

do make it secure before you ship it to someone else. And you know, like a leaking PI, leaking sense data that that is, you know, very damaging. So before you ship it, make it secure and scalable, but they're just testing it. It's fine. And so I find increasingly startups will systematically pursue innovations by building 20 prototypes to see what works, right? Uh because I I I know that there's some angst in AI. A lot of proof of concepts don't make to production. But I think by driving the cost of a proof of concept low enough,

it's actually fine if lots of proof of concepts don't see the light of day. And I know that um the mantra move fast and break things got a bad rep because you know it broke things. And some teams took away from this that you should not move fast, but I think that's a mistake. I tend to tell my teams to move fast and be responsible. And I think they actually lots of ways to move really quickly while still being responsible. And in terms of the AI assistance uh coding landscape, I think was it three four

years ago code autocomplete right popularized by GitHub copilot and then there was a cursor windserve generation of AI enabled ids which great use winds and cursor quite a lot um and then starting I don't know six seven months ago uh there started to be this new generation of highly agentic coding assistants uh including that she's using o3 a lot for coding um cloud code is fant Fantastic. Since quad 4 release, it's become and and ask me again in a few months, I may use something different. But the tools are evolving really rapidly, but I think

uh cloud codeex this is a new generation of highly agentic coding assistance that is making developer productivity keep on growing. And the interesting thing is if you're even half a generation or one generation behind actually makes a big difference compared to if you're on top of the latest tools and I find my team is taking really different approaches to software engineing now compared to even three or six months ago. One surprising thing is we we're used to thinking of code as this really valuable artifact because it's so hard to create but because the cost of

software engine is going down code is much less of a valuable artifact as it used to. So I'm on teams where you know we've completely rebuilt a codebase three times the last month right because it's not that hard anymore to just completely rebuild a codebase pick a new data schema is fine because the cost of doing that has plummeted some of you may have heard of Jeff Bezos's terminology of a two-way door versus a one-way door. A two-way door is decision that you can make. If you change your mind, come back out, you know, reverse

it relatively cheaply. Whereas a one-way door is you make a decision and you change your mind is very costly or very difficult to reverse. So choosing the software architecture of your tech stack used to be a one-way door. Once you built on top of a certain tech stack, you set a database schema, really hard to change it. So that used to be a one-way door. I don't want to say it's totally a two-way door, but I find that um my team will more often build on a certain tech stack a week later, change your mind,

let's throw the code base away and redo it from scratch on a new tech stack. I I don't want to overhype it. We don't do that all the time. There are still costs to redoing that. But I find my team is often rethinking what is a one-way door and what's now a two-way door because the cost of software engineering is so much lower now. And maybe going a little bit beyond software engineering, I I I feel like this actually a good time to empower everyone to build of AI. Uh over the last year, a bunch

of people advised others not to learn to code on the grounds of AI were automated. I think we'll look back on this as some of the worst career advice ever given because as better tools make software engineing easier, more people should do it, not fewer. So when many decades ago the world moved from punch calls to keyboard and terminal that made coding easier. When we moved from assemblies high level languages like cobalt um there actually people arguing back then that now we have cobalt we don't need programmers anymore like people actually wrote papers to that

effect but of course that was wrong and programming languages made it easier to code and more people learn to code text IDs ID is the AI coding assistant um and as coding becomes easier more people should learn to code. I have a controversial opinion which is uh I think actually it's time for everyone of every job role to learn to code and in fact on my team you know my CFO my head of talent my recruiters my front desk uh uh person all of them know how to code and I actually see all of them

performing better at all of their job functions because they can code and I think um I'm probably a little bit ahead of the curve probably most businesses are not there yet but in the future I think we empower everyone to code a lot of people can be more productive I want to share with you One lesson I learned as well on on why we should have people learn to do this which is um when I was teaching generative VI for everyone on Corsera, we needed to generate background art like this uh using midjourney and you

know one of my team members uh new art history and so he could prompt midjourney with the genre the palette the artistic inspiration had a very good control over the images he generated. So we end up using all of Tommy's generated images. Whereas in contrast, I don't know art history. And so when I prompt, you know, image generation, I could write, please make pretty pictures of robots for me, right? And and I could never have the control that my collaborators could. And so I couldn't generate as good images as he could. And I think with

computers, one of the most important skills of the future is the ability to tell a computer exactly what you want. So they'll do it for you. And will be people that have that deeper understanding of computers that will be able to command a computer to get the outcome you want. And learning to code, not not that you need to write the code yourself. Steer AI to code for you seems like it will remain the best way to do that for a long time. with software engineering becoming much faster. The other interesting dynamic I'm seeing is

that the product management work getting user feedback deciding what features to build that is increasingly the bottleneck and so I'm seeing very interesting dynamics in multiple teams over the last year a lot more of my teams have started to complain that their bottlenecks on product engineering and design because the engineers have gotten so much faster some interesting trends I'm seeing three four five years ago Silicon Valley used to have these slightly suspicious rules of thumb but nonetheless rules of thumb will have 100 p.m. to four engineers or 1 PM to seven engineers was this like

PM product manager to engineering ratio right which should take with a grain of salt but it was typical of a 1 PM to six seven engineers and with engineers becoming much faster I don't see product management work designing what to build becoming faster at the same speed engineers I'm seeing this ratio shift so literally yesterday one of my teams came to me and for the first time when we're planning headcom for a project this team proposed to me not at 1:00 PM to four engineers but to have 1 PM to 0.5 engineers. So the team

actually proposed to me I still know no this is a good idea for the first time in my life that I saw you know managers proposed to me having twice as many PMs as engineers was a very interesting dynamic. I I still don't know if this proposal I heard yes is a good idea but I think it's a sign of where the world is going and I find as PMs that can code or engineers with some product instincts often end up doing better. The other thing that I found important for startup found for startup leaders

is because engineing is going so fast. If you have good tactics for getting rapid feedback to shape your perspective what to build faster that helps you get faster as well. So um I'm going to go through a portfolio of tactics for you know getting product feedback to keep shaping what you will decide to build. And we're going to go through a list of the faster maybe less accurate the slower more accurate tactics. So the fastest tactic for getting feedback is look at the product yourself and just go by your gut. And if you're a subject

matter expert, this is actually surprisingly good you if you know what you're doing. A little bit slower is go ask three friends or teammates to get feedback to play with your product and get feedback. Um little bit slower is ask three to 10 strangers you know for feedback. Um, it turns out for when I built products, one of the most important skills I think I learned was how to sit in the coffee shop, how to sit in a when it's traveling, when I travel, I often sit in the hotel lobby. It turns out learn to

spot places of high foot traffic and very respectfully, you know, grab strangers and ask them for feedback on whatever I'm building. This used to be easier when I was less known. When when people recognize you, it's a little bit more awkward. I found that um I've actually sat with teams the hotel lobby very high foot for traffic and you know very respectfully ask strangers hey we're building this thing do you mind taking a look oh and I actually learned in a coffee shop there a lot of people working a lot of people don't want to

be working so we give them excuse to be distracted they're very happy to do that too but I've actually kind of made tons of product decisions in a hotel lobby or a coffee shop with collaborators just just just like that send prototypes to 100 testers you if you have access to logic group of users and prototype to more users and these are these get to be slow and slower tactics and I know Silicon Valley you know we like to talk about AB testing of course I do a ton of AB testing but contrary to what

many people think AB testing is now one of the slowest tactics in my menu because it's just slow to ship it yeah it depend on how many users you have right so and then uh the other thing is um as you use anything but the first tactic some teams will look at the data they make a decision but the missing piece is When I AB test something, um, I don't just use result of AB test to pick product A or product B. My team will often sit down and look carefully at the data to hone

our instincts to speed up to improve the rate. I wish we're able to use the first tactic to make high quality decision. Often sit down and think, gee, I thought, you know, this product name will work better than her product name. Clearly, my mental model the users wrong. to really sit down and think to update our mental model using all of that data to improve the quality of our guts on how to make product decisions faster. That turns out to be really important. All right, so talked about um concrete ideas, speed up engineering, speed up

product feedback. This is one last thing I want to touch on which is I've seen that understanding AI actually makes you go faster. Um and and here's why. As a AI person, maybe I'm biased to be pro AI, but I want to share you why. So it turns out that when it comes to mature technology like mobile, you know, many people have had smartphones for a long time. We kind of know what a mobile app can do, right? So many people including nontechnical people have good instincts about what a mobile app can do. If you

look at mature job roles like sales, marketing, HR, legal, they're all really important and all really difficult. But you know, there are enough marketers that that have done marketing for long enough and the marketing tactics haven't changed that much in the last year. So there are a lot of people that are really good in marketing and it's really important really hard but that knowledge is relatively diffused because you know the knowledge of how to do HR like it hasn't changed dramatically you know in the last six months but AI is emerging technology and so the

knowledge of how to do AI really well is not widespread and so teams that actually get it that understand AI do have a advantage over teams that don't whereas if you need if you have an HR problem you can find someone you know that knows how to do it well probably but If an AI problem, knowing how to actually do that could put you ahead of other companies. So things like what accuracy can you get for a customer service chatbot? You know, should you prom fine tune a workflow? Um how do you get a voice

out to low latency? There a lot of these decisions that if you make the right technical decision, you can like solve the problem in a couple days. They make the wrong technical decision, you could chase a blind alley for three months, right? And and one one thing I've been surprised by, it turns out if you have, you know, two possible architecture decisions, it's one bit of information. It feels like if you don't know the right answer, at most you're twice as slow, right? One bit, you know, try both. It feels like one bit of information

can at most buy you a 2x speed up. And I think in some theoretical sense that is true. But what I see in practice, if you flip the wrong bit, you're not twice as slow. You spend like 10 times longer chasing a blind alley. which is why I think going in to have this right technical judgment, it really makes startups go so much faster. The other reason why I find staying on top of AI really helpful for startups is um over the last two years we have just had a ton of wonderful genai tools or

genai building blocks right partial list but prompting workflows evals guardrails rack voice act async programming lots of ETL embeddings fine-tuning graph DB how to integ models there's a long and wonderful list of building blocks that can quickly combine to build software that no one on the planet could have built, you know, even a year ago. And this creates a lot of new opportunities for starters to build new things. So when I learned about these building blocks, this is actually a picture that I have in mind. If you own one building block, like you have a

basic white building block, yeah, you can build some cool stuff. Maybe you know how to prompt. So you have one building block, you build some amazing stuff. But if you get a second building block like you also know how to build chat bots. So you have a white Lego brick and a black Lego brick, you can build something more interesting. Um if you acquire a blue building brick as well, you can build something even more interesting. Get few red building bras, maybe a little yellow one, more interesting, get more building bras, get more building bras,

and very rapidly the number of things you comb combine them to into grows kind of combinatorily or grows exponentially. And so knowing all these wonderful building blocks lets you combine them in much richer combination. One of the things that deep learn does so I actually take a lot of deep learn courses myself you know to because work with great we work with I think like pretty much all the leading AI companies in the world and cert and and um and uh try to hand out building blocks. Um but when I look at the deep learning

course catalog this is actually what I see. And whenever I take these courses to learn these building blocks, I feel like I'm getting new things that can combine to form kind of combinatorally or exponentially more software applications that were not possible just one or two years ago. So just to wrap up, this is my last slide. I then want to take questions if if y'all have any. I find that there are many things that matter for startup, not just speed. But when I look at the startups that AI fund is building, I find that the

management team's ability to execute at speed is highly correlated with its odds of success. And some things we've learned to get you speed is, you know, work on concrete ideas. Um uh it's got to be good concrete ideas. I find that as a as executive, I'm judged on the speed and quality of my decisions. Both do matter, but speed absolutely matters. rapid entrying with AI coding assistance makes you go much faster but that shifts the bottleneck to getting user feedback on the product decisions and so having a portfolio of tactics to go get rapid feedback

and if you haven't learned to go to coffee shop and talk to strangers it it's not easy but then just just be respectful right just be respectful of people that's actually very valuable skill for entrepreneurs to have I think and I think also um staying on top of the technology buys you speed all right with that let me thank Thank you very much. [Applause] Happy questions. As AI advances, do you think it's more important for humans to develop the tools or learn how to use the tools better? Like how can we p position ourselves to

remain essential in a world where you know intelligence is becoming democratized? I feel like AGI has been overhyped and so for a long time there'll be a lot of things that humans can do that AI cannot and I think in the future the people that are most powerful are the people that can make computers do exactly what you want it to do and so I think staying on top of the tools some of us will build tools sometimes but there were a lot of other tools others will build that we can just use but so

people that know how to use AI to get computers to do what you want it to do will be much more powerful, not worry about people running out of things to do, but um people that can use AI will be much more powerful than people that don't. Hey, so well first of all uh thank you so much. I have a huge respect for you and I think that you are true inspiration for a lot of us. My question is about uh the future of compute. So as we move towards uh more powerful more powerful AI,

where do you think that comput is heading? I mean we see people saying let's ship GPUs to space. Some people talking about nuclear power data centers. What do you think about it? There's something I'm debating what I wanted to say in response to the last question about kind of AGI about maybe I'll answer this and a little bit the last question. So it turns out there's one framework you can use for deciding what's hype and what's not hype. I think over the last two years there's been a handful of companies that um hyped up certain

things for promotional PR fundraising influence purposes. And because AI was so new, um, handful of companies got away with saying almost anything without anyone fact-checking them because the technology was not understood. So, one of my mental filters is there's certain hype narratives that make these businesses look more powerful that's been amplified. Um, and so, for example, this idea that um, AI is so powerful, we might accidentally lead to human extinction. That's just ridiculous. But it is a hype narrative that made certain businesses look more powerful and it got you know ramped up and actually helped

certain businesses fundraising goals. AI is so powerful soon no one will even have a job anymore. Just not true, right? But again that made these businesses look more powerful got hyped up or we are so powerful so when the hype narrative we're so powerful that by training a new model we will casually wipe out thousands of startups. That's just not true. Yes, Jasper ran into trouble. Small number of companies got wiped out. But it's not that easy to casually wipe out thousands of startups. AI needs so much electricity. Only nuclear power is good enough for

that. You know, that wind solar stuff not that's just not true. So, I think a lot of this um GPUs in space, you know, I don't know. It's like um go for it. I think we have a lot of room to run still for terrestrial GPUs. Uh yeah, but but I think uh uh some of these hype narratives are have been amplified that that I think uh are a distortion of what what actually will be done. There's a lot of hype in um AI and how and nobody's really certain about how we're going to be

building the future with it. But what are some of the most dangerous biases or overhyped narratives that you've seen people talk about or get uh poisoned by that they end up running with that we should try to avoid or be more aware of and allow us to have a more realistic view as we are building this future. So I think the dangerous AI narrative has been overhyped. Uh AI is a fantastic tool, but like any other powerful tool like electricity, lots of ways to use it for beneficial purposes. Also some ways to use it in

harmful ways. I find myself not using the term AI safety that much. Um not because I think we we should build dangerous things, but because I think safety is not a function of technology, it's a function of how we apply it. So like electric motor you know you can't the maker of electric motor can't guarantee that no one will ever use it from unsafe downstream toss like use electric motor can be used to build a Dallas machine electric vehicle can be used to build a smart bomb but the electric motor manufacturer can't control how be

used downstream. So safety is not a function of the electric motor as a function of how you apply it and I think the same thing for AI. AI is neither safe nor unsafe. It is how you apply it that makes it safe or unsafe. So instead of thinking about AI safety, I often think about responsible AI because it is how we use it responsibly hopefully or irresponsibly that determines whether or not what we build with AI technology ends up being harmful or beneficial. And I feel like sometimes that the really weird corner cases that get

hyped up in the news. I think just one or two days ago there was a Wall Street Journal article about AI losing control of AI or something. And I feel like that article took uh corner case experiments run in a lab and you know sensationalized it in a way that I think was really disproportionate relative to the lab experiment that was being run and unfortunately technology is hard enough to understand that many people don't know better and so these hype narratives do keep on getting amplified um and I feel like this has been used as

a weapon against open source software as well right which is really unfortunate. Thank you for your work. I think your impact is remarkable. Uh my question is um as aspiring founders, how should we be thinking about business in the world where anything can be disrupted in a day? Whatever great mode, product or feature you have can be replicated with VIP code and competitors in basically hours. It turns out when you start a business, there are a lot of things to worry about. The number thing I worry about is uh are you building a product that

users love? Um it turns out that when you build a business there are lots of things to think about the go to market channel competitors technology mode all that is important but if I were to have a singular focus on one thing it is are you building a product that users really want until you solve that you know is very difficult to build a valuable business. After you solve that the other questions do come to play. Uh, do you have a channel to get to customers? What is pricing long-term? What is your moat? I find

that moes tend to be overhyped. Actually, I find that more businesses tend to start off with a product and then evolve eventually into a moat. But consumer products rand is somewhat more defensible. Um, and if you have a lot of momentum, it becomes harder to catch you. But enterprise products sometimes if you have a uh maybe mo is more of a consideration if they're channels that are hard to get into enterprises. So I think um sorry when when AI fund looks at businesses we actually wind up doing a fairly complex analysis of these factors and

writing a you know two to six page narrative memo to analyze it before we decide whether or not to proceed it or not. And and I think um uh all of these things are important, but I feel like at this moment in time, the number of opportunities, meaning the amount of stuff that is possible that no one's built yet in the world, seems much greater than the number of people with the skill to build them. So definitely at the application layer, it feels like there's a lot of white space for new things you can build

that no one else seems to be working on. And I would say, you know, focus on building a product that people want, that people love. Um, and then figure out the rest of it along the way. Although this important figure along the way. Uh, hi professor. Uh, thanks for your wonderful speech. Uh, I am a Andagramress researcher from Stanford and I think your uh, metaphor in your speech is very interesting. You said the uh, current AI tools are like bricks and are be uh, and can be built upon accumulation. However, so far it is difficult

to see the accumulative functional expansion of the uh integration of AI tools because they often align on the stacking of functions based on uh intent distribution and are accompanied by dynamic problems of tokens and time overhead. So um which is uh which is different from static engineering. So what do you think will be the perspective of a possible agent 2 accumulation effect in the future? But hey just just some quick remarks to that right you mentioned agent uh OM token cost my most common advice to developers is to first approximation just don't worry about how

much tokens cost only a small number of startups are lucky enough to have users use so much of your product that the cost of tokens becomes a problem it can become a problem I've definitely been on a bunch of teams where the cost you know users like our product and we started to look at our right geni uh uh bills and it was definitely climbing in a way that really became a problem. But it's actually really difficult to get to a point where your token usage costs are a problem. And for the teams I'm on

where we were lucky enough that users made our token cause a problem, we often had engine solutions to then bend the cursor and bring it back down through prompting fine-tuning USDs by optimize or whatever. And then what I'm seeing is that I'm seeing a lot of agentic workflows that actually integrate a lot of different steps. So for example, if you build a customer service chatbot, we'll often have to use prompting, maybe optimize some of the results in DSPI, build evals, build guard rails, maybe the customer service chatbot needs rag up part of the way to

get information to feedback to the user. So I actually do see these things grow. But one tip for many of you as well is I will often architect my software to make switching between different building block providers relatively easy. So for example, um have a lot of products that build on top of OM, but sometimes you point to a specific product and ask me which OM are we using? I honestly don't know because we built up evals and when there's a new model that's released, we'll quickly run evals to see if the new model is

better than the old one. And then you'll just switch to the new model if the new model does better on evals. And so the model we use week by week, you know, sometimes our engines will change it without even bothering to tell me because the eval show the new model works better. So it turns out that switching cost for foundation models is relatively low and we often architect our software. Oh, AI suite is open sourcing that my friends and I worked on to make switching easier. Um, switching cost for the orchestration platforms is a little

bit harder. Uh but I find that preserving that flexibility in your choice of building blocks often let you go faster even as you're building more and more things on top of each other. Um so hope that helps. Thank you so much. In the world of education in AI, there are two paradigms mostly. So one is AI can make teachers more productive. Uh automating grading and automating homeworks. But another school of thought is that there'll be personal tutors for every student. So every student can have one tutor that gets feedback from an AI and gets personal

questions from them. So how do you see these two paradigms converge and how would education look like in the next five years? I think everyone feels like a change is coming in edtech but I don't think the disruption is here yet. I think a lot of people are experimenting with different things. So you know Corsera has Corsera coach which actually works really well. Um deep learn is more focused on teaching AI also has some built-in chat bots. Um a lot of teams experiment of autograding. Oh there's an avatar with me on the deep learn website

you can talk to if you want. Uh deep learn.ai. And then I think for some things like language learning with you know speak Dolingo that has become clearer some of the ways AI would transform it for the broader educational landscape the exact ways that AI would transform it I see a lot of experimentation I think what key learning which I've been doing some work with is doing is is very promising for K12 education but I think uh what I'm seeing is frankly tons of experimentation but the final end state is still not clear. I do

think education will be hyperpersonalized. Uh but that workflow is an avatar, is a text chatbot, what's the workflow? I think um I feel like the hype from a couple years ago that with AGI soon and it will be all so easy. That was hype. The reality is work is complex, right? teachers, students, people do really complex workflows and for the next decade we'll be looking at the work that needs to be done and figuring out how to map it to agentic workflows and education is one of the sectors where this mapping is still underway but

it's not yet mature enough to the point where the end state is clear. So I think I think we should all yeah just keep working on it. All right. All right. Thank you so much Andrew. Thank you. Uh hey my question is I think AI uh it has a lot of great potential for good but there's also a lot of potential for bad consequences as well such as exacerbating economic inequality and things like that and I think a lot of our startups here while they'll be doing a lot of great things will also be you

know just by virtue of their product be contributing to some of those negative consequences. So I was curious how do you think you know us as AI builders should kind of balance our uh product building with also the potential societal downsides of some AI products and essentially how can we uh both move fast and be responsible as you mentioned in your talk look in your heart and if fundamentally what you're building if you don't think it'll make people at large better off don't do it right I I know it sounds simple but actually really hard

to do in the moment but AI fund we've killed multiple projects projects not on financial grounds but on ethical grounds where there are multiple projects we looked at the economic case is very solid but we said you know what we don't want this exist in the world and we just killed it on that basis so I hope more people will do that and then I worry about uh bring everyone with us one thing I'm seeing is um people in all sorts of job roles that are not engineering are much more productive if they know AI

than if they don't and so for example on my marketing team my marketers they know how the code. Frankly, they they were running circles around the ones that don't. So, then everyone learned to code and then they got better. But I feel like um trying to bring everyone with us to make sure everyone is empowered to build with AI. That'll be an important part of what all of us do, I think. Um I'm one of your big fans and thank you for your online courses. Your courses make the deep learning like uh much more accessible

to the world. And my question is also about education. uh as AI becomes more powerful and widespread, there seems to be a growing gap between what can actually do and what people perceive it. So what do you think about like is it important to educate the general public about deep learning stuff and not only like uh educate those technical people and make people understand more what really uh what AI really do and how it works. I think that knowledge will diffuse deep learn AI we want to empower everyone to build with AI. So we're working

on it. Many of us work on it. I'll just tell you what I think is the main d I think there are maybe two dangers. One is if you don't bring people with us fast enough, I hope we'll solve that. There's one other danger which is um it turns out that if you look at the mobile ecosystem, mobile phones, it's actually not that interesting. And one of the reasons is there are two gatekeepers, Android and iOS. And unless they let you do certain things, you're not allowed to try certain things on mobile. And I think

this, you know, hampers innovators. These dangers of AI have been used by certain businesses. They're trying to shut down open source because a number of businesses that love to be a gatekeeper to large scale foundation models. So I think hyping up dangers, supposed false dangers of AI in order to get regulators to pass laws like the proposed SP 1047 in California, which thank goodness we shut down, would have put in place really burdensome regry requirements that don't make anyone safer, but would make it really difficult for TS to release open source and open weight software.

So one of the dangers to inequality as well is if these regulatory you know awful regry approaches and I've been in the room where some of these businesses said stuff to regulators that was just not true. So I think that um some of these arguments the danger is if these regulatory proposals succeed and end up siphoning regulations leaving us with a small number of gatekeepers where everyone needs the permission of a small number of companies to fine-tune the model prompt in a certain way that's what will cipher innovation and prevent the diffusion of this information

to let lots of startups you know build whatever they want responsibly but the freedom to innovate so I think so long as we um prevent this line of attack on open source open weight models from succeeding and we we've made good progress but the threat is still there then I think eventually we'll get to the diffusion of knowledge and we can hopefully then bring everyone with us but this fight to protect open source we've been winning but the fight is still on and we still have to keep up that work to to protect open source

thank you all very much it's wonderful thank