Mozilla’s President Urges Consumers To Learn The Risks Of Different AI Systems

[Music] hey there we're here with Mark Surman uh president of Mozilla and we're gonna be talking about open source AI mark thank you so much for joining us and great to meet you I'm happy to be here thanks for having me on Rich sure so uh in 2020 long before chat GPT Mania you wrote a paper about trustworthy AI uh tell us what what does that mean to you well you know Milla as as you might know makes Firefox but we didn't make Firefox for its own sake we made it because we really wanted to

make sure the internet was something you know in the hands of people right that people are protected but people also can make choices and in the the Firefox era that meant priv withy it meant being able to make a web page met all those things and trustworthy AI was really you know I was trying to say what does it look like to have that Firefox effect in the AI era and there's two things that for us make up trustworthy AI one the idea of accountability which is something often when people talk about ethical AI responsible

AI means if something bad happens uh that you know the people who made that system are held accountable and that's something we're working through how do we build that into AI systems how do we build that into our laws but the other half in addition to accountability is agency can we control how the AI Works can we see how the AI works is it working on our side of the table at a broad philosophical level those are the two things that make up trustworthy AI for US agency and accountability and how do you really really

measure that on you know on a not broad level on on a more granular level what what what benchmarks should we use well that's a big thing between a a concept like trustworth Ai and like daytoday of building stuff is really tricky and so that's the process that you know Society is going through right now how do as we make AI in everything we do do we build the expectation that it's being used in a trustworthy way and so we started to look for just indicators of that so one might be privacy which is both

actually something that um helps with HC I can choose whether the system is private and you know helps with accountability if you break my privacy there are privacy laws in a lot of countries um and so like thinking about that in the design of an AI system and so if you really dig down when you think about something like ample intelligence it's being advertised as uh you know more private than say you know what you would get from open AI or Google that is like in the direction of trustworthy one of the things Milla is

doing is working with people who are trying to really popularize that level of privacy inside of an AI system so not only if you got a fancy new iPhone could you expect that what you're typing to that chat bot stays on your device or with you but that that would be true of you know more affordable systems or um you know all kinds of systems so with Milla Ventures one of the companies I'm most proud that we invested in is called flower Ai and they're basically taking that same technology that's in apple intelligence that's coming

into iPhone that keeps your AI data on the phone they're doing that as open source and they're making it something that any developer could build with so that's one example privacy in an AI system is one Element would look for that's for the AI but it's also you know transparency being able to see how things were built what data is inside it's also you know the ability to kind of audit and check how did the system work and so there's like lots of things we can kind of go down the line of elements of what

make a trustworthy AI system um and I you know I think we're looking at the next few Decades of trying to have this dance between moving fast with Innovation and also doing it in a way that is trustworthy yeah and you you mentioned apple and apple intelligence you know they're a company that really sort of markets their their their privacy prowess you like do you think they are holding uh holding their end of the bargain on uh on that when it comes to AI I think apple is better than most when it comes to privacy

and certainly Mozilla has long stood for privacy and we have great products yeah and we have great products that are privacy protecting in in Firefox and uh think things like our VPN the flip side is Apple while being good at privacy and I I do trust them is incredibly closed incredibly expensive and so another value that we look for in terms of you know getting trustworthy ey out there is doing trustworthy things but in an open source way so that it's not only you know the most expensive technology that is private or trustworthy or that

you can kind of check for bias and so on but that really is everywhere and so if you look at a contrast to Apple you might look at meta who's moving in the open source Direction but are people going to trust them and so for us it's actually you know openness and trustworthiness openness and privacy both need to be out there in the system and we stand for both right and and you you mentioned open source there's there's been a very fiery debate in the AI community over o open source and and then closed Source

what just just so we have it like what what is your definition of Open Source well the definition of Open Source that we've had for you know almost 25 years now is any software that allows you to use it in an unencumbered way so you can use it for anything you can use it freely that you can um study it or you can look inside it you can see the code you can see how it works that you can modify it so you can take it and build it into something else combine it with other

things and that you can share it so like after I've modified it I can give it out again to somebody else and those are what are called The Four Freedoms uh of kind of free software or open source and that's you know Linux is built on that Wikipedia is built on that Firefox is built on that and8 trillion dollars in value has been created from open source because it's what the tech industry is built on when we come to AI it's still those same four things but it's a little bit different than software because it's

not just that the code needs to meet those four criteria but you also need the models and the data sets to be things that you can use and share and modify and so on so we're in a process really of kind of debating that figuring out as an industry what is open source we an AI I think the answer is that it's those same four things use study modify share applied to the software parts of AI to the models of AI and actually the data sets or at least the kind of the data set recipe

that goes into an AI model and the open source initiative which originally kind of provideed the definition of of Open Source is about to release its final open source AI definition and it basically is what I just said right and the the open source initiative actually this week uh criticize meta you know for quote unquote polluting the term um what what is what is your take on that well we're with the open source initiative on that I mean LMA was absolutely a boon I mean you had open Ai and others really locking up a lot

of what had happened in Innovation uh in the AI space on top of a lot of open research from the last couple of decades and you know llama did come along and say Here's a performant llm that is open but it is actually not open source in a full sense so we use llama we have something called Lama file that that we've released as soft for you can run llama on a local machine we're fans but we're also critics of the license because the license has a bunch of things that aren't open source at 700

million users which sounds like a lot of people the license stops being in effect so on that basic idea that you can use it for anything which is a rule of Open Source that's not true at 700 million users it's kind of just stops being open source and there's a couple of Clauses in the license like that if you just imagine Mark Zuckerberg you know had built Facebook on top of Linux and then when he got to 700 million users linis Toral who invented Linux would like walk up to his door in Paulo Alto and

knock on a and hold up in a bag for money like you know that that wouldn't have been open source and actually nobody would have built their company if that were the rules nobody would have built their company on Linux and so I do think it's it's not so much that they're polluting it but we really do think they need a different license a truly open source license if they're going to call Llama open source and we'd welcome that so so what do you think the the fix here for them is you know you you

you started to talk about this but is it you know do they change the the framing do they stop using the term open source or or do they or do they have a different license I think they just need to change their license I mean Mark Zuckerberg got wealthy and built a lot of great stuff on top of real open source software if they're going to contribute back uh that's great but they need to do it on the same terms as we continue to have this this debate in the in the industry what do you

think the biggest misconceptions are of of Open Source well one of the misconceptions although it's starting to turn around is that open source AI is more dangerous than closed Source AI that there might be more risks with open source AI than closed Source Ai and of course anytime we have a new technology especially a new powerful technology like AI there are risks I mean deep bakes or biased technology or or kind of misuse of the technology down the line and you know the the idea that open source is more dangerous has been used to sort

of convince Regulators at times I Al almost happened in California to write laws that would really shut down open source or would be biased against open source and what's happened is a lot of research has come out including research that fed into um some some decisions and guidance um by the Department of Commerce federally that said actually there's no difference there's no real marginal risk no extra risk between open source Ai and closed Source AI at this stage in time and so it's really important to look at the evidence look at the research and see

that we do have to look at what of the rest around AI Society but not to kind of scapegoat or try to shut down open sources being particularly the problem that's a misconception and where where do you think that misconception comes from well if I'm cynical uh I think the misconception comes from people whose companies are closed Source uh going and lobbying and saying trust us with safety and you know the idea that one or two or three companies are going to have Monopoly on safe AI to me that feels like really risky thing for

society to do the idea that you would actually you know use what's good of that open source that it's transparent and that it's collectively owned and everybody contributing to the safety of the core infrastructure of our society to me historically that's actually what works out and you you you mentioned regulation and um I I think you you've said in the past that that you're for you know regulation that puts control in the hands of users what like what what do you think smart regulation would look like in in AI so when we think about AI

regulation we think about three things competition privacy and safety and so think about competition like that's something that really matters to forbs readers because it's about business and we want to make sure that um markets are open in the AI era in a way that they kind of um haven't been in the social media era in many regards and it's a chance to get it right we're really happy to see in the executive order that came from the White House earlier this year they were calling out competition in AI from early on so that's one

area that is really important another is privacy like you don't actually have Federal privacy rules in the US uh but you know this is technology that we're very intimate with where we're sharing all of our information and so having privacy rules that talk about our relationship between ourselves and tech companies ourselves and AI that's actually really important and the third is around safety you know a lot of people like the eui act or the caloria with s1047 have been trying to build regulations on AI for safety and you know that makes sense to us but

it should be based on evidence and should be based on a kind of harms based approach should be based on a harms based approach so the EU AI act looks at specific high-risk areas that AI might use use like looks at specific high-risk areas where AI might be used like in healthare or immigration or policing and so you know in those kind of settings you know you have to take extra precautions as a company whereas the California law really was a blanket sort of if something bad happens that is unspecified the company will be responsible

and that's not tenable like we need to be regulating for particular risks and harms and not just on kind of sound regular basis about AI safety what companies do you think are doing the best job in upholding open source standards so you know when we talk about trustworthy Ai and we think about you know what missil is trying to to do in the world there are people who are doing well in open source and there are people who are doing well in some as other aspects of trust worthy Ai and we kind of look at

the whole ecosystem both as people who are starting to build AI into our own products people who are investors and uh in the AI space and so I say you know some of the the interesting folks on the open source side of it are you know clearly hugging face which is a place where you can go and get all the open- source AI staff um also companies like mistr who you know were the some of the first folks to put out commercial open source AI models we actually use mstr and some some Milla products uh

as a kind of llm inside um and then I think many of the people who are most interesting of from a side are not companies at all and so the Allen Institute uh for AI in Seattle or there's now one in friends called kuati or CTI I never get the the pronunciation right but those are nonprofit open source AI Labs that are building you know models that are from end to end open um and much like Milla Foundation or Linux Foundation or the wiky media Foundation those are players who are trying to build something open

source that everybody can kind of owned in some ways by it being you know held in a public interest organization and I think those are the people are doing really the most exciting fully endtoend fully transparent AI work and then there's lots on the responsibility and uh you know kind of trustworthy part you know some that you see with things like apple intelligence you see a lot of really cool AI governance companies coming up like Creo Ai and Fiddler AI um so there's there's just a lot I mean if you go and read the air

Street Capital whats at this deck every year of like here's all the it's like 250 page deck and they just put it out recently all that's going on in AI there's just so many interesting companies coming out and on the flip side what companies are are doing the worst job upholding those standards well the the trick of the last few years or that that you know sort of dastardly planing if if you wanted to be cynical of the last decade was you know open AI took the word open and it's hard to kind of say

open source Ai and not sometimes skip over the source and say open AI but they're incredibly closed and they not only themselves are a closed Source company that you know everything is only available behind an API in a black box which there's nothing inherently wrong with that but they did start out with the prise of being this nonprofit research lab that was going to do open source and you know back in 20 18 2019 Sam mman said no that's not the way it's going to work and they took what was a spirit of open research

like let's say the Google Transformer paper which really inspired you know the whole GPT series of uh of models that uh open AI has built on they took all of that Spirit of open science and kind of locked it all up and so you know certainly if what we need is which I believe we do an open public ecosystem of AI they're going the opposite direction and and just to set the stakes like what what in in your eyes is is the danger of a closed AI system well it's not that there's a danger of

a closed AI system we've got closed software closed technology in all aspects of the world and I use some of it too it's not that open is the only thing we should use the danger is having an exclusively closed also exclusively commercial ecosystem for technology so we talk about public AI as well as open source AI we just put out a paper on this and public AI is this idea that there should be a counterbalance to the closed and Commercial AI in our society much like you know you have cars and you have buses or

you have NBC and you have PBS in most areas of society we have both commercial and public options closed and open options and in the you know the previous era in the internet you had Linux you had Firefox you had Wikipedia as public options nonprofit run really scaled reliable technology or or Internet services and for us the danger is going into this AI era where we only have closed in commercial options and no public options and that's why we are investing in trust for the AI ourselves as a public interest player but it's also why

we think that folks like the Institute for AI are really exciting because they're building that public Lane in this AI era and you know you you mentioned uh open open AI um the launch of chat GPT is kind of what you know really spurred this you know this this current AI frenzy um how has Mozilla changed since then well we've been trying to changed basila since well before chat GPT and it takes a long time to go from being you know a web organization and public interest and and Commercial organization we're we're a hybrid focused

on the web and you know we really were focused on the web as something we want to make sure worked for all of humanity to also focus on how do we do that same thing in the AI era but I would say you know we started that in about 2018 2019 put out our trustworthy AI paper started building open source data sets and working with people on things like auditing but what's sped things up is as there's been more attention and more resources we've started to build trustworthy AI thinking and product and investment into everything

we're doing and so we've gone much further towards being a company standing for an organization standing for uh Ai and commity service so you're starting to see trustworthy AI open source AI showing up in Firefox we set up a separate AI R&D company we've got Billa ventur which has got about 30 or 35 AI companies that's invested in so I guess the big thing is you know we've known we want to bend AI in a better direction for a while now I would say as there's more friendy it's just sped us up um because it

feels even more urgent well Mark thanks so much for joining us thank you so much really appreciate it