Michael Kearns: Algorithmic Fairness, Privacy & Ethics | Lex Fridman Podcast #50

The following is a conversation with Michael Kern's he's a professor at the University of Pennsylvania and a co-author of the new book ethical algorithm that is the focus of much of this conversation it includes algorithmic fairness bias privacy and ethics in general but that is just one of many fields that Michael's a world-class researcher in some of which would touch on quickly including Learning theory or the theoretical foundation of machine learning game theory quantitative finance computational social science and much more but on a personal note when I was an undergrad early on I worked with

Michael on an algorithmic trading project in competition that he led that's when I first fell in love with algorithmic game theory while most of my research life has been a machine learning human robot interaction the Systematic way that game theory reveals the beautiful structure and our competitive and cooperating world of humans has been a continued and inspiration to me so for that and other things I'm deeply thankful to Michael and really enjoyed having this conversation again in person after so many years this is the artificial intelligence podcast if you enjoy it subscribe on YouTube give

It five stars on Apple podcasts supported on patreon or simply connect with me on Twitter at lex Friedman's both Fri D ma n this episode is supported by an amazing podcast called pessimists archive Jason the host the show reached out to me looking to support this podcast and so I listened to it to check it out and I listened I mean I went through it Netflix binge style at least 5 episodes in a row it's not one of my favorite podcast and I Think it should be one of the top podcasts in the world frankly

it's a history show about why people resist new things each episode looks at a moment in history when something new was introduced something that today we think of as commonplace like recorded music umbrellas bicycles cars chests coffee the elevator and the show explores why freaked everyone out the latest episode on mirrors and vanity still stays with they think about vanity in the modern Day of the Twitter world that's the fascinating thing about the show is that stuff that happened long ago especially in terms of our fear of new things repeats itself in the modern day

and so has many lessons for us to think about in terms of human psychology and the role of Technology in our society anyway you should subscribe but listen the pessimist archive I highly recommended and now here's my conversation with Michael Kern's you mentioned reading Fear and Loathing in Las Vegas in high school and having more or a bit more of a literary mind so what books non-technical non computer science would you say had the biggest impact on your life either intellectually or emotionally you've dug deep into my history I see deep yeah I think well

my favorite novel is Infinite Jest by David Foster Wallace which actually coincidentally much of it takes place in the halls of buildings right around us Here at MIT so that certainly had a big influence on me and as you noticed like when I was in high school I actually Stephen started college as an English major so was very influenced by sort of badge genre of journalism at the time and thought I wanted to be a writer and then realized that an English major teaches you to read but it doesn't teach you how to write and

then I became interested in math and computer science instead well in your new book ethical Algorithm you kind of sneak up from a algorithmic perspective on these deep profound philosophical questions of fairness of privacy in thinking about these topics how often do you return to that literary mind that either you had yeah I'd like to claim there was a deeper connection but but there you know I think both Aaron and I kind of came at these topics first and foremost from a technical angle I mean you know I'm kind of consider myself primarily and Originally

a machine learning researcher and I think as we just watched like the rest of the society the field technically advanced and then quickly on the heels of that kind of the the buzzkill of all the antisocial behavior by algorithms just kind of realized there was an opportunity for us to do something about it from a research perspective you know a more to the point in your question I mean I do have an uncle who is literally A moral philosopher and so in the early days of our technical work on fairness topics I would occasionally you

know run ideas behind him so I mean I remembered an early email I sent to him in which I said like oh you know here's a specific definition of algorithmic fairness that we think is some sort of variants of Rawls II in fairness what do you think and I thought I was asking a yes-or-no question and I got back there kind of classical philosophers responsive well It depends if you look at it this way then you might conclude this and that's when I realized that there was a real kind of rift between the ways philosophers

and others had thought about things like fairness you know from sort of a humanitarian perspective and the way that you needed to think about it as a computer scientist if you were going to kind of implement actual algorithmic solutions but I would say the algorithmic solutions take care of Some of the low-hanging fruit sort of the problem is a lot of algorithms when they don't consider fairness they are just terribly unfair and when they don't consider privacy they're terribly they violate privacy sort of algorithmic approach fixes big problems but there's though you get when you

start pushing into the gray area that's when you start getting into this philosophy of what it means to be fair that's starting from Plato what what is justice kind of Questions yeah I think that's right and I mean I would even not go as far as you want to say that that sort of the algorithmic work in these areas is solving like the biggest problems and you know we discussed in the book the fact that really we are there's a sense in which we're kind of looking where the light is in that you know for

example if police are racist in who they decide to stop and frisk and that goes into the data there's sort of no undoing that Downs by kind of clever algorithmic methods and I think especially in fairness I mean I think less so in privacy where we feel like the community kind of really has settled on the right definition which is differential privacy if you just look at the algorithmic fairness literature already you can see it's gonna be much more of a mess and you know you've got these theorems saying here are three entirely reasonable Desirable

notions of fairness and you know here's a proof that you cannot simultaneously have all three of them so I think we know that algorithmic fairness compared to algorithmic privacy is gonna be kind of a harder problem and it will have to revisit I think things that have been thought about by you know many generations of scholars before us so it's very early days for fairness I think so before we get into the details of differential privacy and then the Fairness side I mean linger on the philosophy but do you think most people are fundamentally good

or do most of us have both the capacity for good and evil within us I mean I'm an optimist I tend to think that most people are good and want to do to do right and that deviations from that or you know kind of usually due to circumstance to people being bad at heart with people with power are people at the heads of governments people at The heads of companies people at the heads of maybe so financial power markets do you think the distribution there is also most people are good and have good intent yeah

I do I mean my statement wasn't qualified to people not in positions of power I mean I think what happens in a lot of the you know the the cliche about absolute power corrupts absolutely I mean you know I think even short of that you know having spent a lot of time on Wall Street and Also in arenas very very different from Wall Street like academia you know one of the things I think I've benefited from by moving between two very different worlds is you you become aware that you know these were it's kind of

developed their own social norms and they develop their own rationales for you know behavior for instance that might look unusual to outsiders but when you're in that world it doesn't feel unusual at all and I Think this is true of a lot of you know professional cultures for instance and and you know so then you're maybe slippery slope is too strong of a word but you know you're in some world where you're mainly around other people with the same kind of viewpoints and training and worldview as you and I think that's more of a source

of you know kind of abuses of power then sort of you know there being good people and evil people and and it's somehow the evil people are The ones that somehow rise to power that's really interesting so it's the within the social norms constructed by that particular group of people you're all trying to do good but because it's a group you might be you might drift into something that for the broader population it does not align with the values of society that kind of that's the word yeah I mean or nothing you drift but even

the things that don't make sense to the outside world don't Seem unusual to you so it's not sort of like a good or a bad thing but you know like so for instance you know on on in the world of finance right there's a lot of complicated types of activity that if you are not immersed in that world you cannot see why the purpose of that you know that activity exists at all it just seems like you know completely useless and people just like you know pushing money around and when you're in that world right

you're you and you learn More you your view does become more nuanced right you realize okay there is actually a function to this activity and force in some cases you would conclude that actually if magically we could eradicate this activity tomorrow it would come back because it actually is like serving some useful purpose it's just a useful purpose that's very difficult for outsiders to see and so I think you know lots of professional work environments or cultures as I might put It kind of have these social norms that you know domain sense to the outside

world academia is the same right I mean lots of people look at academia and say you know what the hell are all of you people doing why are you paid so much in some cases at taxpayer expenses to do you know to publish papers and military reads you know but when you're in that world you come to see the value for it and but even though you might not be able to Explain it to you know the person in the street alright and in the case of the financial sector tools like credit might not make

sense to people like is it's a good example of something that does seem to pop up and be useful or or just the power of markets and just in general capitalism yeah and Finance I think the primary example I would give is leverage right so being allowed to borrow to sort of use ten times as much money as you've actually borrowed right so so that's an Example of something that before I had any experience in financial markets I might have looked at and said well what is the purpose of that that just seems very dangerous

and it is dangerous and it has proven dangerous but you know if the fact of the matter is that you know sort of on some particular time scale you are holding positions that are you know very unlikely to you know loo you know they're you know that your value at risk their variance is like 1 or 5 Percent then it kind of makes sense that you would be allowed to use a little bit more than you have because you have you know some confidence that you're not going to lose it all in a single day

now of course when that happens we've seen what happens you know not not too long ago but but you know but the idea that it serves no useful economic purpose under any circumstances is definitely not true we'll return to the other side of the coast Silicon Valley And the problems there as we talk about privacy as we talk about fairness at the high level and I'll ask some sort of basic questions with the hope to get at the fundamental nature of reality but from a very high level what is an ethical algorithm so I can

say that an algorithm has a running time of using Big Oil notation and login I can say that a machine learning algorithm classified cat versus dog with 97% accuracy do you think there Will one day be a way to measure sort of in the same compelling way as the big ol notation of this algorithm is 97% ethical first of all many rif for a second on your specific and login examples so because early in the book when we're just kind of trying to describe algorithms period we say like ok you know what's an example of

an algorithm or an algorithmic problem first of all I could sorting right yeah I'm a bunch of index cards with numbers On them and you want to sort them and we describe you know an algorithm that sweeps all the way through finds the the smallest number puts it at the front then sweeps through again finds the second smallest number so we make the point that this is an algorithm and it's also a bad algorithm in the sense that you know it's quadratic rather than n log n which we know is optimal for sorting and we

make the point that sort of like you know so even within the Confines of a very precisely specified problem there's you know there might be many many different algorithms for the same problem with different properties like some might be faster in terms of running time some I use less memory some might have you know better distributed implementations and and so the point is is that already we're used to you know in computer science thinking about trade-offs between different types of quantities and resources and there being You know better and worse algorithms and and our book

is about that part of algorithmic ethics that we know how to kind of put on that same kind of quantitative footing right now so you know just to say something that our book is not about our book is not about kind of broad fuzzy notions of fairness it's about very specific notions of fairness there's more than one of them there are tensions between them right but if you pick one of them you can do something Akin to saying this algorithm is 97% ethical you can say for instance the you know for this lending model the

false rejection rate on black people and white people is within 3 percent right so we might call that to a 97% ethical algorithm in a 100% ethical algorithm would mean that that difference is 0% in that case fairness is specified when two groups however they're defined are given to you that's right so the and and then you can Sort of mathematically start describing the algorithm but nevertheless the the part where the two groups are given to you I mean unlike running time you know we don't in a computer science talk about how fast an algorithm

feels like when it runs true we measure an ethical starts getting into feelings so for example an algorithm runs you know if it runs in the background it doesn't disturb the performance of my system it'll feel nice I'll be okay with it but If it overloads the system will feel unpleasant so in that same way ethics there's a feeling of how socially acceptable it is how does it represent the moral standards of our society today so in that sense and sorry to linger on that for some high low philosophical question is do you have a

sense we'll be able to measure how ethical and algorithm is first of all I didn't certainly didn't mean to give the impression that you can kind of measure You know memory speed trade-offs you know and and that there's a complete you know mapping from that on to kind of fairness for instance or ethics and and accuracy for example in the type of fairness definitions that are largely the objects of study today and starting to be deployed you as the user of the definitions you need to make some hard decisions before you even get to the

point of designing fair algorithms one of them for instance is deciding who it Is that you're worried about protecting who you're worried about being harmed by for instance some notion of discrimination or unfairness and then you need to also decide what constitutes harm so for instance in a lending application maybe you decide that you know falsely rejecting a credit worthy individual you know sort of a false negative is the real harm and that false positives ie people that are not credit worthy or are Not going to repay your loan to get a loan you might

think of them as lucky and so that's not a harm although it's not clear that if you are don't have the means to repay a loan that being given a loan is not also a harm so you know you know the literature is sort of so far quite limited in that you sort of need to say who do you want to protect and what would constitute harm to that group and when you ask questions like will algorithms feel ethical one way in which They won't under the definitions that I'm describing is if you know if

you are an individual who is falsely denied alone incorrectly denied a loan all of these definitions basically say like well you know your compensation is the knowledge that we are we are also falsely denying loans to other people you know other groups at the same rate that we're doing it's to you and and you know there and so there is actually this interesting even technical tension in The field right now between these sort of group notions of fairness and notions of fairness that might actually feel like real fairness to individuals right they they might really

feel like their particular interests are being protected or thought about by the algorithm rather than just you know the groups that they happen to be members of is there parallels to the big o-notation of worst-case analysis so is it important to looking at the worst violation of Fairness for an individual is important to minimize that one individual so like worst case analysis is that something you think about or I mean I think we're not even at the point where we can sensibly think about that so first of all you know we're talking here both about

fairness applied at the group level which is a relatively weak thing but it's better than nothing and also the more ambitious thing of trying to give some individual Promises but even that doesn't incorporate I think something that you're hinting at here is what a chime I'll call subjective fairness right right so a lot of the definitions I mean all of the definitions in the algorithmic fairness literature are what I would kind of call received wisdom definitions it's sort of you know somebody like me sits around and things like okay you know I think here's a

technical definition of fairness that I Think people should want or that they should you know think of as some notion of fairness maybe not the only one maybe not the best one maybe not the last one but we really actually don't know from a subjective standpoint like what people really think is fair there's you know we've we've just started doing a little bit of work in in our group that actually doing kind of human subject experiments in which we you know ask people about you know we ask them Questions about fairness we survey them we

you know we show them pairs of individuals in let's say a criminal recidivism prediction setting and we ask them do you think these two individuals should be treated the same as a matter of fairness and to my knowledge there's not a large literature in which ordinary people are asked about you know they they have sort of notions of their subjective fairness elicited from them it's mainly you know kind of scholars Who think about fairness no right and I'm making up their own definitions and I think I think this needs to change actually for many social

norms not just for fairness right so there's a lot of discussion these days in the AI community about interpretable AI or understandable AI and as far as I can tell everybody agrees that deep learning or at least the outputs of deep learning are not very understandable and people might agree that sparse linear models With integer coefficients are more understandable but nobody's really asked people you know there's very little literature on you know sort of showing people models and asking them do they understand what the model is doing and I think that in all these topics

as these fields mature we need to start doing more behavioral work yeah which is so one of my deep passions of psychology and I always thought computer scientists will be the the best future Psychologists in a sense that data is especially in this modern world the data is a really powerful way to understand and study human behavior and you've explored that with your game theory side of work as well yeah I'd like to think that what you say is true about computer scientists and psychology from my own limited wandering into human subject experiments we have

a great deal to learn not just computer science but AI and machine learning more specifically I Kind of think of as imperialist research communities in that you know kind of like physicists in an earlier generation computer scientists kind of don't think of any scientific topic as off limits to them they will like freely wander into areas that others have been thinking about for decades or longer and you know we usually tend to embarrass ourselves yes in those efforts for for some amount of time like you know I think reinforcement learning is a good example Right

so a lot of the early work in reinforcement learning I have complete sympathy for the control theorist that looked at this and said like okay you are reinventing stuff that we've known since like the 40s right but you know in my view eventually this sort of you know computer scientists have made significant contributions to that field even though we kind of embarrassed ourselves for the first decade so I think if computer scientists are gonna Start engaging in kind of psychology human subjects type of research we should expect to be embarrassing ourselves for a good ten

years or so and then hope that it turns out as well as you know some other areas that we've waded into so you kind of mentioned this just the linger on the idea of an ethical algorithm of idea of group sort of group thinking an individual thinking and we're struggling that there's one of the amazing things about Algorithms and your book and just this field of study is it gets us to ask like forcing machines converting these ideas into algorithms is forcing us to ask questions of ourselves as a human civilization so there's a lot

of people now in public discourse doing sort of group thinking thinking like there's particular sets of groups that we don't want to discriminate against and so on and then there is individuals sort of in the individual life stories the Struggles they went through and so on now like in philosophy it's easier to do group thinking because you don't you know it's very hard to think about individuals there's so much variability but with data you can start to actually say you know what group thinking is too crude you're actually doing more discrimination by thinking in terms

of groups and individuals can you linger on that kind of idea of group versus individual and ethics and and is it good To continue thinking in terms of groups in in algorithms so let me start by answering a very good high level question with a slightly narrow technical response which is these group definitions of fairness like here's a few groups like different racial groups may be gender groups may be age what-have-you and let's make sure that you know from none of these groups do we you know have a false negative rate which is much higher

than any other one Of these groups okay so these are kind of classic group aggregate notions of fairness and you know but at the end of the day an individual you can think of as a combination of all of their attributes right they're a member of a racial group they're they have a gender they have an age you know and many other you know demographic properties that are not biological but that you know are are still you know very strong determinants of outcome and personality in the light So one I think useful spectrum is to

sort of think about that array between the group and this individual and to realize that in some ways asking for fairness at the individual level is to sort of ask for group fairness simultaneously for all possible combinations of groups so in particular so in particular yes you know if I build a predictive model that meets some definition of fairness by race by gender by age by What-have-you marginally to get a slightly technical sort of independently I shouldn't expect that model to not to discriminate against disabled Hispanic women over age 55 making less than fifty thousand

dollars a year or annually even though I might have protected each one of those attributes marginally so the optimization actually that's a fascinating way to put it so you're just optimizing the one way to achieve the optimizing fairness for Individuals just to add more and more definitions of groups at each and it's right along so you know at the end of the day we could think of all of ourselves as groups of size one because eventually there's some attribute that separates you from me and everybody from everybody else in the world okay and so it

is possible to put you know these incredibly coarse ways of thinking about their nests and these very very individualistic specific ways on a Common scale and you know one of the things we've worked on from a research perspective is you know so we sort of know how to you know we in relative terms we know how to provide fairness guarantees at the coarsest end of the scale we don't know how to provide kind of sensible tractable realistic fairness guarantees at the individual level but maybe we could start creeping towards that by dealing with more you

know refined subgroups I mean we we gave a Name to this phenomenon where you know you protect you you you enforce some definite definition of fairness for a bunch of marginal attributes or features but then you find yourself discriminating against a combination of them we call that fairness gerrymandering because like political gerrymandering you know you're giving some guarantee at the aggregate level yes but that when you kind of look in a more Granular way at what's going on you realize that you're achieving that aggregate guarantee by sort of favoring some groups in discriminating against other

ones and and so there are you know it's early days but there are algorithmic approaches that let you start creep and creeping towards that you know individual end of the spectrum does there need to be human input in the form of weighing the value of the importance of each kind of group so for Example is it is it like so gender say crudely speaking male and female and then different races are we as humans supposed to put value on saying gender is 0.6 and racist 0.4 in terms of in the big optimization of achieving fairness

is that kind of what humans I mean most of you know I mean of course you know I don't need to tell you that of course technically one could incorporate such weights if you wanted to into a definition of fairness you know fairness Is an interesting topic in that having worked in in the book being about both fairness privacy and many other social norms fairness of course is a much much more loaded topic so privacy I mean people want privacy people don't like violations of privacy violations of privacy cause damage angst and and bad publicity

for the companies that are victims of them but sort of everybody agrees more data privacy would be better than less data privacy and and you don't Have these somehow the discussions of fairness don't become politicized along other dimensions like race and about gender and you know you know whether we you and you know did you quickly find yourselves kind of revisiting topics that have been kind of unresolved forever like affirmative action right sort of you know like why are you protecting and some people will say why are you protecting this particular racial group and and

others Will say what we need to do that as a matter of retribution other people will say it's a matter of economic opportunity and I don't know which of you know whether any of these are the right answers but you sort of fairness is sort of special in that as soon as you start talking about it you inevitably have to participate in debates about fair to whom at what expense to who else I mean even in criminal justice right um you know where People talk about fairness in criminal sentencing or you know predicting failures to

appear or making parole decisions or the like they will you know they'll point out that well these definitions of fairness are all about fairness for the criminals and what about fairness for the victims right so when I basically say something like well the the false incarceration rate for black people and white people needs to be roughly the same you know there's no Mention of potential victims of criminals in such a fairness definition and that's the realm of public discourse I just listened to two people listening intelligent squares debates us edition just had a debate they

have this structure we have a old Oxford style or whatever they're called debates those two versus two and they talked about affirmative action and it was the is incredibly interesting that it's still there's really good points on every side Of this issue which is fascinating to listen yeah yeah I agree and so it's it's interesting to be a researcher trying to do for the most part technical algorithmic work but Aaron and I both quickly learned you cannot do that and then go out and talk about and expect people to take it seriously if you're unwilling

to engage in these broader debates that are entirely extra algorithmic right there they're not about you know algorithms And making algorithms better they're sort of you know as you said sort of like what should society be protecting in the first place when you discuss the fairness an algorithm that uh that achieves fairness whether in the constraints and the objective function there's an immediate kind of analysis you can perform which is saying if you care about fairness in gender this is the amount that you have to pay for in terms of the performance of the system

Like do you is there a role for the statements like that in a table and a paper or do you want to really not touch that like you know we want to touch that and we do touch it so I mean just just again to make sure I'm not promising your your viewers more than we know how to provide but if you pick a definition of fairness like I'm worried about gender discrimination and you pick a notion of harm like false rejection for a loan for example and you give me a Model I can definitely

first of all go on at that model it's easy for me to go you know from data to kind of say like okay your false rejection rate on women is this much higher than it is on men okay but you know once you also put the fairness in to your objective function I mean I think the table that you're talking about is you know what we would call the Pareto curve right you can literally trace out and we give examples of such plots on real datasets in the Book you have two axes on the x-axis

is your error on the y-axis is unfairness by whatever you know if it's like the disparity between false rejection rates between two groups and you know your algorithm now has a knob that basically says how strongly do I want to enforce fairness and the less unfairly you know we you know if the two axes are err and unfairness we'd like to be at 0-0 we'd like to zero error and zero fair unfairness simultaneously anybody who Works in machine learning knows that you're generally not going to get to zero error period without any fairness constrain whatsoever

so that's that that's not gonna happen but in general you know you'll get this you'll get some kind of convex curve that specifies the numerical trade-off you face you know if I want to go from 17 percent error down to 16 percent error what will be the increase in unfairness that I've Experienced as a result of that and and so this curve kind of specifies the you know kind of undaunted models models that are off that curve are you know can be strictly improved in one or both dimensions you can you know either make the

error better or the unfairness better or both and I think our view is that not only are are these objects these Pareto curves you know there's efficient frontiers as you might call them not only are they Valuable scientific objects I actually think that they in the near term might need to be the interface between researchers working in the field and and stakeholders and given problems so you know you could really imagine telling a criminal jurisdiction look if you're concerned about racial fairness but you're also concerned about accuracy you want to you know you want to

release on parole people that are not going to recommit a violent crime and you don't Want to release the ones who are so you know that's accuracy but if you also care about those you know the mistakes you make not being disproportionately on one racial group or another you can you can show this curve I'm hoping that in the near future it'll be possible to explain these curves to non-technical people that have that are the ones that have to make the decision where do we want to be on this curve like what are the relative

merits or value of having Lower error versus lower unfairness you know that's not something computer scientists should be deciding for society right that you know the people in the field so to speak the policymakers the regulator's that's who should be making these decisions but I think and hope that they can be made to understand that these trade-offs generally exist and that you need to pick a point and like and ignoring the trade-off you know you're implicitly Picking a point anyway right right you just don't know it and you're not admitting it it's just a link

out on the point of trade-offs I think that's a really important thing to sort of think about so you think when we start to optimize for fairness there's almost always in most system going to be trade-offs can you like what's the trade-off between just to clarify they've been some sort of technical terms thrown around but a sort of a Perfectly fair world why is that why will somebody be upset about that the specific trade-off I talked about just in order to make things very concrete was between numerical error and some numerical measure of unfairness in

what is numerical error in the case of just likes a predictive error like you know the probability or frequency with which you release somebody on parole who then goes on to recommit a violent crime or keep incarcerated somebody who would Not have recommitted a violent crime so in case of awarding somebody parole or giving somebody Perl or letting them out on parole you don't want them to recommit a crime so it's your system failed in prediction if they happen to do a crime okay so that's the performer that's one axis right and what's the fairness

axis so then the fairness axis might be the difference between racial groups in the kind of false false positive predictions namely people that I kept incarcerated predicting that they would recommit a violent-crime when in fact they wouldn't have right and the the unfairness of that just to linger it and allow me to in eloquently to try to sort of describe why that's unfair why unfairness is there the the unfairness you want to get rid of is the in the judges mind the bias of having being brought up to society the slight racial bias the racism

that exists in The society you want to remove that from the system another way that's been debated is equality of opportunity versus equality of outcome and there's a weird dance there that's really difficult to get right and we don't as what the firm ative action is exploring that space right and then we this also quickly you know bleeds into questions like well maybe if one group really does recommit crimes at a higher rate the reason for that is That at some earlier point in the pipeline or earlier in their lives they didn't receive the same

resources that the other group did right and that and so you know there's always in in kind of fairness discussions the possibility that the the real injustice came earlier right earlier in this individuals life earlier in this group's history etc etc and and so a lot of the fairness discussion is almost the goal is for it to be a corrective mechanism to account For the injustice earlier in life by some definitions of fairness or some theories of fairness yeah others would say like look it's it's you know it's not to correct that injustice it's just

to kind of level the playing field right now and Nanyan coarser a falsely incarcerate more people of one group than another group but I mean do you think just it might be helpful just to demystify a little bit about the diff bias or unfairness can come into Algorithms especially in the machine learning era right and you know I think many of your viewers have probably heard these examples before but you know let's say I'm building a face recognition system right and so I'm you know kind of gathering lots of images of faces and you know

trying to train the system to you know recognize new faces of those individuals from training on you know a training set of those faces of individuals and you know It shouldn't surprise anybody or certainly not anybody in the field of machine learning if my training dataset was primarily white males and I'm training that mmm the model to maximize the overall accuracy on my training data set that you know the model can reduce its air or most by getting things right on the white males that constitute the majority of the data set even if that means

that on other groups they will be less accurate okay now there's a bunch Of ways you could think about addressing this one is to deliberately put into the objective of the algorithm not to not to optimize the air or at the expense of this discrimination and then you're kind of back in the land of these kind of two-dimensional numerical trade-offs a valid counter-argument is to say like well no you don't have to there's no you know the the notion of the tension between air and Acuras here is a false one you could instead just go

out and Get much more data on these other groups that are in the minority and you know equalize your dataset or you could train a separate model on those subgroups and you know have multiple models the point I think we would you know we try to make in the book is that those things have cost too right going out and gathering more data on groups that are relatively rare compared to your plurality or more majority group that you know it may not cost you in the accuracy of the model But it's gonna cost you know

it's gonna cost the company developing this model more money to develop that and it has also cost more money to build separate predictive models and to implement and deploy them so even if you can find a way to avoid the tension between error and accuracy training a model you might push the cost somewhere else like money like development time research time and alike there are fundamentally difficult Philosophical questions in fairness and we live in a very divisive political climate outrage culture there is uh all right folks on 4chan trolls there is social justice warriors on

Twitter there is very divisive outraged folks and all sides of every kind of system how do you how do we as engineers build ethical algorithms in such divisive culture do you think they could be disjoint the human has to inject your values and then you can optimize over Those values but in our times when when you start actually applying these systems things get a little bit challenging for the public discourse how do you think we can proceed yeah I mean for the most part in the book you know a point that we try to take

some pains to make is that we don't view ourselves or people like us as being in the position of deciding for society what the right social norms are what the right definitions of fairness are our main Point is to just show that if society or the relevant stakeholders in a particular domain can come to agreement on those sorts of things there's a way of encoding that into algorithms in many cases not in all cases one other misconception though hopefully we definitely dispel is sometimes people read the title of the book and I think not unnaturally

fear that what we're suggesting is that the algorithms themselves should decide what those Social norms are and develop their own notions of fairness and privacy or ethics and we're definitely not suggesting that the title of the book is ethical algorithm by the way and they didn't think of that interpretation of the title that's interesting yeah yeah I mean especially these days were people are you know concerned about the robots becoming our overlords the idea that the robots would also like sort of develop their own social norms is you know Just one step away from that

but I do think you know obviously despite disclaimer that people like us shouldn't be making those decisions for society we are kind of living in a world where in many ways computer scientists have made some decisions that have fundamentally changed the nature of our society and democracy and in sort of civil discourse and deliberation in ways that I think most people generally feel are bad these days right so but they had to make so if We look at people at the heads of companies and so on they had to make those decisions right there has

to be decisions so there's there's two options either you kind of put your head in the sand and don't think about these things and just let they all go and do what it does or you make decisions about what you value you know open injecting moral values into that with look I don't never mean to be an apologist for the tech industry but I think it's it's a little Bit too far to sort of say that explicit decisions were made about these things so let's for instance take social media platforms right so like many inventions

in technology and computer science a lot of these platforms that we now use regularly kind of started as curiosities right I remember when things like Facebook came out in its predecessors like Friendster which nobody even remembers now the people people really wonder like what why would anybody want To spend time doing that you know what I mean even even the web when it first came out when it wasn't populated with much content and it was largely kind of hobbyists building their own kind of ramshackle websites a lot of people looked at this this is like

what is the purpose of this thing why is this interesting who would want to do this and so even things like Facebook and Twitter yes technical decisions were made by Engineers by scientists by executives in the design of those platforms but you know I don't I don't think 10 years ago anyone anticipated that those platforms for instance might kind of acquire undo you know influence on political discourse or on the outcomes of election and I think the scrutiny that these companies are getting now is entirely appropriate but I think it's a little too harsh to

kind of look at history and sort of say like oh you should have been Able to anticipate that this would happen with your platform and in this sort of gaming chapter of the book one of the points we're making is that you know these platforms right they don't operate in isolation so like that unlike the other topics we're discussing like fairness and privacy like those are really cases where algorithms can operate on your data and make decisions about you and you're not even aware of it okay things like Facebook and Twitter These are you know

these are these are systems right these are social systems and their evolution even their technical evolution because machine learning is involved is driven in no small part by the behavior of the users themselves and how the users decide to adopt them and how to use them and so you know you know I'm kind of like who really knew that the you know in until until we saw it happen who knew that these things might be able to influence the outcome of Elections who knew that you know they might polarize political discourse because of the ability

to you know decide who you interact with on the platform and also with the platform naturally using machine learning to optimize for your own interest that they would further isolate us from each other and you know like feed us all basically just the stuff that we already agreed with and I think it you know we've come to that outcome I think largely but I Think it's something that we all learned together including the companies as these things happen you asked like well are there algorithmic remedies to these kinds of things and again these are big

problems that are not going to be solved with you know somebody going in and changing a few lines of code somewhere in a social media platform but I do think in many ways there are there are definitely ways of making things better I mean like an obvious recommendation That we we make at some point in the book is like look you know to the extent that we think that machine learning applied for person purposes in things like newsfeed you know or other platforms has led to polarization and intolerance of opposing viewpoints as you know right

these these algorithms have models right and they kind of place people in some kind of metric space and and they place content in that space and they sort of know the Extent to which I have an affinity for a particular type of content and by the same token they also probably have that that same model probably gives you a good idea of the stuff I'm likely to violently disagree whether it be offended by okay so you know in this case there really is some nod you could tune it says like instead of showing people only

what they like and what they want let's show them some stuff that we think that they don't like or that's a Little bit further away and you could even imagine users being able to control this you know just like a everybody gets a slider and that slider says like you know how much stuff do you want to see that's kind of you know you might disagree with or is at least further from your interests I can it's almost like an exploration button so just get your intuition do you think engagement so like you staying on

the platform you because thing engaged do you think Fairness ideas of fairness won't emerge like how bad is it to just optimize for engagement do you think we'll run into big trouble if we're just optimizing for how much you love the platform well I mean optimizing for engagement kind of got us where we are so do you one have faith that it's possible to do better and two if it is how do we do better I mean it's definitely possible to do different right and again you know it's not as if I think that doing

something different than optimizing for engagement won't cost these companies in real ways including revenue and profitability potentially short-term at least yeah in the short term right and again you know if I worked at these companies I'm sure that it it would have seemed like the most natural thing in the world also to want to optimize engagement right and that's good for users in some sense you want Them to be you know vested in the platform and enjoying it and finding it useful interesting and or productive but you know my point is is that the idea

that there is that it's sort of out of their hands as you said or that there's nothing to do about it Never Say Never but that strikes me as implausible as a machine-learning person right I mean these companies are driven by machine learning and this optimization of engagement is Essentially driven by machine learning right it's driven by not just machine learning but you know very very large-scale a be experimentation where you gonna have tweaked some element of the user interface or tweaked some component of an algorithm or tweak some component or feature of your click-through

prediction model and my point is is that anytime you know how to optimize for something you'll you you know by def almost by definition that Solution tells you how not to optimize for it or to do something different engagement can be measured so sort of optimizing for sort of minimizing divisiveness or maximizing intellectual growth over the lifetime of a human being very difficult to measure that that's right so I'm not I'm not claiming that doing something different will immediately make it apparent that this is a good thing for society and in particular I mean ethical

one way of Thinking about where we are on some of these social media platforms is it you know it kind of feels a bit like we're in a bad equilibrium right that these systems are helping us all kind of optimize something myopically and selfishly for ourselves and of course from an individual standpoint at any given moment like what why would I want to see things in my newsfeed that I found irrelevant offensive or you know or the like okay but you know maybe by All of us you know having these platforms myopically optimized in our

interests we have reached a collective outcome as a society that were unhappy with in different ways let's say with respect to things like you know political discourse and tolerance of opposing viewpoints and if Mark Zuckerberg gave you a call and said I'm thinking of taking a sabbatical could you run Facebook for me for four six months what would you how I think no Thanks would be the first response but there are many aspects of being the head of the the entire company there are kind of entirely exogenous to many of the things that we're discussing

here yes and so I don't really think I would need to be CEO at Facebook to kind of implement the you know more limited set of solutions that I might imagine but I think one one concrete thing they could do is they could experiment with letting people who chose to to see more stuff in Their newsfeed that is not entirely kind of chosen to optimize for their particular interests beliefs etc so the the kind of thing is I could speak to YouTube but I think Facebook probably does something similar is they're quite effective at automatically

finding what sorts of groups you belong to not based on race or gender so on but based on the kind of stuff you enjoy watching and it gets a YouTube serve it's a it's a difficult thing for Facebook or YouTube To then say well you know what we're going to show you something from a very different cluster even though we believe algorithmically you're unlikely to enjoy that thing so if that's a weird jump to make there has to be a human like at the very top of that system that says well that will be long-term

healthy for you that's more than an algorithmic decision or or that same person could say that'll be long-term healthy for the platform the platform for the platform's Influence on society outside of the platform right and they you know it's easy for me to sit here and say these things yes but conceptually I do not think that these are kind of totally or should they shouldn't be kind of completely alien ideas right there you know we you could try things like this and it wouldn't be you know we wouldn't have to invent entirely new science to

do it because if we're all already embedded in some metric space And there's a notion of distance between you and me and every other every piece of content then you know we know exactly you know the same model that tells you know that dictates how to make me really happy also tells how to make me as unhappy as possible as well right the the focus in your book and algorithmic fairness research today in general is on machine learning like we said is data but and just even the entire AI feel right now is captivated with

machine Learning with deep learning do you think ideas in symbolic AI or totally other kinds of approaches are interesting useful in the space have some promising ideas in terms of fairness I haven't thought about that question specifically in the context of fairness I definitely would agree with that statement in the large right I mean I am you know one of many machine learning researchers who do believe that the great successes that have been shown in machine learning Recently are great successes but they're on a pretty narrow set of tasks I mean I don't I don't

think were kind of notably closer to general artificial intelligence now than we were when I started my career I mean there's been progress and and I do think that we are kind of as a community maybe looking a bit where the light is but the light is shining pretty bright there right now and we're finding a lot of stuff so I don't want to like argue with the Progress that's been made in areas like deep learning for example this touches another sort of related thing that you mentioned and that people might misinterpret from the title

of your book ethical algorithm is it possible for the algorithm to automate some of those decisions sort of higher-level decisions of what kind of like what what should be fair what should be fair the more you know about a field the more aware you are of its limitations and so I'm pretty Leery of sort of trying you know there's there's so much we don't all we don't know in fairness even when were the ones picking the fairness definitions and you know comparing alternatives and thinking about the tensions between different definitions that the idea of kind

of letting the algorithm start exploring as well I definitely think you know this is a much narrower statement I definitely think the kind of algorithmic auditing for Different types of unfairness right so like in this gerrymandering example where I might want to prevent not just discrimination against very broad categories but against combinations of broad categories you know you quickly get to a point where there's a lot of a lot of categories there's a lot of combinations of n features and you know you can use algorithmic techniques to sort of try to find the subgroups on

which you're discriminating the most and Try to fix that that's actually kind of the form of one of the algorithms we developed for this fairness gerrymandering problem but I'm you know partly because of our technology our sort of our scientific ignorance on these topics right now and also partly just because these topics are so loaded emotionally for people that I just don't see the value I mean again Never Say Never but I just don't think we're at a moment where it's a great time for Computer scientists to be rolling out the idea like hey you

know you know not only have we kind of figured fairness out but you know we think the algorithm should start deciding what's fair or giving input on that decision I just don't laugh it's like the the cost-benefit analysis to the field of kind of going there right now it just doesn't seem worth it to me that said I should say that I think computer scientists should be more Philosophically like should enrich their thinking about these kinds of things I think it's been too often used as an excuse for roboticists or cantatas vehicles for example to

not think about the human factor or psychology or safety in the same way like computer science design algorithms that be sort of using is an excuse and I think it's time for basically everybody to become computer scientists I was about to agree with everything you Said except that last point I think that the other way of looking at is that I think computer scientists you know and and and many of us are but we need to wait out into the world more right I mean just the the influence that computer science and therefore computer scientists

have had on society at large just like has exponentially magnified in the last 10 or 20 years or so and you know you know before when we were just thinking Tinkering around amongst ourselves and it didn't matter that much there was no need for sort of computer scientists to be citizens of the world more broadly and I think those days need to be over very very fast and I'm not saying everybody needs to do it but to me like the right way of doing it is to not to sort of think that everybody else is

going to become a computer scientist but you know I think you know people are becoming more sophisticated about Computer science even laypeople yeah you know though I think one of the reasons we decided to write this book as we thought 10 years ago I wouldn't have tried this because I I just didn't think that sort of people's awareness of algorithms and machine learning you know the general population would have been high and I mean would you would have had to first you know write one of the many books kind of just explicate alais audience First

now I think we're at the point where like lots of people without any technical training at all know enough about algorithms machine learning that you can start getting to these nuances of things like ethical algorithms I think we agree that there needs to be much more mixing but I think I think a lot of the onus of that mixing needs to be on the computer science community yeah so just to linger on the disagreement because I do disagree with You on the point that I think if you're a biologist if you're a chemist if you

are an MBA business person all of those things you can like if you learn to program and not only program if you learn to do machine learning if you know energy data science you immediately become much more powerful the kinds of things you can do and therefore literature like the library Sciences like so you're speaking I think deaf I think it holds true well you're saying For the next two years but long term if you're interested to me if you're interested in philosophy you should learn to program because then you can scrape data you can

and study what people are thinking about on Twitter and then start making those awful conclusions about the meaning of life right I just I just feel like the access to data the digitization of whatever problem you're trying to solve is a fundamentally change what it means to be A computer scientist I mean computer scientists in 20 30 years will go back to being donald knuth style theoretical computer science and everybody would be doing basically they kind of exploring the kinds of ideas the exploring in your book it won't be a computer sighs yeah yeah I

mean I don't think I disagree not but I think that that trend of more and more people and more and more disciplines adopting ideas from computer science learning how to code I think That that trend seems firmly underway I mean you know like an interesting digressive question along these lines is maybe in 50 years there won't be computer science departments anymore because the field will just sort of be ambient in all of the different disciplines and you know people will look back and you know having a computer science department will look like having an electricity

department or something that's like you Know everybody uses this it's just out there I mean I do think there will always be that kind of canoe style core - yeah but it's not an implausible half that we kind of get to the point where the academic discipline of computer science becomes somewhat marginalized because of its very success in kind of infiltrating all of science and society and the humanities etc what is differential privacy or more broadly algorithmic privacy algorithmic privacy More broadly is just the study or the notion of privacy definitions or norms being encoded

inside of algorithms and so you know I think we count among this body of work just you know the literature and practice of things like data anonymization which we kind of at the beginning of our discussion of privacy say like okay this is this is sort of a notion of algorithmic privacy it kind of tells you you know something to go do with data but but you know our View is that it's and I think this is now you know quite widespread that it's you know despite the fact that those notions of anonymization kind of

redact the in coarsening are the most widely adopted technical solutions for data privacy they are like deeply fundamentally flawed and so you know to your first question what is differential privacy differential privacy seems to be a much much better notion of privacy that kind of avoids a lot of the Weaknesses of anonymization notions well while still letting us do useful stuff with data what's anonymization of data so by anonymous a ssin i'm you know kind of referring to techniques like i have a database the rows of that database are let's say individual people's medical records

okay and i want to let people use that data maybe i want to let researchers access that data to build predictive models for some disease but I'm worried that that will leak you know sensitive information about specific people's medical records so anonymization broadly refers to the set of techniques where i say like okay i'm first gonna like like i'm gonna delete the column with people's names I'm going to not put you know so that would be like a redaction right I'm just redacting that information I am going to take ages and I'm not gonna like

say your exact age I'm gonna say whether You're you know zero to 10 10 to 20 20 to 30 I might put the first three digits of your zip code but not the last two etc etc and so the idea is that through some series of operations like this on the data I anonymize it you know another term of art that's used is removing personally identifiable information and you know this is basically the most common way of providing data privacy but that's in a way that still lets people access the some variant form of the

data So at a slightly broader picture as you talk about what does the not immunization mean when you have multiple database like with a Netflix prize when you can start combining stuff together so this is exactly the problem with these notions right is that notions of Adana anonymization removing personally identifying information the kind of fundamental conceptual flaw is that you know these definitions kind of pretend as if the data set in question is the Only data set that exists in the world or that ever will exist in the future and of course things like the

Netflix prize and many many other examples since the Netflix applies I think that was one of the earliest ones though you know you can redefine oh that were anonymized in the data set by taking that anonymized data set and combining with other allegedly anonymized data sets and may be publicly available information about you for people who don't know the Netflix prize was what was being publicly released this data so the names from those rows were removed but what was released is the preference or the ratings of what movies you like and you don't like and

from that combined with other things I think foreign posts and so on you can case it was specifically the Internet Movie Database where where lots of Netflix users publicly rate their move you know their movie preferences and so the anonymized data In Netflix when kaneen and it's it's just this phenomenon I think that we've all come to realize in the last decade or so is that just knowing a few apparently irrelevant innocuous things about you can often act as a fingerprint like if I know you know what what rating you gave to these 10 movies

and the date on which you entered these movies this is almost like a fingerprint for you is the see of all Netflix users there were just another Paper on this in science or nature of about a month ago that you know kind of 18 attributes I mean my favorite example of this this was actually a paper from several years ago now where it was shown that just from your likes on Facebook just from the taunt you know the things on which you clicked on the thumbs up button on the platform not using any information demographic

information nothing about who your friends are just knowing the Content that you had liked was enough to you know in the aggregate accurately predict things like sexual orientation drug and alcohol use whether you were the childhood divorced parents so we live in this era where you know even the apparently irrelevant data that we offer about ourselves on public platforms and forums often unbeknownst to us more or less acts as signature or you know fingerprint and that if you can kind of you know do a join between that kind of Data and allegedly anonymize data you

have real trouble so is there hope for any kind of privacy in a world where a few likes can can identify you so there is differential privacy right what is differential differential privacy basically is a kind of alternate much stronger notion of privacy than these anonymization ideas and it you know it's a technical definition but like the spirit of it is we we compare to to alternate worlds okay so let's suppose I'm a researcher and I want to do you know I there's a database of medical records and one of them's yours and I want

to use that database of medical records to build a predictive model for some disease so based on people's symptoms and test results and the like I want to you know build a Probab you know model predicting the probability that people have disease so you know this is the type of scientific research that we would like to be allowed to continue and In differential privacy you act ask a very particular counterfactual question we basically compare two alternatives one is when I do this I build this model on the database of medical records including your medical record

and the other one is where I do the same exercise with the same database with just your medical record removed so basically you know it's two databases one with n records in it and one with n minus one records in It the N minus one records are the same and the only one that's missing in the second case is your medical record so differential privacy basically says that any harms that might come to you from the analysis in which your data was included are essentially nearly identical to the harms that would have come to you

if the same analysis had done been done without your medical record included so in other words this doesn't say that bad things cannot Happen to you as a result of data analysis it just says that these bad things were going to happen to you already even if your data wasn't included and to give a very concrete example right you know um you know like we discussed at some length the the study that you know the in the 50s that was done that created the that established the link between smoking and lung cancer and we make

the point that like well if your data was used in that Analysis and you know the world kind of knew that you were a smoker because you know there was no stigma associated with smoking before that those findings real harm might have come to you as a result of that study that your data was included in in particular your insurer now might have a higher posterior belief that you might have lung cancer and raise your premiums so you've suffered economic damage but the point is is that if the same analysis been done without With all

the other n minus-1 medical records and just yours missing the outcome would have been the same your your data was an idiosyncratic eleum crucial to establishing the link between smoking and lung cancer because the link between smoking and lung cancer is like a fact about the world that can be discovered with any sufficiently large database of medical records but that's a very low value of harm yeah so that's showing that very little harm Is done great but how what is the mechanism of differential privacy so that's the kind of beautiful statement of it well what's

the mechanism by which privacy's preserve yeah so it's it's basically by adding noise to computations right so the basic idea is that every differentially private algorithm first of all or every good differentially private album every useful one is a probabilistic algorithm so it doesn't on a given input if you Gave the algorithm the same input multiple times it would give different outputs each time from some distribution and the way you achieve differential privacy algorithmically is by kind of carefully and tastefully adding noise to a computation in the right places and you know to give a

very concrete example if I want to compute the average of a set of numbers right the non private way of doing that is to take those numbers and average them and release like a Numerically precise value for the average okay in differential privacy you wouldn't do that you would first compute that average to numerical Precision's and then you'd add some noise to it right you'd add some kind of zero mean you know gaussian or exponential noise to it so that the actual value you output is not the exact mean but it'll be close to the

mean but it'll be close the noise the you add will sort of prove that nobody can kind of reverse engineer Any particular value that went into the average so noise noise is the Savior how many algorithms can be aided by making by adding noise yeah so I'm a relatively recent member of the differential privacy community my co-author Aaron Roth is you know really one of the founders of the field and has done a great deal of work and I've learned a tremendous amount working with him on it growing up field already yeah but it's now

it's pretty mature but I must admit The first time I saw the definition of deferential privacy my reaction was like well that is a clever definition and it's really making very strong promises and my you know you know at first saw the definition in much earlier days and my first reaction was like well my worry about this definition would be that it's a great definition of privacy but that it'll be so restrictive that we won't really be able to use it like you know We won't be able to do compute many things in a differentially

private way so that that's one of the great successes of the field I think isn't showing that the opposite is true and that you know most things that we know how to compute absent any privacy considerations can be computed in a differentially private way so for example pretty much all of statistics and machine learning can be done differentially privately so pick your Favorites machine learning algorithm back propagation and neural networks you know cart for decision trees support vector machines boosting you name it as well as classic hypothesis testing and the like and statistics none of

those algorithms are differentially private in their original form all of them have modifications that add noise to the computation in different places in different ways that achieve differential privacy so this really Means that to the extent that you know we've become a you know a scientific community very dependent on the use of machine learning and statistical modeling and data analysis we really do have a path to kind of provide privacy guarantees to those methods and and sort of we can still you know enjoy the benefits of kind of the data science era while providing you

know rather robust privacy guarantees to individuals so perhaps a a slightly crazy question but If we take that the ideas of differential privacy and take it to the nature of truth that's being explored currently so what's your most favorite and least favorite food hmm I'm not a real foodie so I'm a big fan of spaghetti I forget it yeah on what what do you really don't like umm I really don't like cauliflower well I love golf okay but is one way to protect your preference for spaghetti by having in Formation campaign bloggers and so on

a boat's saying that you like cauliflower so like this kind of the same kind of noise ideas I mean if you think of in our politics today there's this idea of Russia hacking our elections what's meant there I believe is BOTS spreading different kinds of information is that a kind of privacy or is that too much of a stretch no it's not a stretch I have not seen those idea you know that is not a technique that to my knowledge will Provide differential privacy but but to give an example like one very specific example about

what you're discussing is there was a very interesting project at NYU I think led by a Helen missin bomb there in which they basically built a browser plugin that tried to essentially obfuscate your Google searches so to the extent that you're worried that Google is using your searches to build you know predictive models about you to decide what ads to show you which they might Very reasonably want to do but if you object to that they built this widget you could plug in and basically whenever you put in a query into Google it would send

that query to Google but in the background all the time from your browser it would just be sending this torrent of irrelevant queries to the search engine so you know it's like a weed and chaff thing so you know out of every thousand queries let's say that Google was Receiving from your browser one of them was one that you put in but the other 999 were not okay so it's the same kind of idea kind of you know privacy by obfuscation so I think that's an interesting idea doesn't give you differential privacy it's also I

was actually talking to somebody at one of the large tech companies recently about the fact that you know just this kind of thing that there are some times when the response to my data needs to be very Specific to my data right like I type mountain biking into Google I want results on mountain biking and I really want Google to know that I typed in biking I don't want noise adage to that and so I think there's sort of maybe even interesting technical questions around notions of privacy that are appropriate where you know it's not

that my date is part of some aggregate like medical records and that we're trying to discover important correlations and Facts about the world at large but rather you know there's a service that I really want to you know pay attention to my specific data yet I still want some kind of privacy guarantee and I think these kind of obfuscation ideas are sort of one way of getting at that but maybe there are others as well so where do you think will land in this algorithm driven society in terms of privacy so sort of China like

Chi Fuli describes you know It's collecting a lot of data on its citizens but in the best form it's actually able to provide a lot of sort of protects human rights and provide a lot of amazing services and its worst forms it can violate those human rights and and limit services so what do you think will land on so algorithms are powerful when they use data so as a society do you think we'll give over more data is it possible to protect the privacy of that data so I'm optimistic About the possibility of you know

balancing the desire for individual privacy and individual control of privacy with kind of societally and commercially beneficial uses of data not unrelated to differential privacy or suggestions that say like well individuals should have control of their data they should be able to limit the uses of that data they should even you know there's there's you know fledgling discussions going on in research circles About allowing people selective use of their data and being compensated for it and then you get to sort of very interesting economic questions like pricing right and one interesting idea is that maybe

differential privacy would also you know be Bo a conceptual framework in which you could talk about the relative value of different people's data like you know to demystify this a little bit if I front of build a predictive model for Some rare disease and I'm trying to you I'm gonna use machine learning to do it it's easy to get negative examples because the disease is rare right but I really want to have lots of people with the disease in my data set okay but but and so somehow those people's data with respect to this application

is much more valuable to me than just like the background population and so maybe they should be compensated more for it and so you know I think these are kind Of very very fledgling conceptual questions that maybe will have kind of technical thought on them sometime in the coming years but but I do think well you know to kind of get more directly answer your question I think I'm optimistic at this point from what I've seen that we will land at some you know better compromise than we're at right now where again you know privacy

guarantees are a few far between and weak and users have very very little Control and I'm optimistic that we'll land in something that you know provides better privacy overall and more individual control of data and privacy but you know I think to get there it's again just like fairness it's not going to be enough to propose algorithmic solutions there's gonna have to be a whole kind of regulatory legal process that prods companies and other parties to kind of adopt solutions and I think you've mentioned the word control and I Think giving people control that's something

that people don't quite have and a lot of these algorithms that's a really interesting idea of giving them control some of that is actually literally an interface design question sort of just enabling because I think it's good for everybody to give users control it's not it's not a it's almost not a trade off except you have to hire people that are good at interface design yeah I mean the other thing that has to Be said right is that you know it's a cliche but you know we who is the users of many systems platforms and

apps you know we are the product we are not the customer the customer our advertisers and our data is the prod okay so it's one thing to kind of suggest more individual control of data and privacy and uses but this you know if this happens in sufficient degree it will upend the entire economic model that has supported the internet to date And so some other economic model will have to be you know will have to replace it so the idea of markets you mentioned by exposing the economic model to the people they will then become

a market they can be participants in participants in and and you know this isn't you know this is not a weird idea right because there are markets for data already it's just that consumers are not participants in there's like you know there's sort of you know publishers and content Providers on one side that have inventory and then they're advertised on the others and you know you know Google and Facebook are running you know they're pretty much their entire revenue stream is by running two-sided markets between those parties right and so it's not a crazy idea

that there would be like a three sided market or that you know that on one side of the market or the other we would have proxies representing our interest it's not you Know it's not a crazy idea but it would it it's not a crazy technical idea but it would have pretty extreme economic consequences speaking of markets a lot of fascinating aspects of this world arise not from individual humans but from the interaction of human beings you've done a lot of work in game theory first can you say what is game theory and how does

help us model and study yeah game theory of course let us give credit where it's due they don't comes From the economist first and foremost but as I've mentioned before like you know computer scientists never hesitate to wander into other people's turf and so there is now this 20 year old field called algorithmic game theory but you know game game theory first and foremost is a mathematical framework for reasoning about collective outcomes in systems of interacting individuals you know so you need at least two people to get started in game theory and many People are

probably familiar with prisoner's dilemma as kind of a classic example of game theory and a classic example where everybody looking out for their own individual interests leads to a collective outcome that's kind of worse for everybody then what might be possible if they cooperated for example but cooperation is not an equilibrium in prisoner's dilemma and so my work and the field of algorithmic game theory more generally in these areas kind of Looks at settings in which the number of actors is potentially extraordinarily large and their incentives might be quite complicated and kind of hard to

model directly but you still want kind of algorithmic ways of kind of predicting what will happen or influencing what will happen in the design of platforms so what to you is the most beautiful idea that you've encountered in game theory there's a lot of them I'm a big fan of the field I Mean you know I mean technical answers to that of course would include Nash's work just establishing that you know there there's a competitive equilibrium under very very general circumstances which in many ways kind of put the field on a firm conceptual footing because

if you don't have equilibria it's kind of hard to ever reason about what might happen since you know there's just no stability so just the idea that stability can emerge when there's Multiple or that it means not that it will necessarily emerge just that it's possible right it's like the existence of equilibrium doesn't mean that sort of natural iterative behavior will necessarily lead to it in the real world yeah maybe answering a slightly less personally than you asked the question I think within the field of algorithmic game theory perhaps the single most important kind of

technical contribution that's Been made is the real the the realization between close connections between machine learning and game theory and in particular between game theory and the branch of machine learning that's known as no regret learning and and this sort of provides a fray a very general framework in which a bunch of players interacting in a game or a system each one kind of doing something that's in their self-interest will actually kind of reach an equilibrium And actually reach an equilibrium in a you know a pretty you know a rather you know short amount of

steps so you kind of mentioned acting greedily can somehow end up pretty good for everybody or pretty bad or pretty bad it will end up stable yeah right and and you know stability or equilibrium by itself is neither is not necessarily either a good thing or a bad thing so what's the connection between machine learning and the ideas well if we kind of talked About these ideas already in in kind of a non-technical way which is maybe the more interesting way of understanding them first which is you know we have many systems platforms and apps

these days that work really hard to use our data and the data of everybody else on the platform to selfishly optimize on behalf of each user okay so you know let me let me give what the the cleanest example which is just driving apps navigation apps like you know Google Maps and ways where you know miraculously compared to when I was growing up at least you know the objective would be the same when you wanted to drive from point A to point B spend the least time driving not necessarily minimize the distance but minimize the

time right and when I was growing up like the only resources you had to do that were like maps in the car which literally just told you what roads were available and then you might have Like half hourly traffic reports just about the major freeways but not about side roads so you were pretty much on your own and now we've these apps you pull it out and you say I want to go from point A to point B and in response kind of to what everybody else is doing if you like what all the other

players in this game are doing right now here's the the you know the the route that minimizes your driving time so it is really kind of computing a Selfish best response for each of us in response to what all of the rest of us are doing at any given moment and so you know I think it's quite fair to think of these apps as driving or nudging us all towards the competitive or Nash equilibrium of that game now you might ask like well that sounds great why is that a bad thing well you know it's

it's known both in theory and with some limited studies from actual like traffic data that all Of us being in this competitive equilibrium might cause our collective driving time to be higher may be significantly higher than it would be under other solutions and then you have to talk about what those other solutions might be and what what the algorithms to implement them are which we do discuss in the kind of game theory chapter of the book but but similarly you know on social media platforms or on Amazon you know all these algorithms that are Essentially

trying to optimize our behalf they're driving us in a colloquial sense towards some kind of competitive equilibrium and you know one of the most important lessons of game theory is that just because we're at equilibrium doesn't mean that there's not a solution in which some or maybe even all of us might be better off and then the connection to machine learning of course is that in all these platforms I've mentioned the optimization that They're doing on our behalf is driven by machine learning you know like predicting where the traffic will be predicting what products I'm

gonna like predicting what would make me happy in my newsfeed now in terms of the stability and the promise of that I have to ask just out of curiosity how stable are these mechanisms that you game theories just The Economist's came up with and we all know that economists don't live in the real world just Kidding sort of what's do think when we look at the fact that we haven't blown ourselves up from the from a game theoretic concept of mutually assured destruction what are the odds that we destroy ourselves with nuclear weapons as one

example of a stable game theoretic system just to prime your viewers a little bit I mean I think you're referring to the fact that game theory was taken quite seriously back in The 60s as a tool for reasoning about kind of Soviet US nuclear armament disarmed ative date on things like that I'll be honest as huge of a fan as I am of game theory and it's kind of rich history it still surprises me that you know you had people at the RAND Corporation back in those days kind of drawing up you know two by

two tables and one the row player is weekend oh the US and the column player is Russia and that they were taking seriously you know You know I'm sure if I was there maybe it wouldn't have seemed as as naive as it does at the time you know seems to have worked which is why it seems naive well we're still here we're still here in that sense yeah even though I kind of laugh at those efforts they were more sensible than than they would be now right because there were sort of only two nuclear powers

at the time and you didn't have to worry about deterring new entrants and who was developing the Capacity and so we have many we have this it's definitely a game with more players now and more potential entrants I'm not in general somebody who advocates using kind of simple mathematical models when the stakes are as high as things like that and the complexities are very political and social but but we are still here so you've worn many hats one of which the one that first caused me to become a big fan of your work many years

ago is Algorithmic trading so I have to just ask a question about this because you have so much fascinating work there in the 21st century would what role do you think algorithms have in space of trading investment in the financial sector yeah it's a good question I mean in the time I've spent on Wall Street and in finance you know I've seen a clear progression and I think it's a progression that kind of models the use Of algorithms and automation more generally in society which is you know the things that kind of get taken over

by the algos first are sort of the things that computers are obviously better at than people right so you know so first of all there needed to be this era of automation right we're just you know financial exchanges became largely electronic which then enabled the possibility of you know trading becoming more algorithmic because once you know The exchanges are electronic an algorithm can submit an order through an API just as well as a human can do at a monitor quickly it can read all the data so yeah and so you know I think the the

places where algorithmic trading have had the greatest inroads and had the first inroads were in in kind of execution problems kind of optimized execution problems so what I mean by that is at a large brokerage firm for example one of the lines of business Might be on behalf of large institutional clients taking you know what we might consider difficult trade so it's not like a mom-and-pop investor saying I want to buy a hundred shares of Microsoft it's a large hedge fund saying you know I want to buy a very very large stake in Apple and

I want to do it over the span of a day and it's such a large volume that if you're not clever about how you break that trade up not just over time but over perhaps multiple Different electronic exchanges that all let you trade Apple on their platform you know you will you will move you'll push prices around in a way that hurts your your execution so you know this is the kind of you know this is an optimization problem this is a control problem right and so machines are a better we know how to design

algorithms you know that are better at that kind of thing then a person is going to be able to do because we can take volumes of Historical and real-time data to kind of optimize the schedule with which we trade and you know similarly high frequency trading you know which is closely related but not this optimized execution where you're just trying to spot very very temporary you know miss pricings between exchanges or within an asset itself or just predict directional movement of a stock because of the kind of very very low-level granular buying and selling data

in in The exchange machines are good at this kind of stuff it's kind of like the mechanics of trading what about the can machines do long terms of prediction yeah so I think we are in an era where you know clearly there have been some very successful you know quant hedge funds that are you know in what we would traditionally call you know still in this the stat ARB regime like so you know stat are referring to statistical arbitrage but But for the purposes of this conversation what it really means is making directional predictions in

asset price movement or returns your prediction about that directional movement is good for you know you you have a view that it's valid for some period of time between a few seconds and a few days and that's the amount of time that you're gonna kind of get into the position hold it and then hopefully be right about the directional movement and You know buy low and sell high as the cliche goes so that is a you know kind of a sweet spot I think for quant trading and investing right now and has been for some

time when you really get to kind of more warren buffett style timescales right like you know my cartoon of warren buffett is that you know warren buffett sits and thinks what the long-term value of Apple really should be and he doesn't even look at what Apple's doing today he just decides You know yeah you know I think that this was what its long-term value is and it's far from that right now and so I'm gonna buy some Apple or you know shorts and Apple and I'm gonna I'm gonna sit on that for 10 or 20

years okay so when you're at that kind of time scale or even more than just a few days all kinds of other sources of risk and information you know so now are talking about holding things through recessions and economic cycles wars can Break out so there you have to install a human nature at 11:00 yeah and you need to just be able to ingest many many more sources of data that are on wildly different timescales right so if I'm an hft I'm a high-frequency trader like I don't I don't I really my main source of

data is just the data from the exchanges themselves about the activity in the exchanges right and maybe I need to pay you know I need to keep an eye on the news right because you know that can Sudden cause sudden you know the the you know CEO gets caught in a scandal or you know gets run over by a bus or something that can cause very sudden changes in but you know I don't need to understand economic cycles I don't need to understand recessions I don't need to worry about the political situation or war breaking

out in this part of the world because you know all you need to know is as long as that's not gonna happen in the left next 500 milliseconds Then you know my models good when you get to these longer timescales you really have to worry about that kind of stuff and people in the machine learning community are starting to think about this we held a we did we jointly sponsored a workshop at 10:00 with the Federal Reserve Bank of Philadelphia a little more than a year ago on you know I think the title is something

like machine learning for macroeconomic prediction You know macroeconomic referring specifically to these longer timescales and you know it was an interesting conference but it you know my it left me with greater confidence that we have a long way to go to you know and so I think that people that you know in the grand scheme of things you know if somebody asked me like well whose job on Wall Street is safe from the bots I think people that are at that longer you know the time scale and have that Appetite for all the risks involved

in long term investing and that really need kind of not just algorithms that can optimize from data but they need views on stuff they need views on the political landscape economic cycles and the like and I think you know they're they're they're pretty safe for a while as far as I can tell so Warren Buffett yeah I'm not seeing you know a robo Warren Buffett anytime so she'd give him comfort last question if you could go Back to if there's a day in your life you could relive because I made you truly happy maybe you

outside family boy otherwise do you know what what day would it be what can you look back you remember just being profoundly transformed in some way or blissful I'll answer a slightly different question which is like what's a day in my life or my career that was kind of a watershed moment I went straight from undergrad to doctoral Studies and you know that's not at all a typical and I'm also from an academic family like my dad was a professor or my uncle on his side as a professor both my grandfather's were professors all kinds

of majors to philosophy yeah all over the map yeah and I was a grad student here just up the river at Harvard and came to study with less valiant which was a wonderful experience but you know I remember my first year of graduate school I was generally pretty unhappy And I was unhappy because you know at Berkeley as an undergraduate you know yeah I studied a lot of math and computer science but it was a huge school first of all and I took a lot of other courses as we've discussed I started as an English

major and took history courses and art history classes and had friends you know that did all kinds of different things and you know Harvard's a much smaller institution than Berkeley and it's computer science Department especially at that time was was a much smaller place than it is now and I suddenly just felt very you know like I'd gone from this very big world to this highly specialized world and now all of the classes I was taking were computer science classes and I was only in classes with math and computer science people and so I was

you know I thought often in that first year of grad school about whether I really wanted to stick with it or not and you know I Thought like oh I could you know stop with a masters I could go back to the Bay Area into California and you know this was from one of the early periods where there was you know like you could definitely get a relatively good job paying job at one of the one of the tech companies back you know that were the the big tech companies back then and so I distinctly

remember like kind of a late spring day when I was kind of you know sitting in Boston Common and kind of really just kind of chewing over what I wanted to do with my life and I realized like okay you know and I think this is where my academic background helped me a great deal I sort of realized you know yeah you're not having a great time right now this feels really narrowing but you know that you're here for research eventually and to do something original and to try to you know carve out a career

where you kind of you know choose what you want to Think about you know and have a great deal of Independence and so you know at that point I really didn't have any real research experience yet I mean it was trying to think about some problems with very little success but but I knew that like I I hadn't really tried to do the thing that I knew I'd come to do and so I thought you know I'm gonna I'm gonna stick I'm gonna you know stick through it for the summer and you know and and

and that was very formative because I Went from kind of contemplating quitting to you know a year later it being very clear to me I was going to finish because I still had a ways to go but I kind of started doing research it was going well it was really interesting and it was sort of a complete transformation you know it's just that transition that I think every doctoral student makes at some point which is to sort of go from being like a student of what's been done before to doing you know your own thing

And figure out what makes you interested in what your strengths and weaknesses are as a researcher and once you know I kind of made that decision on that particular day at that particular moment in Boston Common you know the I'm glad I made that decision and also just accepting the painful nature of that journey yeah exactly exactly and in that moment said I'm gonna I'm gonna stick it out yeah I'm gonna stick around for a while well Michael looked up do you work for a long time it's really talk to you separation get back in

touch with you - and see how great you're doing as well thank thanks a lot appreciate you