[Music] greetings everybody this is our first pit principal lecturer and we are covering reflexivity today or more specifically reflexivity for accountability [Music] so in today's talk we're going to cover what is reflexivity why does it matter and what does it entail what are some of the challenges how does it relate to the other principles Etc but first I want to do a little bit of review you'll remember this which I like to think of as the pit braid the pigtail or ponytail right in the middle right that's supposed to represent two processes intertwining over time
the process of Technology development and the process of public interest articulation and that what hops out at the end the outcomes are the desired outcomes we hope and these are public interest outcomes so we have two meta principles the first one we made a bigger deal of at the beginning of the semester and that is alignment because alignment is what results when you're able to integrate and intertwine these two processes uh in an effective manner but of course we also have the process of integration so we can say that the two meta principles new two
meta pit principles are integration and Alignment integration would be the process alignment would be the product and they're really just two sides of the same thing but this is what all the other principles really are attempting to ensure and bring about namely that public interest articulation is intertwined with technology development in a synergistic Manner and an ongoing manner so uh just to remind you uh here are the four or eight pit principles we can think of them as four because they are related reflexivity is the process that hopefully will get us more accountability anticipation is
the process that will hopefully get us more preparedness inclusion is the process that will get us more social robustness and responsiveness we said was the process that will get us more alignment but since we want to make integration and Alignment the two sort of overarching meta principles let's change alignment to adaptability eyes with me good so these are our principles in a nutshell you can think of them as four or five eight or ten but remember we're on a journey here to discover these principles what they mean and how they show up uh in both
expected and unexpected places and ways and also for you to discover additional principles or additional emphases that you can bring to my attention and to one another's attention in your discussions in papers so what is reflexivity what is reflexivity it is a special kind of awareness a certain certain type of awareness and we can break it down uh in sort of two respects one is what is the object of awareness and the other is what is the subject of the law of awareness who is aware and how is that person aware so in terms of
the object this is the the content what are you aware of and basically it comes down to um this abstract concept of dynamic interaction so in reflexive awareness you're aware of a dynamic interaction and this could be between two or more different things and there are different versions of this but basically uh we can talk about self and other we can talk about reality and its apprehension there's a number of themes that are dynamically interrelated and the two we're just going to focus on here is the idea that in the act of trying to understand
reality um I'm going to change reality and reality is going to change me okay so there's this idea that reality is bending back on itself through my attempt to know it my attempt to understand it my attempt to model it and this is the idea that sometimes um known as the snake biting its tail or who bought us um that somehow reality is bending back on itself now there's a number of ways that reality can do this and mechanisms that bring it about and there's a number of different ways that this principle can be represented
one of my favorite is a equals a that if you want to understand reflexivity I'll tell you what it is it's a equals a the idea that a equals a um that's a little misleading in two respects one is that it might be a equals a prime um uh but that would you know but the idea of reality depending on back onto itself suggests that a is reality and a is reality and reality bends back on itself okay um but the other uh potentially problematic aspect to a equals a is that it doesn't really tell
us that there's this time Dimension okay if I'm a designer and I design and I'm a product of my environment and I design products in a certain way those products are going to be a product of the environment but that is going to be something that unfolds over time first I have to become produced by my environment then I have to produce a product okay and both of these are going to take time and the product then is going to interact in the environment in such a way that there's all these sort of loopy uh
recursive and reflexive activities that mean that the final outcome is something that is affected by its environment as much as it affects its environment hook hopefully you guys are still holding on for the ride here um so that's that's understanding reflexivity as the object of awareness okay we're somehow aware of these Dynamic feedback loops when I design something I'm going to affect society um but since Society has affected me before that I'm going to design it in such a way that Society is affecting Society through me and of course these are in limited limited ways
right because one part of society affects me but I'm not aware of the other part of society because I grew up in this side of town not that side of town and therefore when I'm designing my product it's going to affect not just the side of town that I'm from but also the other side of town that I haven't visited okay so this is why it's problematic and this is why we want to make reflexivity not just a truth about the world a theoretical uh statement but also a practice that somehow what I as a
practitioner or what my organization is aware of or what a an institution that has developed organically over time what that thing that being is aware of cognitively the nature of the awareness is a certain type of way in which these activities these Dynamic interrelations show up in my day-to-day life in my regular practice as a technology developer as a policy developer okay somebody who's participating in pit okay and so this is why we refer to it as a cognitive disposition as opposed to deliberative thought where I sit down and I think very carefully like we
are now a equals a a means reality and b means the representation of reality okay uh so so reflexive awareness in this respect is not a theoretical disposition it's a tacit disposition it's a way I have of approaching the world okay it's not when I stop everything and I start thinking very carefully that can certainly be a process slow thinking or deliberate thinking can certainly be a process that can be more or less reflexive okay but when we talk about reflexivity as a quality of civic-minded technologists it's a quality that they can take with them
into the stressful high stakes High paced process of Technology development now reflexivity comes from and is you know shows up in all these different areas of study and practice social theory which is where it comes from qualitative research where it's it's used economics psychology Consciousness studies run-of-the-mill mindfulness not to put it down but just to sort of say mindfulness is is definitely a practice it's not a high theoretical activity although the Consciousness studies might scientists might disagree with me and even Alchemy which I think is where this image of the uru baras comes from I'm
not going to go into these I'll give you some videos to sort of get you into some of these but I just wanted to give you a sense of how pervasive it is so why the heck does it matter well it matters simply because in technology or policy development and public interest articulation the conditions for unintended consequences are constantly arising they're arising because of the technology developers in part their dispositions their behaviors their patterns of thought and yet these very people who are involved in the process are often unaware of both the potential unintended consequences
that could come down the line and the ways in which they are contributing to the conditions that arise for the possibility of unintended consequences to occur now when we get into the stir uh process we will sort of locate all of this onto decision making and so if it helps you now to think about reflexivity and why it matters just related to decision making right because people make decisions and those decisions give rise to consequences both intended and unintended both immediate and delayed therefore it's important how I think about my decision making and what I
think about when I'm putting my attention into that decision making and thinking not just consciously and deliberately but also tacitly and quickly so we can sort of return to this idea uh that we've now seen a couple of times and just sort of remind ourselves that along the way of Technology development and along the way of public artists are interest articulation whether they're intertwined or not both of these processes whether separate or together give rise to conditions that can lead to unintended consequences and these can be undesirable consequences and they can be consequences that undermine
or go against the public interest we've talked about them before they can be conflicts of interest but they can also be uh social structural historical cultural inputs that somehow are decisions function as regulators of the outcomes that will occur later so examples of these sort of the conditions for unintended consequences to arise can be cultural historical and other inputs in the same way that ruha Benjamin uses that phrase that term in her book race against technology sorry race after technology which I had you read the introduction to and we can think of other examples but
in every case the point is that these are invisible these inputs to technology development are below detection okay we're not aware of them either because we don't have time we're not trained to see them we don't believe they exist we're super stressed out and and we're focused on something else okay um and that our own assumptions the very assumptions that enable us to show up to work and and know some mathematics know some Team Dynamics uh know something about product development and marketing and sales and Market uh studies these very assumptions can also work against
perplexive awareness right sometimes this is because our assumptions serve us only in terms of getting product to Market but they don't serve Us in terms of getting pit to Market they can also show up in the form of fallacies and we've talked about some of those fallacies technological determinism volitional determinism Etc there's also this idea of eye methodology this is uh basically a default position that technology designers and product designers fall back on when they do think about whether consciously or unconsciously when they do think about the user they will assume that the user is
like them or they will assume that they are an adequate representation of the user and they'll simply rely on their own experiences their own knowledge so this is an idea from outshorn and others who are STS Scholars Science and Technology studies Scholars and they Define eye methodology as a design fraction practice in which designers consider themselves as representative of the users and rely on personal experience so just to remind you the four pit aspirations each of these are infused with a need for reflexive awareness right advancing the public interest using technology producing technology that considers
the public interest developing policies that promote pit and training people to be civic-minded technologists all right I'm not going to spend a lot of time on these but again the idea that technology development and policy development and public interest articulation are largely the products of One's Own cognitive dispositions okay and the cognitive dispositions of One's group and one's organization and one's culture okay and so if we want to get technology working for the public interest and working with public interest articulation if we want public interest articulation to actually work in and be representative of the
public interest then we need to be aware of these inputs that come through our cognitive mechanisms our cognitive orientations are cognitive interest sorry our cognitive disposition so what is reflexivity involved well primarily we said that it is a form of awareness and that we can think of it both in theoretical and in Practical terms and theoretical or formal terms um reflexivity is the statement that inputs to technology development and policy development and articulation can common hidden forms undetected forms and these can affect outcomes down the line later okay but the Practical or tacit knowledge and
dispositional awareness basically takes the form of situational awareness and awareness of how inputs that might be coming from historical cultural or psychological sources show up in our daily decision making and situations so exercising reflexivity and reflexive awareness requires qualities of attentiveness vigilance patience and persistence okay and it also requires a capacity for cognitive and emotional discomfort and even disruption why well we talked about cognitive dissonance earlier how being aware suddenly of something that runs contrary to your working assumptions that do serve you but now you're aware of the opposite statement that could undermine you and
now you've got two statements that seem to counsel one another out but they're both sort of true they're both uh they've both got utility or they both have are demanding they both have some claim on the truth and that that is painful all right there's there's cognitive dissonance and so I you know have to have a capacity to withstand that pain to sort of be suspended in the moment when I'm having two thoughts that don't necessarily go together um and you know this is hard enough to begin with it's not something something that that most
people are very good at and even experts and and professors and highly developed and thoughtful people can be very bad at not shutting down when there's cognitive or emotional dissonance but add to that the fact that we're supposed to have reflexivity and reflexive awareness under conditions that are not necessarily conducive to these qualities of patients persistence vigilance and these capacities for discomfort okay and we'll open up these conditions in a minute but but you know you can just just think about how how just you know everyday uh strategy technology development and design marketing Innovation uh
entrepreneurship these are activities that do not necessarily encourage a sort of broader sweeping awareness of what how are my assumptions influencing my environment how are my social and cultural assumptions going to potentially influence the social and cultural environment that my policy or my technology is going to live in so reflexivity where you know why would anyone have it okay and and you know the truth is that we do have it all the time okay it's just in select ways right so there has to be I believe a motivation for reflexivity to be exercise and this
motivation comes from some care or some desire right this is pretty basic so you're with me so far right I'm only going to sort of be reflexively aware of certain things if I care about something that being reflexively aware of these things is somehow going to serve or if I desire to be reflexively aware or I desire something and I recognize it being reflexively aware is going to help me get what I desire so this care this desire could be in the form of a commitment that I have to getting a technology out or having
a technology function in a certain way that increases equity in the world or it might come from a value I have or it might come from pleasure that I have that's it's pleasurable for me to have reflexive awareness right which you can actually get there even though it's painful and it's it introduces discomfort if you get used to that and you associate that with doing the right thing or making the world a better place suddenly pushing your Edge and being emotionally and uh cognitively uh sort of disrupted can actually be a pleasurable thing it can
give you energy it can also come from empathy empathy for the user can come from a sense of Ethics do the right thing these are principles I want to adhere to people can be reflexive because they have a sense of themselves that this is this is the way to maintain my Integrity I want a certain kind of character and I want it to have integrity and to do that I have to think about uh the broader implications of my actions it can come from a disposition towards curiosity I want to know and so I'm going
to take risks in order to gain that knowledge or a desire to understand the world okay um or it could come from a desire to just constantly keep learning or to improve myself a self-development and personal growth now these last two uh curiosity and understanding can be aided tremendously if we want to understand where reflexivity comes from to begin with hence how we could get more of it in ourselves and in others then we can sort of go to the writings of the American philosopher John Dewey who really took this concept of inquiry and made
it a very important fragment pragmatic philosophical principle and so I like to say it's the desire to inquire that we want to sort of view as a resource okay and by inquire I mean what do we meant by inquiry which is really um as he talks about it in in several of his books including one called how we think inquiry is a human activity that is engaged in when uh you are in the midst of getting something done and there's a problem that emerges you become aware of the problem or you become aware that a
problem is starting to emerge he calls it a problematic situation so in the process of working on anything I become aware of a problematic situation and then according to Dewey I stop what I'm doing or I add another channel to what I'm doing and I start thinking about the problem and how to overcome it and that's inquiry and then I might test things and then finally I burst through so basically according to Dewey inquiry is just to form problem solving and it happens when we become aware of a problem so if we're going about our
way and we're developing a technology and everything's fine then we don't have any problems and we're not conducting inquiry but of course as you know problems are constantly arising day to day week to week month to month even moment to moment and so we're constantly engaged in problem solving so that means that there's constantly inquiry going on and when you're performing inquiry use can step out of yourself and say okay what am I taking for granted what am I missing here so that's reflexive awareness and then the idea of understanding the world um as well
as self-improvement self-development both of these Concepts both of these motivations are really nicely articulated by Peter sengay in the Fifth Discipline one of the first four disciplines is what he calls systems thinking and this is a very nice book it's very friendly introduction and very useful introduction to systems thinking in a business and organizational environment but what he calls the Fifth Discipline is actually he terms It Personal Mastery and this is the ability to engage in systems thinking and apply it but also to realize that you are part of this system and the way you
perceive the system is going to uh limit or determine or regulate how you then design strategies policies and actions and artifacts so this idea of perception is very very important we know that from the policy Sciences framework perception is is very much part of what the policy scientist practitioner but also the actors that he or she is studying are going to be influenced by perception is really important and all these things influence perception so it's very important to be aware if we want to solve problems according to Dewey or you know in the new discipline
of public problem solving right how we become aware of the problem how we Define that problem and how we understand its solution sets and the conditions that give rise to the problem and to the potential solution these are very much they're not just scientific applications of an algorithm or a model they involve our awareness of Our Own Thing thinking our own values and how our thinking and values and choices and decision making is influenced by other cultural organizational economic and social factors and settings and vice versa that even if we're solving a problem the public
needs us to solve the way in which we solve that problem can be better or worse based on how aware how reflexively aware we are okay so uh all right reflexivity involves awareness uh to exercise that awareness there are certain conditions qualities and capacities and there's a certain type of motivation that's required in the first place so when it results what does it result in well it results in integration for alignment okay because the the the more reflexive we can be during this process of Technology development a public interest articulation and the intertwined braiding of
them we're increasing the opportunities for integration and alignment the capacities for integration and alignment and the probabilities that we will have integration and Alignment so you can think of this as a quantitative argument that the more quantity of reflexivity you have the more probable you know is the outcome that you're going to have integration and Alignment okay but you can also think of it in a qualitative manner the deeper that reflexivity is the the more it can sort of stay open to to inputs that that might not have been aware of the deeper the integration
and the deeper the alignment can be between technology development and public interest articulation we're also and this is where the second part of the principle accountability comes from reflexivity enhances the conditions for accountability okay now most discussions of reflexivity that you'll encounter will not mention accountability and when I give you the first sort of basic definition of accountability you may not think of reflexivity in the first case right the standard way to think about accountability is that it is a way it's a mechanism to ensure responsibility it's synonymous with answerability blameworthiness liability and it tends
to carry with it an expectation or a demand or a requirement for account giving giving an account to an interested or disinterested party that says oh this is how I made my decision this is how I ran my team this is why I made the investment the way that I did okay so accountability if you're accountable to someone or to a an institution therefore you're more likely to do the right thing or to think more carefully about your actions okay that's that's fine but there's also a notion of accountability that is a little more informal
and it's what I like to think of as taking responsibility having regard for considering taking into account okay in decision making in daily professional practice high levels and low levels okay so I want to keep both of these ideas alive when we think about accountability as a pit principle okay because I really don't want to send the message that the pit principle of accountability means that we have to set up harsh punishments and sanctions for people who don't do the right thing we're going to come in and punish them and only then are we going
to get them to do the right thing there certainly is a place for coercion and enforcement okay but the simple fact is that it's often not going to work when we need it to work the most and so I think we want to redefine accountability to ourselves so that it has this idea of considering if I take things into account then I'm having accountability what does that even mean okay well in the first case it's easy to imagine what we might be accountable to right the public interest in the form of goals objectives criteria and
values that we or our organizations or our investors or clients have previously committed to and stated okay so by being more reflexively aware throughout the pit process we can increase our accountability to goals and objectives and commitments that we've already made we can also increase our accountability to others so stakeholders especially those who do not have a voice okay the more we know about these stakeholders more we can sort of become reflexively aware of them the more likely we are to be aware of our own ignorance and to seek out additional knowledge or to sort
of delay making a decision and maintain flexibility and then finally this is perhaps the most interesting way to close the tail and close the loop and have the snake bite the tail accountability to the self okay by being reflexively aware during the pit process we can be accountable to our own values and commitments both on a personal and professional level but also I should say both on a personal and a professional level now but the other meaning of accountability taking into account right this is the idea that we can be accountable by taking things into
account okay no one is necessarily going to tell us to do this or be able to Monitor and see if we're doing it or Force us to start being more you know taking things into account One Way versus the other this really has to be uh personal choice and personal habit and cultural uh ethos ways of doing things this is take things into account because that's how we are trained and that's how we have trained ourselves okay okay now there's two ways in which reflexivity helps us take things into account in ways that might increase
accountability one is that we might take more things into account or different things into account that we otherwise wouldn't have we might think about race or economic status or accessibility in a way that we we previously just not in a way but when we previously hadn't thought about those things we might now start thinking about them okay and that's encapsulated in this idea of plus respicora that's Latin for take more into account take more things into account and it's actually an ethical injunction that was formulated by this person Carl Mitchum who's an American philosopher of
technology and he is a professor at the Colorado School of Mines and also in the year for the European graduate school he's written a lot on philosophy of engineering and philosophy of technology and he came up with this idea plus respicura take more into account when you're developing and designing your technology okay so more things that you otherwise wouldn't or different things that you otherwise wouldn't there's also a way in which we can be more accountable and that is by taking things into account in a different way than we did before and that's my idea
of Midstream modulation and that's the idea that we're going to take things into account not as a sort of obligation that we have to spend time thinking about and then check the box then move back to what we were doing okay but we take things into account not by adding a new activity but simply by enhancing the activities that were already participating in as technology Developers so if if I use the example of ethics technology developers don't necessarily think about ethics when they're developing their technology okay for good reason because ethics is sort of this
science it's a it's a it's a it's a philosophical exercise it's a discipline you have to have an ethical framework you have to have the rules of reasoning you have to sort of think about the problem and the Dilemma and then and it's really quite you know moral philosophy is quite a cognitive uh exercise and so when you're developing technology you tend to just be thinking about or doing science or doing Innovation or doing public policy development you tend to be thinking about the task at hand so when we think about ethics we're also not
generally thinking about technology so when we do technology we're not thinking about ethics and when we think about ethics we're generally not doing technology okay Midstream modulation says no let's do and think them both together but then we have to sort of shift into a different mode of thinking and that's this tacit practitioner mode of thinking where we're moving instead of having slow deliberate thinking this is fast automatic thinking this is why I call it a cognitive disposition thank you so why isn't everybody reflexive all the time in a constantly expanding manner well there are
many barriers and challenges to reflexivity and being reflexively aware whether it's for pit purposes or just general problem solving and maybe the biggest is this idea of attention being Limited and that there are continual demands made on our attention tension is a limited resource and a person only attention is a resource and a person has only so much of it according to Matthew Crawford who wrote a book called The World beyond your head on becoming an individual in an age of distraction so Matthew Crawford is one of the people who's given rise to the idea
of an attention economy okay and that's this idea that uh there's only so much attention that goes around it's a limited resource and you know in order to have productivity and to get things done and to get businesses off the ground and maintaining and competing and and participating in the economy we have to really be aware of how attention is allocated but stress uh is one of the things that results from the attention economy which is ironic because then we have less energy for attention we put less of our energy into attention and we put
more of it into being aware or rather we put our attention not so much on the task at hand but on the things that we're worried about and especially if um managers uh directors sponsors CEOs Regulators people that were accountable to are telling us that they want us to pay attention to cost they want us to pay attention to time to markets then these are going to compete with our attention and they are often going to be the the thing that wins out right either because of stress or because of authority that we're beholden to
some of these competing demands are simply going to have rise to a higher level of priority than the others and of course as we've talked about already culture and training can set us in dispositions and reinforce those dispositions to only think about certain things in certain ways and of course our identity our very identity is at risk when we talk about cognitive and emotional disruption and our ability our capacity to handle discomfort and disruption and risk taking especially when the team is not necessarily ready to keep having another Criterion added to the table or another
consideration another what if question to come up they want to move forward and get the job done and here you are saying oh but wait should we think about this aspect of the user right you have to have a certain capacity to take a risk one of the cultural models that we've talked about before is the linear model of innovation that says you know just put money in put your head down do your job apply your expertise do what we paid you to do and then throw it over the wall and move forward don't stop
and Gunk off The Works don't ask these questions that we're not supposed to talk about getting ahead of myself getting ahead of myself when I teach you the stir method I um I'll tell you a story about that that I might not tell you then and that is that we are often told by scientists and Engineers who do the stir process you make us think about things we're not supposed to think about okay so being reflexively aware means thinking about things being aware of things that were normally not trained to think about now add to
that more uh barriers and challenges and that is that reflexivity can't be enforced or coerced you just can't make people think about certain things you can do all sorts of horrible things to them but their thoughts are the one thing that they have autonomy over and I believe that we shouldn't try to nudge and manipulate and sort of overly influence how people think we can do that through nudging but it's morally problematic and again it's very difficult and then the flip side of that is that too much reflexive awareness can be overwhelming and it can
shut people down and therefore it can be Proto it can be counterproductive okay we know this through the well-known phrase paralysis through analysis if you think about too many options either you've spent all your time thinking and you haven't done anything or you realize the magnitude of your decision and now you're afraid to act because you because of fear of failure if you're doing the wrong thing and also something called decision fatigue which gets us back to the idea of attention as a limited resource there's only so much willpower that we have in a day
and so if we're going to attend to things that might seem to be peripheral instead of central that might reduce our ability to really focus on the central things this again is something that will return to in stir but it's a very real um um condition a very real challenge so just a couple of tips I'm not worried about you guys as my students needing to have reflexive awareness I think you're already reflexively aware of my disposition and open to that this is more for you uh to take a leadership position with other team members
with Partners collaborators whether it's a formal or an informal leadership position so how to work in an organization where reflexivity is a limited quantity and there are a lot of factors that are working against it well first of all do pay attention to existing motivations and Investments because going back to John Dewey what people are invested in they're going to work to bring about and only when there's a problem that falls across their Pathway to getting that done only then are they going to engage in inquiry only then are the human become reflexively aware so
this is a resource that we have to work do not assume that you or a formal policy or a top-down demand or a great argument or a powerful emotional appeal can actually change somebody else's thinking and motivation that is actually ironic whoops that's ironic because reflexivity is something that's sort of voluntary and it occurs and so if you're trying to bring about reflexive awareness there's there's you know by by sort of being by forcing it um then you're not really respecting the the principle of reflexivity to begin with I put the stirrer icon up here
just because a lot of these tip all of these tips actually come out of my uh my experience with this dirt program so instead of doing those things listen to what people are saying and what they care about and what their problems are and look for opportunities for the reflexivity that already exists or that can be enhanced and grown including existing practices of problem solving okay these could be practical decisions problems these are also opportunities formal and informal assessments okay when it's time to write a report to stakeholders only then do we think oh gosh
did we think about this do we take that into account how much does it really cost how much is it going to weigh Etc right these are formal opportunities for assessment this is an opportunity to enhance reflexive awareness and there are informal assessment opportunities that are constantly arising and if you can attune yourself to them those are opportunities to say yeah what else should we be thinking about here okay other forms of expectations or practical requirements for people to be accountable to things or ideas or people these are our opportunities to support reflexivity so how
does it interact with the other pit principles well in brief reflexivity can be supported by anticipation if I'm anticipating unintended consequences that can enhance my reflexive awareness and if I run a stakeholder awareness Workshop or I talk I interview potential users as as several of you have talked about in the discussions this can enhance my reflexive awareness okay but we don't always have the luxury of anticipation or inclusiveness and so um reflexivity can support us to say you know maybe we should be more anticipatory here maybe we should learn more about inclusiveness maybe we should
talk to some stakeholders okay but when even that when reflexive awareness doesn't lead to anticipation and inclusiveness because we can't afford it or because it's just not on the table it's not an option well this is controversial but I'm going to say that reflexivity can substitute for anticipation and or inclusiveness it's better than nothing okay it it takes not just our experiences and our assumptions but are reflected experiences and assumptions and it says okay what do I know and if nothing else it says what do I not know let me discover my ignorance let me
discover my non-knowledge let me discover the unknown unknowns let me be put a category aside for that okay and then in the end of the day reflexivity can very much be a direct line to responsiveness because if you're aware of something and it's not something you want to have happen you might take action right away to change that here's a nice diagram from a couple Scholars frya and flipsa 2019 that appeared in the Journal of responsible Innovation and I can send you the uh reference if you're interested they look at a number of responsible Innovation
principles including transparency which for us is a luxury we don't have unless we're doing public policy development we won't have transparency so if we're doing private technology development we assume we won't have transparency about the technology but we can sort of see how for them uh inclusion supports reflexivity and reflexivity directly supports responsiveness but I would add to this that anticipation can also enhance reflexivity and that reflexivity can enhance inclusion can help us go get information talk to stakeholders and reflexivity can also lead to an enhancement support patient there you have it guys that's our
talk on reflexivity for accountability stay real human and totally awesome