2.5 Hour Symposium: Safety Culture, Lean, and DevOps

I'm so glad that we are finally doing this this has been a dream of John Wells and I for many many years so assembled today we have an incredible group of experts Sidney Decker professor at Griffith University in Brisbane where he founded the safety science Innovation Lab and is also honorary professor of psychology at the University of Queensland he's the author of many books on safety including his most recent book The safety anarchist dr. spear someone who's influenced my thinking more than anyone else I can think of senior lecturer at the MIT Sloan School of

Management dr. spear is a senior fellow for the Institute of Healthcare Improvement and was a four-time recipient of the Shingo prize for research excellence he is the author of the high velocity edge and has extended his research beyond manufacturing to research and development workplace Safety health care and many more dr. Richard cook he's a practicing physician research scientist at The Ohio State University Department of integrated systems engineering dr. Cook has been focusing on resilience and safety and is well known in the DevOps community for his chapter in the web operations books why complex systems fail

thank you so much for being here John Willis my longtime partner in crime John Willis is the vice president of DevOps and digital Practices at SJ technologies prior to SJ technologies he was the director of ecosystem development for docker which he joined after the company he co-founded socket Blaine was acquired in 2015 John was on the founding team behind the first DevOps days and as a co-author with me on the DevOps handbook Thank You Jean what an adventure we have been on for years thank you and part of that is this event today which I

think you and I have discussed as potentially Historic so to the illustrious panel here each one of you are associated with an incredible body of knowledge that you've created or at least contributed to creating could each one of you describe in a couple of minutes your journey dr. Decker my cell with you I've made a commitment and that is to write books faster than my fan base can keep up reading them and so that keeps me busy so and I seem to succeed at that or they just don't read It all but no I so

my mum thought it was good for me to study psychology at some point that seemed to fit and so I started with that then after psychology PhD and engineering after that I got professor got to be professor pretty early in life and and and that became sort of an existential crisis Jean isn't oh my don't have to do this you know another 34 years and and so then I I thought I joined the opes Community which had represented for me at least and and became an airline pilot on the side while I was Dean and

professors are flying about a day a week out of Copenhagen down to the Mediterranean in North Africa and actually uhm get to do the stuff that you talk about and so rather than dev I became ops and the the the fantastic advantage of that obviously is is that you're always you'd be the first to arrive at the scene of the accident Sorted sharpens the mind a little bit in the the having 189 lives in your hands while you're making these decisions while you have to create a safety climate or culture within your little operational unit

is much more real than just talking about it and so since then I have been seeing the things that we I think should talk about across a whole range of industries and and patterns repeat themselves and there's commonalities that I hope to discover it With the others here today that I think and can mean something for the wider communities and then just to be specific in a wide variety of industries these are dangerous oh sorry you know you're right yeah high consequence high consequence domains and so we're where people die if things go wrong and

lots of people can die a lot of innocent people can die and so of course in aviation we kill in big batches and so but that's true for petrochemicals the Mining and transportation general some health care although I'm certainly not the expert of the table for that but so so yeah excellent a good point dr. spear oh yeah hey so I got to graduate school as part of the generation that was worried about Japan it was late 1980s Japanese companies we're decimating American companies and the question was why if you look at the explanation back

in the well the early eighties it was cultural Attributes which I then quickly got dismissed but then it was technological people were looking for the secret robot the secret source of automation it might be secret tools and algorithms for production control to bring stability to otherwise chaotic systems and turned out that anything that someone could buy robot for example anyone else could buy and there was a whole wealth of research by some real luminaries a guy named Jay Jay Kumar Steve wheelwrights when those Other characters we said well you know the technology may be important

maybe necessary but it's not sufficient to explain the gaps in performance it really comes down to people and how people are embedded in and managing these very very complex dynamic systems for high performance and so the reason we went to look at Toyota's Toyota was one of these companies which was decimating the big three the explanation for that was at first cultural was a Japanese thing which in fact is actually not but was explained that way or it was technology you know lean production control tools when we got to Toyota we realized that they had

created an organization and a culture of tremendous scientific energy I think the first one of our early quotes with my mentor Kent bone we wrote that Toyota figured out how to create a community of scientists where they're constantly discovering and so you'd Asked um you know our transition over the years so I think when I started my work has probably the Toyota guy and then maybe for a brief period an auto guy and then an industry guy but really my career has converged down to the singularity of advocating for the scientific method that everything we

do represents our best understanding and our best understanding is imperfect and so what we have to do is expose our best understanding to ridicule quickly and Often to figure out where we're wrong so we can correct on that and just dwell in the past moment I think what caught so many people's attention was the Harvard Business Review paper the decoding of DNA the production system yeah based on your peach deduct oral thesis that was where he worked on the plant floor of a Tier one toe to supply over six months yeah that is correct yeah

yeah I mean I I reek woat that all the time I think It's my favorite line in that report is that you know it was a community of scientists continually experimenting yeah which is that's what Facebook does that's what Google does for today yes let me just add one more annotation on it because I think it's actually profoundly important because often times we think about science and knowledge work more generally as the domain of the elites whether it's you know in this country it's not by class but it's by Education and credentials or what zip

code you happen to live in and what Twitter realizes that you can if you manage properly you can harness the experimental exploratory instincts of everybody regardless of their pedigree and when you do that with all your people and your competitor is doing it with some 5% of their people you're bringing a much bigger team and a much better team so you're gonna win consistently Oh fantastic thank you dr. Spear dr. cook I have a misspent youth in the computer industry back in the days of punch cards and finite element models and things like that and

at some point along there I decided to torpedo my career and go into medicine so I couldn't I was trying to get out of what I thought was a fast-changing area with lots of innovation and new things that were required lots of learning and moved to a fairly slow-paced area where nothing changed very much and so I went Off to medical school I I've had a fairly productive 30-year relationship with David woods at Ohio State University also had influenced on Sydney and in the process we've been looking at a variety of issues related to how

systems fail and a better understanding of how systems fail initially promoted by or based upon studies of big technological failures like Three Mile Island but then branching out into a whole variety of Different areas and over my career I've looked at military systems and telecommunication systems and semiconductor wafer fabrication and a variety of healthcare settings and just a whole bunch of different industries and at the failures and all these different places sometimes very spectacular ones sometimes relatively minor ones beginning in the 1990s when patient safety became a popular term or a popular area of concern

it became Clear that there was a kind of disconnect between our vast capability to do things on the one hand and the fact that the things that we were doing were also causing injuries and harm on the other and the juxtaposition of those two things became a really profound source of of discontent and and and and disillusion and this generated a whole series of is Alishan from who your disillusion disillusioned from the among the population that wanted to be able to Use these services and have been told that they were safe a disillusion among the

theorists who couldn't understand how the system which seemed to be producing all these good things was also producing the bad ones disillusion among the regulators and and virtually all the parties and I think it's that it was the confluence of that sense of decease that generated the patient safety movement and I had the privilege of participating in some of the early stages of that Trying to bring in what we'd already known about how systems fail how complex systems fail in the past I'm also a practitioner I'm an anesthesiologist I put people to sleep and wake

them up and I've been doing that for going on 30 years it's been a very humbling experience to be in contact with this in this way we share the kinds of experiences that city talked about and that the people here at this conference what Be experiencing that as many of them you know not only talk about the work but actually do the work and experience what that's why and I think it's a very important perspective to have because you understand what the real what's really at risk what's really matters and also gives you a big

appreciation for how uncertain and how ill-behaved the world very often is right now my focus is on trying to understand where the resilience and systems comes from and We've come sort of full circle from trying to focus on failures to trying to understand why there aren't so many failures the thing I'm pretty well known for saying is that the surprising thing is not that your systems sometimes fail it's surprising that they ever work at all and that's really true given the levels of complexity and their piece of change the kinds of varieties of challenges that

they encounter it is surprising that the systems that we have Work and we're trying to understand more about that and as we do we're developing relationships with a variety of different groups there's a collection that I call a snafu catchers which is a consortium of companies and people interested in trying to understand this and we've been working on trying to expand that we could talk a little more about what the sources of resilience are but I think that the key thing is that we find them here in the DevOps world And we can identify those

and bring those forward and there's much in common and there's big resonance between what we find there and what we find across other industries do you know what's your quote about Murphy's Law uh yeah this is important one obviously that you know Murphy's Law of use he says that which can go wrong will go wrong well that's actually not the case and it backs off exactly on what Richard says that which can go wrong usually goes right it's Actually the latter issues at this but and that's very true that which can go wrong usually goes

right and then the risk is that we draw the wrong conclusion yes so yeah yeah yeah each one of you for better or for worse it has been made incredible contributions to your respective fields and I suspect that when people think about you they think about what the work that you did maybe ten years ago as opposed to work that you're doing now so could each one Of you take an opportunity to say may our specific Corrections you might want to broadcast out to the universe in terms of people know me for X but I'd

rather be known for Y dr. cook if applicable I'm not sure many people know me so I don't think I feel like I have to have to respond to that well you did make it has been said it's I'm convinced that that what matters now is trying to understand how it is that people are able to make the systems that we have Work and understanding that so both a deep problem and a very interesting one and given the kinds of challenges that we're facing and we're likely to face it's critical work that has to be

done now and we are just beginning that process and that's what we're focused on and you know the reason why if applicable many of in our community we see the work that you're doing with dr. woods with the velocity community talking about you Know specific application of this body of work to you know what we do every day so in many ways oh my I think I understand is actually very contemporary not just the crazy doctor showing up at velocity conference dr. spear I suspect many people just know you as the Toyota Production systems person

the HP article from 1999 so how about you what is the correction that you would make yeah the corrections so um I think people should think about two things feedback and God Perfectly and here's what I mean says uh you know man plans God laughs and the reason man plans is he tries to bring his best understanding to situations because in the absence of planning they were pretty much guaranteeing failure God laughs because we bring in perfect understanding and it turns out God or mother nature of the universe gets a veto on everything we do

so I think it's back to the feedback which is if we start with the assumption that Everything we plan is flawed then what we really should be doing is seeking aggressively energetically relentlessly if you back as early as possible we don't do that in fact as we plan we start making all sorts of social capital emotional investments in the the accuracy of the plan and when we start doing that we start rejecting the possibility of feedback and once we start rejecting the possibility of feedback we forget again And when we put the plan at the

practice God will laugh at us the universe will have its veto so I'll answer the question in it but there's there's multiple things that need that oh yeah yeah but just to finish your your your um answer your question so when I first started this work is very shopfloor operations focused and you know because we had gone in to try and understand why Toyota after all that study had been so poorly and imitated and we realized that In these very complex dynamic operations they had infused an enormous amount of feedback to trigger self correction and

self improvement and I think what we've done and this is managed to generalize from that is not just the shop floors no matter where you do your work you have to solicit feedback quickly because the universe will get into video someday and doctor just to make one concrete point yeah we're saying is that it is not for just a domain of highly repetitive Mechanistic work of people working with the hands it is applicable to people who work with them yes so actually gene your your point is a great one is that I think when people

looked at Toyota for an example they look at the end which is cars coming off the line more you know not more often than not almost always defect-free compliant with expectations etc etc and they say oh this must be applicable to repetitive work which it is but then they make the false Conclusion it's applicable only to repetitive work we tilt your owners famous quote you know it's my own narrative but they he let the American cars come in and they asked him why why do you let them come watch how we do things right and

I in you know my own kind of narrative is he kind of Snickers and says they can copy our process but they'll never understand our culture yeah that's right so so you know connecting those two points right which Is by the time you actually have cars coming off an assembly line every 50 seconds you know almost always compliant right you've gone through this whole set of processes of designing the car designing the system designing the logistics system actually putting building those things physically training people resourcing equipment and when that first car comes off the line

let alone the thousands your multiple years and billions of dollars in your Investment of thinking and what most people do is don't expose their thinking you know two three four five years in advance of that first car so then maybe they'll they'll say oh what we need to do is put feedback on the system at the very very end but by then you've already baked in failure or guaranteed success I had a boss who said you raced cars he says when the race car crashes into the wall and blows up into flames he said the

mistake wasn't made in that turn it Was by made seven eight turns before right that predestined or in designing a car the good burst into flames [Laughter] mixed metaphors here but I just a little aside I think we had joked that the benchmark for this event was the famous debate about morality by Michel Foucault and know'm Chomsky I I think we have them beat I'm having so much fun how about you there's Michel Foucault in the question that you asked Yeah I said we should forgot paraphrase and translate which is which is probably helpful that

a man cannot be man sorry a person cannot be held to something he or she said ten years ago can a man not change his mind about they just did so I can't exactly so but before I say anything I mean I think so so Steve these are these are gorgeous points but I beg to differ on various philosophical practical grounds because so first of all reading external agency or some Conspiracy by god or mother nature or the universe into the adversarial things that we that we that we encounter is is hugely infantile right they

might think and and I think people let's be clear are you all right absolutely but um but that doesn't matter that's we're here to talk it's about the top right we have no ego in abstraction and if we have egos in this then it's not gonna be good debate of course it's Right I'm Dutch I'm direct you know there's no anyway so but the point is I think it is misleading if we want to do something about this and try to understand why are the sources of failure and success lie is to say well you

know we're imperfect we try our best but then there's this external force that somehow screws us up it's the source of failure and success is both in ourselves in our own complex systems and so and then and then to suggest that we Ignore feedback I mean that is entirely inconsistent with all the evidence that I sit on but as soon as we deploy something we seek all kinds of feedback but we count only what we can count we don't count what counts and so the feedback that we do solicit is not helpful to us understanding

how the thing we're gonna go wrong and so finally I mean this debate was was done like like 250 years ago me Voltaire right Voltaire 1755 out of Lisbon Earthquake essentially ridiculed the whole notion that the best laid plan plan of mice and men go to go to crap because of some external agency I don't think we need to have that again so I just question you but but I'm going to take a crack at it I don't I think he means God in a metaphor sentence no but I think let's not put it a

label on it that's all it's external to us right because that is the illusion that gets us to think completely wrong about these Things we are the source of both success and failure and yes a Sydney I think I think my attempted metaphor was not as clear as it needed there you go language matters matters what I what I mean to say is the universe operates the way the universe operates you put a forward we're part of it if we're not we put forces on masses and they predictably accelerate in proportion to both the mass

and the force and their other rules written into the fabric of the natural Universe we are part of that what our understanding of it is grossly imperfect and if you look at anything that fails unless it was some intended and designed by a sociopath it fails not due to intent our own intent to do stew or our lack of understanding then when we sit here today and we're whether were five minutes five hours or five years out from the actual event were characterized by an enormous amount of ignorance about what actually will Transpire so but

the and so I have to throw some big words on the table but the the Newtonian Cartesian ontology that this is right that there is such a thing as a possibility of perfect understanding is deeply misleading there isn't in a complex system there is no such thing it is all imperfect you only know right and so I think I think of it as there's a true not and if I'm always pointing nonsense nonsense that's a bit it doesn't matter you're constantly Approving to get there they're using scientific thinking so you're always course correcting it's like

like the optimist and just me just to put it worse I haven't questioned yet by the way yeah it's irrelevant or fault well no no it was Steve's fault because I needed to have one thing I mean I think we will agree that entropy exists that I mean without even getting into the quantum mechanics on certain launches no that Would suggest that there is these four complexity I just don't see them as I think contradictory so let me let me see if I can find some something that adds some clarity to this and I hope

this is an attempt I'm not sure it's going to work a seventies point about complexity I think is an important one which is that we are dealing with complex systems and therefore they both we both take action on them and they take action on us we Are not it's not a deterministic world we cannot create a system we can influence different things we can try different approaches but there are responses in the world that changed the way those things play out and alter the kinds of pathways so we are not in a position where it

is simply our intent but our ability to understand both the way that our intentions are going to play out in the short run but also the kind of responsiveness that the system Has and how it's going to going to react to that there the bottom line here is that this is about adaptive systems and people are adapting and the they are adapting in this world to the various kinds of changes that are going on and when we trace those patterns of adaptation we discover various kinds of patterns that we can sometimes deploy I think that

it says it is a mistake for us to become too focused on a kind of sort of Philosophy of how this works because I think the danger is that we begin to think about making progress by by in a kind of semantic way right I think one of my cores of why I wanted to do this because I think we had a discussion will you you thought that lien was a deterministic model and I was not by the what I've read from dr. spear not what I've made from right Mike Rother my Crothers toward O'Connor

and so it is the disconnect here that you don't like The God or true north but its direction okay because I think and please help me if I'm wrong is the the motto from the two favorite authors in lean you and my co-author and I think you say the same exact thing which is it is a non deterministic model and there's some direction Mike calls it a true north a challenge and it's a direction that he constantly he says this is direction this is my current state and hold off before you yeah because I'm Gonna

I'm gonna get the part that the challenges and certainly give me a minute yeah he says there's a current condition and then there's a target condition and then I go into the gray zone and the gray zone is the scientific method of constantly proving and just course correction to get to this direction I want to get the single piece flow I may never get there but if I'm always using scientific method and I embrace uncertainty to get there which Is scientific thinking then I'm working my way toward you don't get to talk all right I

mean I want to hear dr. sphere here yeah yeah might so yeah and it's not deterministic I mean we are dealing with systems which are complex and dynamic they keep changing and they keep changing I guess in an adaptive fashion in that they never return to an initial state um that no matter what you do it's always a change state so there's a layperson when I first heard the term Homeostasis you know consistent state said oh and implies that there's some sort of balance in biological systems and once you get to that balance it just

stays there one of things that's been impressed upon me of course is that there's never balance that you exert yourself your body adapts you eat something your body adapts and it gives the appearance of balance because of all these adaptive mechanisms going on Biologically one of the things that's been impressed upon me even further is that these adaptive mechanisms happen at the the large scale in terms of release of hormones and that sort of thing but an even smaller scale adaptation within cells and even adaptation at the level of genetics and so it's this idea

that um you know your state at time T is your starting point for T plus delta-t but no matter what happens at T plus delta-t even if you leave the situation alone or Act on it you never go back to the state of T you go to the state of T plus delta T plus another delta T and so again to your point it's not deterministic actually you're constantly searching now that's the biological system but the system's you embedded yourself an aircraft the aircraft don't go up and someone deterministically designs an aircraft it depends on

a tremendous amount of adaptive control to keep a plane going where the pilot wants and it Has not complex it's only complicated well I think so I think I think youyou you're quibbling over some of the semantics the issue is the plane will not go where you intend unless you have feedback at multiple nested levels you have it at the individual components and components within systems meaning that you as a pilot have to make some critical decisions but far fewer than the decisions that are being looped into the system itself so that's where I said

With feedback and again the reason for feedback being so critical for people making decisions is the commitment we make today is built on a tremendous amount of ignorance about what's gonna happen and just one last thought cuz you know if I'm gonna pull in fields here I mean the reason those fellows got Nobel Prizes for option theory is because they lie quantified right but you know that they they quantified and Merton also don't forget My T guys right but um the reason they got Nobel Prizes for that is they said they acknowledge what we all

know which is what's gonna happen in the future is unknown today to cert varying extents and there's consequently value to have be able to make a judge not a judgement a decision tomorrow when I'm closer to what's actually tomorrow is going to be like and what would they were able to do was quantify the value of that delay on the decision so again it's another way Of saying in the moment I am a characterized by and played by an enormous amount of ignorance about what's gonna happen there and the only way to get closer and

closer to what I intend to happen there is to make sure that I'm open to feedback at very quick cycle times frequently in slower cycle times less frequently but in the absence of feedback for sure whether it's biologically or technological I will crash in the absence of correct and I'm Gonna make an observation there's a technique calling going to the balcony and looking at the dance floor I would just look to give ourselves a permission if there are no objections so while here for a little bit because I think it would be a great goal

to be able to say all right I think we have some view of the universe you know that that the universe is uncertain complex I just want to make that about plane ten feet paint as a intermediate goal let me Talk about does that work for you yeah I think I think that the problem of course is that when we say things are uncertain or we say they're complex we are not our heads but we don't have a complete agreement about what all the implications of those things are or how they play out and what

that implies for how we should proceed that sounds like semantics to me right well maybe it might not but but but but let me let me back up here and tell you What motivates some of the work that we do and why I think it plays in the devops world what we find is that when people design systems and build systems in a variety of different settings their understandings are as you point out imperfect or incomplete in a variety of ways but their goals are usually pretty straightforward and the goals that they pursue are the

ones that help shape the kinds of structures that they build and in many cases what they build our Structures that achieve some aspects some component of that sort of stuff but miss represent or miss characterize or underestimate the potentials for various kinds of failures in a variety of different ways and so we end up with a system that's well designed to produce something and we focused on that something and yet we may not be at the same time building a system that is for instance sustainable over a long period of time or capable of weathering

of Various serious kinds of storms or that does not produce some epiphenomena some other things that are actually damaging or problematic and as a consequence we end up discovering those things later on we we build systems and we try to make them as as robust as we think we need to but but then our experience with these sinks begins to demonstrate that we are miscalibrated and we can take Air France 447 or 47 we can take Three Mile Island we can take a variety of failures night Capital and a variety of others as examples of

where we've we've really misunderstood the implications of what it is that we are building in fundamental ways that prove that produce new paths to failure that we did not really comprehend at the time that we're build you know it is interesting thread here because it is what we have to manage and what we create I would say the most relevant piece of DevOps has been lean because lean has more Influence my I think your stuff on complexity and on how we operate and how we deal with complexity once we have this you know a hundred

thousand or million servers but the most dominant model that is influenced everything I've been doing in this space since I've been doing it is the kind of single piece flow Eric Ries is Lean Startup which comes directly from there about how do we deliver things and a lot of that is built on the scientific Thinking of I'm going to build I want to create there was an early story of a company that I worked with that I was they had this thing about where to get the bars and he wrote this web interface and they

filed Eric Ries Lean Startup model and what they found was that didn't work it actually they became this thing for we're happy hours were right so they cost corrected and they but physical the goal was to be like a iphone application for where to find a Cool bar and all that and what they found was there - minimal Viable Product and kind of sign really just Lean Thinking was it got got them to a place where there where they were really making the money was you know identifying where all the you know happy hours or

so and that that model of lean is just influenced the stuff you work on when you have clumped like a night capital is so complex so the trading firm small configuration error caused Eight serve I mean I'll get yelled at like yeah but eight servers this system admit and then missed the eight server but we know that's not that wasn't the reason let me let me actually pause I mean so I thought what you submitted yeah as a characterization of the world as it exists full of uncertainty incorrect assumptions whether we build something or run

something can we all agree that that's a fine place to be and to mechanics do we all accept that Doesn't seem anything in controversy there's nothing controversial about know they only I think the only nuance that I would like to add and I don't think rich would disagree would be that once we discover that our previous models are no longer valid because we are now discovering that through a particular failure these systems create pathways that we haven't foreseen once we've made that discovery I think the only thing that I would like to call for is

equal Human ëthey because it doesn't mean that we are now closer to some to some ground truth of full knowledge and perfect knowledge to use to you Steve's word of how the system actually does work because that will be within complex systems will be forever out of our reach right nor does it mean that that pathway will repeat itself it could be possible do we all agree on a statement or with quail who have are there any qualifications or can we declare that we Have we're all in agreement as that as a frame if you

can't agree with that it's not a complex system theory is pretty clear on that yeah yeah I think we're always plagued by ignorance and the sooner we find that out the better just because we learn something doesn't mean we learn something that we're getting towards some an end point where eventually we will acquire complete perfect now again all right let me let me take that one step further now and so This connects to the to the lean idea and I and I want to I want to propose an idea which would might might link these

things together in a way that would make them at least congruent the the notion of lean depends very much on the idea of value mapping the the ability to understand what is valuable is crucial because it sets the guidance for what we are trying to pursue that a reasonable statement so it's a reasonable statement based on the reading of the lean Literature it's an inaccurate and this is not you this is the literature it's an intact and inaccurate understanding of what Toyota created I can't possibly talk about Toyota but I could but we used lean

a lot in the US and and so let me just add let me just say that I think the problem excuse me the problem that we arrive at very often is the issue of what is value and how do we define value and and what we and the thing that I'm concerned about Irrespective of the underlying theory about how things ought to work is that in application very often what this is is a is a means of essentially eliminating everything that does not add a bottom-line why don't we start by having Sydney you educate us

on safety culture and then Richard I'll ask you to expound reinforce or maybe even paint how your your perspective is a little bit different so Sydney why you educate us well educate is a that puts the bar Pretty high I suppose but what we've learned over the last 2025 years is that a safety culture is a culture that allows the boss to hear bad news it is that time he's a huge literature but most of it is wrong and some of it is useless but but that remark does cut across a lot of the data

a culture that allows the boss to hear bad news now there's two things that that we need to look at then how do you actually create possibilities for the boss to hear bad News good news any news that has everything to do with leadership and creating a climate of psychological safety where people feel safe to speak up and won't lose their face or reputation for suggesting that things are not as they may have seemed and that is actually and the evidence is quite clear and this is that's actually the easy part it's not easy it's

certainly not easy but it's the easy part compared to Knowing and deciding what is bad news and this goes back to some of the discussions of a feedback for example that we've already had what counts as bad news right and and and so the issue that we see is that people in complex dynamic socio technical systems or a nice isn't really normalize right if you get certain results and you get them over and over and over you normalize to them and in some sense the it becomes very difficult to understand that in That normalization what

counts as bad me I've just that well and it's determining is a normalization of deviance that the that yeah but it's absolutely normalization of deviance is the word deviance because it is no longer seen as deviance otherwise why would you normalize right and so only from the outside in hindsight is it easy to that's all that was deviant you guys normalize it you are bad we are good right we somehow have the gods our view Over this thing which is why I get sort of upset when we when we make that frame to all there's

always someone who knows better well guess what there isn't not in complexity right and so but normalization so what is bad news what counts as bad news and the only thing you can do as far as we know is to keep a discussion about risk a life invite in different perspectives get people from different backgrounds today is that Normal right and so and then you have a discussion about it yeah this is normal what do you do well we do this oh really that sounds really risky to us right and then out of that discussion

you can help calibrate what you will see is bad news um John I almost we had this great conversation over lunch about you know at Google just how much research they did to try to figure out what makes great teams Great died you want to summarize there yeah I mean it was you know from reading it was just incredible on research from you know they don't almost every engineering at Google they engineer an anthropologist they they just went to every domain and you know the expected simplicity answer was psychological safety I mean there was a

longer version of it but was like just created an environment of psychological safety allow the junior person has been there Day one with the senior person who's been there like forty thousand years the usual is I don't think that works company because you know it goes because an IT like it's just it has always been this horrible like why is that person talking right now I have been running this system for thirty years you should not be allowed to talk for another eight years but but in it but Google wanted to make sure that anybody

could raise their hand at any time and say and in fact it Was encouraged in fact if everybody agreed there was a problem thing is Johnny and then out then we can leave this because it's that's that's dealt with but we quickly get lost in in sort of elevating or legitimating the voice from below and that's really that's really nice and crew resource management does that and you know hospitals are full of initiatives to try to make that happen but I think it overlooks the fact that if the boss the one at the top of

The pecking order right of the of the of the tree doesn't feel safe to lose face and say actually I don't know I mean the climate of psychological safety applies to then a whole night of the lower lows I mean again we were having this conversation at lunch like is there a perfect ideal anyway no never but if the if the model is moving towards where everybody's least trying to be and from top to bottom again I'll throw my toilet and my understanding was everybody had a Mentor to your relationship right across the board right

so it creates a psychological safety in the ideal and nobody has any idea but you try to move towards a place and that was what Google was doing is Google perfect absolutely not in fact some of the recent stuff that's happened there's been horrific but are they trying to spend incredible amount of energy in learning you know they believe that it's somehow immersing Psychological safety throughout the organization so Richard why you react to Sydney's canonical definition of Industrial Safety yeah I mean this is a topic that's been that people have talked about and done research

on and there's a I mean they're journals on the thing and so I can't possibly do it justice and in you know in a single phrase when we watch people who are working in situations where there's hazard around clearly one of the Problems that they have is that they are under pressure to do things in spite of the fact that that hazard is present and they're I seem always trying to make a distinction or a decision about whether or not the hazard is close enough large enough and potent enough significant enough to warrant that they

change the behavior that they are doing in some particular way and in most cases what we find is that after accidents after things that come to attention big events And you could take big oil platform burns and you could take care plane crashes or you could take catastrophic outcomes in hospitals is that after accidents everybody will agree that we were miscalibrated about where we should be relative to production that we should have backed off of production and we should have been paying more attention to the hazards that were around the problem is that that it's

clear in retrospect it's not so clear and Prospect we understand when the outcome is bad that there may have been opportunities to avoid that outcome but we over rate or put too much emphasis on those opportunities and how salient they should have been this is called hindsight bias and it's a major problem that we have the problem is to do this looking forward and it's even more difficult because in most of the world's especially in the world of DevOps there's so much change going on That the pattern of hazard and risk that actually exists out

there is not well understood and shifts and changes shifting and changing all the time and and so to remain apprised of that and to be dealing with it in a thoughtful way relative to where we are in our production demands is a constant problem and that what safety is is is how people work that problem out that's what safety is safety is how people work out the difference between Where they understand the hazards and and and risks lie and the kinds of things that are driving them to encroach on those boundaries and city in your

uh mmm DevOps days presentation you actually talked about you had just horrific scenes of people falling into you know equipment you know resource extraction you know game you know killed and right I mean this is these are the just one example of when you talk about like you know the real risk of death and My caption that accurately yeah but what is fascinating if we go on what Richard said is that the sheer presence and clear presence of that risk or hazard where the difference between living and dying can be half a second and literally

I've done investigations where that is exactly the case doesn't necessarily sensitize people teams organizations to to to it and so there's no privilege that they have the devil's doesn't happen oh yeah in fact there's even a Tory mmm-hmm I think it's true even to a serve like that the way the person died was at as part of a routine maintenance he had to crawl into the Machine and was horrible and resulting his death I mean that's just hints at the crawls into the machine every day made it work but there's everyday there's a special bit

there's the problem is acute in DevOps in ways that are I think quite different than the example of the machinery and that is it's exceptionally hard in our World to know what a small change is they're all small and we live in a world at the in the IT setting where where the change where where we have lot change management we're very good at talking about this if the change is going to have this big impact we have to go through all these different rules as I said but but very often when things fail it's

not that kind of change that people are doing they're looking at saying I was just following the standard Procedure here in fact you'll read that in some of the accident reports they say people were using a well-established procedure negotiation tonight and it goes boom in the night and so one of the difficulties that we have I think that is a real serious problem in the DevOps world is there's the calibration about where those risks and hazard actually lie and how they are changing in fact I'm just there's a couple areas that I just would like

to explore before go to Steve but you know what you just talked about was I do that if I understand correctly these are the characteristics of complex and tightly coupled systems right small changes where the failure patterns are just hard to predict and also of systems that have incredible brittleness and by their design and so we have circumstances where we accumulate hazards that I could sometimes call it dark debt we accumulate this dark debt over long Periods of time when when when the bash shell was written it was an exercise of a weekend and trying

to put together something that would help me do some particular task but now it's integrated into just about every step of every procedure that we use it turns out that the bash shell has a vulnerability in it that nobody realized at the time nobody would have thought of exploiting that vulnerability nobody would have looked for that vulnerability does it come back To bite us in spades and the maybe is another concrete example the British Airways failure right something went down and in the power of procedures they knocked out the most critical business operations of the

organization for three days stranding 70,000 passengers all across the UK and the CEO of the International airline group said it's not a power failure it's not a technology failure I mean it was a it was a power failure Any rational explanation this would say this is a the most profound business failure right this is a maybe I should say failure an accident and and if we look at if we look at what's happening it seems to me that it's clear that we are and we are becoming ever more dependent upon systems linking them up in

new and inventive ways and take advantage of opportunities and in so doing creating an ever more brittle and more difficult To assess infrastructure such that when these events occur they are astonishing surprise taunting surprises they people don't say oh I knew that was going to happen people go what it did what somebody must have powdered up I worked with configuration management for many years some of the tools that you use to configure infrastructure and the the rule of thumb was that you monitored an infrastructure maybe every 30 minutes we used the tool once at a

large one of The largest banks in the world then we turned it on to about ten seconds and what they thought they had from a figuration management was a model there was a centralized model how they thought every machine looked ever every 30 minutes but in their mind at any given time everybody said this is what the system looks like we turned it down to even was less than 10 seconds we found out beyond that there were a billion Unchained unplanned changes a Day that nobody knew about mm-hmm right and a billion this is a

lot one of the largest banks right and so like it's not a question whether you write it brittle or you crack it brittle it is just brittle and then you know I don't I know a little about the butterfly effect but a billion unplanned changes a day or it's gonna create you you couldn't even calculate what that cost in our pecks cost you were yeah so one of the mistakes we make is the all right so Where do you get whether it's resilience or agility there's a dynamic piece the feedback piece which is I'm tracking

differently than I thought let me find out in ten seconds rather than thirty minutes right so that's the feedback there's a structural piece to which often gets overlooked which is our brains ability to understand the complexity of the thing in front of us is pretty limited and if we a couple everything to everything else our Ability to anticipate and visualize the dynamics of those things but even when they're subject to conditions we hadn't anticipated that's pretty much impossible so we're always going to be shocked and surprised so there's a structural approach to that which is

tight boundaries and nesting which is and so whether it's a person or a technology have a qualification with a clear definition around what the operating boundaries are where that Qualification actually means something and then to the feedback part on there two opportunities one is feedback on is the person behaving in a way that appears to be qualified and if they're not then we should investigate why but also are they operating inside at the edge or across one of those boundaries because if they're operating across one of those boundaries we haven't qualified them there that that's

that's beyond the frontier from what we know and Understand to what we don't know and don't understand and your point about this piece of software I wrote it for a purpose and actually probably qualified it pretty well it's only for that purpose that looks it's working is doing this thing is very narrow an application doesn't affect my oh let's use it everywhere whenever qualified it for everywhere qualified it for here yeah right and and and so I think it's just you know we I think we referred to Before which is uh what do you need

to do which is I think it's this notion of constantly be paranoid that you're venturing to a place which you haven't yet explored and consequently whether you into a space or into conditions whatever else it is which is why you constantly then you know you want to be in a ten second world from feedback and not a 30-minute world for feedback because you want to know sooner than later That you ventured into a territory which is poorly chemically that that was just an exercise for discovery because it wasn't actually the technology couldn't support that I

don't know but think about but the general principle right so you know and again the president been a practicing physician so I'll be careful on this but would you rather do you know check your blood sugar is once a day once an hour once a minute or just have your body do it you know and it has sort Of continuous fashion and I think most of us would prefer to have a continuous fashion because it means we can have a couple of donuts for breakfast and not worry about going into shock if we're checking it

once a day we have to have a very very programmed existence in tightly bound in circumstances because if we violate any of that we've run into real trouble but let me give you because it is a example one of the big problems is that As you try to achieve tighter and tighter control to - in order to have the benefits say better control of your blood sugar you be you begin to discover characteristics of systems that now you didn't previously know about so we know for example that pursuit of tight control of glucose has associated

with it a whole bunch of side effects that are actually very dangerous and and we are struggling with trying to figure out exactly how often we should do this the I think that the the the ability of our technology to carry us into regions that we didn't expect would be problematic is greater now than ever has been in the past without a doubt and I'm gonna just give myself permission to put a question out there and if things go wrong I'm just trying to withdraw it yeah but the one thing that John you reminded me

of is that you know which is a hindsight bias I mean did this thing with that what used to be I didn't think somewhat Obscure at least uh from my perspective I mean it's come to the forefront you have Thinking Fast and Slow being a New York Times best seller you know just how that's been integrated into so many disparate domains is that consistent with kind of the view of bias in the safety community or is that inconsistent no I think it's largely inconsistent I think that that the emphasis on bias right now is a

social reaction to a variety of social problems trying to Lodge them not in in the usual causes but in other ones and I think this is way way overblown I I'm very unhappy with the way that that discussion is gone and I think particularly the idea that we can unbias or D bias people through various kinds of things is is almost nonsense the the the immediate need jerk that I have is that I am always suspicious of binary systems people say oh there is this and then there is that Hang on first of all is

it binary to begin with what about everything in between right and so when it comes to human cognition simplifying it down like that and then thinking that there is a perfect world which is unbiased relative to which we are somehow deficient I think sets is on the completely wrong discussion and yeah my reading of that is that those guys didn't necessarily write intentionally prescriptively initially It was descriptive they look at all the ways in which are you know even with the best intentions our mind will lead us down in a bad direction in my take

on Kanaan aaron was that it was that he just wanted you to understand whether you were doing system which was quick thinking yeah and not not that they were saying there was a binary approach again this I think is the same theme all the time is that all he was trying to say was make sure your system to type Thinking was that you were breaking down something as opposed to just jumping to it yeah but Sydney's point is a good one right which is those guys invested a huge amount of time documenting all the ways

in which we act with confidence when we really shouldn't have come yeah really should be walking real slow right now but we don't and they actually explain why we might have developed those mechanisms for proceeding with confidence and speed when in fact we Shouldn't again you know one of the big concerns is that anytime something ends up in the hands of management then it becomes a transaction I'm gonna buy this and we can talk later about you know the I'm gonna transact them by lean tools or whatever else it is but it's almost like people

said I'm gonna transact I'm gonna buy behavioral economics which if they actually understood it would realize it's not for sale when it's not for sale but if you're making a quick decision to Buy it you're beautiful so Sydney and Richard thank you for teaching us everything that we need to know about Industrial Safety yep Steve can you teach us about lean maybe the common wisdom and maybe we're lean might have led us astray yeah sure so um let me do that slightly out of water more chronological because I think there was a sort of a

peak and understanding about where the ability to deliver more value to society with less effort originated And then I think there was somewhat a valuing or you know bottoming out on that and then what the peak should be so if you go back to the 1970s right so that was an era of existential crisis for manufacturing and there was this advocacy for getting advanced technology to deal with that existential crisis and now you know our technological solutions are in the form of AI and machine learning but back then it was computer numerical control machines and

Automation this thing and that thing and there was big investment in that and then scholars started to notice that people are buying exactly the same stuff but getting hugely different returns from it so I had a colleague JJ Kumar who did a study of numerically controlled machining and found that people buying the same gear but that on any given day the folks who used it well had 40 times 40 X the product variety and three times the productivity of People buying exactly the same equipment and then some other guys who I am pleased to consider

you know colleagues and mentors so as Clarke Kaizen will write wrote a book called dynamic manufacturing and what they were trying to point out is same story line which was if you control for the product technology and the process technology there are these extraordinary differences in the ability effectiveness of certain organizations to deliver Societal value versus others and the dynamic piece was that they were talking about a learning dynamic all right so that's going on in and this is coming out of you know these folks are at Harvard Business School and over at MIT you

get this guy John Kraft SiC who starts documenting the difference in effectiveness so John when he was a master student I think now he's he runs Honda America North America he does the documentation to say to realize that Apples to apples comparison you have most auto final assembly plants with the same number of people in the same amount of space excetera etc making more or less him amount of product every day but there was this small number on mostly Toyota but not all Toyota which on any given day half the people were doing twice the

work and this is actually the origin of the term lean John was trying to come up with a title for his master's thesis and his first article And so it was a natural if Henry Ford was the Godfather of mass production and these guys were still producing but they were producing with far less effort much more than if Henry Ford was the originator of mass production and mass production was what was being practiced at Ford General Motors Chrysler and elsewhere then this was lean in comparison so the term lean originally was descriptive of the phenomenon

less in more out and around the same Time a guy similar timing it i might he was doing his doctorate so he wrote a dissertation on a thesis this guide john paul McDuffie who's at Penn he did another set of studies I said you know this phenomenon was witchcraft sake documented it's about learning and his first in a head and really a seminal article was comparing problem solving in different plants it was the same problem in different plans and it gets back to this issue of feedback and learning Which was in one extreme people would

take reports and you know they at some point processed the reports and they take every presentation of the same symptom as having the same sources is the same problem and the other extreme was the the plant which every time there was let's say a a leak around the gasket on a window they treated each leak individually as having his own idiosyncratic cause and necessitating his own idiosyncratic investigation not Surprisingly the folks who were learning a lot more a lot fast or their performance was in this category of half and twice out so if you think

about the time frame forgive me yeah the which one had better performance the one that treated each leaky thing each probably didn't treat until you proven otherwise treat each problem as idiosyncratic and then if it turns out they're not that's fine but the other one started with the preconception bias maybe maybe not but Start with the preconception that every time there's a leak it must be it must be a leak problem as opposed to lots of problems which manifest themselves as leaks so so anyway that puts you late 1980s and so what's the story between

craft stick and McDuffie which is there's wildly wild divergences in performance and the root cause of that is the learning dynamic of the organization which again is one of the few times MIT and Harvard agreed Because the folks at Harvard are writing dynamic manufacturing dynamic alright so that that's definitely a peak so then what happens people start studying these lean manufacturers and quickly the focus goes on to the shop floor and the things you can see so what can you see on the shop floor you can see oh they have continuous flow they have value

streams they have just-in-time pole systems Kanban cards standard work for s you know which is the placement of things Where you actually need them and II start seeing this identification of tools for the purpose of stability taking it to a ridiculous degree so you know if you've had the experience you go into facilities and say oh we're very lean and you see people have actually put masking tape on their desk as to where the coffee cup will go and where their pen goes and their stapler you know and that kind of thing that's like a

logically diagnosable by the way but You'll see that extreme it of course what that meant was they viewed this as a set of transactional decisions to buy tools in order to get stability but you know to the continuous reference to complexity and the dynamic nature of systems stability is an illusion there never is stability there's always adaptation and so now they're taking any sort of sort thing about it you have these very very complex technologies which are inherently unstable which is Why we have instrumentation and control and you have those existing and ecosystems which are

also inherently unstable they're constantly changing now you're trying to take an organization put on top of that to manage this complex dynamic environment a highly structured rigorous compliance audited system and not surprising there's a gross mr. Matt this is my point about God gets a veto is that the reality you know whether it's mother nature God the Universal reality reality gets a veto over your arrogance that you can somehow control it as opposed to adapt to it and just as a little sidenote I think in the airline industry there are similar rituals or I'd like

the stapler that means there was a whole bunch of compliance rules about what you can and can't bring in the cockpit yeah you have to have everything into it yes let me give credit to the airline industry and and no offense but take Shots and medicine because if you look at the data these guys do some very dangerous stuff and if you actually contemplated how dangerous it is what they do you'd walk everywhere you never get on an airplane if you actually started thinking like yeah I want to fly it's such and such a speed

it's such and such an altitude and inside of a thing which was made with so many parts I'm kept oh yeah yeah but we've pressurized made by People some of whom who had fights with their spouses the day they went in before they went to work that day you didn't you'd walk right or swim but you never ever fly but anyway it's a very very probably one of the safer human activities to do well where is that is because these folks have written codes and regulations and salt and all right this is structural this it's

the stage but why do they do that so they can see aberration right and and so you know the Way I understand checklists and your world is that you declare a checklist this is how we prepare to get a plane underway because you may as well write down what we think is the best approach but also we want to use the checklist to say oh wait a second worth to step 14 and and it's not occurring the way we expected we're outside that boundary we know you know we're now in an unqualified situation but when

is it happen when these checklists so then Someone says wow those guys do really good work it's really safe and it's amazing what they do every day um let's bring this to an environment where we're not nearly as reliable health care so you get checklist in a checklist manifesto and I think that was probably the the problem is that so what happens in the translation now it's that it's something that you impose on others which team comes up with a checklist and the rest of us have to adhere to the Checklist and we're dealing with

a patient and you know and each patient is idiosyncratic in some fashion we're going through the checklist and the checklist now says oh hold on you know you have a point here of non-compliance now the Av aiding aviation community would say whoa wait a second we're outside our bound where we've qualified let's take a deep breath here and figure out what's different than we expected what happens When people look at these dynamic learning tools or dynamic feedback diagnostic tools and treat them as adherence compliance tools they whoa no no keep going well yeah but the

checklist says we're you know some of them is wrong I know I'll keep going because the checklist says you have to go from step 14 to 15 regardless of what's actually happening at 14 so anyway tying this back to your question where did the lien people go Off off routes right so what was the first evidence from the 70s and 80s wow there's something here about the lowering dynamic and then what it crashed it can McDuffie say wow you know if you've had at that learning dynamic your ability to deliver value is orders of magnitude

greater than anybody else and then what happens is that people start documenting and it's actually very hard you know and I think we've all had this experience it's very hard to sell Learning you know how do you price that I'd like to buy a unit learning please yeah well what wouldn't you know but but but you can sell a value stream right and you can sell a 5s exercise and you can sell a standardization exercise so you crash down to this is all about standards compliance audit enforcement now again you know anytime we start talking

about really outstanding organizations and it doesn't matter sector it keeps coming back to man they Seem to be figuring out stuff far faster than anybody else whether it's Apple introducing new products or Google introducing new things like this or Toyota new product whatever it whatever it is it's like man you know they got to a point of understanding the market and understanding how to meet the needs of that market better faster than anybody else which gets us back to that peak which is about the learning dynamic hey just you said something that I mean my

Jaw dropped was they documented what they could see it can't reminds me of uh you know some of the work that you and John also are doing in terms of there's a stuff that we can see that the below the line above the line and we can you just react to that you know I think one of the one of I mean there's another term that sometimes used called cargo call ting which is a to try and obtain the the results that someone has by mimicking The behaviors that you can recognize that they have and

this is a an old term that that's got a long history I won't go into but I think we cargo cult a fair amount because we see successes around us and we say gee this is what they're doing let's try and do that here without having an appreciation for the fact that number one we're only seeing a small representation of what's actually going on and number two the conditions that they exist here and the conditions that Exist over there and exist in our place may be quite different and so I think this has been one

of the big problems now I also think that that we live in a complex enough world especially in the DevOps world where where we don't have a clear idea of what's actually running in the system at any one moment and I have to I saw your presentation at devops days Detroit and you had made this claim that everything below that above the line are all the visible behaviors the Interactions the ticketing systems whatever and well below and the mental models of the Panda madhumatl sorts of works yeah and then and then he said everything below

the line are the code and the environments in a database and whatever then in your theatrical way you said everything below the line doesn't exist and yes it was and they actually matter a lot and I want to and I want to I want to point this out which is I think we are agreed That learning is crucial the Crestor the question is in part what kind of learning matters how much of our time and effort is spent on learning and how much can we devote to that and and how can we direct our attention

so that the learning is productive and one of the problems that we have in large complex systems is it's hard to know what is it worth learning about hmm and and we have tried to use the events that occur as messages coming from the system The incidences of accidents and problems this message is being sent from below the line to above the line and then decoding those things in fact I just made a blog post called incidents or our untyped pointers you know it's a it's a void star type device an incident points to some

location but it doesn't tell you what's at the location and it's up to you to go and make sense out of that in a variety of ways you can do that well you can do that badly we've Been very interested in how it is that the people who are working with these systems are incorporating that experience into their understanding of where the vulnerabilities are because that's a shifting pattern and brother I'm gonna expose my ignorance I mean I thought I sort of understood this above the line below the line but below the line are the

systems the code above the line are the the things you can see the line is the representations of the System so the line it's a line of representation it's essentially all the screens you take all the screens and you line them up together that's the line above the line are people interactions conversations mental models goals a variety of things below the line are the things that are behind those screens and you never can see that and it's inaccessible to you and I think what I just learned from the the Steve's assertion is that it turns

out the stuff Above the line that you can't see either like mental models learning behaviors they were dead oh I mean at the point I was gonna make when you said yeah I mean Mike where you took me to where lean going wrong it's probably the same place where agile went wrong with Six Sigma went wrong the ideal of that stuff you said something I thought was profound which is it was the commercialization of how to teach people to do it it became at least common denominator was I had to Train you lean yes I

went in and I came in with charts and I did this and that's exactly where agile the agile manifesto is a beautiful thing today most people in doubt I was community get really kind of irate when a scrum master comes in and talks about how he needs to do something Six Sigma I think comes from Deming yeah and and but I was at G capital I worked at GK I went to Croton ville right I saw how GE was trying to do it and it was ugly yeah so I mean you know you know Deming

another MIT guy so you got a Beaver's right now but if you look at Lee I think it's somewhere in the Midwest first yeah I will claim them anyway so uh we don't know how to deal with it because MIT is now a very cool places well you know and so we're still having that socialize but uh but Deming if you look at his writing he kept talking about profound on knowledge and a lot of the tools he helped create which were appropriate for High volume low variety repetitive manufacturing you know control charts and that

kind of thing were opportunities to see aberration deviance to trigger learning so that you could develop profound knowledge now and if you look at Demings you know he had fourteen points I think and his other writing it was about the responsibility of leaders to lead their organization so that it was constantly developing profound knowledge so what happens with Deming Deming becomes a Six Sigma which becomes belts which becomes like this Karate Kid thing you know I'm a green belt I'm a brown belt you know so and why are you a brown belt because my control

charts have more variables than I was at Joey and I had I had actually had a green belt and I was responsible for a group of just early DevOps now this is like early 90s right so it wasn't deficit was a crazy operations group and I had to define my Six Sigma How many defects for every month and the guy next door to me has appeared to me manager he owned a server farm and his was uptime and it used to drive me nuts because he would just have uptime as his defects and and I'm

like that's just opening the door closed the door in other words but that's how bad it was we had one group nobody in the organization understood that electricity was his Sigma my electricity was the culture of people creating this on Tuesday writing This how many lines of code and and so it couldn't have gotten worse off of more Deming I I've studied a little bit about Deming where he started and where GE was let me ask you a question though about about this sequence because there there's something in it that I think is very interesting

and you named a list of things it goes back to you know Quality Assurance quality improvement call me and then Six Sigma and all these things and and lean seems to have been Caught up in this to some degree which is this sense that there is this thing that we are pursuing that is a actually very valuable but we can't we in our once it becomes popular once it becomes important or recognized the need to appear to be embracing that is so great that we adopt the language or we adopt a program and claim to

have it which then in some ways sort of sterilizes the thing itself it makes it impossible for an outsider to know where lean really is Or where we're different so I actually you're prompting the thought so actually the attempt to broadly apply completely inverts the thing to the point I don't know thought of it quite this way but uh so Deming if his advice was how to drive in the direction of profound knowledge it was because you were driving away from profound ignorance and it had to be right and the Toyota folks and so the

only way to drive from ignorance to knowledge is acknowledge or ignorant Right and and in the case of the Toyota folks this whole notion of standard work what but with all these different devices and tools to find error was a way to find ignorance all right now all right those thing about this so that you have at least these two trees streams of thought which is about identifying the fact that you just don't know what you're doing right yeah they're really an extremely you're an apt all right and then it comes over into the broader

Society where we give people credentials and it starts in you know it starts in kindergarten right you know you did something nice you get a lollipop and then right and then you get a little star and then you get letter grades in elementary school and new because it has to be more precise when you get to high school you know yep don't have letter grade you have number grades because ninety five versus ninety-four is more precise than a minus vs. na right all Right so any your whole career is about getting credentials improving that you've

qualified for the credentials and getting the credentials where you're gonna you know or pursuing the credentials where you think you're gonna get them because you don't want to get in a situation where you try for a credential and then fail because then you're a failure right so now you have these folks who've developed a system Which is premise on recognizing your ineptitude and they're trying to deliver it to people who are the last thing they'll ever admit is ineptitude right and so now you have these people who all they're trying to do is demonstrate credential

eyes somehow demonstrate competence trying to absorb a system which is dependent on recognizing ineptitude so they can't do that so actually I think it I think it's um it's a sabotage I don't really thought about it this way but these people who are potage yeah what by home of the learn is it good to the non learners right good they've left their own devices they wouldn't ruin this thing and then apply it in a really bad destructive ways but it's even worse than that right they're not left to their own devices they're taking it as

a Trojan horse right maybe that's what it is lien is the Trojan horse for what way good gives instruction yeah let me Follow on that because I have a question that flows from that yeah which is what kind of advice then given that that's a risk and we understand that that's a risk in it and in fact it might even be there's a kind of moral risk because if I propose something that can be easily adopted but gained so that I can have the credential but not the thing right then I'm it I'm contributing to

that right now contributing to the delinquency of an industrial suicide by The way as authors this is a profound I was going to skirt that right there but the question the question is this how could we construct or how would you suggest that that the direction or the guidance be constructed to avoid that how do you how do you take how do you make a presentation of a theory or an idea in such that it did it is somehow difficult to hijack or to door to make sterile oh so all right so this sounds like

the answer you'd expect from a guy Wearing glasses and a bow tie from MIT you got to teach people the scientific method because it and over to your field and Aviation too right which is you're constantly plagued by the fear that you don't understand a situation so what do you do you construct an experiment with more control than less preferably but construct an experiment and II engage with an expression if your best ideas to find out you're wrong right and it turns Out the scientific method and the power of refutation is the source of a

lot of great human discovery and in the absence of it we'd still be suffering cholera and endemic scale and having surgery with that anesthesia right but when we get over into the so it's a loosely used term here the social sciences as writers but certainly consumers of that stuff we don't make all right so here's the thing all right except for the people who go out and buy all these sort of like Herbal supplement kind of stuff most of us are really really happy that the FDA required pharmaceutical companies to prove through all these gateways

that their thing is actually safe and effective right like YPO opioid right yeah yeah well all right so that's a beautiful day it's perfect right no no nobody who's ideal no but you get my point is that so that's one that that but this is why there's such Legitimate anger about that aside from the human suffering it's like the system failed you told me that there was a system that demonstrated safety and effectiveness and this stuff fails on the safety thing and and you didn't adhere to the standard you espoused that you're gonna um constantly

challenge new ideas you know representatives drugs or not but so all right so as laypeople we're really skeptical of stuff that comes at us which hasn't demonstrated Let's say safety and effectiveness it hasn't been subject to refutation and cleared all those hurdles of making it through without being refuted but then we walk through the airport or wherever else it is going Amazon we buy all these books which is how to be a successful charming and witty also and a lot of that advice there's actually no strong empirical test right you know and and and and

so here's the thing people gonna write books and run consulting Firms because I want to generate revenue and if people gonna buy that crap you know then people will sell that crap so really who's the burden on its on the burden is on us when we read something and say hey wait wait wait a second where was the opportunity to refute that idea and if we read through something and see there's no opportunity refute idea we should reject the idea so anyway scientific method you actually after a little period of despair and when you Say

authors might be morally contributing to a profound wrong but you know that actually what I was actually then followed by a feeling of hopefulness because one of the most recent books along this domain is the Lean Startup by Eric Ries and it is it is all about the scientific method I mean it's all about it's not PDCA mean it will be I mean actually comes to Deming again PDCA right is you're you know that is you know that is your Classic scientific method right plan dude way right is what is the least amount of work

you can get to confirm that there is actually demand for this thing that you're trying to build right and the purpose is to state hypothesis confirm that there's demand and then minimum product right exactly as you learn which is one example where what I guess I was trying to I was I was saying this dr. Lander point here's an example of a Best-selling book that actually does teach very much a scientific methods the point has influenced a generation of startups that all speak the language of Lean Startup I wonder if one I took coin don't

I think is great is like I don't think we do this enough just I know in history like read a book and look for what's and why it didn't work I know what was wrong yeah I you know you eat a night I like you know all three of you I read your complex systems originally I Fell in love with it I read your stuff I'm the original human guide to human failure I read your book I loved it I actually was guilty of not stepping back and being a critical thinker about I thought that

was really a profile point about yeah but there's there's there should be room for inspiration there should be room for for taking an idea and running with it in ways that that cuts it any any moment that we that we idolize right literally idolatry Right we we we surrender to something being the final answer to all of our problems we we are idolizing the scientific method here I mean you read pull fire ovens against method right you understand Tom Kuhn also by the way from a doctor from a reputable institution for and and it suggests

that this can go off the rails in all kinds of ways you explain that yeah I'm not sure why scientist method could go it's an it's an utterly irrelevant discussion for the All right so but but re Tom Kuhn that's all better than the Bible by the way right so big fan is by the way no scientific method for evolution thank you thank you thank you okay so the method is only as good as the questions that we know to ask because without the questions the method is gonna be useless or anyway hidden so it

would keep asking the same sort of questions within a particular part that would be Tom's argument in st. paradigm we're not gonna Discover anything new we're gonna give it locked into a rut cell opioids as if it's the greatest thing right and so this is anyway it's a uremic point I was gonna ask another question and I'm sort of leading up to this with the question about whether or not there's some moral responsibility for proposing things that can be misused which is that there is a probably a moral value to proposing things that are not

easily misused or that can be used in effective ways and So what what that opens up to me is the issue about whether or not the work that we are doing in DevOps and the kinds of activities that we're doing now have a moral and ethical quality to them that is somehow not yet in the discussion I you can think about this in two different ways one is that that the products that we are using now have have begun to have profound influences on the society they were in and so Facebook may have been a

social Network at one point but now it clearly is a tool for political influence right and and we can't escape those things and we even though we didn't plan it that way these are the outcomes that we are seeing because of the reach of our technology and because of the way it's connected so we have I think the question is do we have a some sort of moral or ethical responsibility as we are building these things and and how we are making them so one of the reasons I Wanted to do this right is because

I actually was when I first the first time I met dr. woods and I naively went up to him and I heard him present at velocity and I was like oh so excited because I read a little bit about lean and some you know I think I'd read brought this book I read something and I saw lean as more of a non-deterministic more of embracing uncertainty and when I heard dr. woods I was like oh my god this is like lean in and respectfully he kind of Like lean no and and then I you know

I thought about I talk to gene quite often about I read a lot more I read your book I ran into you in in Brisbane and I you gave an amazing presentation about a walk through kind of safety history and I grabbed you off to say it I says everything I just saw it looks like a non-deterministic mono and you like absolutely and we had a further conversation we had a podcast and I said to you can you do me one favor can you We serve because I asked you had you studied Deming and you

said no at the time and I said could you just research Deming and then just let me know either tell me I'm totally off-base because if you embrace Deming then I know I've got the Nexus of everything here there's and and yeah and you caught you about a year later you sent me an email and you said you were right and you said by the way and like to my heart it's in the safety anarchists now And so that when I said to Jean I'm like I really want to try to get this conversation and

so the things I try to look for work what are we blameless I think is something that we all agree on psychological safety we all agree on you have to rely on the audits you're way behind the car right and so I guess that I wanted to try to drive to one of the things that we do agree on because that was the purpose of trying I think we've had an amazing Conversation but that was the thing I wanted to drive is I wanted to find out what are the you know your your Nick sorry

like look what are we what are we all what do we all have right at this table right until we're proven wrong mm-hmm right strong opinions weakly held that's nice but that's exactly the scientific strong declaration of what you're willing to give up you know that's a nice way because I think there was some sort of Like when you acknowledge you said oh yes I did read Demings did you read them out of crisis yes and you know said that's just 14 points and you were like this does fit what we're trying to say now

a very nice that's true it so in the safety anarchist some of the things that I certainly rely on is is the point of blamelessness that if you if you if you create a culture or a setting in which you blame people for making errors or not not doing or living up to the spec You're just gonna make life more miserable for yourself and your system but you might it's it's a to manages it's like being in your pants right it feels really good and like a great relief at the moment okay but you'll look

like a fool you start to stink very quickly so so that that's not Deming would probably not to put it that way but I think it would recognize you're the church-going guy okay I'm the son of a preacher woman but the and so Hangul We've already talked about some other ones that were in the safe Garin Argost help me well Demmings you 14 points yeah yeah but I'm not gonna recite their minds here that's like well I mean it was I think some of the things that I think like he hated MBAs you or either

this this idea of this kind of deterministic like that I could create a KPI or he said NBA's because they knew terminology was invented now in Silicon Valley it's called okay ours by the way Okay all right same thing yeah you know but he hated that right because it was deterministic like that this shall happen and we'll check in a month or four months well anything another very important one that is very much related to that is you know said if you have to rely on audits to know what's going on you're way behind the

curve and it because the audits I mean Dave wood says it beautifully right it's it's a it gives you a very distant View of operations and not authentic you an authentic view you got to be in there you have to go you have to go operational which is why I think Richard and I somehow I mean it's not that we valorized the native that we valorized the person actually doing the dangers and dirty work but we certainly recognize doing it ourselves that there is incredible value to understanding that perspective from the inside out right the

animate the perspective of the Antiperspirant would help out so you know I came up a list of things that I think that we're in the areas of commonality oh sorry yeah psychological safety blamelessness integration of scientific method to improve daily work reduction of bureaucracy and certification yeah it's just I'm thinking they're sort of accomplices decisions and flow and it's and I'm in favor of safety and quality to the problem my concern here is that that These are these are aspirational qualities and things that we would we would like to embrace as as values and I

agree that the we that they are valuable things but when we actually watch the way that they get enacted in real working situations we find that many people will claim to have a blameless postmortem when it's not blameless at all no there are many people who will claim that they do they have a psychologically safe work environment When in fact it's a psychologically very hazardous work environment and and revelation after revelation both in media and from our studies suggests that it's a lot more conflicted down at the sharp end than that that simple kind of

statement but should that make us cynical or rebellious it makes us very cautious about accepting taking at face value claims that that exists in a particular location I think it also should ask we should ask why it is that Those things are so difficult to obtain maintain in those places rather than say oh yes they have that they're just doing it the right way I'm particularly concerned about the the fact that we are struggling to have this kind of openness and transparency in a world that is becoming increasingly opaque organizationally structurally the the world has

become much more opaque as time is going on we have much more difficulty seeing into things as I would Give you at the examples of of the trouble of the reports on various kinds of outages that we actually get the kinds of information that we get about how companies make this is just a backtrack a second when you say you know you're troubled by the fact that you can't confirm the behaviors our as reported I mean I think to Steven this point it's like well you should be able to tell from the outcomes right if

if we really care about the outcomes you Showed up with a safer more productive more innovative place I mean it's actually I think to this point though that the decisions and actions that lead to outcomes occur so far in advance that if you wait many of them if you wait to see the outcome it's too late what you really need to do is see leading indicators of the right behaviors the right actions okay so and and that becomes really really hard I think it was something you said earlier it was About the how do I

value or discount an experience I might have and and the thing is in the moment I can put a pretty concrete value on social ridicule mmm you know and I can put a pretty high concrete value on social acceptance so I'm in the moment to call out and say you know I don't understand what's going on this doesn't feel right you know and again you we were talking about biases and how we try to protect her so one of the things that come at econo men in Traverse keys is this asymmetry between how much we're

willing to pay for a reward versus how much we're willing to pay to insure against the risk so in the moment I'm in a lot of social risk that's gonna really drive my thinking in my action versus well this might be good someday but I'm not really quite sure and even if it is you know how much value well you write there's a fig leaf nature to the fully agree with both of You which which is probably no fun but but it's an appointment which is if we rely on a supposedly psychologically safe workplace to

to catch the types of trouble that should have been designed out of the system way before we were relying on on the the last line of defense if you want to use that language to be both clairvoyant and heroic and I think that that's but above the pay grade that we're willing to fork out yeah I mean what it does is it really Puts a huge burden right so I've been making a lot of ridicule of a transactional view of management that oh I can buy an initiative ridicule that sounds for highly descriptive okay well

you know but it puts a huge burden on leadership to cultivate and model I think that was a point you were making earlier which is not just to say oh you know John can speak up any you can speak up anytime you don't want but yeah I won't know and I know everything yeah to Be seen to know everything right right which and I think the thing the very few in this gets to your point why is it so hard to see I think we understand why it's so hard to see but the few organizations

which consistently are much better than anybody else you'll find leaders who go out of their way to humble themselves it's exactly ready that's exactly right I mean you know and never cease doing it right you know in all right so you know You get stuff like oh the the serpent leader and it's almost like varisai where the nobility would dress up is the peasantry you know I mean we expensive hasn't read J and then gorge themselves Yeah right at gorge themselves anyway right they try something but it's like somehow you know Oh noblesse oblige as

I know it said right you know so I'm offering you that yeah what do I say yeah I'll bring you the cup of tea but knowing that We're faking it yeah right as opposed to you know true humility and modesty and again encouraging making it safe for others and demonstrating what it looks like that displays the cynical opposite and responding to your question John I think one of the things that I would say as we look at resilience and try to understand how what resilience become where the sources of resilience are an organization and by

resilience I Generally mean the sustained sustained adaptive capacity but if we say where does resilience come from it doesn't come from a thing it comes from a whole bunch of different things that interference different sorts of timescales some of which are very long yeah so there are there are things that have timescales that are on the order of decades or even centuries and and that's you know professional organizations our sense of responsibility what it means to Be a participant in these sorts of activities the kinds of education do no harm ethic people thousands of you

all of these things all of these things take very long times to develop and and I don't foresee a time in which we are so good at predicting the future and the systems that are in it they were going to be able to take away the need for people to make difficult decisions at the point of contact and so what I think we have to think about is what do we Need to do to prepare people to be in those roles Sydney's a pilot and a pilot has a set of technical competencies has some abilities

to do different kinds of things you saw such things in certain ways but what we also depend upon the pilot to do is to make the right kinds of decisions based upon the right kinds of values and judgments and we those come from long kinds of it experience being part of that that group of people who are pilots and what does it mean by A pilot and having taken it yourself what it means to be a physician or what it means to nurse and one of the things that I think that is truly missing in

the world of DevOps is what it means to be a DevOps person in the sense of belonging to a group that has such an ethic and a profession and a basic set of values that transcend the current employer or the current employment setting and I think we rely in the space of and for Operating our most hazardous and most valuable systems on people like that I mean look at the mill kind of training that one gets to be a military person who's in charge of large weapon systems or a pilot or something like that but

we've done very little but in terms of building that kind of but you can't engineer that in and part of the problem is this is a fast forward professional develop within a couple decades no no so so the but the problem then is is if you Start engineering it I think we're back to what was some some ideas that Steve was suggesting that we all we give them is a see me lay of values and we sell them the made-in-china glitzy plastic version of it right oh no we have these values right but they enough

they don't even know how do you avoid them can you fast forward but I would propose fast right - agree with you I agree we don't and in you know in an IT has just been a zigzag right we're not really Cause those few science we don't we don't hardly do science now we do a little science because of some of the stuff that's happened coming out of Google I'm gonna go back to the way you were describing Toyota Toyota created a way of thinking that allowed people who weren't like in today's weather they did

to have the pedigree right they did it there right didn't they create what he they they had created the sharp edge that allowed people do scalable delivery Of things that wasn't it solved yeah so I think this point about identifying what you both were saying about identifying with professionalism of a group my experience is that you will meet out in the industrial world a lot of people who might have worked for the General Motors Toyota joint venture more often than not they were General Motors employees I very rarely meet X Toyota employees out in the

world look footworks and hey I used to work for Toyota I think it is because they know when they leave Toyota they will not find the same professional ethos wherever they go I don't know about the IT world nearly as well as that world but I find it also you rarely find people who used to work at Apple you know I used to work at Apple and you should hire me because I've got some because again I'm willing to bet the folks who work at Apple realize that if they leave they leave not only they

Leave that professional ethos and that's a huge sacrifice for them you know ditalion back to Toyota because that's what you got to started so um people again have often made the mistake of copying Toyota's shop for tools and practices my experience with Toyota is that they explain the shop floor tools and practices first by the emotional and even moral elements and then say oh yeah by the way it helps us you know control Our inventory so when you ask the Toyota the real luminary is there I mean kind of like the Jedi Knights you know

obi-wan Kenobi's of Toyota so what do you guys do a pull system yeah there'll be a hand wave you know it synchronizes this dynamic system but you know I said aha because someone decides to give up a third to half a year's salary to buy one of our products now if we have a factory organized like a normal factory no one Will get a sense of the work they're doing is in response to the trust given to us by the person who's investing three six nine months worth of earnings to buy our car or the

pull system when someone takes a car and that triggers the production of the pieces that go into that car that's right that means that someone who's two and three and four layers back in our supply chain will understand that someone trusted them that day and don't you want to go To work knowing that people trust you to do your job right and oh by the way we make cars yeah it was really it was in the original Dutch contrast between Sloane and Ohno right the Sloane I think said see you know we're not in general

motion we're not in the business making cars when abyss making money I was like we're not in business making cars we're in the business making great people who great ya know so that's exactly so I'll just Add this I recently had a chance to attend the 25th anniversary celebration for a group within Toyota originally called the Toyota supply support center now it's called the Toyota Production system support center right so think about it this TPS thing requires support right I mean that's why they exist to support this management system and the folks who work in

that organization their job is to teach this way of managing other people some and suppliers A little bit in their plants they have a sister organization which really does the plant work and then two unaffiliated organizations like food banks and people who build housing for emergency you know post flood relief and that kind of thing it's a couple of things about this event I'm sitting there with my mentor Kent Bowen right and he looks around it's like 300 people at dinner the Secretary of State of Texas is attending and they've flown people from you know

long Retired leaders of Toyota have come to this event he said son of a gun look at the big deal they're making about an organization which has never had more than 25 people it was alright and then so what and what is this TPS thing the Toyota Production system that they're the support support is for so all the presentations people come in as you know the Toyota Production system what's it about it's about engaging people and problem-solving to meet society's needs And whether it was a guy who said it in Japanese was translated in English

I've said it in Japanese which was came out of his mouth as broken English a guy just said it in English it all came back to engaging people as problem solvers to meet society's needs we're there for like eight hours no one mentioned cars yeah well and eight hours it didn't come I ones that they happen to sell cars time today I heard oohs and ahs things he was Saying you were like ah so I mean based on what he just said in that description what's the pot let's play the knick game was beautiful the

positive things no but but so where safety thinking has been going in the last few years and and I mean Richard is he's been instrumental in driving in that direction as well is to indeed don't see people as you're a problem that you need to control through tighter procedure realizations know people are your problem solvers Right they do these dangers and dirty work every day they solve the problems for you don't think that you should you should tell them what to do because they actually know what to do that's why you hired them right to

ask them what they need what do you need to be successful right and so and the third one is and we talked is we rather than than pursuing the negatives and trying to reduce the number of negatives try to understand why things go right and enhance the Positive capacities that make it so I think that is entirely consistent yeah I I think that that I might have a little reluctance which is based upon the idea of tying this to a corporate organization or a corporate structure because I think that in some ways what I'm talking

about the people and and their their capacities and resilience should not be dependent it shouldn't arise just simply because they have an association with a Particular firm it was the transected my vital teacher I think and I think that if that pilots our pilots who happen to fly for a particular airline where particular uniforms they were and if I were going to chart a path for DevOps if somebody said to me what should we do to ensure the future of DevOps it would not be build a better tool chain or a way I think everybody

would it we all agree it would be find ways so that the young People who are entering into this can get the kind of professional mentoring and apprenticeship experiences over a long enough period of time that they can actually develop those skills but could we learn from that model I could now I'm with you that like you know pilots have an ethos of course pilots not corporation but there's is there enough there from what he described to learn to try to extend and then I think that everything we've done so far in DevOps Is to

try to learn from lean we spent a lot of time trying to draw wrong and right examples i thought but is there enough work inside companies that you would find that there's much more heterogeneity within companies than most of us would would like every time and that there are places of great strength and places of great weakness yeah and the question for us really should be where are those places of great strength and why are So strong what is the kind of thing that makes that so functional and in the few places that I've been

able to study and I've looked at a dozen or maybe a little less than that and brother I can give you access to 23,000 okay kind of study that I'm talking about doing is actually going and watching the work really happen and spending time with people looking at particularly at the problems that they have is is that there are in such places a CAD ray of experts of People who have long enough experience there and high enough levels of trust from above and from below that they become the kind of of they're the kind of

bellwethers know what matters and people trust them they trust their judgment they trust their discretion they trust their behavior in ways that allow them that give them good models for them to emulate and that they pursue that and I think structuring environments like that is going to Become more and more important I just want to make sure that I heard the last 30 minutes correctly because you know I actually rattled off a list of terms are more associated with safety culture than turned it you know in a spectrum and the ones that were more associated

with lean Toyota so forth and what I heard was just wild agreement I was like that was too easy even though it was more aspirational I mean is that I hear just what I wanted to hear that we have Something wrong I'll put it for you a different way I'd say forget about lean and forget about safety culture stop using those terms just drop them forget them onto your bottom third there are there they're there they miss they they misdirect us more than they help us yeah we have to go and look into this work

itself and understand it in the kind of detail that allows us to see how people are grappling with these problems and Find ways to make that grappling easier more effective better for them and and I I think it's a mistake to conflate this with some sort of some sort of almost almost a political agenda if you will it's wrong term it makes for great TV by the way well it might but but but you know I think when we use terms like that safety culture lien and I think we evoke in people a whole bunch

of responses and I'm not sure that's what we want to do I Think what we want to do is we want to say to them we're really interested in the problems that you have carrying out the work that you're trying to do can you show us more about where those problems come from can you tell us what the kinds of risks and hazards that are surrounding it we have to begin to do that irrespective of what kind of labels we leader-1 but this is the problem of scale right it goes back to why did lien

fell right maybe they weren't bad actors At scale sometimes you have to retract it down something I've got Sidney Dekker he's speaking at my conference well who is he okay what do I do now he's well he's a safety is a factor safety engineering okay I come in I want to help you how are you gonna help so I kind of psyche with you but I think we we have to put abstractions on things and and you know I think the BIOS and even like I predicted most of those people who tried To Train lean

we're not bad actors they just had to come in with a suitcase and do something to teach it is and yeah late labels matter a lot yeah i and and and you know um you know is a phrase you've used before but to take a term and unpack it and be habitual in the unpacking of terms I think is important because our natural tendency is to have something which we've um described with a lot of detail and granularity then we label it and in the moment of label That's okay because we've collectively arrived at that

description and now we can put a label on it we all know what that means but then we take it and we give the label to somebody else and I haven't gone through the packing process mmm so now they unpack them in their own particular way so anyway depending on just plain English or whatever language is your language I think is important maybe that though lady Strauss came up it says I Think one of the kings of anthropology a Devi Strauss came up with a criterion that is perhaps puts the bar really high or really

low I don't know what he said is it good to think with very French way of phrasing and obviously but is it good to think with is safety culture good to think with or is it actually bad to think with visit does it send us on a rabbit hole is situation where and it's good to think with is root cause good to think with And we all have opinions about that I there's I think there's a need for a balance a little bit you don't want to reinvent the wheels through sort of an assumption of

grounded theory that simply by going in and watching the wolves mate for long enough we can build theory about how these things work and don't work for them absolutely have to be sensitive deeply sensitive to through the operations of the actual group but it is it is by the way naive to think That we would do that even we don't bring any terms that we have no models and background anyway because we do of course our observations our discussions are driven by things we've seen before and is it spending a lot of time left while

helping my kids with maths right and and and and and and turns out labels are wicked important you know if an angle is corresponding or same side interior whatever but again all the social pressure is I haven't believed I'm how I can get through it marital pressure right so the the point is is that there's a concept and there's a fluency which depends on understanding the symbols hmm but there's a process to that and and you can see it with the kids if they use the symbols too soon right they don't eat you stumble UK

they don't have the fluency to apply the ideas later on and I think that's what we're very true very and and I think that goes to you made root cause is Probably not good to think with I think it is misleads all kinds of things that's why here's why I do actually posit for just no I know because we're gonna run out of time meta for a moment actor but like I say the observation right if I were to plot the number of frowny faces you know to happy faces I would say just even the

course of the last few hours right the number of he faces going up the number of frowny faces going down right so I think just Even in this time together we're circling what does it have to do with anything fun just in terms of like is it safety or Athene is it's a happy sort of we're sort of transcending that later but but harmony is is a false goal to have when it comes to a group by the way it's pretty bad groupthink and then you anyway so we've got so okay so backtrack so is

it good to think with it's a great question to ask and then at some point somebody needs to make a judgement is it Good to think with is it not good to think with and there I think it is useful to take guidance from those who've been there before in fact it would be stupid to not take guidance for those who've been there before and and health care has been down the root cause halt Aviation's been down the root cause halt what we find is obvious I mean root call a cause first of all is

a choice is not something you find right here at : angles is it beautifully of course is Something you create you invent you construct you say that's the cause right versus this is the cause well of course also has if you think in Newtonian terms is also an effect of another cause where do you stop and they're already the problem like what is the root when does it stop right there is no root because it's it only stretches as far as your imagination right and so what we then have to reel back is say where

can you make meaningful changes in the stuff That you discuss and discover right that would be you know an awesome persons question I mean you should the root cause is pretty much not only where you start looking but where you begin where you become beginning to losing confidence you can do anything about it and so the root cause is obviously the big bang but that's not really where we need to go back to fix all of this right so so I think and the root cause I also have objection to it because I think it

Often gets misinterpreted as playing towards this transactional oh I found the cause hi you right and also and then blame is easily connect to it and the problem is that that person either didn't have the right thing a transaction made the wrong decision and that's the cause that's language right but if you really think it through why does what if things go wrong is because we didn't know how to make them go right I mean if you really have to go To root cause the root cause of everything is ignorance I just didn't understand and

in the moment when I had to act I didn't know what I was trying to achieve or or I was or I didn't know how to achieve it right and and and so you go root cause like a lot of people do it gets down to a technical fault oh we need to buy that gizmo because if we had that gizmo we wouldn't have had that problem or that person needs training because if that Person had that training we wouldn't have had that problem depends the question if it's so obvious now that you should have

that business what didn't you have that gizmo because she didn't know you needed it right you know is it so anyway I think that frames are different great highfalutin ideas but for the vast majority the population you know their cultures don't accommodate the they you are they have the wrong tool to think what do we do then yeah I Mean I'm certainly sympathetic with that but I guess I would say that I think again this depends a little bit upon our loose notion of what culture is and how badly we use that now a bad

data structure usually term I would claim that these people are doing that learning anyhow and they're encountering these things and learning about them very often in spite of our management systems in spite of our organizational structures they're not learning because Of them they're learning other things and they're not always learning what they what we would hope that they would learn sometimes they're learning what not to say and what not to raise but the idea that people are not learning is I think a mistake people are learning all the time they're learning they're avid learners and

they constantly learn yeah the question is do they learn in ways that are productive and I think I think that I would argue that this is where This this professional kind of requirement actually plays a role and if you get a group of tech people together and you get down a you know out of the out of the structure of a formal meeting and you say to them tell me about the problem that you had with us and so I hear that was a pretty bad outages oh yeah that was really terrible you know we

were really done and they'll tell you this story that's very very persuasive about all the things went wrong all the Difficulties that they had and all the risk what do tech people do when they get together they tell stories like this it's what we did last night at dinner right we've sat around and told told horror stories why because that's how we learn we really do we really do pay attention to stuff we really are concerned about it and we learn about this not because somebody says you go learn we do it because that's who

we are Interesting rich the the ethical commitments and the values you spoke of earlier that the profesh profession lacks because it's too young actually I think emerge from these stories yes they tell it because the stories sit on a bedrock of assumptions of what is right and what is wrong because that Alton believes the ethical question we could write better we can make that rule way that's exactly the value is that we can make that better but we have to we have To go where they are it was interesting I find that very hopeful because

yeah you and I had a conversation in Detroit and I had made this claim that I think it's going to be our field the technology field that will deploy these wonderful thinking patterns mental modeled lessons it will be the first to deploy them at industrial scale and I think you know the fact that you know we've all does this conference where 1,400 people converge upon you know There's an intrinsic motivation for them to learn right they're here because they feel a part of something bigger they you know you've heard they're almost a plaza when Ron

Forrest from Nike said you know the first thing when they started talking about these developed patterns was wait a minute does that mean have to wear pager wait no pagers are for the other people but he said the most important thing is that we had to had create a cultural responsibility you Build it you're responsible for it and it was just you know your passion and you know John auspi you know his passion um the stakes are high how varying his title at Google I don't know if he's still there it was a chief economist

so I didn't your company has a chief economist and he said this was about five years ago he said we're in a state of combinatorial innovation mm-hmm you know that's how we created the Cambridge you know we call the Cambrian explosion MIT I'm reason is because were yeah can't you say Cameron higher you came bread cheese Oreo but but but I think your passion is is absolutely right I mean we we have to get this right because all the things the scale of what we're you know I don't like everything that we're going to do

is going to be affected by somebody running some computers and work infrastructure everything and in the ethos and how we do it yes we have To get passionate and put our fists down and figure out how to get but these are the kind of people I want in the room hmm - you know that's I always tell you I wish I could kidnap you and just put you in IT and take you away from industrial if I'll give my wife a job I want to say John those people are already there and the biggest job

that we have is to find them to raise up what it is that they're doing - to make that make it clear that that is what we value As well those people are there I encounter them every day I encountered them here oh yeah in fact I'm just amplify that point in the context of where are the amplification points I mean I think the people here they've all self-selected right they they care about this in fact is it even the language of you know historical significance the you know these are the change agents it's a

you know these are people who are taking sometimes vast risks to overcome very Powerful and common systems and so as I would submit that what would a powerful ingredient to have when you have a bunch of people raising their hand saying I am one of these people who I'm willing to take personal risk yeah so the cup is definitely half full you're here we're all here right I mean I mean I mean I know you look at like you look at that ops and you have these questions about DevOps but but I think to your

point right I mean I think we can I mean like Jean said there like there are self selected people here who are coming here try to find the right navigation like I think that I think DevOps has a has a variety of opportunities that are really revolutionary one of the important ones is the DevOps is at least potentially a great equalizer across a set of social class gender and race activities or services that that we might be able to find some common ground in and where where the the idea of a Meritage Krisi that's unconnected

to those things as possible there's a way to go and and this would be this would be a revolutionary change for our society not not just for the out way we do IT i think devops has the opportunity to engage a whole bunch of bright people in ways that are are much more productive for them and things that they have done in the past or that they might end up doing and so i think it is i think it's a beautiful source of the kind of Complex relationships and technologies and things that that young people

are fascinated with and would love to be involved in and so i think the question for us is how do we open those doors how do we bring them in how do we give them the opportunities how do we mentor them so that they become those those those reasonable rational thoughtful judge ethical ethically judgmental people who we want to be running the systems in the I think you know we've already answered You've you spoke at EV upstate Detroit you we finally got you to speak and you know Enterprise DevOps this morning people got to hear

stuff they're gonna read your book you spoke you two years ago I think we're doing it I mean I think we gotta get better we got to get more precise but even further than all right so and maybe just to resonate this point a little further people still tell me about how much in this community they remember your talk yes they always been Up the Godzilla slide about what decimation of the US office history looks like I'm but I mean every they saw the relevance of your teachings same with you Sydney I mean they're these

are the people who make the mappings they know how this will help them advance their cause so my question I think I'm gonna John I can't give you a magic wand we have these people for a couple more hours and maybe even more if you could Wave a magic wand what would you do to tap their expertise and their network and their you know everything that they've accumulated in the course of career if you could wave a batch one what would each one of them do for you to help advance this cause I I think

this you know jokingly call this an atom smasher right and I I want to get people who have really good ideas on how to deal with these complex systems that we live with and Get them you know I think we've accomplished a lot here in this discussion I think we did continue discussion you know you asked me this morning about this and I said oh this year is easy where do we see who we bring next year yeah let's get some of these other people that have really good ideas but but like let's just keep

atom-smashing this stuff because I do think the common good pieces you three brilliant men or a really good and we Talked about the influences where but some of the IT people who call ourselves computer scientists who really weren't who were just bouncing like pinball that's right and we got somewhere now we're really important like Facebook right and we've got infrastructure and when Amazon goes down commerce breaks like oh now we're now I know you need the grownups to help us right and the grown-ups honestly are you you you and a bunch of other people that

are in other Fields and and we need your help it's what we're trying to do with snafu catchers I mean that was the idea about trying to get to build an industry consortium that brings people together to talk about and understand these things the key thing that though is that the John is that we don't have the answers that you're seeking all right so I'm not looking for the answer okay looking for the direct you don't have the advice of the answers and you're and Those are in that in the people who are already out

there the answers are over there yeah but the questions need asking I think I think we don't have all the right questions either but with some inspiration of having been in some of these things or we have some questions to ask that might elicit the sorts of answers yes that we want that I think I think the the blast radius of your presentations morning I I suspect and as I go through the community over the next year maybe we'll have major impact III believe that just like the blast radius view two years ago right so

yeah there's no we don't have right you guys aren't all right I don't all right genes got it right but I think the only way we get better you know I kind of jokingly called these super patterns there are these super patterns that in these disciplines that you know that seem to just show up you know In you know Halak receivin like I don't want to go there but you know and even argue whether you guys hate it or like it but that you look at all these disciplines and they seem to be these super

patterns and if we can just keep kind of condensing them and then exploding them I think we get stronger you have that's my magic wand right and my quest for even more concrete I mean I'd love to hear any concrete advice you know unsolicited advice that you would Give us as a community from each one of you not question to ask questions we need to seek I mean what advice given the information that you have yeah what advice can you give a group of people who are leading these you know they're putting themselves in personal

risk to help overcome incumbent systems that I mmm so I'll take a shot at this the general advice is to this gets back to your the ethics of a profession is to worry about The consequences of getting it wrong mm-hmm and I'll give you an example so I'm working with folks at a pharmaceutical company and you try to understand why someone becomes an or guineo Studies organic chemistry gets a PhD and then does chemistry at a pharmaceutical company and what worries them each day so work with a woman named Beth who was describing discovering a

high school friend on Facebook and then realizing through his post that his Mother had gotten an aggressive form of cancer and she was promised six months and the promise wasn't met and she passed faster than that what is the risk Beth is addressing each day is the risk that her work doesn't progress fast enough to help someone in a situation live longer live better do more all right so um that's the example of what is the risk of not getting it right and the thing about your world versus the more traditional industries more Traditional industries

affect travel as fast as physical stuff and the more effect you wanted to have the more stuff you had a move which actually meant it's going to move slower and slower your stuff travels at the speed of light or close to it and so the ability to have really profound effect on very large scale very quickly I mean the risk of getting it wrong is staggering and you know I certainly hope we just like you were saying about pilots they get on Board a flight with 189 souls with them you know what is the risk

of things going wrong well in some of the situations where there's this IT is more than 189 the risks are extraordinary and I'd like them to worry about that and and NIT it's not just one plane going down is every plane dropping those guys all at the same time I think these global or staying on the ground or staying the ground yeah it's having it's having the screens That display all the locations of all those planes suddenly go boy yeah and just make sure I heard the advice correctly is what you're saying is that can

have negative consequences just because the spread is unprecedented vast but there's an inverse that change positive change can happen also at that's a beautiful idea is that the speed of light show me that and yeah I mean it's gonna play on what we what we've been discussing is you Know these ace the ace symmetries of biases and how much we will insure against risk versus invest in a gain and so I I think people should have be highly motivated by the risk of not getting it right I mean yeah we can always dream if you

know well if I get it right you know the gain of that but that's less motivating then what happens if it goes wrong and if I'm dwelling on that just challenge for a second would that's basically aren't you educating People being scared to the point of paralysis and no no no well paralysis comes about when you're scared and you have no idea what to do right I think that I think we're playing a lot with the etymology of words right so now how are in coward comes from the same thing was your brain locks up

it's like Oh situation don't know what to do freeze so I'm not advocating freeze I'm actually advocating action that and this is where the stories come in you got a Max is the opportunity for any stories telling right because it is very difficult to generically sensitize people people to the risk of getting it wrong actually right and we know this it's it they'll go blunt and it won't be it won't inform their action or their ethic but knowing about stories where somebody did almost the same thing and actually did something really bad or whatever might

sensitize them to it and So storytelling I think conferences like these shouldn't but could build in more space for story - let me tell you the disaster were almost or the near miss right and and how did aviation get as safe as it did because it's it has the courage and honesty and openness and protection to start telling these these near misses let me tell you how I almost screwed this up so we need to systematically hunt down every hope in your industry and let me finish the Recommendation system Attica Lee hunt down everything in

your industry that puts downward pressure on honesty openness to disclosure and learning through these storyteller I have chills I think the thing is you know when you said earlier like when you were flying a plane you knew right there what was the risk of getting it wrong because somebody had been there but NIT very rarely do we really understand I mean I even like the night capital I'm Not sure the system administrator knew that day that that company was going to go out of business in 24 hours right but I think to your point and

Richard your point I think and like I think the storytelling but somehow get people to think about like this isn't just a computer this isn't just a program you are now you don't have to do it in sort of a badger English no no no no I know I think good if you do the bad dream then we've gone down the negative yeah no I Totally agree I read your book only not all other authors out there buddy the no I I think that's yeah you can't be badgering you can't be negative it can't be

you know kind of you know some type of justice Scalla you did this wrong but the same token I think the messaging that I'm hearing is that we need people to really understand the impact if you're an uber you're the impact of getting wrong as you shut down commerce for around the World I mean if you really get it wrong nobody get you know if like everybody if in San Francisco and there's no more cabs and you get it wrong nobody's getting anywhere and then what happens when nobody's getting anywhere people are missing fights anyway

I have to uh I might expose my reaction I don't explain my reaction and I I think one thing that we should do for next year's programming once it's a primary Mechanism here to encourage experiential learning is to experience reports but there is definitely the survivor bias you know we are picking the best stories where the you know the hero vanquishes the villain and the risks and I it occurred to me right that there is a whole dimension by elevating the sense of danger by telling the stories right it is uh that is a need

that must be served so I I know and and we had that discussion last night over dinner is Actually quite a serious one there is there is a second victim trauma issue in in you know there's suicide issues it has a lot to do with of course the loneliness of the profession and not seeing feeling part of or in any practical natural sense being part of a professional context that that ups that risk but but allowing people to tell these stories in a safe space is is incredibly and I know that all we have to

do is ask I will have a look at my Long I was it to do that what other advice would you give us thank you Steve for that I know exactly what to do with that I would like you to I would like you to try and create opportunities for reflection not just for storytelling yeah for opportunities for reflection about where we are and what we're doing and what our systems are actually being used for I don't think we can probably do this alone or inside individual firms I think That the culture of a firm

the environment of a firm the circumstances of a firm are unique and create a particular ambience in an environment and I think we need to find ways to break out of the firm based approach to things and that may mean insertions collections groups professions to span different things but I want to want to try and not put I don't think all the burden of this beyond the corporate manager and I think We should find a way to spread that burden across our the collection of people who do this this should be for example part of

the educational system when people are studying computer science in their schools and colleges it needs to be part of that curriculum and and it needs to be part of the kinds of discussions that we have it should be part of the interviewing strategy when we're talking to people about how they're going you know what what kinds Of things they want to do or what kind of experience I had I'm a big fan of the you know the circles of control influence a concern and yeah one is encouraged to focus on what you have control over

you know so if we gave you a venue here at the DevOps Enterprise summit to you know you know where you had people with interest in that could you help us structure that kind of exercise something I think we could but I think what we would do is we would Call on the people that we already know from the industry and the people who are involved in this stuff the folks who participated for instance in the stellar report preparations that we just got done yeah these are people who are articulate about the kinds of issues

that arise and are comfortable doing it and have those kinds of qualities so yes I think we can help as a specific you're saying John all spa might be a good candidate Joe knows well but there are Also people you know and and people who are not immediately visible to this community because they don't they don't give presentations as none know but are actually leaders and thinkers so one of the things that we noticed in one of the conversations that we haven't we track the slack record of an outage and how people were talking and

trying to solve it okay and one of the things that we noticed was that although there are very few communications from one particular Person when that person spoke everybody listened yeah and when you can find people like that you found incredible repositories of knowledge wisdom insight and if you can find those people and bring them forward that's a resource that you could call on and and we should be looking for and trying to find those people I know exactly where to find them okay good you