Complexity Explorer Lecture: David Krakauer • What is Complexity?

So what is complexity what I'm going to try and do is section the concept to look at it from a multitude of different angles which I think is appropriate for any area of inquiry we will do this is we'll look at complexity as a discipline the way we might look at physics as a discipline well look at the Domain that complexity is investigating we'll also look at its methods it's epistemology the kinds of mathematics that you use in complexity science and then relate it to one of the more fundamental concepts related to complexity and that's

emergence so complexity in the disciplines if I asked you to define biology or geography how would you do it it's extremely hard to do and it's no easier for complexity science so the way to do this I think is to break up a discipline into its constituent parts or its preferred methods and approaches and we're going to do that for a number of disciplines and then finally do it for complexity So here are some disciplinary traits how quantitative is a field is it obsessed with measurement and calculation as a natural scientists have tended to be

or is it more satisfied with the qualitative narrative type of account now reductionist relates to explanations that feel fulfilling by virtue of presenting the parts of a system so you go down levels to account for the level of Interest that's what we mean by reductionism so a physicist might say to understand gravity I need to understand gravitons or to understand electricity electrons and so forth another question which gets confounded with reduction is compression which is are the fields amenable to compressive mathematical description can you write down short equations that capture the essence of the phenomenon

that's not the same thing as reductionism it's a different kind of reduction a reduction to short compressed equations and finally how historicist is the field how important is history In accounting for a phenomenon how far back do you have to go and of course in biology uh we feel you have to go way back to understand traits in organisms we have phylogenetic explanations less so in physics and so that if you like quartet of characteristics helps us to define a discipline so let's consider physics so physics is obviously a very quantitative field everything is presented

in terms of numbers and measurements and so forth it's also a very reductionist because the search for Grand unified theories or fundamental theories typically consist in looking for elementary constituents for example the standard model in physics the minimum number of particles and Fields and having done that presenting that in mathematical terms using very elegant very compressed mathematical formula f equals m a Maxwell's laws and so on so physics has that characteristic that sometimes described as the back of the envelope like calculations short calculations based on fundamental constituents that are highly quantitative Okay so systems biology

is highly quantitative it's also very reductionist you try to understand traits in terms for example of their genetic or epigenetic factors and it's also very historicist you understand things in a taxonomical or phylogenetic framework but it's not very compressive it doesn't present regularities in terms of um very short elegant mathematical equalities then there's something like biological anthropology or biological linguistics and here they're slightly less quantitative there's there's less data than there would be for example in genomics they tend to be reductionist trying to understand things again in terms of biological factors that contribute to behavior

very historicist phylogenetic and also somewhat compressive that is using fundamental evolutionary theories like kin selection to understand Behavior and having explained all of those to try and illustrate that all these fields should be understood in terms of how much they weight different factors or traits where does complexity science fit well complexity science is very quantitative by and large uh it's very Historical we're studying you know adaptive agents and it's very compressive because we're looking for mathematical theories that capture essential regularities but what we are not is reductionist right we're not looking down levels to explain

the level of Interest and that's one of its defining features that we'll come back to at some point when we talk about emergence so having talked a little bit about what complexity might mean as a discipline let's talk about what complexity science studies that is the domain the territory of inquiry that establishes the way it looks and feels so if we look at classical mechanics physics this is the study of very ordered processes and you can write down equations that describe the orbits of the planets and the Stars in a very compressive compact form that

means low complexity in this case complexity relates to in in some sense the number of pages of equations required to describe the regularity of Interest so in that sense Newton's laws Are very compressed when you get to quantum mechanics that introduces more stochasticity more Randomness and the equations even though they're still classical or correspondingly slightly more complicated interestingly if you introduce lots of Randomness you can also write down a very compressed description in terms of statistical mechanics and thermodynamics so these two limits or in some sense the limits of the physical world and that's why

physics has been so effective at theorizing about phenomena but if you now look at the Domain where noise and regularity compete the complex domain what happens we don't really know we need uh entirely new kinds of theories to describe this intersection where Frozen accidents dominate that is the world of nature or of culture and so here's some examples on the left classical mechanics just a little bit more randomness the wave equation in quantum mechanics on the far right where you get a lot of randomness the description of the entropy of a system and again in

the Middle where that c is written Some new mathematics some new description is required that respects the complex domain now what's happened in the 21st century is that two very distinct approaches have evolved to deal with complexity on the one hand you have machine learning AI that encodes whole libraries of big data sets with billions of parameters that produce within a very circumscribed range highly predictive Solutions on the other hand you have complexity science which tries to do something closer to what physics was trying to do that is a smaller number of essential processes and

equations which describe regularities but don't predict so it looks as if we've reached this point of bifurcation where you have to make a decision I can either go down the path of prediction and lose understanding and comprehensibility or go down the path of mechanism and understanding and lose prediction and I think the open question that we're all dealing with is could we reconcile these two different approaches to the complex Domain here are some examples of methods and Frameworks in complexity science that have been invented to deal with the complex domain and his for example scaling

Theory um that is what patterns or regularities span multiple different orders of magnitude in space and time agent-based models which take seriously the idea of agency or reflexivity that is the things that we study in the complex domain have teleology they have purpose they have function and that's not true in physics a network theory that takes seriously the collective Dynamics are of complex systems and of course one of the interesting things about these three is that they find application so in scaling Theory we can explain how long organisms live how many species we typically find

in a in a unit area we can even apply scaling Theory to social phenomena where we're interested in how patent production for example scales as a function of City size Network theory is used ubiquitously uh In this particular case to study political polarization or the spread of disease and agent-based models are the preferred computational tools for looking at things like swarming flocking and congestion in cities so of Interest here is that even though the complex domain doesn't yield to these highly compressive formalisms they prove to be extremely useful in studying real world problems so when

we talked about the complex domain what we were talking about is the structure of reality what we call ontology but then there's a question of how we understand that reality how we describe it how we mathematize it the structure of knowledge itself and we call that epistemology and complexity science has a very interesting epistemology in 1960 a prominent physicist working in quantum mechanics Eugene vigna wrote a paper called the unreasonable effectiveness of mathematics in the physical sciences and vigno is very interested in this perplexing observation that you can Invent in mathematics freely through your imagination

and yet somehow that imagination that imaginary object can predict regular patterns in the natural world that have nothing to do with you so how is it that mathematics is so effective at explaining and predicting the real world and we can place that in a slightly more mathematical framing by saying that what amazed vigna is that models with very few parameters that is highly compressed very parsimonious models could predict phenomena very precisely and that's what's represented on these two axes here the x-axis showing the number of parameters and the axes coming out towards you how predictive

that model is and at the top there is an example of what he was amazed by Maxwell's equations here's another example from the founder of the Santa Fe Institute working in particle physics this is Marigold man and Murray wrote down uh Ali algebra a mathematical formalism to explain symmetries in this case eight-fold symmetries captured by something called su3 and by manipulating these lead groups he was able to predict Particles that had never been observed before so exactly to vigna's point somehow the mathematics generated a solution that didn't seem to be present in the mathematics to

begin with another example is the work of Paul Dirac this is the Dirac equation it's a relativistic wave equation it takes quantum mechanics and special relativity and merges them and he solved this equation and discovered negative energy states and he used those solutions to infer the existence of angimatter so no one has seen antimatter these equations were derived to describe the Ordinary World that we can measure and observe and yet they predicted something extraordinary so that's the world if you like the epistemological world that's made possible by the simple domains of physics right the two

edges of that graph I showed earlier which are either perfectly regular or perfectly random that's what they allow us to do that most of the world that we care About the social world the biological world and so on isn't like that it has a different ontology right it combines noise and Order and so what do we do now um it's not the unreasonable effectiveness of mathematics it's the unreasonable ineffectiveness of mathematics in dealing with a complex domain the key to effective core screening is that you don't lose predictive efficacy by losing degrees of freedom by

losing parameters so there are special domains where averaging is actually permitted and one very good example of that is the domain of scaling so in scaling Theory you get equations that look a bit like equations from physics this looks a bit like f equals m a this is basal metabolic rate scales as mass raised to the three-quarter powers and those threes and fours there are actually the dimensions of space three divided by the dimension displays plus one fractal Dimension so it's very physical um and you can derive these equations through a mathematics um of perivation

theory uh that would be Very familiar to the world of physics and here's an example of what that looks like and so scaling Theory gives us insights into the complex domain by using the concept of core screening very effectively but again as I said for many phenomena that's not an option and so much of complexity science does something different instead of trying to find parsimonious models like f equals m a or b equals m raised to the three quarter power it asks what gives rise to those structures in the first place what allows for the

possibility in the complex domain of course screening or what doesn't what produces the structure that we want to theorize about so let me make that quite explicit now and with an example if you think about machine learning and the performance of algorithms like alphago Alpha zero underlying all those lines of code and all those hundreds of millions if not billions of parameters is a very simple idea the idea of reinforcement learning and that can be written down in just a few lines of code that's just a few lines of mathematics So in that sense This

is highly compressive it's not the particular model that finally is instantiated but how the parameters are tuned and the same thing goes for biology people sometimes say evolutionary theory is not predictive well it's not predictive in the sense that you could predict a giraffe or a flea or a bacterium but all of them were subject to the same optimization principle in their local environment natural selection and drift and so what we're looking for is a parsimonious description of the algorithm or the process that produces the object physical science theorizes about the object parsimoniously we're theorizing

in some sense about a process that gives rise to an object a process that gives rise to a theory so in that sense complexity science is meta theoretic one of the concepts that one hears a lot about when talking about complexity is emergence it's it's its nearest relative in a certain sense and just like complexity it generates a lot of perplexity so I want to explain What emergence is now one very simple way of defining emergence is you're dealing with an emergent phenomenon when there's no need to look under the hood and I use that

in the following sense if your car stops um you're not quite sure why and the most natural thing to do is to check whether you've run out of gas so it's a kind of reductionism there because you're saying you understand why a car stops uh you need to look at its parts and when I phenomenon is strongly emergent you don't need to look under the hood and let me give an example this equation is the so-called fermat conjecture and it took hundreds of years to be solved and this is Andrew Wiles who finally solved it

uh between 1993 and 95 in fact the first solution had an error in it which freaked him out and then he was able to correct it but you can ask you know how did he do it how did he solve this theorem and let me show you quickly some pages from his proof here's one page where he establishes the relationship between the thermic conjecture and elliptical forms He goes through a whole series of ingenious deductive steps recruiting unexpected areas of mathematics until he finally arrives at the conclusion which is the proof of the Therma conjecture

now this proof is presented to us only in terms of mathematics right the language of mathematics is sufficient to establish The credibility of the result you don't have to look under the hood of Andrew Wiles to determine whether or not this proof is right or not for example we don't need to do brain science on Andrew Wilds we don't have to say you know the reason why the proof is correct is because he was expressing a lot of Serotonin or dopamine or this particular neural circuit was being recruited that would be interesting that would be

something that you might want to know but has nothing to do with the correctness of the proof similarly uh whilst his economic circumstances or the particular Market that he's working in and the University of Princeton who's paying his salary none of this is relevant to the correctness of the proof uh and neither is his nationality or his ideology so here's an example Where correctness operates entirely at the level of mathematics and moving below mathematics for example doing sort of particle physics on Andrew Wilder's brain might be interesting but is not Illuminating with respect to whether

the theorem has been proved or not now exactly the same thing goes for for go so here's the code that went into Alpha zero um on the left uh and here's some algorithms that it's using on the far right there's uh the algorithm playing Lee so dong if you wanted to understand whether Alpha zero was playing well you would observe the game you are you articulate the strategy and tactics at the level of the game you wouldn't have to go back down into the code to determine whether or not the move is a good one

because this particular for Looper is present just like Andrew Wiles and the Brain the phenomenon exists at the level of the game even though all that code is in support of it as Andrew Wilder's brain was in support of the proof this raises a very interesting question why is it that if my car doesn't read Its destination we feel it's appropriate to look under the hood why is it for example in the case of biological psychology or Psychiatry if you're interested in depression or mood swings you might think it's appropriate to look at the endocrineal

system look under the hood of behavior but when it comes to mathematics and its correctness there's no need and the basic idea here is that um there are multiple levels in any complex systems for example in the case of alpha zero there's Hardware there's Computing machines um on that Hardware you have software for example you might be running python or C or lisp code but what that code is encoding is strategy and each of those levels is described by a different language and each level corresponds to a different kind of understanding and the key idea

here is like the key idea that I mentioned earlier in relation to scaling which is core screening that that as you move from one level to the next you're projecting many states onto a smaller Number of states so a computer has transistors with many binary States code has a smaller number of variables and the strategy spaces in some sense the most compact of all that's the level in some sense which Theory exists to the extent that that level operates somewhat autonomously we often describe that level as a level of a discipline so that takes us

back to the beginning ontologically in terms of the structure of reality all of reality sits on physics sits on quantum foam so ultimately that's reality but we don't describe the world exclusively in terms of quantum mechanics we have theories of Harmony and Melody right we have theories of prosody we have the theory of evolution and all of those theories are made possible by emergence if emergence wasn't operating then we'd always have to go down to the lowest level right and the question then becomes can we theorize about the mechanisms that make strong emergence possible So

just to make that a bit more explicit what's implied by emergence is what we sometimes call functional closure or dynamical sufficiency or horizontal entailment this is the the jargon of emergence and what it's saying is if you've picked a level that's legitimately emergent then that's the only level you need to know to understand its future development right and that's what this illustration is trying to show any given level is necessary and sufficient to predict its own future so let's just look at the case of go so at the highest level go is a strategic tactical

game and if you are an expert watching the board and analyzing the game there would be no additional value in asking what computer language was alphago written in what how many for Loops were there in the code go into the level of the software would not be Illuminating in understanding how the game is being played similarly going down another level to The level of hardware saying what's that GPU doing uh when the game is being played that wouldn't illuminate the progress of the game it's happening it's necessary it's the physical substrate upon which the algorithms

and the strategies are being run but they're not the level at which you would understand the game and so in this particular diagram that arrow that goes from one level to the other is signifying that to understand the development of the game it is sufficient to analyze the state of the board and that means that go shows emergence because I don't need to go down to the level of the software there's no additional value in terms of the unfolding of the game and if that were not true um if the only way I could understand

what Alpha zero was doing was to look at the state of the hardware You could argue that's just a bad design there's a bug there's something wrong with the system so in in fact in many cases that we're interested in we're trying to build emergence in we're trying to build levels that are Protected from the lower levels so when does emergence fail so we've looked at the case of go and we've said that in go the board is sufficient and we don't have to look at the software or the hardware but let's imagine now that

you turned up the temperature in the room you turned it up so high that the cooling system of the gpus and CPUs failed and now Alpha zero is producing pathological moves bugs that are a consequence of overheating at that point the board is no longer sufficient the Strategic conception of the game is no longer sufficient you have to look at the software and particularly the hardware to understand the behavior and so emergence is even more subtle concept because it only apply applies within a certain range of operational parameters and when you move outside of that

range then you the protection breaks you get leakage across levels and you're forced to go down to a lower Level it's partly why in physics subatomic physics reductionism is practiced at very high energy scales because you're trying to look at the energy scales where you need to do reductionism and so a big and deep open question is in the world of complex phenomena how robust if you like is the emergent level and when are we forced to go down to the microscopic description and this is a little bit beyond this discussion but there are mechanisms

that that create protectorates so each of these levels in some sense screens off variation at a level below preventing leakage which means you don't have to look down so for example a lot of people might study as I said earlier a psychological phenomena in terms of the endocrine system hormones that's a good example where screening off is not taking place where you don't have protection so to understand the future of behavior you'd have to go down and look at the future of physiology right and What's so interesting about this is it opens up a dimension

of complexity which is a kind of pluralism because it's suggesting that there could be a science that explains why diversity of epistemology is necessary so let me just give you two examples um of these functionally closed protectorates one from physics and that's the ideal gas law this is a law that relates pressure and volume and temperature numbers of particles and this law applies irrespective of the gas that you're studying right and the reason why you can write down this equation is because there are theories Newton's Second Law the so-called flux theorem the equipartition theorem these

are all the mathematical elements that prove protection and say you know what you can study the gas level of the so-called variables estate volume entropy pressure temperature you don't have to go down to the particles and their particular energy states another example closer to complex systems is something like Zip floor and zip floor is a very well known regularity for example in bodies of text That says that there's a relationship between the rank order of a word and its frequency so for example uh the second most common word in a text is half as common

as the most common the third most common one-third as frequent and so on so there's this relationship it's not a zip floor turns out that it doesn't matter what text you study you find it and so a little bit like the ideal gas law there's a set of network theories combinatorial theories probabilistic theories that tell you why irrespective of the text you expect to find this scaling relationship so again this is a instance of of mechanisms of protection so now we can ask a question of how all these things come together how does the complex

domain the epistemology of complex systems and emergence relate and the key Point here is that by and large when you study complexity science you're theorizing about emergent levels and in particular you're theorizing about those emergent levels using Algorithmic theories and so if we were theorizing about the mind we would like to know that it was not necessary to describe the brain and then we want to know what the variational principle or the optimization principle is that's appropriate for mind so you might have a learning theory that is quite independent of the substrate of the brain

doesn't really matter because the mind is a protected state and to understand its future Evolution you don't need to look at the future evolution of the brain now it might not be true but then it wouldn't be a truly emergent phenomenon and so that's how in some sense these various Concepts come together and why complexity is so closely related to emergence just to sort of conclude here like everything else complexity is complex and there's no simple definition we can look at it through this lens of a discipline and that means to look at its characteristics

not to define it which I find futile We can look at the world the ontological reality the complexity Sciences are interested in and that's that world that's perched between order and disorder we can ask what kinds of mathematics or computation that world requires that complexity epistemology and then how that relates to emergence and in particular this fact that most of what we study are these emergent properties that is new levels of organization which are in themselves sufficient uh to understand their future states that um they encode a certain kind of entailment such that you don't

have to go down levels as physics does through reductionism to get a good understanding of how they operate so what is complexity what I'm going to try and do is section the concept to look at it from a multitude of different angles which I think is appropriate for any area of inquiry we will do this is we'll look at complexity as a discipline the way we Might look at physics as a discipline well look at the Domain that complexity is investigating we'll also look at its methods it's epistemology the kinds of mathematics that you use

in complexity science and then relate it to one of the more fundamental concepts related to complexity and that's emergence so complexity in the disciplines if I asked you to define biology or geography how would you do it it's extremely hard to do and it's no easier for complexity science so the way to do this I think is to break up a discipline into its constituent parts or its preferred methods and approaches and we're going to do that for a number of disciplines and then finally do it for complexity so here are some disciplinary traits how

quantitative is a field is it obsessed with measurement and calculation as a natural scientists have tended to be or is it more satisfied with the qualitative narrative type of account now reductionist relates to explanations that feel Fulfilling by virtue of presenting the parts of a system so you go down levels to account for the level of Interest that's what we mean by reductionism so a physicist might say to understand gravity I need to understand gravitons or to understand electricity electrons and so forth another question which gets confounded with reduction is compression which is are the

fields amenable to compressive mathematical description can you write down short equations that capture the essence of the phenomenon that's not the same thing as reductionism it's a different kind of reduction a reduction to short compressed equations and finally how historicist is the field how important is history in accounting for a phenomenon how far back do you have to go and of course in biology uh we feel you have to go way back to understand traits in organisms we have phylogenetic explanations less so in physics and so that if you like quartet of characteristics helps us

to define a discipline so let's consider physics So physics is obviously a very quantitative field everything is presented in terms of numbers and measurements and so forth it's also a very reductionist because the search for Grand unified theories or fundamental theories typically consist in looking for elementary constituents for example the standard model in physics the minimum number of particles and Fields and having done that presenting that in mathematical terms using very elegant very compressed mathematical formula f equals m a Maxwell's laws and so on so physics has that characteristic that sometimes described as the back

of the envelope like calculations short calculations based on fundamental constituents that are highly quantitative okay so systems biology is highly quantitative it's also very reductionist you try to understand traits in terms for example of their genetic or epigenetic factors and it's also very historicist you understand things in a taxonomical or phylogenetic framework but it's not very compressive it doesn't present regularities in terms of um very short elegant mathematical Equalities then there's something like biological anthropology or biological linguistics and here they're slightly less quantitative there's there's less data than there would be for example in genomics

they tend to be reductionist trying to understand things again in terms of biological factors that contribute to behavior very historicist phylogenetic and also somewhat compressive that is using fundamental evolutionary theories like kin selection to understand Behavior and having explained all of those to try and illustrate that all these fields should be understood in terms of how much they weight different factors or traits where does complexity science fit well complexity science is very quantitative by and large uh it's very historical we're studying you know adaptive agents and it's very compressive because we're looking for mathematical theories

that capture essential regularities but what we are not is reductionist right we're not looking down levels to explain the level of Interest and that's one of its defining features that we'll come back to at some point When we talk about emergence so having talked a little bit about what complexity might mean as a discipline let's talk about what complexity science studies that is the domain the territory of inquiry that establishes the way it looks and feels so if we look at classical mechanics physics this is the study of very ordered processes and you can write

down equations that describe the orbits of the planets and the Stars in a very compressive compact form that means low complexity in this case complexity relates to in in some sense the number of pages of equations required to describe the regularity of Interest so in that sense Newton's laws are very compressed when you get to quantum mechanics that introduces more stochasticity more Randomness and the equations even though they're still classical or correspondingly slightly more complicated interestingly if you introduce lots of Randomness you can also write down a very compressed description in terms of statistical mechanics

and thermodynamics so these Two limits or in some sense the limits of the physical world and that's why physics has been so effective at theorizing about phenomena but if you now look at the Domain where noise and regularity compete the complex domain what happens we don't really know we need uh entirely new kinds of theories to describe this intersection where Frozen accidents dominate that is the world of nature or of culture and so here's some examples on the left classical mechanics just a little bit more randomness the wave equation in quantum mechanics on the far

right where you get a lot of randomness the description of the entropy of a system and again in the Middle where that c is written some new mathematics some new description is required that respects the complex domain now what's happened in the 21st century is that two very distinct approaches have evolved to deal with complexity on the one hand you have machine learning AI that encodes whole libraries of big data Sets with billions of parameters that produce within a very circumscribed range highly predictive Solutions on the other hand you have complexity science which tries to

do something closer to what physics was trying to do that is a smaller number of essential processes and equations which describe regularities but don't predict so it looks as if we've reached this point of bifurcation where you have to make a decision I can either go down the path of prediction and lose understanding and comprehensibility or go down the path of mechanism and understanding and lose prediction and I think the open question that we're all dealing with is could we reconcile these two different approaches to the complex domain here are some examples of methods and

Frameworks in complexity science that have been invented to deal with the complex domain and his for example scaling Theory um that is what patterns or regularities span multiple different orders of Magnitude in space and time agent-based models which take seriously the idea of agency or reflexivity that is the things that we study in the complex domain have teleology they have purpose they have function and that's not true in physics a network theory that takes seriously the collective Dynamics are of complex systems and of course one of the interesting things about these three is that they

find application so in scaling Theory we can explain how long organisms live how many species we typically find in a in a unit area we can even apply scaling Theory to social phenomena where we're interested in how patent production for example scales as a function of City size Network theory is used ubiquitously uh in this particular case to study political polarization or the spread of disease and agent-based models are the preferred computational tools for looking at things like swarming flocking and congestion in cities so of Interest here is that even though the complex domain doesn't

yield to these highly Compressive formalisms they prove to be extremely useful in studying real world problems so when we talked about the complex domain what we were talking about is the structure of reality what we call ontology but then there's a question of how we understand that reality how we describe it how we mathematize it the structure of knowledge itself and we call that epistemology and complexity science has a very interesting epistemology in 1960 a prominent physicist working in quantum mechanics Eugene vigna wrote a paper called the unreasonable effectiveness of mathematics in the physical sciences

and vigno is very interested in this perplexing observation that you can invent in mathematics freely through your imagination and yet somehow that imagination that imaginary object can predict regular patterns in the natural world that have nothing to do with you so how is it that mathematics is so effective at explaining and predicting the real world and we can place that in a slightly more mathematical framing by saying that what amazed vigna is that models with very Few parameters that is highly compressed very parsimonious models could predict phenomena very precisely and that's what's represented on these

two axes here the x-axis showing the number of parameters and the axes coming out towards you how predictive that model is and at the top there is an example of what he was amazed by Maxwell's equations here's another example from the founder of the Santa Fe Institute working in particle physics this is Marigold man and Murray wrote down uh Ali algebra a mathematical formalism to explain symmetries in this case eight-fold symmetries captured by something called su3 and by manipulating these lead groups he was able to predict particles that had never been observed before so exactly

to vigna's point somehow the mathematics generated a solution that didn't seem to be present in the mathematics to begin with another example is the work of Paul Dirac this is the Dirac equation it's a relativistic wave equation it takes Quantum mechanics and special relativity and merges them and he solved this equation and discovered negative energy states and he used those solutions to infer the existence of angimatter so no one has seen antimatter these equations were derived to describe the Ordinary World that we can measure and observe and yet they predicted something extraordinary so that's the

world if you like the epistemological world that's made possible by the simple domains of physics right the two edges of that graph I showed earlier which are either perfectly regular or perfectly random that's what they allow us to do that most of the world that we care about the social world the biological world and so on isn't like that it has a different ontology right it combines noise and Order and so what do we do now um it's not the unreasonable effectiveness of mathematics it's the unreasonable ineffectiveness of mathematics in dealing with a complex domain

the key to effective core screening Is that you don't lose predictive efficacy by losing degrees of freedom by losing parameters so there are special domains where averaging is actually permitted and one very good example of that is the domain of scaling so in scaling Theory you get equations that look a bit like equations from physics this looks a bit like f equals m a this is basal metabolic rate scales as mass raised to the three-quarter powers and those threes and fours there are actually the dimensions of space three divided by the dimension displays plus one

fractal Dimension so it's very physical um and you can derive these equations through a mathematics um of perivation theory uh that would be very familiar to the world of physics and here's an example of what that looks like and so scaling Theory gives us insights into the complex domain by using the concept of core screening very effectively but again as I said for many phenomena that's not an option and so much of complexity science does something different Instead of trying to find parsimonious models like f equals m a or b equals m raised to the

three quarter power it asks what gives rise to those structures in the first place what allows for the possibility in the complex domain of course screening or what doesn't what produces the structure that we want to theorize about so let me make that quite explicit now and with an example if you think about machine learning and the performance of algorithms like alphago Alpha zero underlying all those lines of code and all those hundreds of millions if not billions of parameters is a very simple idea the idea of reinforcement learning and that can be written down

in just a few lines of code that's just a few lines of mathematics so in that sense This is highly compressive it's not the particular model that finally is instantiated but how the parameters are tuned and the same thing goes for biology people sometimes say evolutionary theory is not predictive well it's not predictive in the sense that you could predict a giraffe or a flea or a bacterium but all of them were subject To the same optimization principle in their local environment natural selection and drift and so what we're looking for is a parsimonious description

of the algorithm or the process that produces the object physical science theorizes about the object parsimoniously we're theorizing in some sense about a process that gives rise to an object a process that gives rise to a theory so in that sense complexity science is meta theoretic one of the concepts that one hears a lot about when talking about complexity is emergence it's it's its nearest relative in a certain sense and just like complexity it generates a lot of perplexity so I want to explain what emergence is now one very simple way of defining emergence is

you're dealing with an emergent phenomenon when there's no need to look under the hood and I use that in the following sense if your car stops um you're not quite sure why and the most natural thing to do is to check whether you've run out of gas So it's a kind of reductionism there because you're saying you understand why a car stops uh you need to look at its parts and when I phenomenon is strongly emergent you don't need to look under the hood and let me give an example this equation is the so-called fermat

conjecture and it took hundreds of years to be solved and this is Andrew Wiles who finally solved it uh between 1993 and 95 in fact the first solution had an error in it which freaked him out and then he was able to correct it but you can ask you know how did he do it how did he solve this theorem and let me show you quickly some pages from his proof here's one page where he establishes the relationship between the thermic conjecture and elliptical forms he goes through a whole series of ingenious deductive steps recruiting

unexpected areas of mathematics until he finally arrives at the conclusion which is the proof of the Therma conjecture now this proof is presented to us only in terms of mathematics right the language of mathematics is sufficient to establish The credibility Of the result you don't have to look under the hood of Andrew Wiles to determine whether or not this proof is right or not for example we don't need to do brain science on Andrew Wilds we don't have to say you know the reason why the proof is correct is because he was expressing a lot

of Serotonin or dopamine or this particular neural circuit was being recruited that would be interesting that would be something that you might want to know but has nothing to do with the correctness of the proof similarly uh whilst his economic circumstances or the particular Market that he's working in and the University of Princeton who's paying his salary none of this is relevant to the correctness of the proof uh and neither is his nationality or his ideology so here's an example where correctness operates entirely at the level of mathematics and moving below mathematics for example doing

sort of particle physics on Andrew Wilder's brain might be interesting but is not Illuminating with respect to whether the theorem has been proved or not now exactly the same thing goes for for go so here's the code that went into Alpha zero um on the left uh and here's some algorithms that it's using on the far right there's uh the algorithm playing Lee so dong if you wanted to understand whether Alpha zero was playing well you would observe the game you are you articulate the strategy and tactics at the level of the game you wouldn't

have to go back down into the code to determine whether or not the move is a good one because this particular for Looper is present just like Andrew Wiles and the Brain the phenomenon exists at the level of the game even though all that code is in support of it as Andrew Wilder's brain was in support of the proof this raises a very interesting question why is it that if my car doesn't read its destination we feel it's appropriate to look under the hood why is it for example in the case of biological psychology or

Psychiatry if you're interested in depression or mood swings you might think it's appropriate to look at the endocrineal system look under the hood of behavior but when it comes to mathematics and its correctness there's no need And the basic idea here is that um there are multiple levels in any complex systems for example in the case of alpha zero there's Hardware there's Computing machines um on that Hardware you have software for example you might be running python or C or lisp code but what that code is encoding is strategy and each of those levels is

described by a different language and each level corresponds to a different kind of understanding and the key idea here is like the key idea that I mentioned earlier in relation to scaling which is core screening that that as you move from one level to the next you're projecting many states onto a smaller number of states so a computer has transistors with many binary States code has a smaller number of variables and the strategy spaces in some sense the most compact of all that's the level in some sense which Theory exists to the extent that that

level operates somewhat autonomously we often describe that level as a level Of a discipline so that takes us back to the beginning ontologically in terms of the structure of reality all of reality sits on physics sits on quantum foam so ultimately that's reality but we don't describe the world exclusively in terms of quantum mechanics we have theories of Harmony and Melody right we have theories of prosody we have the theory of evolution and all of those theories are made possible by emergence if emergence wasn't operating then we'd always have to go down to the lowest

level right and the question then becomes can we theorize about the mechanisms that make strong emergence possible so just to make that a bit more explicit what's implied by emergence is what we sometimes call functional closure or dynamical sufficiency or horizontal entailment this is the the jargon of emergence and what it's saying is if you've picked a level that's legitimately emergent then that's The only level you need to know to understand its future development right and that's what this illustration is trying to show any given level is necessary and sufficient to predict its own future

so let's just look at the case of go so at the highest level go is a strategic tactical game and if you are an expert watching the board and analyzing the game there would be no additional value in asking what computer language was alphago written in what how many for Loops were there in the code go into the level of the software would not be Illuminating in understanding how the game is being played similarly going down another level to the level of hardware saying what's that GPU doing uh when the game is being played that

wouldn't illuminate the progress of the game it's happening it's necessary it's the physical substrate upon which the algorithms and the strategies are being run but they're not the level at which you would understand the game and so in This particular diagram that arrow that goes from one level to the other is signifying that to understand the development of the game it is sufficient to analyze the state of the board and that means that go shows emergence because I don't need to go down to the level of the software there's no additional value in terms of

the unfolding of the game and if that were not true um if the only way I could understand what Alpha zero was doing was to look at the state of the hardware You could argue that's just a bad design there's a bug there's something wrong with the system so in in fact in many cases that we're interested in we're trying to build emergence in we're trying to build levels that are protected from the lower levels so when does emergence fail so we've looked at the case of go and we've said that in go the board

is sufficient and we don't have to look at the software or the hardware but let's imagine now that you turned up the temperature in the room You turned it up so high that the cooling system of the gpus and CPUs failed and now Alpha zero is producing pathological moves bugs that are a consequence of overheating at that point the board is no longer sufficient the Strategic conception of the game is no longer sufficient you have to look at the software and particularly the hardware to understand the behavior and so emergence is even more subtle concept

because it only apply applies within a certain range of operational parameters and when you move outside of that range then you the protection breaks you get leakage across levels and you're forced to go down to a lower level it's partly why in physics subatomic physics reductionism is practiced at very high energy scales because you're trying to look at the energy scales where you need to do reductionism and so a big and deep open question is in the world of complex phenomena how robust if you like is the emergent level and when are we forced to

go down to the Microscopic description and this is a little bit beyond this discussion but there are mechanisms that that create protectorates so each of these levels in some sense screens off variation at a level below preventing leakage which means you don't have to look down so for example a lot of people might study as I said earlier a psychological phenomena in terms of the endocrine system hormones that's a good example where screening off is not taking place where you don't have protection so to understand the future of behavior you'd have to go down and

look at the future of physiology right and what's so interesting about this is it opens up a dimension of complexity which is a kind of pluralism because it's suggesting that there could be a science that explains why diversity of epistemology is necessary so let me just give you two examples um of these functionally closed protectorates one from physics and that's the ideal gas law this is a law That relates pressure and volume and temperature numbers of particles and this law applies irrespective of the gas that you're studying right and the reason why you can write

down this equation is because there are theories Newton's Second Law the so-called flux theorem the equipartition theorem these are all the mathematical elements that prove protection and say you know what you can study the gas level of the so-called variables estate volume entropy pressure temperature you don't have to go down to the particles and their particular energy states another example closer to complex systems is something like Zip floor and zip floor is a very well known regularity for example in bodies of text that says that there's a relationship between the rank order of a word

and its frequency so for example uh the second most common word in a text is half as common as the most common the third most common one-third as frequent and so on so there's this relationship it's not a zip floor turns out that it doesn't matter what text you study you find it And so a little bit like the ideal gas law there's a set of network theories combinatorial theories probabilistic theories that tell you why irrespective of the text you expect to find this scaling relationship so again this is a instance of of mechanisms of

protection so now we can ask a question of how all these things come together how does the complex domain the epistemology of complex systems and emergence relate and the key Point here is that by and large when you study complexity science you're theorizing about emergent levels and in particular you're theorizing about those emergent levels using algorithmic theories and so if we were theorizing about the mind we would like to know that it was not necessary to describe the brain and then we want to know what the variational principle or the optimization principle is that's appropriate

for mind so you might have a learning theory that is quite independent of the Substrate of the brain doesn't really matter because the mind is a protected state and to understand its future Evolution you don't need to look at the future evolution of the brain now it might not be true but then it wouldn't be a truly emergent phenomenon and so that's how in some sense these various Concepts come together and why complexity is so closely related to emergence just to sort of conclude here like everything else complexity is complex and there's no simple definition

we can look at it through this lens of a discipline and that means to look at its characteristics not to define it which I find futile we can look at the world the ontological reality the complexity Sciences are interested in and that's that world that's perched between order and disorder we can ask what kinds of mathematics or computation that world requires that complexity epistemology and then how that relates to emergence And in particular this fact that most of what we study are these emergent properties that is new levels of organization which are in themselves sufficient

uh to understand their future states that um they encode a certain kind of entailment such that you don't have to go down levels as physics does through reductionism to get a good understanding of how they operate