Statistical Mechanics Lecture 1

[Music] Stanford University okay if there are no more questions tonight we begin the study of statistical mechanics now statistical mechanics is not really modern physics it's premodern physics it's modern physics and I assure you it will postmodern physics it's Probably the second law of Thermodynamics will probably Outlast anything that uh that comes up time and time again the second law of Thermodynamics has been sort of our um our guide poost our Guiding Light if you like uh to um to know what we're talking about and to make sure we're making sense statistical mechanics and thermodynamics

may not be as sexy as the higs BOS on but I assure you it is at Least as deep [Music] um it's a lot deeper my particle physics friends shouldn't uh disown me it's a lot deeper it's a lot more General and it covers a lot more ground and explaining the world as we know it and in fact um without statistical mechanics we probably would not know about the higs Bosone all right so um with that little starting point what is statistical mechanics about well let's go back a step the laws of physics the basic

laws of physics Newton's Laws principles of classical physics classical mechanics the things that uh were in classical mechanics course quantum mechanics and so forth those things are all about Predictability perfect predictability now you say well in quantum mechanics you can't predict perfectly and that's true but there are some things you can predict perfectly and um those things are the predicta bles of quantum mechanics again as in classical mechanics you can make your predictions with maximal let's call it maximal um Precision or maximal whatever it is predictability if you know two things if You know the

starting point which is what we call Initial conditions and if you know the laws of evolution of a system if you can measure the or if you know for whatever reason the initial starting point of a closed system a closed system means one which is either everything or it is sufficiently isolated from everything else that the other things in the system don't influence it if you have a closed system if you know the initial Conditions exactly or at least with uh whatever Precision is necessary and you know the laws of evolution of the system you

have complete predictability and that's all there is to say now of course in many cases that complete predictability would be totally useless having a list of the position and velocities of every particle in this room would not be very useful to us the list would be too Long and uh subject to rather quick change as a matter of fact so you know you can see while while the basic laws of physics are very very powerful in their predictability they also in many cases can be totally useless uh for actually analyzing what's really going on statistical

mechanics is what what you use basically probability Theory statistical mechanics let me say first Of all it is just basic probability Theory statistical has applied to physical systems but when is it applicable it's applicable when you don't know the initial conditions with complete Perfection it's applicable uh when you it may even be applicable if you applicable if you don't know the laws of motion with infinite precision and it's applicable when the system you're investigating is not a closed system whether it's interacting With other things on the outside in other words in just those situations where

ideal predictability is impossible then what do you resort to you resort to probabilities but because the number of molecules in this room is so large and probabilities tend to become very very precise predictors when numbers when the laws of when the laws of large numbers are applicable statistical mechanics itself Can be highly predictable but not for everything um as an illustration you have a box of gas the box of gas might even be an isolated closed box of gas it has some energy in it particles rattle around if you know some things about that box

of gas You can predict other things with great Precision if you know the temperature you can predict uh the uh the energy in the box of gas You can Predict the pressure these things are highly predictable but there are some things you can't predict you can't predict the position of every molecule you can't predict when there might be a fluctuation a fluctuation which uh you know fluctuations are things which happen which don't really violate probability Theory they're the sort of tales of the probability distribution things which are unlikely but not impossible Fluctuations happen from time

to time in the sealed room every so often uh an extra large group an extra large density of molecules will appear in some small region bigger than the average someplace else the molecules will be less dense and fluctuations like that are hard to predict You can predict the probability for a fluctuation but you can't predict when a fluctuation is going to happen uh it's exactly the same sort of Thing flipping coins flipping coins is a good example is probably our favorite example for thinking about probabilities uh if I flip a coin a billion times you

can bet that approximately half of them will come up heads and half will come up Tails within some margin of error but there will also be fluctuations every now and then if you do it enough times a thousand heads in a row will come up can you predict when That thousand heads will come up no but can you predict how often a thousand heads will come up yes not very often uh so that's what statistical mechanics is for it's for making statistical probabilistic predictions about systems which are either too small contain elements which are too

small to see too numerous to keep track of usually both too small to see by see I mean you know you it's true you can see Some pretty small things molecules but uh uh so they they may not be too small to see but there were too many of them there are too many of them to keep track of and that's when we use probability Theory or statistical mechanics um we're going to go through some of the basic statistical mechanics uh applications not just applications the theory the laws of thermodynamics the laws of statistical mechanics

and then How they apply the gases liquids solids uh whether we will get the quantum mechanical systems or not I don't know but um but just the basic uh the basic ideas okay in incidentally another thing which is very striking is that generally speaking over the history of certainly over my history in physics um and I'm sure this goes back to to the middle of the 19th Century sometime all great physicists all of them were masters of statistical mechanics it may not have been the sexiest thing in the world but they were all masters of

it why first of all because it was useful but second of all because it is truly beautiful it is a truly beautiful subject of physics and Mathematics and um it's hard not to get caught up in it not hard not to get uh not to fall in love with it the reason I teach it is not for you it's for me I Love teaching it I love teaching it I teach it over and over and over again and in a sense my life is consisted of um learning and forgetting and learning and forgetting and learning

and forgetting statistical mechanics so here's my opportunity to learn it again okay um let's begin with what I usually call a mathematical interlude in this case it's not an interlude it's a starting point and I'm just going to Make some extremely brief remarks which you all know at least I think you all know them about probability just to have uh you know just to let level the ground what are we talking about and um what I am not going to explain because I don't think anybody can explain it is why probability works why does it

work if you ask why it works the first answer will be it doesn't always work you may have a probability for something and you test It out and sometimes it doesn't work those are called the exceptions so the answer to the question is why does it work well doesn't always work it mostly works except when it doesn't when doesn't it rarely how rarely every so often but there is a calculus of probability a mathematical theory of probabilities and we'll talk about it a little bit okay so we'll take probability To Be A Primitive concept basically

primitive Concept and we'll suppose that there is a space of some sort a space of possibilities the space of possibilities could be the space of outcomes of experiments or it could actually be the space of states of a system a state of a system could be the outcome of an experiment if the experiment consists of determining the state of the system then the state of the system is the outcome so we have a space and let's Call that space let's label the elements of that space with a little ey okay for example if we were

flipping coins I would be either heads or tails if we were flipping Dy you know like in dice dies of dice in dice uh I would run from 1 to six if there were two dyes then uh then we would have uh enough uh enough indices to keep track of two dice and so forth so I is the space of possibilities of outcomes or the space of possible states of a System and if we are ignorant um statistics always has to do with ignorance you don't know everything and so you assign probabilities to outcomes and

so we assign a probability P of I to the I outcome to the answer to our question okay what are the rules for p ofi what does p ofi have to satisfy and let's for the beginning at least in the beginning let's imagine that I enumerates some discrete finite Collection of possibilities later on we can have an infinite number of possibilities or even a continuously infinite number of possibilities but for the time being uh I might run from one to n n possibilities and the rules are first of all P sub I has to be

be greater than or equal to zero negative probabilities we don't like them don't know what they mean okay Next the summation over I of P sub i p of I should be one that means that the total probability when you add everything up all possibilities should be one you certainly should get some result okay next now this is a kind of hypothesis this is the law of large numbers that if you either make many replicas of the same system or do the same experiment over and over of very very large number Of times right and

take all of the outcomes which gave you all of the experiments which gave you the I outcome that's some number let's call it n of n of I that's the number of times that the experiment turned up the I possibility and you divide it by the total number of Trials total number of Trials means the sum of all I or just the total number of Trials that the limit of this this is a physical hypothesis it's a physical hypothesis it can go wrong if n is not large enough but in the limit of large n

n n goes to Infinity in the limit of very very of course n never goes to Infinity you never get to do an infinite number of experiments but nevertheless we're kind of idealizing we're assuming we can do so many experiments that the limit n Goes to Infinity um is effectively uh been reached then that is p of I so P ofi controls by assumption the ratio of the N of I's okay everybody happy with that you use this all the time I think well sometimes we use it okay now let's suppose that there is a

quantity let's call it f ofi it's some quantity that's associated with the I State we can assign it we can make it Up for example if our system is heads and tails and nothing but heads and tails we could assign F of heads and call it + one and F of tals and call it minus one if our system has many many more States we may want to assign a much larger number of possible FS but f is some function of the state it's also a thing that we imagine measuring it could Be the energy

of a state or it could be the moment of a state given a state of some system it has an energy would be called in that case perhaps e of I or it could be the momentum or it could be something else it could be whatever whatever you happen to like to uh to think about then an important quantity is the average of F ofi and the average of F ofi I will use the quantum mechanical Notation for it even though we're not doing quantum mechanics it's a nice notation physicists tend to use it all

over the place mathematicians hate it um just put a pair of brackets around it it means the average the average value of the quantity averaged over the probability distribution it has a definition its definition is that it's the sum of I of f of I weighted with the Probability for example if and incidentally the average of f ofi does not have to be any of the possible values that F can take on for example in this case where F of heads is + one and f f of taals is minus one and you flip a

million times and the probability is a half of heads and a half for tails the average of f will be zero zero is not a possible outcome to the experiment there's no rule why the Um why the average should be one of the possible experimental outputs but it is the average and this is its definition it's each value of f is weighted with the probability for that value of f you can write it another way you can write it as a sum over I of f of I times the number of times that you measure

I divided by the total number of Measurements that's what P of I is in the limit that there are a large number of measurements okay so that's defined to be the average that's our mathematical preliminary for today that's all we uh that's all I wanted to uh to level a playing field by making sure everybody knows what a probability is and what an average is because we'll use it over and over okay let's start with let's start With coin flips I always start with coin I start every single class with coin flips even when I'm

teaching about the pigs bowz on okay if I flip a coin a lot of times well whether I flip a coin a lot of times or not the probability for heads is usually deemed to be 1/2 and the probability for tails is usually also deemed to be 1/2 why do we do that why is a why is it a half and a half What's the what's the logic there what the what logic tells us that and in this case it's symmetry it's the symmetry of the coin of course no coin is perfectly symmetric and even

making a little Mark on it to distinguish a heads and tails uh biases it a little bit but apart from that tiny tiny bias of marking the coin with maybe just a tiny little scratch the coin is symmetric heads and tails are symmetric With respect to each other and therefore there is no no no reason no rationale for when you flip a coin for the uh for it to turn up heads more often than tails and so it's symmetry quite often I might even say always in some deeper sense but at least um at least

in many cases symmetry is the thing which dictates probabilities probabilities are usually taken to be equal for configurations which are Related to each other by some symmetry symmetry means if you you know you act with a symmetry you reflect everything you turn everything over that uh that the system behaves the same way okay another example besides coin flipping would be dice flipping dice flipping instead of having two states has six states one die and we could imagine coloring them let uh we color the faces red yellow blue And then on the back green uh purple

and orange okay that's our that's our Dy and it's been colored uh we don't have to keep track of numbers we can keep track of colors and what is the probability that uh that when we flip the die flip it into the air hits the ground what's the probability that it turns up up red well it's 1 six right there are six possibilities they're all symmetric with respect to each other we use the principle of symmetry to tell us That the P of each I they're all equal and they're all equal to 16 but what

if there is no symmetry what if really the die is not symmetric for example what if it's weighted in some unfair way or what if it's been cut uh with faces that are not nice and parallel cubes then what's the answer the answer is symmetry won't tell you you may be able to use some deeper underlying Theory and to use some concept of Symmetry from the deeper underlying Theory but in the absence of some uh some something else the there is no answer the answer is experiment do this experiment a billion times keep track of

the numbers assume that things have converged and that way you measure the probabilities you measure the probabilities and thereafter you can use them you can use them if you you keep a table of them and then you can use them In in the next round of experiments or you may have some theories some deep underlying Theory which tells you where like quantum mechanics or statistical mechanics statistical mechanics tends to rely mostly on symmetry as we'll see so if there's no symmetry to guide you or to guide your um implementation of probabilities then it's experiment okay

now there's another answer there's another possible answer This answer is often frequently invoked and it's a correct answer under other circumstances it can have to do with the evolution of a system the way a system changes with time so let me give you some examples of what it might have to do let's take our six-sided Cube and assume that our six-sided cube is not symmetric it's not symmetric uh but we know a rule we know That if we put that Cube down on the table it's not a cube when we put that die down on

the table and we stand back this thing has this habit of jumping to another state and jumping to another state and jumping to another state it's called the law of motion of the system the law of motion of the system is that whatever it is at one instant at some next instant it will be something else according to a definite rule the instance could be seconds it could be Microc seconds or whatever but imagine a discrete sequence and let's suppose there's a law a genuine law that tells us how this Cube moves around for example

if it's red now we've done this over and over many times in different contexts but it is so important that I feel a need to emphasize it again this is what a law of motion is it's a rule telling you what the next configuration will be given it's a rule of updating of updating Configurations red goes to blue blue goes to yellow yellow goes to Green green goes to Orange Orange goes to purple and purple goes back to Red okay given the configuration at any time you know what it will be next and you know

what it will continue to do of course you may not know the law maybe all you know is that there is a law of This type all right so you know what I'm going to do next what am I going to do next I'm going to draw this law as a diagram you've all seen me do this in other contexts so let's do it we have red let too hard to draw squares red blue green orange what happened oh yellow yellow orange purple all right so a law like this can be just represented by a

set of lines Connecting a set of arrows red goes to blue blue goes to Green Green goes to yellow yellow goes to Orange Orange goes to purple purple goes back to Red making the Assumption now that there's a discrete time interval between such events I am not assuming that the cube has any symmetry to it anymore the cube may not be symmetric at all it may have points uh you know one one Edge one face may be tiny another Face but if this is the rule to go from one configuration to another and each step

takes let's say a microsc I might know I might have no no idea where I begin but I can still tell you if I let's say it's a microsc a microsc and my job is to catch it at a particular instant and ask what the color is I don't know where it started okay but I can still tell you the probability for each one of these Is6 doesn't have to do with symmetry well maybe it does have to do with some symmetry but in this case it wouldn't be the Sy the symmetry of the structure

of a d it would just be the fact that as it passes through these sequence of states it spends one sixth of its time red one sixth of its time blue one sixth of a Time green and if I don't know where it starts and I just take a flash uh you know a flash shot of it my probability will be one 16 now that one six did not really depend on knowing the detailed law for example the law could have been different let's make up a new law red goes to Green Green goes to

Orange Orange goes to yellow yellow goes to purple purple goes to blue and blue goes back to Red this shares with the previous law that there's a closed cycle of events in which you pass through each color once Before you cycle around you may not know which the law of law of nature is is for this system but you can tell me again that uh that the probability will be 1 six for each one of them so this prediction of six doesn't depend on knowing the starting point and doesn't depend on knowing the law of

physics it's just important to know that there is a particular kind of law are there possible laws for the system which will not give you one six Yes let's write another law red blue green yellow orange purple this rule says that if you start to Red start with red you go to Blue if you start with blue you go to green if you and if you get the green you go back to red or if you start with purple you go to yellow yellow to Orange Orange back to purple notice in this case if you're

on one of these two cycles you stay there Forever if you knew you were on the upper cyc cycle if you knew you had started doesn't matter where you start but if you knew that you started on the upper cycle somewhere then you would know that there was a one-third probability to be red a onethird probability to be blue and a onethird probability to be green and zero probability to be red yellow or orange on the other hand you could have started with the second cycle you could Have started with purple might not have known

where you started but you knew that you started in the lower triangle in the lower cycle here then you would know the probabilities of 1/3 for each of these and zero for each of these now what about a more General case the more General case might be that you you know with some probability that you start on the upper triangle here and with some other Probability on the lower triangle in fact let's give these triangles names let's call this triangle the plus one triangle and this one the minus one triangle okay just giving them names

attaching to them a number a numerical value if you're here something or other is called plus one if here something or rather is called minus one all right now you have to append you have to start with something you've got to get from someplace else it doesn't follow from Symmetry and it doesn't follow from cycling through the system some probability that you're either on cycle plus one or cycle minus one where might that come from well flipping somebody else's coin over here flipping a coin over here might decide which of these two or it might

be a biased coin Co so you will have a probability to be + one and a probability to be minus one these two probabilities are not probabilities for individual colors they're Probabilities for individual Cycles Okay now what's the probability for blue the probability for blue begins with the probability that you're on the first cycle times the probability that if you're on the first cycle you get blue that's 1/3 so the probability for blue red or green is 1/3 the probability that you're on the first cycle and likewise the Probability that your yellow will be this

is the probability for red blue or green in this case and this * 1/3 will be the probability for purple yellow or orange okay so in this case you need to supply another probability that you've got to get from somewh else this case here is what we call having a conservation Law in this case the conservation law would be just the conservation of this number for red blue and green we've assigned the value + one that plus one could be the energy or it could be something else I tend to call it the zilch for

some reason I call everything a zilch if there is no name for it uh yeah so anyway let's let's think of it as the energy for to keep things uh familiar the energy of these three configurations might all be + one and The energy of these three configurations might all be minus one and the point is that because the rule keeps you always on the same cycle that quantity energy zil whatever we call it is conserved it doesn't change that's what a conservation law is conservation law is that the configuration space the space of possibilities

divides up into Cycles like this now the Cycles don't have to have equal size here's another case 1 2 2 three four you go around this Way and then the two guys over here go into each other so uh red goes to Blue Goes to Green goes to purple that's the upper cycle here and the lower cycle is uh yellow goes to Orange goes to Yellow goes to Orange still we have a uh conservation law here it's just the number of states with one value of the conserved quantity is not the same as the as

the number of states or the other Value but still it's a conservation law and again somebody would have to supply for you some idea of the relative probabilities of these two where that comes from is part of the study of statistical mechanics and the other part of the study has to do with saying if I know I'm one one of these tracks how much time do I spend with each particular Configuration that's what determines probabilities and statistical mechanics some a priori probability from some Wares that tells you the that tells you the probabilities for different

conserved quantities and cycling through the system Yeah question no no okay um so so far you're assuming that within any conservation Arena if you will the probabilities of all the stat Are the same the time spent in each state is the same right so it's it's completely deterministic laws are completely deterministic this would be classical physics laws completely deterministic no real ambiguity of what the state is except you're kind of lazy you didn't determine the initial condition your um your timing wasn't very good each state only lasts from microsc uh you're a lazy guy and

you uh and you only have a Resolution of a of a millisecond but uh but you're a nevertheless you're able to take a very quick flash picture and uh and pick out one of the states that's that's the circumstance that we're talking about question yeah um if we take two pictures is it reasonable to then assume that if the first picture indicated that we are in one cycle the the L later one should indicate the same cycle since it couldn't get out of it yes that's a good Assumption um yes right so once you determine

the value of some conserved quantities then you know it and then you can reset the probabilities for it okay um unless all right so let's let's talk about let's talk about honest energy for a minute yes if we have a closed system uh to represent the closed system I will just draw a box closed now closed means that it's not an interaction with anything else and therefore can be Thought of as a whole universe unto itself okay it has an energy the energy is some function of the state of the system whatever determines the state

of the system now let's suppose we have another closed system which is built out of two identical or not identical versions of the same thing now if they're both closed systems there will be two conserved quantities the energy of this system and the energy of this system and They'll both be separately conserved why because they don't talk to each other they don't interact with each other the two energies are conserved and uh you could have probabilities for uh each of those individuals but now supposing they're connected they're connected by a little tiny tube which allows

energy to flow back and forth and there's only one conserved quantity the total energy and it's sort of split between the two of them and you Can ask what's uh you can then ask what is the probability given a total amount of energy you could ask what's the probability that the energy of one subsystem is one thing and the energy of the other subsystem is the other if the two boxes are equal you would expect uh you would expect on the average they have equal energy but you can still ask what's the what's the probability

for a given energy in this box Given some overall piece of information so that's a circumstance where it may be that giving giving the probability for which cycle you're on now which cycle you're on I'm talking about the cycle of of one of these systems here May be determined by thinking about the system as part of a bigger system and we're going to do that that's important but in general you need some other ingredient besides just cycling around through the system here to tell you the relative Probabilities of conserved quantities okay so we're off and

flying with statistical mechanics there are bad laws by bad laws I don't uh not in the sense of domma or any of those kind of laws but uh but in the sense that the rules of physics don't allow them and you all know what they are they're laws that violate the conservation of information the most primitive and basic uh rule of Physics the conservation of information conservation of information is not a stand conservation law like this it's the rule that you keep that you can keep track you can keep track both going forward and backward

so let's just mention that again it's all worked it's all described in the classical mechanics book I'm just reviewing it now but let's take a bad law it's a possible law by bad I Mean 1 two 3 4 1 2 3 4 5 six so these are the faces of the die again but the rule is wherever you are this is red wherever you are you go to Red even if you red you go to Red okay we'll discuss in a moment what's wrong with this law but this law has one of the features that

it has is it's not reversible it's not reversible in the sense that you can go from Blue to red but you Cannot go from red back to Blue so in that sense it's not reversible You can predict the future wherever you are the future is very simple for this particular law wherever you are you'll next be at Red you can make it more complicated you could make a few you can make it more complicated but uh this law always winds up with red it's it's a bad law because it loses track of where you started

whereas these laws don't lose track if you know that You've gone through 56 56 and a half Cycles then you know that if you started at Red you'll come back and you can tell exactly where you'll be and you can also tell where you came from you can tell not only where you'll be but exactly where you came from this law you can't say where you came from this is a law that loses information and it's exactly the kind of thing that classical physics does not allow quantum physics also Doesn't allow the quantum mechanical version

of it so the rule that this type of rule that this type of law is unallowed I give a name to it as I said many times there is no name for it because it's just so basically primitive that everybody always forgets about it it's so basic I call it the minus first law of physics and I wish it would catch on people should start using it I mean it Is really the most basic law of physics that information is never lost that distinctions or differences between states propagate with time and you never lose track

in principle if you have the capacity to follow the system uh because you may be too lazy to follow the system that's your problem but nature doesn't have that problem nature allows uh in principle that you can reconstruct where you came from all right there uh so that's a bad Law what how do you tell the good laws from the bad laws just by diagrammatics here it's very simple good laws every state has one incoming arrow and one outgoing Arrow an arrow to tell you where you came from and an arrow to tell you where

you're going okay so that's uh uh those are good laws in classical mechanics Continuum classical mechanics there is a version of this same law any anybody know the name of that version of That of the theorem that goes with the conservation of uh of information it's called lille's theorem we've studied it in classical mechanics but let me give you a count example to lille's theorem um friction is an apparent uh contradiction wherever you start you come to rest it's sort of like saying wherever you start you come to Red wherever you start you come to

rest Well you may not know exactly where you are but you always come to rest that seems like a violation of the laws that tell you that uh that distinctions have to be preserved but of course it's not really true what's really going on is that when you run the Eraser through here it's heating up uh the surface here and if you could keep track of every molecule you would find out that the distinctions between starting points is recorded but let's imagine now that There was a fundamental law physics by a fundamental law I mean

you know Rock Bottom fundamental law uh for a series of particles for a collection of particles and the equations of motion for the particles were this DC X that's the position of the particle somewhere by dt^ 2 that's called acceleration we could put a mass in but the mass is not doing anything there's a Lot of particles so I'll label them I oh we've used I to label State I should not do that let's call call it n little n the nth particle and what is that equal to it's equal to minus some number gamma

we've seen that number before in another context Times dxn by DT anybody remember what kind what this formula represents friction uh viscous drag again it has The property that if you start if you give a if you start with a moving particle it will very quickly come almost to rest it takes it'll exponentially come to rest pretty quickly and so if all particles in a gas for example satisfied this law of physics it's perfectly deterministic it tells you what happens next but it has the unfortunate consequence that every partic just comes to Rest um that

sounds odd it sounds like no matter what temperature you start the room it will quickly come to zero temperature that doesn't happen this uh is a perfectly good differential equation but there's something wrong with it from the point of view of conservation of energy there's something wrong with it from the point of view of thermodynamics if you start a closed system and you start it running you Start with a lot of kinetic energy temperature we usually call it it doesn't run to zero temperature that's not what happens in fact um this is not only a

violation of energy conservation it looks like a violation of the second law of Thermodynamics it says things get simpler you start with a random bunch of particles moving in random directions and you let it run and they All come to rest what you end up with is simpler and uh requires less information to describe than what you started with as very very much like um everything going to red and among other things it violates the second law of Thermodynamics which generally says things get worse things get more complicated not less complicated okay so that's um

all right now there's another way to say Another important way to say this rule that every state has to have one arrow in and one Arrow out the thing that I called either the minus first law or the conservation of information supposing we have a collection of states and we assign to them probabilities probabilities P of State one p of State State 2 p of State 3 and so forth for some subset of the states Not all of them some subset of them all the others we say have probability zero okay so for example um

we could take our D and assign red yellow and blue probability a third and green orange and pink or whatever it was probability zero where we got that from doesn't matter we got it from somewhere somebody secretly told us in our ear it's either red yellow or blue and I'm not going to tell you which all right and now you follow the system You follow it as it evolves whatever kind of law of physics as long as it's an allowable law of physics after a while and you're following it in detail you're not constrained by

your laziness in this case you are capable of following in detail then what is the probability what are the probabilities at a later time well if you don't know which the laws of physics are you can't say of course but you can say one thing you can say there Are three states with probability 1/3 and three states with probability zero they may get reshuffled which ones uh were probable and which ones were improbable but after a certain time there will be those same three not the same three states but there will continue to be three

states which have probability and the rest don't so in general you could characterize these information conserving theories by saying supposing You assign some subset of the states let's say m out of n States let's say there are n States altoe that's the total number of states and now we look at some M where m is less than n and we say for those M States the probability for those M States is 1/ M for these states and zero for all the others you understand why I say one / M if there are M States equally

probable Then each one has probability 1 / M and all the remaining have probability zero then the number of states which have non-zero probability will remain constant and the probabilities will remain equal to 1/ m is that clear is that obvious that should be obvious the states may reshuffle but that uh that uh the number with non-zero probability will remain fixed that's a characterization a different characterization of um of the Information conserving laws for the information non-conserving laws everybody goes to R to Red you may start with a probability distribution that's uh one over five

for red green purple orange and yellow and then a little bit later there's only one state that has a probability and that's uh and that's uh red okay so that's that's uh this is another way to describe probab um information Conservation and we can quantify that we can quantify that by saying let M be the number of states which all have equal under the assumption that they all have equal probability let M be the number let's give it a name occupied States states which have nonzero probability with equal probability and then M what is m

characterizing m is characterizing your ignorance the bigger m is if m is equal to n that means equal probability for Everything maximal ignorance if m is equal to 12 n that means you know that the system is in one out of half the states you're still pretty ignorant but you're not that ignorant you're less ignorant what's the maximal what's the minimum amount of ignorance you can have that you know precisely what state it's in in which case m is what m is one you know that it's in one particular State okay all right so m

is A measure of your ignorance really m in relation to N is a measure of your ignorance and associated with it is a concept is the concept of entropy and we come to the concept of entropy notice entropy is coming before anything else entropy is coming before temperature it's even coming before energy entropy is more fundamental in a certain sense than any of them although in a certain sense it's uh we we'll Discuss uh entropy in a minute but s is a lot logarithm of M logarithm of the number of states that have an appreciable

probability more or less all equal for the specific uh circumstance that I talked about that entropy is conserved all that happens is the states which are occupied reshuffle but there will always be M of them with probability 1/ M okay so That's that's that's where we are and that's the conservation of entropy if we can follow the system in detail now of course in reality we may be again lazy lose track of the system and uh we might have after a point lost track of the equations uh and lost track of our timing device and

so forth and so on and we may wind up we may have started with some a lot of knowledge and wound up With very little knowledge that's because again not because the equations um cause information to be lost but because we just weren't careful perhaps we can't be careful perhaps there are too many degrees of freedom to keep track of so when that happens the entropy increases but it simply increases because our ignorance has gone up not because anything has really happened in the system which has um if we could follow it we would find

That the entropy is conserved okay that's that's the concept of entropy in a nutshell we're going to we're going to expand on it we're going to expand on it a lot we're going to redefine it with a more careful definition uh but what does it measure it measures approximately the number of states uh that have a non-zero probability okay the bigger it is the less you know what's the maximum value of S log in log in now of course n could be infinite you might have an infinite number of states uh and if you do

then there's no upper bound to the amount of ignorance you can have but you know in a world with only n States your ignorance is bounded so the notion of Maximum entropy is a measure of how many states there are altogether now I said that entropy is deep and fundamental and so it is but There's also an aspect to it which is um makes it in a certain sense less fundamental it's not just a property of a system it's a property of a system and your state of knowledge of the system it depends on two

things it depends on characteristics of the system and it also depends on your state of knowledge of the system or the state of knowledge of the system uh so keep that in mind okay now let's talk about Continuous mechanics mechanics of particles moving around with continuous positions continuous velocities how do we describe that how do we describe the space of states of a mechanical system you know a real mechanical system particles uh and so forth we describe it as points in Phase space we learned about phase space phase space consists of positions and momenta momenta

in simple context momentum is masstimes veloc velocity so Roughly speaking it's the space of positions and velocities let's draw it P is momentum goes that way and this axis is a standin for all of the momentum degrees of freedom if there are 10 to the 23rd particles there are 10 to the 23rd PS but I can't draw more than one of them well I could draw two of them but then I wouldn't have any room for the q's for the x's and horizontally the positions of the part Which we can call X X or P

doesn't matter all right a point here is a possible state of the system if you know a point here you know a position and a velocity and you can predict from that you if if you know the forces okay let's start with the analog of a probability distribution which is zero for some set of states and constant or the same for some other set for some smaller set well some fraction of the States all have the same probability and the other states have zero probability we can represent that by drawing a a patch in here

a sub region in the phase space and say in that sub region there's equal probability that the system is at any point in here and zero probability outside that's a sort of situation where you may know something where you may know something about the particles that They're in some sub region here for example uh you you know that all the particles in this room are in the room right so that puts some boundaries on what x are you may know that all of the particles have momentum which are within some range that confines them to

this way so a typical bit of knowledge about the room might be represented at least approximately by saying that the zero Probability to be outside this region and a probability equal I won't say one but equal probability to be in there okay now what happens as the system evolves as a system evolves X and P change the equations of motion say that that XMP change if you start over here you might go to here if you start nearby you'll go to some nearby point and so forth and the motion of the system with time is

almost like a fluid flowing in The phase space if you think of the points of the phase space as fluid points and let time go the phase space moves like a fluid and in particular this patch over here the let's call it the occupied patch the occupi patch becomes some other patch that other patch after a certain amount of time the system now is known to be in here after a certain amount of time we now know that the system is in here not in here Anymore and that it has equal in some sense equal

probability to be anywhere in there okay there's a theorem that goes with this the theorem is called leoville theorem and what it says is that the volume of phase the volume in Phase space the amount of volume of this region in the XP space and keep in mind the XP space may be high dimensional if not just to if it were two-dimensional we would think of it as the area if it were how face space is never Three-dimensional it's always even dimensional it has a p for every X so the next more complicated system would

be four-dimensional when I speak of the volume in Phase space I mean the volume in whatever dimensionality the phase space is if you follow the phase space in this manner here Leo's theorem you can go back to leoville theorem it's in the uh it's in the classical mechanics lecture Notes it occupies I think a whole lecture I think you follow and it tells you whatever this evolves into it evolves into something of the same volume in other words roughly speaking the same number of states it's the immediate analog of the discrete situation where if you

start with M States and you follow the system according to the equations of motion you will occupy the same number Of states afterwards as you started with there'll be different states but you'll preserve the number of them and the probabilities will remain equal so the rule is not the rule the theorem says that the volume of this occupied region will stay the same and a little bit better it says that if you start with a uniform probability distribution in here it will be uniform in here okay so there's a very very close Analog between the

discrete case and the continuous case and this is what prevents this kind of fundamental equation from being this kind of equation an equation where everything comes to rest that can't happen okay why not let's see why it can't happen let's just look on this Blackboard and see why imagine yeah imagine that no matter where you started you ended up with P Equals z that would mean every point on here got mapped to the xaxis it would mean that this entire region here would get mapped to a one-dimensional region and a onedimensional region has zero area

so lille's theorem prevents that what it says in fact is if the blob squeezes in One Direction it must expand in the other direction the situation for the moving eraser is that if the phase space of the Eraser gets shrunk it means somebody else's some other components in the phase space the probability distribution is spread out what are the other components in this case it's the PS and X's of all the molecules that uh that are in the table so for the case of of the Eraser there's really a very high dimensional phase space and

as the Eraser may come to rest almost rest so that the phase space squeezes This way it spreads out in the other directions the other directions having to do with the other uh hidden microscopic degrees of freedom okay so there we are with information conservation minus first law of physics and uh let's uh pass let's not go to the zeroth law let's jump the zeroth law we'll come back to the zeroth law anybody know what the zeroth law Says well I'll tell you what it says we haven't defined what thermal equilibrium is okay but it

says whatever the hell thermal equilibrium is if if you have several systems and system a is in thermal equilibrium with B and B is in thermal equilibrium with C then a is in thermal equilibrium with C we will come back to that just put it out of your mind for the time being because we haven't Des described what thermal equilibrium Means but we can now jump to the first law Z minus one zero and first law and the first first law is simply energy conservation it is simply energy conservation nothing more it's really simple to

write down it's Simplicity belies its power it is the statement that first of all there is a conserved quantity and the fact that we call that conserved quantity energy uh will play for the moment not such a big role right now but Um let's just say there's energy conservation all right what does the what does that say that simply says de whatever the energy is De by DT is equal to zero now this is the law of energy conservation for a closed system if a system consists of more more than one part in interaction with

each other then of course any one of the parts can have a changing Energy but the sum total of all of the parts will conserve energy so if a system is composed as I drew before of two parts with a link between them and this is called one and this is two then this reads that D E1 by DT is equal to minus D2 by DT I've really written the E1 by DT plus D2 by DT is equal to zero but then I transposed one of them to the right hand side just to indicate just

to make Graphic that if you lose energy on one side you gain it on the other so that's the first law of thermodynamics and that's all the first law of thermodynamics says it says energy conservation now in this context here there's a slightly hidden assumption we've assumed that if a system is composed of two parts that the energy is the sum of the two parts that's really not generally true if you have two systems and they Interact with each other there may be for example forces between the two parts so there might be a potential

energy that's a function of both of the coordinates for example uh the energy of the solar system I'm being very naive I'm thinking of the solar system as two orbiting uh neutronian particles the energy consists of the kinetic energy of one particle plus the kinetic energy of the other particle Plus a term which doesn't belong to either particle it belongs to both of them in a sense and it's the potential energy of interaction between between them in that context you really can't say that the energy is the sum of the energy of one thing plus

the energy of the other thing energy conservation is still true but you can't divide the system into two parts this way on the other hand there are many many context many Contexts where the interaction energies between systems is negligible compared to the energy that the systems themselves have um if we were to divide this tabletop up into blocks okay let's let's think about it divide the tabletop up into blocks how much energy is in each block well the amount of energy that's in each block is more or less proportional to the volume of each Block

how much energy of interaction is there between the blocks the energy of interaction is a Surface effect they interact with each other because their surfaces touch and typically surface area is small by comparison with volume so in many many we'll come back to that we'll come back to that but many many contexts uh the energy of interaction between two systems is negligible compared to the energy of either of them When that happens you can say to a good approximation the energy can just be represented as the sum of two energies of the two parts of

the system plus a teeny little thing which has to do with their interactions and under those circumstances the first law of thermodynamics this is the top to is always true the second uh has that little caveat that we're talking about systems where energy is strictly additive where you add energies does Everybody understand why I say you don't always add energies that sometimes energies are not additive yeah actually I was thinking that we have the same possible problem with the probabilities we assumed that the outcomes were mutually exclusive otherwise the some law well we did other

yeah otherwise the some law doesn't work yeah okay so in all the context which we talk about the D if it's yellow it can't be Red you say orange orange is both yellow and red well we don't count that way yeah so that's correct that no that's that's absolutely correct we made the assumption that what I called States what I called states are mutually exclusive right absolutely okay let's come back to entropy we're not finished with entropy we've done entropy we've done energy we haven't gotten the temperature yet notice the temperature comes in behind entropy

and Even energy comes in behind entropy but uh temperature is a highly derived quantity by by highly derived I mean it's a uh despite the fact that it's a thing you feel with your body so it makes it you know really feel like it's something intuitive it is a mathematically derived concept less primitive and less fundamental than either energy or entropy but we'll come to it let's come back to entropy Um we defined entropy but only for certain special probability distributions let's lay out on the horizontal axis just SCH just to be schematic on the

horizontal axis we will put down all the different states here's IAL 1 here's IAL 2 here's IAL 3 this axis just labels the various States and of course a and vertically let me plot probability okay well the probability of course Is only defined on the on the integers here that's not very good it's some some probability but uh let's uh let's um I don't want to have to draw such a complicated thing every time I want to draw a probability distribution let's just draw a graph some probability distribution or some probability uh for each uh

position now what we did was we defined entropy for a very special case the special case being Where some subset have equal probabilities and the rest have zero probability for example if our subset consists of this group over here of M the whole group being n then all of these have the same probability and their probabilities have to add up to one so the probability is 1/ n we just draw that by drawing a box like that then we Define the entropy to be the logarithm of the number of states in Here generally speaking we

don't have probability distributions like this generally speaking we have um probability distributions which are more complicated in fact they can be anything as long as they're positive and all add up to one so the question is how do we Define entropy in a more General context where the probability distribution looks like this I'm going to write down the formula and then we're going to check that it that it really gives this answer When uh when it should give this answer uh and for today I'm just going to write it down and tell you this is

the definition you'll get familiar with it and you'll start to see why it's a good definition it's representing something about the probability distribution and what it's representing is in some average sense the average number of states which are importantly uh contained inside the probability distribution the narrower The probability distribution the smaller the entropy will be the broader the probability distribution the bigger the entropy will be I'll write it down for you now we'll write it down and then explore it just a little bit tonight all right s for a general probability distribution is first of

all minus that's funny because this is positive but nevertheless the formula begins with minus a sum and it's a sum over all of the States all of the possibilities so it's a sum over I there's a contribution for each place here the probability of I times the logarithm of the probability of I do you remember all right let's let's write something else then you remember that the average of f is equal to the summation over I F of I * P of I this is actually the the average of log P Subi it's the average

of log P subi all right let's work this out let's see what this gives in the special case where the probability distribution is one over [Music] M for M States altogether it has with M and because it has width M it must have height one/ M because all the probabilities have to add up to one all right so let's work this out let's Take the contribution for all the unoccupied States from all the unoccupied States p subi is zero so you get nothing yeah that's right okay so now what's the all right good so let's

uh consider the limit of P so of log p over P or let's just say the limit of no not that's not right the limit of P log p as p goes to zero you know how to calculate That okay I'm going to leave it to you it's a little calculus exercise to calculate uh the limit as p goes to zero of P log p it's zero the point is that p goes to zero a lot faster than log p goes to Infinity log p as p goes to zero goes to is very slow p

goes to zero fast so this goes to zero you're absolutely right though that has to be uh that has to be uh shown but P log p in the limit that p goes to zero is zero okay so with that piece of Knowledge the contribution from states with very very small probability will be very very small and as the probability for those States goes to zero this quantity the contribution will go to zero but what about the ones here which have significant probability they all have the same piece of by and they all have the

same log piece of by log piece of by the log piece of I is logarithm of 1 n M for all of them the p Ofi is also 1/ m not log 1 M but 1 / M so each contribution is 1 M * log 1 M how many contributions are there like that all right so we multiply by m and get rid of the one/ M there there's a minus sign here I carry it along all time M because there are M such terms so the 1 / M cancels and we just left left

with log 1 / M what's log 1/ M minus log M right so that's why the minus sign was put there in the first Place there's no Miracle the minus sign was put there because probabilities are less than one and so the logarithms of them are always negative so you soak up that negative with an overall negative sign and entropy is positive but this is exactly the same answer that we or by the original definition just salal log M logarithm of the number of states but this is a definition now that makes Sense even when

you have a more complicated probability distribution and it is a good and effective it it it's the average of log P for the special case where the probability distribution is constant like this then um then all of the probabilities in here are 1 / M and calculating the average of 1 / M well it's just of log 1/ M just gives you this all right so this is the general Definition of the entropy that's associated with a probability distribution now notice entropy is associated with a probability distribution it's not a thing like energy which is

a property of a system it's not a thing like momentum it's a thing which has to do with a specific probability distribution probability distribution on the space of possible States so that's why it's a little bit Of a more obscure quantity from the point of view of you know intuitive uh definition as I said its definition has to do with both the system and the uh your state of knowledge of the system let's do some examples let's calculate some entropy for a couple of simple systems our first system is just going to be not a

single coin but a lot of Coins so we have capital N coins n of them and each one can be heads or tails Etc and so on as a matter of fact we have no idea what the state of the system is we know nothing the probability distribution in other words is the same for all states complete ignorance absolute ignorance what is the entropy associated with uh such a configuration there are n of these all the probabilities are Equal under the circumstance where all of the probabilities are equal we just get to use logarithm of

the number of possible States the answer here is the logarithm of the number of total States how many states are there all together two to the N right two to the N oh let's I'm sorry I'm going to change definitions for a minute for a reason that you'll see in a minute I'm going to call this little n number of coins is little N over here B stood for the total number of states so if I match terminologies Big N the total number of states is e to the 2 N No 2 to the N

sorry I'm getting tired two to the n two to the N States all together two states for the first coin two states for the second coin and so forth two to the N Al together and the total number of states is capital N what's the what's the entropy given that We know nothing to L to log in in log two s is equal to n log 2 that's that's the logarithm of 2 to the N all right so here we see an example of the fact that entropy is kind of additive over the system it's

proportional to the number of degrees of freedom in this case n * log 2 and we also discover a unit of entropy the unit of entropy is called a bit that's what a bit is in information Theory it's the basic unit of entropy for a system which has only two states up or down heads or tails or whatever the entropy is proportional to the number of bits or in this case the number of coins times the logarithm of two so log two plays a fundamental role in information Theory as the unit of entropy uh it

does not mean that in general that entropy is an integer multiple of log we'll see in in a second it doesn't mean That um okay so that's uh that's this case over here let's try another case let's try another state of knowledge here our state of knowledge was zil we knew nothing this is the maximum entropy maximum entropy logarithm of capital N or logarithm of 2 to the little n which is n log 2 one bit of ENT try for each coin if you like okay let's try something else let's try a state in which

what we know is oh Something else supposing we know the state completely in other words that's the case where m is equal to one that would be the case where we know that the probability is only non zero for one state then m is one and S is zero logarithm of zero so absolute knowledge perfect knowledge complete knowledge corresponds to zero Entropy the more you know the bigger the more you know the smaller the entropy excuse me okay let's uh take a case uh an interesting case let's take our heads and tails again and here's

what we know we know that all of them are heads oh let's begin with that all of them are heads what's the entropy then zero except for one of them which is Tails now supposing we know which one is Tails what's the entropy zero but suppose we don't know which one is Tails equal probability for all for all states which contain one tail and N minus one heads what's the entropy indeed okay so why is it log in how many states are there with non-zero probability the answer is little n it could be this one

it could be this one it could be this one in other words for this particular Situation capital M the number of states that have nonzero probability is just little n all the states have the same probability so we're in exactly this situation except that m is just equal to little n in possible States this one could be tals this one could be taals this one could be tals they all have equal probability so capital M is n and the entropy now is for this situation the Entropy is equal to logarithm of little n notice that

that's not an integer multiple of log 2 in general so in general entropy is not an integer multiple of log two nevertheless log 2 is a good unit uh it's it's a basic unit of entropy it's called a bit s in this case is equal to log n yes the two in the log two comes from the fact that you only have a probability of heads or tail right yeah So if you had three it be log three absolutely absolutely right now computer scientists of course like to think in terms of two for a variety

of reasons first of all the mathematics of it is nice but uh two is uh two is the smallest number which is not one small integer yeah it's the smallest integer is not equal to one um but uh it's also true that the the you know the Physics that goes on inside your computer is connected with switches which are either on or off or uh so uh counting in units of log two is uh very useful excuse me yeah uh it's probably not important but what is the base of the log Ro well it doesn't

yeah the standard this is a this is this is a definition of course this is definition the definition it okay good the Good it depends on who you are if you're an information theorist a computer scientist often or a disciple of Shannon then you like log to the base 2 in which case this is just one One n and it and the entropy here is just n measured in units of bits if you're a physicist then you usually work in base e okay but the relationship is just a multiplicative uh you know log to the

Base e and log to the base two are just related by uh uh by a numerical Factor that's always the same okay so when I write log log I mean log to the base e but very little would change if we used some other base for the logs okay so there we are let me just tell you what the definition is of the entropy in Phase space if we're not talking about [Music] Um if we're not talking about these finite discrete systems like this we're talking about continuously infinite systems phase space and let's begin supposing

the probability distribution ution is just some blob where the probabilities are equal inside the blob and zero outside the blob in other words the simple situation then the definition of the entropy is simply well you could say the logarithm of the number of states but how many states are there in Here clearly a continuous infinite Infinity of them so instead you just say it's the log of the volume of the probability distribution in Phase space s for a continuous system is just defined to be the logarithm of the volume in Phase space now if I

wrote V you might get the sense that I'm talking about volume in space no I'm talking about the logarithm in Phase space the volume in the phase space the phase space is high Dimensional whatever the dimensionality of the phase space is the volume is measured in those kind of in units of uh uh momentum times velocity to the power of the number of coordinates in the system all right but s is equal to the logarithm of the phase Space volume let's call it P phase V phase space that's under the the volume of the region

which is occupied and has a nonzero probability distribution this is the closest analog that we can think of To log m where M represents the number of equally probable states of the discrete system more generally if we want if we have some arbitrary probability distribution the arbitrary probability distribution P what would P probability what would it be a function of it would be a function of all of the coordinates and all of the momenta that's all of the moment and all of the coordinates p's and q's or P's and X's whichever you like all of

them Probability for the system to be located at point of phase space p and X incidentally when you have continuous variables like that do you write that the sum is all equal to one that wouldn't make sense you can't sum over continuously infinite set of uh variables it becomes integral we'll come back to this but let's uh let's just uh spell it out right now that if you have a probability distribution on a phase space the rule Is that the integral of it is to equal one p is really a probability density on phase space

it's a probability per unit uh cell in Phase space and what would you expect the entropy to be I'll give you a hint it starts out with minus where's where's the formula that we had here before let's rewrite the formula that we had before s is equal to minus summation over I of P sub I log of P sub I to go to the Continuum you simply replace Su by integral so there's an integral over the phase bace that's like the sum over I and then probability of p and Q times the logarithm of the

probability but in first approximation I don't mean first approximation numerically I mean first conceptual approximation it's measuring the logarithm of the volume of the blob the Probability blob in Phase space okay why did you move from X to Q oh I I good no mistake In classical mechanics x's and q's are coordinates and PS are momenta but you know that okay so we've now um defined entropy which as we've seen depends on a probability distribution I'm going to go one more step tonight and Define temperature oh uh well we Could stop here I think that's

probably enough for one night next time we'll do temperature and then uh and then discuss the boltzman distribution uh which is the probability distribution for thermal equilibrium we haven't quite to find thermal equilibrium yet but we will you can ask some questions I don't mind some questions now I just I just I have the feeling that I've probably done Enough for one night uh oh that if if you get a head if the probability the next one is a head that this well I haven't made any such assumption I haven't made any such assumption my

only assumption in calculating the um the various entropies was either that you know nothing or I told you what you know uh how you got to know that and what the reasoning was and Whether it had to do with knowing something about the Dynamics and the independence and so forth uh may come into your calculation of what p is but in saying you know nothing the implication was that all states are equally probable and without asking how you knew that um to say that all states are equally probable is closely related to saying That there

are no correlations it does say that if you all right let let's good let's talk about correlations for a moment to say that you know nothing means you know nothing so in particular if you you you begin knowing nothing and you measure one of the coins what do you know about the other ones nothing nothing you started knowing nothing about them you measured one of Them you still know nothing about the other ones of course you know about the one that you've measured now now let's take the other the other case that we studied supposing

um we know that all coins are heads except for one which is Tails and we now measure one of them and we find that it's tails what can we say about some other one it's surely Heads what if we measure that that one is heads what do we know about the other ones probability it changes the probability what's the new probability that one of the other ones is Tails right it's one over n 1 over n minus one instead of one over sorry one over yeah 1 over n minus one so that's correlation that's correlation

where when you measure something you learn Something new or the probability distribution for the other things is modified by measuring something that that's called correlation um for the complete ignorance there is no correlation for any other kind of configuration in general there's very likely to be some uh not necessarily but there's very likely to be some correlation correlation as I said means you learn something about the Probability distribution of other things by measuring the first one or you or you modify the probability distribution good okay I don't know if that's what you asked about or

not but yeah okay all right as that original system there we had two parts once we measured one thing had the conserved quantity again we had the conserved quantity there that once we did one measurement we knew which side we were Yeah incidentally entropy is as additive it's additive it's the sum of the entropies of all the individuals it's proportional to the number of things it's additive Whenever there is no correlation when there's no correlation it's additive now um so uncorrelated systems have additive entropies uh and we'll come back that's a theme that we'll come

back to in here's an interesting Question supposing here's what you know you know that if you measure a coin up then with 3/4 probability that it's too n they're laid out in a row if you measure one of them and it's up then the probability for its neighbors is 3/4 to be down that's what that's what you're given that that's all you know that's all you know is that if one of them is up if any one of them is up its neighbors uh are 3/4 likely to be down it's an interesting uh thing to

try to calculate the entropy of such a distribution um that is correlated of course that is correlated because when you measure one you immediately know something about its neighbor or make up make up your own example like that make up your own example like that and uh and compute the entropy you'll learn something from it okay any other questions before we go Home had information theory in 1949 physicist had entropy a long time before that discuss the relation not particularly the re the the formula here is due to boltzman I think is this the one

that's on boltzman's Tomb I'm not sure no what what's on boltzman's Tomb w What minus W he meant This well he did he did write this this is this is boltzman formula final formula for entropy and the only difference between the Shannon entropy and the boltzman entropy is that Shannon Used Log to now of course Shannon discovered this entirely by himself he he didn't know uh boltzman's work and uh from an entirely different direction from information Theory rather than from thermodynamics but none of it would have Surprised boltzman nor do I think boltzman's definition would

have surprised Shannon so they're really the same thing there's no real point in comparing them because they're comparing them they are the same there's no real difference Shannon may have I don't know whether he did or not put the minus sign in here if you don't put the minus sign in there it's called information if you put the minus sign it's called lack of Information or entropy so I I I don't know which Shannon wrote down but anyway wrote down entropy you did yeah simple uh way to relate to um Heisenberg certainty in terms there's

no no no this is a separate issue this doesn't have to do with quantum mechanical uncertainty um it has to do with the uncertainty implicit in mixed States not the uncertainty implicit in pure States okay I looked it up entropy equals k w oh oh boy yeah there's a conversion factor you know Cal h baral g equals boltzman's constant equal 1 very boltzman's constant was a conversion factor from temperature to um to energy the natural unit for temperature Is really energy but the energy of a molecule for example is approximately equal to its temperature in

certain units those units contain a conversion factor K boltzman I will I'll remind me to talk about k boltzman for more please visit us at stanford.edu