welcome everybody thank you so much the first of all apologies because that did not reserve a large enough room but so we're just going to get cozy but I've delighted to see so many of you this is the first talk in a series dedicated to Shannon and bull as you may know this year is a 200th anniversary of Bulls birth and next year is the 100th anniversary of Shannon's birth so we're celebrating the birthdays together we're very fortunate to have today Robert Gallagher before I go introduced and I just want to mention that next month
we'll have Mademoiselle the head of economic superior is going to give a talk and the month after that we're going to have Medusa done there will also be an event in March this is being done jointly by MIT CS by MIT physics RLE and University College Cork we've been doing all of these together and also on Tuesday at LSE we have a movie about pool which is going to be shown in the evening so if you're interested please do come and with that I'm very happy today to introduce Bob Gallagher and Bob is a member
of National Academy of Sciences National Academy of Engineering you've got lots and lots of awards and he's going to talk really well so thank you for coming Bob and I was also just elected a member of the National Academy of idiotic children so I feel very so I feel very proud one sure but you have to prove your idiot see it's okay I want to talk about both George Boole and Claude Shannon today but I'll be talking mostly about Claude Shannon because old as I am I didn't quite overlap with George Boole he was he
unfortunately died at the age of 49 which is and he did quite a bit before that so that's pretty remarkable so I think he's a pretty fantastic person but the only thing I know about him is what I've read about him and I knew Claude Shannon pretty well I've spent my whole life essentially trying to understand what it was that made Shannon do so many extraordinarily effective things he was very very smart and that helped but there was more than that and that's basically what I want to talk about today what's so bull is famous
for boolean algebra of course which is also called boolean logic what he did I mean logic has been understood basically since the time of Aristotle and probably for a long time before that but logical choice applied with natural language and bool figured out a way to do it in equations which makes all of us as engineers and mathematicians and scientists all very delighted because here was a member of our group essentially who said you can do this with equations too if you think about it a little bit and you think about the way logic is
actually used you think ah this is really very important because if you listen for example to the Republican debates you hear well I'm not going to talk about politics never sworn I won't do that but but if you listen to them what you find is not the examples you always find in a philosophy class of bad logic because those examples are things would say good logic because as Socrates was a philosopher philosophers or men therefore Socrates was a man very sexist but that's the usual example given and then if you said the other way around
Socrates was a man and Socrates was a philosopher and therefore awful Lhasa is our man that's bad logic the trouble with using logic in natural language is people make statements to form a syllogism all of which are very fuzzy and they make them not in order to find the truth themselves but they make them in order to sell what they have decided as its truth by completely other means and because of that logic doesn't work in those other fields it's not it's not because of the lack of equations the lack of equations simply makes it
a little harder to do so but but the the advantage of what bull did was he suddenly stripped boolean logic away from all of these bad uses of logic and he found nice ways of doing it and as I say their logic continues to be abused in politics religion and most non-scientific areas and it's deliberately misused it's misused because people want to give speeches they want to convince other people or something and of course they're going to use whatever technique works for them and most of us as scientists feel that's very unfortunate but that's the
way it is so we live with it okay for Channing also of course created information theory and that was his really major work it was a stunning piece of work it was something which people were just startled by I mean for a very long time people in many walks of life thought gee this really is a fantastic piece of intellectual work this will be used to help linguist understand their field will be helped it will help philosophers to understand what information is Shannon's theory which got called information theory has very little to do with information
it really has to do with data and it has to do with how engineers deal with data and it's a marvelous thing because of the way it does that but if you think this is the way to deal with information I don't think anybody has any idea how to deal with information at this point we try to do our best with it where we try to misuse it for our own ends but that's not that's not what's going to help us too much okay would question with modern communication technology and computer technology and their synthesis
which have all happened in the last 50 years would they have developed as quickly and in the same way without Shannon and bool I really don't know it's a it's a very hard question if you look at history the way it's in the way it has evolved clearly Shannon had a major part to do with it the fact that rule taught us how to how to do logic with equations had a major amount to do with it if they hadn't done it with somebody else have done it anybody's guess my guess is if if Shannon
had started to try to understand switching Theory without boolean logic he would probably have it then have invented boolean logic because that's the kind of person he was so that I think that boolean logic would have been invented in in any case it's rather startling but people didn't use it very much for a hundred years until Shannon came along so so bull in fact did have a great deal to do with all of this and it was kind of kind of amazing what he did I can't really guess what those things are about I'm not
a historian I'm not a person who was going to write the history of what of who really invented the Information Age and I I find such questions crazy because there were so many people involved in so many different parts of this crazy world we live in right now that but I don't want to blame it on anyone I don't want to give anyone credit for it what I do want to understand is what was it that made people like bull and Shannon so enormous Lee creative and that's the whole thing that I'm trying to sort
out and one thing that I just saw a short time ago was Claude Shannon gave a talk entitled creative thinking back in 1952 to a small group of researchers I presume at the theoretical research group at Bell Labs and he told people how to do research well of course it didn't get reported at all because most people think most people who do research things they know how to do research and most of them do to a certain extent some of us sometimes and say well I know how to do research but maybe I could do
it a little better if I learned some of the tricks that other people use so what kind of tricks wasn't that Shannon was using he called them tricks I mean to me they were much more than tricks but he started out with three things that he thought were his they take me a little while to get used to this he started out with three things that he thought every researcher had to know in order to be a capable researcher and there's not much surprise in any of them except the way he stated them and I
want to talk about this a little bit because because what I'm trying to do is to take this short talk that he gave try to make sense of it especially for someone back then who came from a very different culture I mean our culture has changed enormously in the last 60 years or so everybody does things in a totally different way than they did before words mean different things now when he talked about training and experience I was a little put off by that because to me you don't train researchers and trained dogs to stand
on their hind legs and things like that but you help researchers learn to do research but then I thought about a little more and said no Shannon was right if he said helping people learn to do research everybody will say what do you mean by learning and he didn't he didn't really mean learning here all he meant was that people had somehow picked up enough tricks about research they picked up enough knowledge they had they had done the things that students are supposed to do along with learning along with becoming educated you just have to
learn a lot of things you have to learn a lot of facts they're boring you don't like to deal with them but you have to learn what other people know if you want to talk to other people you have to get all this background and that's what he meant went by training I think he didn't mean one person training another person what I think he really meant was with people acquiring the the background the stuff that's no fun that they need and he said you need intelligence and he was just pointing out but if you
have an IQ of 90 you'd probably better look for some field other than creative thinking you're probably not going to be very successful if you were successful there you want to be able to talk anybody into believing what you had done so you need those two things motivation he spent a long time trying to explain that because he clearly didn't mean what most of us think of as motivation of course any motivation to do anything but he was talking about a kind of intellectual motivation he was talking about the inner drive to formulate questions and
find answers thought was really into questioning and answering things he was he would probe on all sides of a problem he would not do what students do when I was teaching here I was always amazed at the enormous number of students who when you give them a problem to do will fill ten pages with equations and if they're lucky they'll come out with the right answer and realize that one of the equations is the right answer and if they're not they won't but they'll never make a mistake and if I write one page of equations
I'll make ten mistakes but at least I asked myself when each question does this is this equation leading me anyplace and students today are so fast and so facil and so well trained that they can just blast through anything without without really getting anyplace because there's so many different ways to go and you have to somehow sort out what the right way is so so he was talking about this inner derive the formulating and finding answers talking about curiosity about fundamental characteristics of things he didn't he wasn't interested in the details he was interested in
really in what made things tick which was the way he often put it there was a need to understand in multiple ways there was a need he got oh and the other thing that he pointed out was the natural part of this that that all of us who ever try to do anything academic have found out for ourselves there's an enormous satisfaction that comes when you finally understand something you've been fighting with for three or four days there's just this wonderful aha I finally see it and it really feels good and he was just telling
people that yeah it if it doesn't feel good to you then you shouldn't be doing it so he had all of these things which were I think sort of important but not not really unusual but then he said he had a number of tricks which he often found useful now to me when I looked at the tricks he was talking about I said gee those weren't tricks those are the basic principles of how to do research so I want to share them with you and maybe you'll agree with him or not the first of them
was simplification you really have to get rid of enough detail including practical aspects of the problem for intuitive understanding the first time I ever talked to Shannon I been screwing up my courage that went and talked to him when I was a young assistant professor for a long time I was I was working on what I thought was a very fascinating problem it was important it was also fun it was fascinating I thought gee this is this is the best thing I've done for a long time I went and talked to him he looked very
puzzled he was trying to understand he was trying to be polite and then he kept saying well I don't quite understand what this is all about is there some way we can simplify it a little bit so we going talk back and forth and finally we got finally we got rid of the most important practical aspect of the problem and I felt a little bad and then and then we got rid of the next most practical aspect of the problem and after that we got rid of the most important theoretical aspect of the problem and
I was just feeling terrible but then after two or three more steps we got down to something where both of us saw what the obvious answer was I saw it because I've been working on it for so long he saw it because he was just smarter than any anybody else I've known and amazingly putting the pieces back the problem was solved I never forgotten that it was and I've tried to do it myself it works an enormous man at the time but what were his other tricks he would find it was similar to a known
problem experience helps in doing that I think I think most of you figure out how to do that pretty quickly on your own there's not much surprised about that it's a something and I'm going to go through an example of some of his research and we see he did all of these things the third one was he said you got to reformulate the problem very often you got to avoid getting in a rut and he he had the sense that for most people including him when he worked on a problem for a while pretty soon
it was like a broken record you know I would go back and back and back would keep will keep going as far as he could go up to this wall and he just memorized all the steps in the argument so that obviously each time he did it he was the same law and he said when that happens to stop forget all of that and do something entirely different look for a totally different way to look at the problem so he that's pretty important too then he said generalize and that's much more important than well it's
it's much more significant and much more general than just generalizing from where you simplify I mean sometimes you start out with this problem you simplify it down to this problem and then off you go generalizing it off there or something you never thought about before or which turns out to be more important than that thing you were working on to start with and another thing with structural analysis which is break the problem into pieces this sounds a little like the simplification but the simplification is not breaking it into pieces a simplification is just getting rid
of most of the problem and just saying let's think of a little sub problem we might make some progress on and then finally inversion and this is something we all do you work back from what you think the answer ought to be now you might think that's unscientific and it is a little unscientific because scientists are supposed to look for the truth they're not supposed to assume the truth but all of us do assume the truth we all say I think the answer ought to be this it's it's amazing anytime you talk to the most
formal mathematicians in the world Russian mathematicians Hungarian mathematicians all of them they will prove theorems with this enormous carefulness everything is done very precisely but then when you talk to them they will come out after a while with will the answer would it be something like this and that's the way they think it's not the way they prove theorems it's the way they think that's the way all of us think it's the only way one can think that one doesn't do that one is not going to make any progress of one if one only makes
statements one is sure off then you're sort of defeated from the beginning okay so so inversion was starting at the end and working back and seeing what's and saying what has to be true to make it what you want to be true be true okay so that's there were some other tricks of chatting then and talk about there which I've sort of discovered from questioning what he was doing all the time by reading his papers and saying well how did he come up with this and one of the most useful according to my beloved is
be interested in several interesting problems at all times namely instead of just giving up and working intensely on one problem have ten problems in the back of your mind be thinking about them be reading things about them wake up in the morning and sort of review whether there's anything interesting about any of them and and invariably when you're asking questions and trying to do what you're trying to do one of them will pop to the fore and it's usually one where you read something about it the day before where you talk to somebody about it
or you read something in a newspaper or you just some peculiar synapse went off in your brain when you're sleeping god knows what doesn't make any difference something triggers one of those problems and you start working on it now why is that so important well I've always found that one of the most difficult things in trying to do research is how do you give up on a problem so many students doing a thesis just beat themselves over the head for year after year after year saying I've got to finish this problem I said I was
going to do it and I'm going to do it well if if it's an experiment you're going to do yes you can do that you can do it more and more carefully if something isn't working you fix it to the point where it works if you're trying to prove some theorems and the theorem happens to be not true then then your chances of success are very low now if you have these 10 problems in the back of your mind and they're just one problem that's been driving you crazy what are you going to do it's
going to sink further and further back in your mind and just because you have more interesting things to do you've gotten away from it after a year you won't even remember you were working on it it will have disappeared that's a far better thing that the reason it it out and say well I don't think I can go any further with this because of this this in this reason because your problem is you don't understand the problem to start with which is why you have to give pointing but you don't understand it well enough to
understand why you were to give up one so you just find that you can do other things which become more interesting temporarily you look for contradictions as well as proof when you're trying to prove a theorem you think you've made a little progress you think you've gotten some insight on it don't don't spend more than an hour being happy about it and telling all your friends about it go back and say okay I'm going to I mean for me it was I'm going to be my father I'm going to find something wrong with this and
and you know all of us have somebody we can point to a professor we hated or I mean every one of us has has somebody they can think of but the point is when you think you understand something blast as many holes in it as you can because then you'll understand it whole lot better okay study what's happening in multiple fields but don't work on what many others are working on if you're working on what problems that other people are working on makes your chances of success much smaller it also means that you have to
work very fast and for somebody like Claude Shannon he didn't like to work fast he liked to work at his own pace which was the kind of pace that led to fantastic results if he had to had to do it in a hurry he just wanted them I don't think he'd have managed with that there's another advantage to that and there's also a disadvantage to it if you work in a field that a very large number of other people are working on and you keep publishing little results in that field pretty soon since the field
is very large you work with ten other people so you're publishing lots of papers you have ten authors on each of these papers and therefore you build up a tremendous list of things you've done and there are still places to count how many papers you Quinn it and you published and say that must be have some effect on how good you are so research well I certainly believe it has a lot of effect on it's very highly correlated with how good a researcher you are the more papers you've published every year the poor a researcher
here you almost have to be good researchers maybe do one really good paper in their lives Shannon if you took away all the nice little things he did and you leave him with only his switching paper and his communication paper he would still probably be the greatest engineer we have seen in the last hundred years I mean you can quarrel with that but but the point as he was one of the great ones on the basis of two papers if you throw out the switching maybe you might not think he was only one of the
two bastards something just on the basis just on the basis of that one paper it was a fantastic research and that one paper took them back about seven years to do it and if you look at his entire published output it wasn't all that high okay so as conceptual questions about everyday things which is what he was doing all the time he was not too busy to look at something strange and say gee that's sort of interesting he'd love to build things and because he loved to build things he would go back and forth between
thinking about them and building them don't assume don't write papers unless you really want to share something fascinating this again has to do with the over publishing business don't assume your readers know everything you do spoon-feeding really is not a bad idea it's if it's hard enough to get your ideas across that if you do what you think is spoon-feeding you're probably going to keep people with you for an awful lot longer than you would otherwise okay so let's go through a quick biography of him you have to do that when you're talking about him
I just doesn't skip it but I feel like it what - he was a nerd he was he was normal but and bright but yet in there of course he was a nerd I mean he's one of the greatest researchers of the 20th century you think he was a athlete in high school or something well because of this he graduated from high school the age of 16 so he was showing some some signs of being bright and it wasn't because he was speeding through it was just because he he did things well he got a
double degree at the University of Michigan and math and electro engineering at age 20 so he was he was little ahead of the pack at this point but that's not fantastic he didn't know what to do at that point but he saw an ad for a research assistant to take care of the differential analyzer at MIT which was which was probably the about the best computer around at that time I mean I mean this was 1930 38 9 1936 I believe so there wasn't much computation around but he thought that would be cool and you
know Vannevar Bush he was he was not like the rest of the Bush clan in fact he wasn't even a part of the Bush plan he was I mean he was a highly respected scientist and he and and and he operated at a very broad level so I don't think Shannon was thinking about this but but you know if you have somebody like that on your side and you want to spend a few years doing something odd you can probably get away with doing it you don't have to worry about the things that most researchers
have to worry about because you have a have a very powerful person in your corner but what this did is he got interested in all the switching that was involved with that differential analyzer yes yes it was a continuous time kind of thing and all these things that went but around continuously but there's a lot of switching to control what went on and he got interested in how the switching worked he taken a course in boolean algebra at Michigan so I was lying when I told you there was no interest in it at all it
was there was some interest but it but but it was not a huge field but he understood it a little bit and from that he started getting interested in switching he wrote his master's thesis and his master's thesis created created a new field he won an award for it for the best engineering paper for somebody under 30 but many people looked at that paper and they said this is this is the most important master's thesis ever read it so I don't know about that but it was certainly it was certainly a very significant piece of
work and it did start a new field and I made him relatively famous they got him a job at Bell Labs and then caught him a job work where people really respected him so he was on his Hawaii his PhD thesis was an algebra theoretical genetics the results were very good but he never bothered to publish it now why didn't he bother to publish it I mean for you guys for the most part if you didn't publish your PhD thesis it would be suicide and a wife for your career but remember he was famous already
if you're if you're famous already you don't have to do any more you could just sit back on your laurels and you can do an interesting thesis and not publish thank you so much that's great ok so so he could sit back in his laurels but also he was interested in other things he he had this business of he had all these things he was interested in the one he was interested in currently was not genetics it was I mean it was a good good PhD thesis but it was not it was not the thing
that was most interesting to him so he just put it aside and forgot about it and it sucked down into the sand this is of his head and that was the end of it so he started working in telecommunication he worked at the Princeton Advanced Study Institute for about a year he was the one no no what did I say Princeton's oh yes yes you're right now no you're absolutely right and and he only worked at places like that I mean after he worked there he was Bell Labs then MIT and that's the only time
when I've ever thought that maybe somebody took a step down when he came to MIT it's a bet then anyway that's he was no ok during the war he worked on fire control he worked on cryptography the funny story about cryptography he wasn't cleared to work one so he he talked to the people who were doing it including touring who was doing a lot of the cryptography worked in but he couldn't tell them I mean he could tell them everything he knew but they couldn't tell him what they knew but anyway he developed mathematical theory
of cryptography fairly early during the war he didn't publish them until the end of the war because it was classified until then many people thought that led to his theory of communication most people who looked into it more and have asked him and have asked his wife and asked other people are pretty convinced that it was the other way around he just wasn't ready to talk about communication at that point because all the pieces had not come together and cryptography at the level that he was interested in it was a would was really a simpler
problem so that was just what he was doing 1948 everything came together for his mathematical theory of communication he wrote it republished it a little bit later and then it was the theory the mathematical theory of communication of course and I do want to talk to you a little bit about one part of that theory I think probably half of you in the room knows much about it as I do and the other half probably don't know anything about it and probably are not that interested in it I want to talk to you about it
because it's a beautiful example of how he did research of how he used all these principles that he knows about in which unfortunately this set of principles which which I view is the is the main principles for doing intellectual research I never talked about in school you never hear about it in grade school you should hear about it then never hear about it in high school you never hear about it in college graduate students never hear about it except you you you're very lucky because you're there learning something that has been hidden for a long
long time and I'm and I'm trying to make it known at this point okay so and in this theory sources are characterized by the bit rate per symbol or per second needed to reproduce the source exactly or within a given distortion measure channels can be characterized by an essentially error-free bit rate called capacity you all know that the third part of it which is not as well publicized but which could be more important because it is a standard binary interface these days between sources and channels almost everything for a very large number of reasons almost
all sources are turned into bits and what you put into channels are bits which you've encoded you don't you don't go directly from a source to a channel you always go to this interface and you know every time you used to talk to somebody about communication they would say how much bandwidth are you using to send your messages you never hear that anymore you rarely hear it people say what bitrate are you sending and people talk about sources and they say what's the bitrate of the source okay so so bitrate has really taken over people
believe in this now and that's that's partly what he did I'm going to let you read this you can probably read it three times as fast as I can say it because this is what he said to start to motivate the work that he was doing and I wanted to put it up here because it's beautiful language and it is just an amazing if you think about it for a long time you won't think of anything he's left out he's put it all in there oh very compactly also it looks nice but it's all there
okay so that's that's your last chance to read it I hope you're all speed readers I'm going to talk about the source representation that he did mostly just to talk about how he did this how he did this business of simplifying oops okay so let's must be a little technical now P of AI is going to be the probability of the letter AI it's the probability of an independent identically distributed letter sequence which is X 1 X 2 X 3 X 4 so what we're starting with is English text for example and we're trying to
say what's the probability that a particular sequence of English tax is going to appear and Shannon said interesting enough the letters are all independent of each other why did he assumed that it was to simplify the problem and he had thrown out all the practical parts of it he thrown out all the linguistic parts of the problem he kept the part but it was using Morse code for example it was recognized that he was the most likely letter in the alphabet and therefore if you are trying to encode things you would make the a single
dot but then he did something very brilliant everybody else that had looked at the problem had looked at Morse code and after looking at Morse code and been partly mathematicians partly engineers they wrote a difference equation to represent the fact that that you are making letters of unequal length word on unequal spaces and all sorts of little details like that and people gotten yeah that was an easy difference equation to solve quite straightforward and they would solve it but where did they go then you see other people would make the simplification but they make a
simplification and a way I mean they throw away the practical part of the problem but they'll also throw away the ability to generalize the problem in any way I mean you probably could have generalized it in some way but it wasn't very difficult and nobody but he did it this way and looks back again so the probability of Shannon would be probability of capital s probability of H probably they and so forth which and since he was assuming these letters were iid what he had was P of n cubed because there were three ends P
of capital S P of H P if P love okay fine that's very simple and then he said well what happens if you have a million letters because in telecommunication you know you're transmitting letters all the time and back then a lot of what you wanted to transmit was English text or or French text or what have you and then he said well in the law of large numbers typical sequences where the sequences are very long are going to have about lengthe scale they're going to have about tau times the probability of I appearances of
the letter I for each I okay so first thing he's done here just to make you quite a remarkable simplification of the problem where he really throw away what looks like the whole problem and then he used similarity in a sense to say I know about the law of large numbers I'm a probably after all and if I make these sequences very long maybe I'll find out something but by using the law of large numbers so he wrote it down the probability of a very long sequence is going to be the product over the letters
in the alphabet of the probability that is assumed for that letter times the number of that letter tau times P sub R now I'm feeding you all and those of you who work with the law of large numbers know that I'm leaving out of a few small things but but but this is essentially what you have and then he said well how do I deal with this product and he knew very well that when you deal with sums of things you can use the law of large numbers and so he converted this to two to
the length of the sequence times the sum TI times the logarithm of P of AI and he said t that looks like an interesting formula and he said it looks like the entropy in physics the fact that it happens to be that the entropy in physics was totally irrelevant here and a lot of physicists got very interested in business and they say aha I can be on the wave of the future because I'm one of those one person out ten thousand who studied thermodynamics and understood what entropy was and I wish I had understood and
entropy when I was studying physics but I never did but this seemed like a very simple idea but anyway here was it was very straightforward and then he realized that all typical sequences have about the same probability cumulatively their probability is 1 therefore they have to be about 2 to the length times H of the typical sequences and by this time he was home because he realized that if we have a very large number of sequences and you want to represent them with binary digits you take this many and alphabet this large and it takes
tau times H at the binary digits to represent it so you're taking H of P binary digits per letter ok the reason I'm going through this the reason I'm leaving out a lot of details is because the argument is really very very simple any any good graduate student if you give them this problem and you spell out what they're supposed to do well enough they're going to solve it ok so this is one of the things that made him famous and one of the interesting things is it's channel coding theorem it was just an extension
of this it was just a generalization so where you have generalization put in all so he was using all of his tricks and by using all of his tricks this theory instead of being very complicated as people thought it was physicists were trying to explain this entropy which is trivial in terms of thermodynamic entropy or or quantum entropy even which is very difficult so anyway that's what Shannon did it was simple and he used all of his tricks now after he did that he started to generalize it again he said well the same argument will
work if I take a Markov chain and I study the probabilities of diagrams I can study the probability of trigrams I can then you'll want to study the probability of words I can use diagrams of words and you read his paper and his and his insight into how to do that how to calculate those those probabilities experimentally I mean it's just a thing of beauty but there's nothing extremely special about it that's something that any bright person would think of after a while okay so let me just I'm going to skip this and just comment
at the end that Shannon's work was brilliantly simple and simply brilliant why not because it was anything complicated about it because he knew these tricks and he knew how to use them and it all turned into something terribly important okay the typical sequence arguments he used work for channels also he used the same approach and he did have to go to a real trick not one of these principles but but the idea of using randomly chosen codes and how he came up with that I don't think anybody has ever figured out this was just his
pure genius that was coming out at that point okay but he always found the right way to look at problems he always built things to study problems he wrote technical papers to study them if you think about what he did every problem he looked at he really found what really see to be the nicest way of looking at them he was the opposite of an applied mathematician many people try to become applied mathematicians thinking thinking that's a good way to apply mathematics to engineering problem he was the opposite of that his trick was in finding
the right models to look at problems applied mathematicians solve other people's models now that's an important thing to learn how to do but that's not what's really important here his business was finding the models and it was also solving the models because it was true sobbing the models they got the insight about it okay so let's go on to bool I don't want to forget him but I'm except I'm going to be fairly brief on the biography he was the son of a cobbler the cobbler was more interested in mathematics and optics than he was
in cobbling and bull was brought up about 120 miles north of that's where am i okay he was about 120 miles north of London he was really self-taught he stopped school very early he started teaching school in those days that happened a schools in boarding schools there wasn't education that everybody had to do incidentally this is his 200th anniversary and it is Shannon's 99th anniversary 100 minus one who cares about the one and and he didn't know anything but he taught himself everything that he learned that schools he were in had pretty good libraries so
he started writing mathematics papers and they kept getting rejected except a kindly editor looked at it and said there was some promise there and helped him learn how to write papers he wrote a paper in 1944 on symbolic algebra symbolic algebra what what 84 for as I say 1944 well that's 1744 he was very precocious you know symbolic algebra is this business of taking symbols like differentiation integration things like that and treating them like outbreak symbols and it's and it works sometimes and he he made some real progress on that problem and he got a
royal medal from the rail society in 1944 for his paper on that and after that his career was made so he was lucky like Shannon after you're after you're famous you can write papers and they get published because because the way the world works and especially today nobody has time to really review papers carefully sad thing but that's that's the way it is if everybody writes 50 papers a year and everybody reads carefully reads 50 papers a year it means that the average number of people who read your papers is going to be 1 there
are three reviewers which means that all you can hope for is to have one of the three reviewers read your paper and the rest of them will read the abstract and say something stupid okay good but you know at this point bull bull had it made so he so he started to do things who are really interesting to him his mathematical analysis of logic first one came out in 1847 and very young then still well and then he was appointed as the professor of mathematics at the University at the at University College and quark that
was not the name then but this was very unusual he didn't have a high school degree he certainly didn't have a college degree he certainly never gotten the doctorate or anything like that and yet he was the professor in mathematics at a at a reputable school great why because he won this award he done one paper and done it really well so everybody said that trumps everything else he's he must be one of us he must be better than us okay so what he'd done with his reduction to equations and his boolean logic was something
which then became famous for the short term as many papers were not all that payment he did some work in probability but it really didn't last at all and it was a long time before all of the paradoxes and probability got worked out okay similarities between Shannon and bull I'm I'm trying to finish by five o'clock and I have four minutes to go so the speed will have to go up a little bit as I have some things I really want to say both of them were recognized when they were very young Shannon wrote this
fantastic paper and switching when he was 22 what have you guys all been doing better question what about and and of course bull was a little older but bull had a few handicaps because he didn't have any schooling magnum opus of each one opened up a new field each one spent about eight years on their magnum opus Shannon on mathematical theory Communication bulan logic their magnum opus was really quite simple in retrospect I mean something you can describe in five or ten minutes and you really have most of it at that point bulls research appears
to indicate that he really understood Shannon's tricks of creative research I mean many people throughout history have understood these rules just ten happened to as a by thought one day gave a seminar where he where he let it out what he was doing and that was nice I want to talk a little bit about simplification because that's sort of the sort of the main trick that he used and it's the most important of them what is simplification what do we mean by it and there are a lot of quotes around about simplification if you look
up on Wikipedia you'll find pages and pages and pages of them and most of them sort of say it's all right to be a simpleton and and they say you don't have to do any more than that that wasn't what Shannon meant at all Weinberg had an interesting commentary on it he said in the study of anything outside human affairs and I don't know why he left out human affairs including the study of complexity which a lot of us are interested in it is only simplicity that can be interesting and the point he was trying
to make is until it becomes simple you don't understand that you can't reason with it it's not part of your system and something complicated out there Einstein usually said things right but I think this was this was his biggest mistake I think aside from his saying that God doesn't roll dice which was probably held back probability theory for a long time but Einstein said everything should be as simple as possible but no simpler well of course but what does that mean how do you tell and so say you're really puzzled by Alfred North Whitehead I
don't know whether you've heard of Whitehead or not he was a philosopher in England he was a co-author with Bertrand Russell of the principles I forget what I forgot what it was called then it was their effort to take all of human knowledge and compress it into one big very heavy volume principia mathematica yes and so so he was a more respected person he was a big deal and his his explanation of it was search for simplicity but mistrusting and that I think I think it really is a home run I want to explain it
and then I'm going to wrap it up searching for simplicity is really searching for an intuitive understanding of something it's it's it's searching for something you can put into your own system a thought of the kinds of things that pop into the foreground often and easily it's a it doesn't go and and the mistrust that he advocates that goes along with it I can't separate the okay mistrusting means that after you simplified something after you put it into your your web of ideas that you can understand all the time what he said at that point
was knock it down mistrusted find out what's wrong with it because that's the way you learned by first building something up by making a common sense to yourself and then finding out what its limitations are finding out where it's weak so that was what he did and he went back and forth according to whitehead between searching for simplicity and then knocking holes in it and then hopefully generalizing an alternative simplification the search and mistrust really leads to a process of success of probing you very the simplification you generalize the reformulating you do all of these
things each step you learn something new and as you learn something new you keep adding to the whole thing until eventually you've made sense out of the whole business China was a grand master of all of that none of the rest of us are Grand Master's but my contention is all of us can practice these things we can all try to do these things I've I'm not a very smart person but I've tried doing these things after Shannon taught me how to do it and you know it works pretty well perhaps we might even teach
our students about creative research instead of pushing them to program more and more complex problems computational resources let us solve incredibly complex problems but do we learn anything I don't know so graduate students may take simplicity for triviality they've been complicated misconception it takes the best students to well people think that it takes the best students to solve the most complex problems it really takes the best students to find the most simple open problems and perhaps if we if all of us try to use hands tricks a little more we will all be better at
creative thinking and we'll all have more fun in life thank you thank you so so much Bob but you only have a couple of minutes for questions and then we shall whisk Bob upstairs building 36 six floors so that everybody can enjoy the reception and he doesn't get bogged down here for the next hour so maybe if we have a couple questions yes he was mildly interested in it he was interested in the general ideas if he never really liked to go that far into that because because it's really more detailed than what he was
really concerned with I mean it was one of those things that was in the back of his mind but there were never things that really brought it up very much yes okay thank you sir forever doc my question is in Shannon's tricks the importance of intuitive understanding has been stressed quite a lot now my question is how do you see the relationship between lack of training and intuition because sometimes the training component can act like a kind of toolbox where you have various components and the permutation and combinations of how you arrange those components lead
to maybe your own intuitive understanding about a problem so if someone say for example bull if he did not have any formal training but he had intuition so how do you see I mean this relationship between intuition and lack of training or over training uh only with great difficulty yeah it's uh I think I think the easiest thing to say about it is that bull was not untrained he spent a great deal of time reading on his own and didn't so in fact he was a highly disciplined person who who worked very hard at knowing
lots of things in building up this this general feeling let me put it this way which might which might make sense to you a person who thinks intuitively you could think of a little bit as somebody who thinks outside of the box and this crazy notion we have is if you want to do something highly creative you ought to get outside of the box really you can't you can't do anything useful outside the box you have to start in the box go out of the box go back in the box go out of the box
it says process the way Ted was talking about and that's where intuition and training come into because the two of them reinforce each other so you have to use both thank you again and we shall see you upstairs building 36 6-4 thanks again Bob you