Differential Equations: The Language of Change

imagine zooming in on a single neuron in your brain at first glance it might seem like a simple device it receives inputs and if they're strong enough it fires an output impulse but this view misses one of the most fascinating aspects of neural Behavior it's intricate Dynamics in time neurons are not static input to Output machines they are sophisticated temporal computers changing their state and responsiveness from moment to moment how a given NE neur response to input depends not only on that very input but also on what this neuron was doing a millisecond ago a

second ago or even minutes in the past this temporal complexity is the key to understanding how the brain generates such a rich variety of thoughts and behaviors a beautiful branch of mathematics that describes exactly these phenomena is called dynamical systems theory it deals with differential equations but despite this intimidation name most of the insights it offers actually can be grasped without a single formula simply by looking at pictures in this video which is the first one in upcoming series we'll lay the foundation we'll introduce what differential equations even are and discuss important Concepts such as

equilibrium points face portraits and limit cycles and in later videos we'll take the groundwork established today and apply it to neuronal Dynamics to gain a geometric intuition behind phenomena such as spiking bursting or adaptation if you're interested stay tuned at its core dynamical systems theory is about studying how things change over time a system the thing that is changing can be pretty much anything a neuron firing a population of animals interacting a leaf floating in the water or if the global climate the key is having a way to describe the system State at any given

point this is where the idea of State variables comes in these are real numbers that when known fully determine the state of the system everything that you might care about for instance consider a ball moving through a three-dimensional space to fully describe its state at any given time Point assuming you know all the forces that act on it like gravity or a drag all you need to know are the three coordinates X Y and Z numbers that describe its position in space as well as the three components of the Velocity how fast the ball is

moving in those three directions once you have those six numbers you fully know the state of the system in the sense that you can unambiguously predict its trajectory arbitrarily far into the future and for a simple pendulum you only need two State variables the angle from the ver vertical and the angular velocity but what about more complex systems how can we even know what state variables are well it depends on the particular system and the level of abstraction you want to characterize it on after all these are all models designed to describe a particular aspect

of the real world while discarding all others as Irrelevant for instance the two State variables for a pendulum are only Justified if we allow ourselves to approximate the pendulum as a single point while if we care about its rotation or its deformations or the temperature we might need a larger set of State variables in the case of a neuron when we are talking about its computations we really care about the electrical properties while discarding other things as irrelevant such as the concentration of sugar floating around or the DNA being repaired meanwhile for a biochemist who

wants to describe and model these very phenomena it is the other way around as you can see choosing the state variables what to describe and what to throw away is kind of an art of its own we'll talk about what the state variables for a neuron are a bit further in the series but for now let's introduce the formalism of differential equations that describe how these State variables actually change over time imagine you're tracking a population of bacteria in a Petri dish initially you start with a thousand and then you count them every hour after

a few hours you might see a pattern like this there is clearly some relationship here it looks like the number of bacteria which is the only one state variable doubles every hour we can express this mathematically in the following way where n of T is the number of bacteria at time T now here's a question for you what would the population size B in between those hourly measurements for example if you only came into the room after just 30 minutes well we know that bacteria don't wait for the 1 hour mark to suddenly double they

are dividing continuously so something must be happening in between those measurements your first guess might be that after 30 minutes the population size will be exactly halfway between 1,000 and 2,000 so about 1,500 but let's dig deeper to really understand what's going on we need to think about how fast the population is growing what we call the rate of change think of it like a speedometer for our bacteria population just like a car speedometer shows kilometers per hour how fast you're moving our bacterial speedometer will show how fast the population is growing in terms of

bacteria per unit time we can denote this rate of growth with a little dot above n like this in our petri dish each bacterium divides independently that means that at any given moment the rate with which new bacteria are appearing is proportional to the number of bacteria already there we can write this as an equation the constant K represents the fraction of the population that are dividing at any given moment and it depends on things like temperature or food availability this simple looking formula is actually a very big deal in mathematics this is called a

differential equation and it relates a value of a quantity to its rate of change differential equations are the language of dynamical systems describing how things change over time from physics to Neuroscience let's zoom in even closer on this concept of a rate of change in mathematics we call it a derivative and there is a special way of writing it as dn/ DT equivalent to the dot notation we used earlier imagine DN as a tiny change in the number of bacteria and DT as a tiny Step in Time the derivative DN / DT represents how much

n changes when we take an infinitely small step DT into the future if you imagine the population size over time as a graph then the derivative the value of our speedometer at any given time corresponds to the slope of the line tangent to the graph at that point in real world of course we cannot have a fraction of a bacterium but it's useful to think of the population as changing smoothly and continuously growing by very small amounts as you take very small time steps into the future it lets us rely on the apparatus of calculus

that describes and analyzes continuous processes now this is where things get interesting when we move from Pure math to the Practical world of computation and modeling we need to make a subtle but a very important shift instead of dealing with infinitely small changes we'll work with still small but measurable steps to highlight this difference we'll use the symbol Delta instead of D so Delta n represents a small but not infimal change to n and delta T A small but not infinitely Small Step in Time this shift is at the heart of numerical methods which we'll

rely on throughout the video series let's see how it works in practice imagine we decide to measure our population every 5 minutes instead of every hour 5 minutes is small enough that we can reasonably assume the rate of change doesn't vary much during that time this corresponds to the idea that if you zoom in close enough on the graph of a curve it starts to look like a straight line let's work through an example suppose we start with a th000 bacteria and we know that k equals 1 when time is measured in hours this means

that our rate of change equation becomes the following so when we have a th000 bacteria they are multiplying at a rate of a th000 bacteria per hour or about 17 every minute after 5 minutes we estimate the population size to become a 1085 bacteria instead now we update our rate of change based on this new population size to become 18 bacteria per minute instead of 17 so we can estimate the population size for the next 5 minute time stamp and we can keep repeating this process to estimate the population for any future time this is

the essence of solving differential equations with numerical methods those of you who immediately recognize this formula as the equation for exponential growth might be wondering why we are spending so much time on this computational approach when this particular equation can be much more easily solved analytically and you'd be right but the reason is simple for most of the differential equations out there including the ones governed in the neuronal Dynamics such an analytical solution that would allow us to Jump Ahead multiple computational steps simply does not exist numerically however we can solve basically any differential equation

imaginable there are several important differences between those two approaches the numerical solution requires us to perform many update steps multiplications and additions to arrive at the result getting estimates for later time demands more computational effort unlike an analytical solution we don't have an explicit formula where we can simply plug in values and get a result instantly another crucial aspect is the finite Precision of our method in our example we used 5 minute steps but for greater accuracy we might need to decrease the step size perhaps updating our estimate every minute or even every second the

more accurate we want our answer to be the more computation it requires there is one more important point to consider in our computations we assumed we somehow knew the underlying value of K that relates the rate of change to the value of the population size in reality however we don't usually have access to parameters like these instead we collect data similar to the table we had in the beginning and we might need to run multiple computations to try different values of parameters to find the one that would fit our data best this process too involves

finite Precision so inaccuracies are inevitable now that we've established what differential equations actually are and how to solve them let's talk about the geometric intuition behind dynamical processes and how these equations can help us gain insight into the behavior of the system to introduce key Concepts let's explore another fundamental model in biology now we're stepping into the world of coupled differential equations B we will have multiple State variables and the rate of change of each depends on all the others consider the interactions between coexisting populations in a predator prey model often Illustrated with rabbits and

foxes as the rabbit population grows foxes have more food leading to an increase in their numbers however more foxes means more rabbits get eaten causing the rabbit population to decline with fewer rabbits the fox population shrinks allowing rabbits to reproduce and the cycle continues this intuitive understanding is certainly compelling but can we build a simple mathematical model that would describe these oscillations let's start by defining our variables X will denote the number of rabbits and Y will mean the number of foxes we are aiming for a system of equations in this form where F and

J are functions relating the rate of change of each variable to the overall state of the system in reality those functions can contain countless complexities and dependencies on other factors however we are seeking the simplest mathematical model that captures the essence of such oscillations let's start with our prey population rabbits reproduce at a rate proportional to their population given us the term ax where a is the positive net growth rate note that there is also an analogous rate for natural deaths proportion to the population size but here we are assuming that the birth rate significantly

exceeds the natural death rate so the total population growth rate is positive rabbits also get eaten by Foxes we account for this with an interaction term minus B * XY in the equation for x this term is proportional to the product of X and Y representing the probability of rabbits and foxes encountering each other multiplying by the fraction of encounters that lead to rabbits being caught thus our equation for the prey population is the following now for the Predators foxes benefit from eating rabbits converting a proportion of that energy into reproduction this gives us the

positive interaction term we also include a term for natural deaths which typically outweights natural births for predators in the absence of prey this results in the following differential equation for y this simple model of a two couped differential equations is a very powerful tool for exploring the geometry of dynamical systems now how do we solve this system as before we'll rely on numerical methods suppose we know the values for all the four parameters a b c and d given an initial State we can compute the derivatives make incremental changes to X and Y repeat this

process this would give us two curves showing how each variable changes Over time however to gain a deeper Insight let's visualize our data differently instead of plotting time as a horizontal axis and population size as a vertical imagine a coordinate plane where x-axis represents the number of rabbits and Y AIS represents the number of boxes notice that there is no time Dimension here each point on this plane represents a possible state of our system at a given instant with time governing the trajectory of switching between the states such a coordinate plane that corresponds to different

states of the system as a whole is known as the phase space in this phase space each point has two corresponding rates of change the instantaneous velocities of the rabbit and fox populations we can represent these as vectors little arrows coming from each point with the horizontal component of the arrow given by the derivative of x and the vertical given by the derivative of y this graphically shows the mapping between the systems State XY coordinates and its rates of change if we plot these vectors for many points we create a vector field showing how the

system's Evolution flows from any starting point this is known as the phase portrait with such visualization we can gain qualitative insight into systems Behavior without doing precise number crunching we can simply just visually follow the arrows to get a feeling for the trajectory with that model in hand let's see what interesting Dynamics we can observe first let's look for special points within our face space where the system settles into balance these are known as equilibrium points and they correspond to both derivative being equal to zero if the system happens to fall in either of those

States it will remain there forever as there is no mechanism for it to get out equilibrium points can be found by setting the right hand side of our equations to zero turning differential equations into a system of familiar algebraic equations with two unknowns that are numbers instead of functions solving this gives us two equilibrium points 0 0 which means that both populations are extinct it's a trivial but mathematically valid solution and there is another nonzero equilibrium which means that there exists a balance between predators and prey now that's more interesting in our face portraits these

equilibria appear as points where all arrows shrink to zero indicating that once you reach that point there is no way out rabbits and foxes will be exactly balanced in each other route with no changes to the population Dynamics but how does the system behave around these points let's start with the origin if we are slightly off of this point let's say a few rabbits but no foxes what happens well our equations tell us that the population of rabbits will continue to grow exponentially while the population of boxes remains zero in the face space that looks

like a trajectory moving right along the x-axis the zero Z equilibrium is unstable the slightest perturbation sends the system away from it in the other direction though if we have a few foxes but no rabbits their population will go down approaching the point of Extinction closer and closer as there is no food for the foxes to reproduce now what about the nonzero equilibrium if we start near that point and follow the arrows in our face portrait we see something remarkable the pop ations cycle endlessly around the equilibrium creating a closed loop this behavior is called

a limit cycle and it will actually be fundamental when we talk about the excitability of neurons notice that we never explicitly put any oscillations into our system of equations there are no trick functions or circles in the initial description limit Cycles emerge from this seemingly simple Dynamic law of interaction between the two variables and the isn't just one there are infinitely many cycling orbits which one you'll end up on depends on the initial conditions so far we have fixed a particular set of parameters varying the values of a b c and d doesn't fundamentally change

the behavior of the system it only affects the position of the non-zero equilibrium as well as the exact shape and amplitude of the limit cycle oscillations slightly modifying our system of equations to account for the limited availability of food to rabbits leads to another interesting Behavior instead of a large number of cycles that can be realized depending on the initial condition there is now a single stable equilibrium which all trajectories converge to defining the stable balance between rabbits and foxes as we have seen even such a simple system as the Predator prey model can exhibit

very rich and complex behaviors the idea of a face portrait gives us a powerful visual tool to understand these Dynamics without getting lost in the mathematical weeds ideas such as the stability of equilibrium points and the emergence of limit Cycles form the foundation of dynamical systems theory applicable to a variety of fields crucially these same ideas can help us understand the intricate workings of the brain neurons like the populations of rabbits and foxes can be described as dynamical systems as as well the state variables might represent things like the membrane potential and the state of

ion channels instead of the population sizes but all the underlying principles remain the same however to truly understand the differential equations that govern the Dynamics of a neuron we will first have to dive into the world of cellular biophysics which deserves a dedicated video so stay tuned for that next adventure in the meantime if you're interested to learn more about the mathematics we talked about today you're going to love the message from out today sponsor brilliant.org brilliant is one of the best places to learn math and physics online they offer a wide range of interactive

courses that make complex topics accessible and engaging such a Hands-On approach means that you're not just passively watching you're actively solving problems and building intuition one course you might find particularly interesting is Vector calculus it is incredibly relevant to what we discussed today remember those portraits with arrows showing the directions of change that's Vector calculus in action this course will help you understand the mathematics behind those Vector fields and much much more from surface integrals to diffusion all in the form of bite-sized interactive lessons but brilliant doesn't stop there they also have courses on computer

science and data analysis no matter what your background and interests are brilliant has something for everyone ready to take your learning to the next level head to brilliant.org Aram konov to get a 30-day free trial of everything brilant has to offer as well as a 20% discount on annual subscription if you like the video press the like button share it with your friends and subscribe to the channel if you haven't already also consider supporting me on patreon where you can find text versions of video scripts containing all the math details as well as the ability

to vote and suggest video topics stay tuned for more computational neuroscience and machine learning topics coming up goodbye and thank you for the interest in the brain