Elegant Geometry of Neural Computations

picture two identical neurons receiving exactly the same input to the naked eye they are indistinguishable same size same shape same basic structure you might expect them to behave more or less identically yet one is sitting quietly at rest while the other is rapidly firing bursts of electrical impulses what causes this strike indifference you might think that there are some fundamental variations in cell electrophysiology or biochemistry that are invisible from the outside but the answer lies in their history what these neurons were doing before that observation well all the other factors including cell biophysics could in

principle be identical unlike simplified neurons in artificial neural networks that act as basic input to Output functions biological neurons have memory their response depends not only on the input but also on the recent history of activity today we will see how this remarkable property arises from the internal dynamics of neurons along with other fascina neurocomputational properties of biological cells using the powerful language of dynamical systems we will gain a geometric intuition that makes these complex behaviors seem Crystal Clear if you're interested stay [Music] tuned let's start by modeling a single neuron in the previous video

we explored the the hkin Hawley model a complete biophysical description of neurons electrical properties if you haven't seen it I highly recommend watching it first as it will be our starting point today but here's a quick recap a neuron's electrical behavior is governed by four differential equations that track how key parameters or state variables change over time the central equation describes changes in membrane voltage driven by ionic currents flowing through the membrane the other three equations monitor the proportion of open ion channels which themselves depend on voltage creating an intricate feedback loop but here's a

catch being four-dimensional this system is impossible to visualize directly to gain intuition we need to simplify it let's start with the gating variables specifically n which represents the fraction of open potassium channels we derived its equation by considering transitions between two states with Alpha and beta representing the rates of gates becoming permissive and non-permissive imagine we freeze the voltage at some value now Alpha and beta become positive numbers while initially there may be an imbalance eventually the system reaches an equilibrium where the two directions of this reaction balance perfectly we can highlight this equilibrium seeking

Behavior by rewriting the equation after expanding and rearranging terms terms we get the following this is mathematically identical but the interpretation of constants is a bit different here and infinity represents the equilibrium value the fraction of open channels at a given voltage when the dust settles and both sides reach their stable balance and to is the time constant telling us how quickly that equilibrium is reached think of it like this if too many channels are open which means that m is greater than n INF the negative rate of change closes them until reaching an equilibrium

if too few are open the positive rate drives them up the smaller the tow the faster this process happens a key Insight emerges when we compare time constants across gating variables the M gates are lightning fast compared to n andh so fast in fact that we can treat them as instantaneous since we have an explicit equation for M Infinity we can simply replace M with its equilibrium value and eliminate one differential equation completely even better it turns out many neurons generate perfectly good spikes without H gates at all their potassium currents alone can counteract sodium

and restore the rest in state without the inactivation gate so by removing H we are left with just the two coupled equations one for voltage and one for potassium Gates this simplification opens up a powerful visualization tool called the phase space imagine a landscape of moving water where each point represents a possible state of our neuron voltage on one axis and potassium channels on another the differential equations act as the laws of fluid dynamics creating currents and addes in this space drop a leaf at any point and it will carried Along by these currents tracing

out the neurons Behavior over time the direction and speed of the flow at each point represented by arrows in our diagram called the face portrait tell us exactly how voltage and channel States change from moment to Moment In This Plane there are points of Tranquility called equilibria where there is no flow in either direction these correspond to states where both derivatives equal to zero some equilibrium points are stable like a ball resting at the bottom of the bowl if you give it a small push the system's Dynamics will naturally drive it back to the equilibrium

these stable points often correspond to the neuron's res in state other equilibria are unstable like a ball balanced precisely at the top of the hill where the slightest perturbation sends it racing away to understand how these equilibria arise let's examin special curves called NK lines regions of the face space where just one variable remains constant the voltage NK line shows all states where voltage remains steady where Inward and outward currents perfectly balance along this curve our arrows Point either straight up or straight down as only the gating variable continues to change similarly the n n

Cline reveals where the fraction of open poti ium channels stays fixed with arrows pointing purely horizontally where these n lines intersect we find equilibrium points finally let's clarify what we mean by injecting current into the neuron any external input whether applied through an electrode or received from other neurons is simply another source of ionic current flowing through the membrane we can incorporate it into the voltage equation as an additional term alongside other currents when we adjust the input current it shifts the V NK line up or down in our face portrait altering the system's Dynamics

if you find it uncomfortable and biologically unrealistic to reason about what applying a current to a neuron means you can think of it as a kind of ion channels whose conductance depends on an external Factor such as a concentration of neuromodulator molecules floating around in extracellular space resulting in persistent current then the concentration of this hypothetical neurom modulator is a knob that we can turn controlling the level of excitation supplied to a neuron while a persistent current does change the face portrait throughout the video we will also think about transient perturbations which can be visualized

as brief instantaneous in fact pulse-like currents which just push the system State horizontally without affecting the face portrait with that mathematical framework at hand let's try to map Concepts from the face space to neural behavior in their default State neurons sit quietly at a stable equilibrium point they resting potential where small pertubations from this equilibrium are quickly dampened out give the neuron a tiny push and it will return to rest the fundamental property of neurons is their excitability if you push the voltage Beyond a certain threshold something dramatic happens the neuron generates an action potential

this isn't just a simple return to rest as before it's a large stereotypical Excursion through the phase space where the voltage Rises explosively potassium channels activate bringing the voltage down and the system eventually resets excitability means that larger perturbations are not dampened they are Amplified by the system's internal Dynamics but it could be that the trajectory doesn't stop at the resting state after completing one Spike instead there are certain regimes typically observed with the sustained input current being sufficiently large when the neuron fires repetitively in the language of dynamical systems this corresponds to a limit

cycle a closed loop in the face space that the neuron transverses again and again the system is no longer SE in a stable resting state it is locked into the repetitive pattern of activity where each loop around the cycle corresponds to one action potential to understand why such transitions arise and what causes the behavior of the neuron we need to take a closer look at the underlying face portrait as mentioned before the stable equilibrium corresponds to the resting state it is also called a stable node since all nearby trajectories flow into it but zooming out

and looking at the nullclines we can see that there are in fact two more points in which they intersect so there must be two other equilibria the problem is that both of these equilibria are unstable so even the tiniest perturbations will cause the state to rapidly diverge from this equilibrium but in which direction where does this runaway trajectory go this is where the roles of these two unstable equilibria become important let's start with the one on the right this equilibrium seems to be right in the center of the limit cycle so no matter in which

direction the trajectory is knocked away from the equilibrium it's going to fall into the Basin of Attraction of the limit cycle wind itself around it and continue looping indefinitely resulting in rhythmic firing but what about the one in the Middle where the ascending part of the voltage nline intersects with the N noine this equilibrium is known as a saddle point indeed if you start near it trajectories are pulled towards it along One Direction and pushed away Along another Direction resulting in the saddle shape there is a special curve in the face plane called the separatrix

which actually converges to the equilibrium if you follow it perfectly this curve acts as a boundary a watershed separating the face plane into two regions distinct basins of attraction trajectories that start inside it get swept into the spiking orbit while the ones that start outside return to rest as you can see the separatrix plays the role of a threshold if the input perturbation is large enough to knock the trajectory into the attraction domain of the limit cycle the neuron will fall into the spiking Behavior if not return to rest however if the push is too

strong that it overshoot shoots the limit cycle attraction domain entirely the trajectory will also return to rest with a spike detour there is thus a sweet spot for pertubation amplitude to switch the neuron into persistent firing you may have noticed an interesting duality in neuronal responses when we perturb a neuron from rest one of the two things can happen either it generates a single Spike and returns to rest or it transitions into persistent firing State this this behavior depends crucially on where the limit cycle is located in the face space and how it interacts with

the separatrix in some neurons the limit cycle exists as a separate orbit carving out a basin of attraction in the face space with the separatrix forming a boundary around this attraction domain for these neurons pertubations of the right amplitude will cause the system to transition from rest to persistent firing this means that under the same input current the neuron can be either resting or repetitively spiking depending on its history which is why such neurons are called by stable this dependency is called hysteresis it's a powerful computational feature allowing individual neurons to serve as kind of

memory elements that maintain their state either rest or firing even after the pertubation that triggered the switch is gone this explains the mystery we saw in the very beginning other neurons however which we call all mono stable have their bi physical parameters tuned in such a way that the limit cycle intersects with the stable equilibrium in these neurons even when a perturbation is strong enough to trigger a spike the trajectory inevitably returns to rest because any wbe spiking orbit must pass through the stable equilibrium in other words if there is a stable equilibrium persistent spiking

is not possible without well-timed perturbations if we want to make such a neuron fire repetitively on its own we need to remove the stable equilibrium by fundamentally changing the face portrait this brings us to The crucial concept of bifurcation which refers to qualitative change in the face portrait such as when equilibria appear disappear or change their stability importantly bifurcations involve changes in the underlying differential equations and thus will change the no lines and Vector fields for example example increasing the persistent current applied to a neuron corresponds to a constant shift in the vine and thus

might cause a bifurcation as we'll see shortly in other words we are changing the landscape of fluid flows unlike merely pushing the leaf around with perturbations let's turn the knob on the applied current and see what happens to the bable neuron first as the voltage NK line shifts up the two points where it intersects with the N NK line the stable node and the saddle approach each other closer and closer until at some critical value of current they fuse and kind of annihilate each other indeed this branch of the voltage nline Now lies fully above

the N nline so the only point of intersection is the unstable equilibrium that is in the middle of the spiking orbit because there is no more rest in equilibrium basically all trajectories get attracted to the Limit cycle so at this current repetitive spiking is the only Behavior the neuron can exhibit this kind of bifurcation observed when a node and a saddle coales is known as the saddle node bifurcation when we do the same thing with a monostable neuron we see a similar picture of the node and the saddle annihilating each other because there is no

more stable equilibrium getting in the way of the limit cycle this neuron can now start to fire persistently this bifurcation is a special and important case of the saddle node bifurcation called saddle node on invariant Circle or snake for short to emphasize that the bifurcation is happening on the cycling orbit waiting to break loose while we just saw how a saddle node bifurcation can lead to repetitive firing there is another excitability route that neurons can take one that is accompanied by drastically different behavior let's look at a neuron with a slight slly different biop physics

of potassium channels which tend to open at lower voltages in this neuron the end nine is shifted to the left so now the voltage nline intersects it only at one point instead of three this means that our face portrait has just one equilibrium in the absence of input current that equilibrium is stable and corresponds to the resting state now as we increase the current this equilibrium slides along the end nline until at a critical point it under go a qualitative change switching from stable to unstable such bifurcations where a single equilibrium suddenly changes stability rather

than colliding with another equilibrium are called andron of hop by forcas these come in two flavors subcritical and super critical in this super critical case when input current is low and the equilibrium is stable all trajectories eventually converge to rest however the Dynamics are not simply linear certain perturbations get Amplified causing the system to take a spike-shaped detour before returning to rest importantly though these are not stereotyped all or non-action potentials with saw previously their shape and amplitude depends on the perturbation size some might look like full spikes While others are more like half spikes

of smaller amplitude this happens because there is no unique spike in orbit but rather multiple possible detour trajectories like neurons with snake bifurcations these neurons are monostable there is no separate stable orbit of repetitive firing and while the stable equilibrium exists the neuron can only generate individual spikes in response to strong perturbations as we increase the input current past the bifurcation point the equilibrium becomes unstable and a small limit cycle emerges around it now all trajectories converge to this orbit large pertubations might cause temporary excursions but the system always returns to these uniform oscillations the

neuron no longer has a single rest in State instead its equilibrium behavior is to produce small rhythmic oscillations a strong push might trigger a larger Spike of variable size but the system will always settle back into its characteristic oscillatory pattern with further increases in input current this limit cycle gradually grows until its amplitude matches that of Spike responses transitioning the neuron into repetitive firing this explains why Spike amplitude can vary the sharp distinction between spikes and non-s spikes that characterizes all or non responses is blurred in these neurons the subcritical case tells a different story

here even before the equilibrium loses stability there exists a large amplitude limit cycle a full spiking orbit somewhere out in the face space however this stable cycle is separated from the stable equilibrium by an unstable cylic curve which acts as a threshold any perturbation that pushes the system Beyond this boundary will cause it to jump into the large amplitude firing orbit this creates by stability the neuron can either rest quietly or fire repetitively at the same input current depending on whether its state lies inside or outside this unstable cycle when we increase the input current

the unstable cycle shrinks around the equilibrium at the bifurcation point this cycle collapses onto the equilibrium itself causing it to become unstable now there is no more barrier protecting the equilibrium and the tiniest perturbations will send the system into repetitive firing the transition is sudden and dramatic unlike the gradual emergence of a cations we saw in the super critical case this brings us to another fundamental distinction in how neurons process information neurons that transition to firing through saddle node bifurcations either pure saddle node or snake are called integrators they gradually accumulate perturbations until their state

crosses the separatrix threshold triggering either a transition to repetitive firing or a single Spike for these neurons the timing and the pattern of inputs don't matter only their cumulative effect in contrast neurons that undergo undrawn of hop by forc are called resonators these neurons exhibit sub threshold oscillations making them sensitive to input timing they preferably respond to inputs that arrive at specific phases of these oscillations and poorly timed excitatory inputs might even suppress their response rather than enhance it think of it like this an integrator neuron is like a bucket filling with water you can

add water quickly or slowly regularly or irregularly anyway once it reaches the threshold it spills a resonator neuron is more like a swing pushing it at random times won't get you very far but even small pushes at the right Rhythm can build up to a large response all right let's try to put everything together we have seen how dynamical systems theory and geometric reasoning about face portraits elegantly explain the diverse zoo of neural Behavior our exploration revealed two fundamental dichotomies in neural Dynamics monostable versus by stable neurons and integrator versus resonator properties these distinctions perfectly

map onto four different kinds of bifurcations a neuron undergoing a saddle node bifurcation is a by stable integrator at a given input current it can either rest or repetitively Spike depending on its history switching it from rest to firing requires sufficient perturbation but the timing of inputs is relevant a neuron with a Snick bifurcation is a monostable integrator the same input current cannot support both behaviors since the resting equilibrium blocks the spiking orbit repetitive firing only emerges when the applied input current is strong enough to an at the stable equilibrium with its neighboring saddle a

neuron exhibiting super critical and drawn of H bifurcation is a monostable resonator it has prominent oscillations around the resting state which means that the timing of pertubation matters and it has certain resonant input frequencies and because the spiking orbit emerges from the destabilized equilibrium only one regime is possible at a given level of Applied current a neuron with a subcritical hop bifurcation is a by stable resonator it combines timing sensitivity with the ability to maintain either resting or spiking States at the same level of input current as the spiking orbit can coexist with the stable

equilibrium while real neurons are undoubtedly much more complicated these fundamental mechanisms help explain their diverse computational roles for example integrator neurons excel in brain regions that need to accurately en code input strength like sensory systems and resonator neurons are prevalent in circuits that need to process temporal patterns or generate rhythmic activity such as those controlling movement or maintaining certain forms of memory as you can see this diversity in neural Dynamics isn't just a mathematical curiosity it is fundamental to brain's computational power this video was mostly based on a wonderful book by Eugene isik cabage called

dynamical systems in science I highly recommend taking a look if you're interested in diving deeper into what we talked about today with more rigorous and extensive treatment and if you're looking for a way to build up mathematical Foundation to tackle books like this you're going to love the message from our today's sponsor brilliant.org brilliant combines interactive visualizations with problem solving to make complex stem topics engaging their Hands-On courses break down challenging Concepts into bite-size l lessons that build your intuitive understanding step by step I particularly want to highlight their linear algebra course which is incredibly

relevant to today's discussion for example analyzing the stability of equilibrium points relies on igen vectors and igon values crucial tools that are essential whether you're studying the biophysics of real neurons or abstract neural networks prent offers a great variety of courses across mathematics computer science and data analysis whether you're just starting out or looking to solidify your understanding of complex topics brilliant has got you covered if you're ready to take your curiosity to the next level head to brilliant.org krenov to get a 30-day free trial of everything brilliant has to offer plus a 20% discount

on your annual subscription if you enjoyed the video share it with your friends subscribe to the channel if you haven't already and consider supporting me on patreon where you can vote for video topics and find extended versions of video scripts stay tuned for more interes in neuroscience and machine learning topics coming up goodbye and thank you for the interest in the break