Welcome back. So I'm really excited to do this lecture on reinforcement learning. I've been wanting to do this for a long time.
Those of you who know me know that I love control theory and machine learning and reinforcement learning is kind of at this sweet spot between these two super important fields. Okay, so reinforcement learning is essentially a branch of machine learning that deals with how to learn control strategies to interact with a complex environment. And one of the ways I think about this, the way I'm going to define this, is that reinforcement learning is a framework for learning how to interact with the environment from experience.
This is a very biologically inspired idea. . .
this is what animals do. So through trial and error, through experience, through positive and negative rewards and feedback, they learn how to interact with their environment. OK good.
So before I jump in I want to show some motivating videos. I really like this one where reinforcement learning is used to learn how to walk in this artificial environment. And there's a lot of papers like this where people use reinforcement learning as kind of an optimization framework to learn how to control a complex system, in this case a bipedal walker, often in a simulated environment.
And this just looks really cool and it's a difficult control problem. This is a really hard non-linear control problem. Now the goal would be to take what you learn here and start to port that over into the real world to make better robots and better actual physical agents that can interact with the world alongside us, to learn how to learn like humans and animals do.
So another video I love. . .
this is my dog Mordecai and my wife has trained him. . .
this is a treat on his nose. . .
to hold the treat on his nose until she says ok, after which he can then grab the treat and eat it. This is not an easy trick to learn and this again this goes to show you anytime you, anybody who's trained an animal, a dog or any other animal, has done some type of reinforcement learning or reinforcement training.
