1.4 How is Reinforcement Learning Structured?

Finally, to wrap up our introduction, let’s take a look at:

How is Reinforcement Learning Structured? 🤔

Taxonomy of Reinforcement Learning

• Lecture 1 – Introduction.
  A discussion on why it matters, what it is, where it is applied, and how it is structured.

• Lecture 2 – Mathematical foundations for understanding Reinforcement Learning.
  Key math topics covered include Set Theory, Linear Algebra, Calculus, and Probability.

• Lecture 3 – Learn to make decisions in one-state environments using the Multi-Armed Bandits framework.
  Key algorithms covered are \(\epsilon\)-greedy, Upper Confidence Bound (UCB), and Thompson Sampling.

• Lectures 4–6 – Learn to make decisions in multiple finite-state environments by leveraging experience and bootstrapping.
  Key algorithms covered are Dynamic Programming (DP), Monte Carlo (MC), Temporal Difference (TD), and n-step Bootstrapping.

• Lectures 7–11 – Learn to make decisions in multiple infinite-state environments using gradient-based learning and classical RL methods (MC and TD).
  Key algorithms covered are Value Function Approximation (VFA), Deep Q-Networks (DQN), Policy Gradients (VPG), Advanced Policy Gradients (APG), and Monte Carlo Tree Search (MCTS).

• Lecture 12 – Conclusion.
  A discussion on advanced topics, real-world applications, and the future outlook of Reinforcement Learning.