Learning Objectives
DATS 6450 – Reinforcement Learning
Lecture 1: Introduction
1.1 Why Should I Learn Reinforcement Learning?
1.2 What is Reinforcement Learning?
1.3 Where is Reinforcement Learning Applied?
1.4 How is Reinforcement Learning Structured?
Lecture 2: Math Foundations
Learning Objectives
2.1 Set Theory
2.2 Axiomatic Probability
2.3 Conditioning
2.4 Independence
2.5 Discrete Random Variables
2.6 Continuous Random Variables
2.7 Probability Distributions
Lecture 3: Multi-Armed Bandits
Learning Objectives
3.1 Multi-Armed Bandit Framework
3.2 ε-Greedy
3.3 Upper Confidence Boundary (UCB)
3.4 Thompson Sampling
3.5 (Optional) Contextual Multi-Armed Bandits (CMAB)
Lecture 4: Dynamic Programming
Learning Objectives
4.1 Markov Chain
4.2 Markov Decision Process (MDP)
4.3 Iterative Policy Evaluation
4.4 Value Iteration
Lecture 5: Monte Carlo
Learning Objectives
5.1 Monte Carlo Prediction
5.2 Exploring Starts Monte Carlo
5.3 On-Policy Monte Carlo
5.4 Off-Policy Monte Carlo
Lecture 6: Temporal Difference
Learning Objectives
6.1 Temporal Difference (TD) Prediction
6.2 SARSA
6.3 Q-Learning
6.4 Double Q-Learning
6.5 (Optional) n-step Bootstrapping
Lecture 7: Function Approximation
Learning Objectives
7.1 Value Function Approximation
7.2 On-Policy Function Approximation
Lecture 8: Deep Q-Networks
Learning Objectives
8.1 Deep Learning
8.2 Deep Q-Networks (DQN)
Lecture 9: Policy Gradients I
Learning Objectives
9.1 Policy Gradient Theorem
9.2 Addressing Sparse Rewards
9.3 Action Selections
9.4 Vanilla Policy Gradient (VPG)
Lecture 10: Policy Gradients II
Learning Objectives
10.1 Trust Regions
10.2 Monotonic Improvement
10.3 Proximal Policy Optimization (PPO)
Lecture 11: Monte Carlo Tree Search
Learning Objectives
11.1 Model-based Reinforcement Learning
11.2 Monte Carlo Tree Search (MCTS)
Lecture 12: Conclusion
Learning Objectives
12.1 Advanced Topics in Reinforcement Learning
12.2 Identify the Reinforcement Learning Application
12.3 Outlook of Reinforcement Learning
Applications
Finance
Pseudo-Labeling
Recommendation Systems
Homeworks
Homework 1
Homework 2
Homework 3
Homework 4
Homework 5
Homework 6
Homework 7
Homework 8
Homework 9
Homework 10
Homework 11
Homework 12
Exams
Exam 1
Exam 2
Final Project
Final Project Overview
Final Project Ideas
References
Learning Objectives
Learning Objectives for Lecture 3: Multi-Armed Bandits
(
Sutton and Barto 2018
)
🎯
Multi-Amed Bandits Framework.
\(\epsilon\)
-greedy.
Upper Confidence Boundary (UCB).
Thompson Sampling.
Bernoulli & Gaussian generated environment using
numpy
.
Taxonomy of Reinforcement Learning
Sutton, Richard S., and Andrew G. Barto. 2018.
Reinforcement Learning: An Introduction
. 2nd ed. Cambridge, MA: MIT Press.
2.7 Probability Distributions
3.1 Multi-Armed Bandit Framework
