Reinforcement Learning (RL) is a subfield of machine learning concerned with how intelligent agents ought to take actions in an environment to maximize the notion of cumulative reward.
Foundations
- Reinforcement Learning - Core definitions, the agent-environment loop, and rewards.
- Markov Decision Processes - The mathematical framework (States, Actions, Transitions).
- Exploration vs Exploitation - The fundamental trade-off in learning.
Mathematical Tools
- Bellman Equations - Recursive decomposition of value functions.
- Dynamic Programming - Solving MDPs with known models (Policy/Value Iteration).
Model-Free Algorithms
- Temporal Difference Learning - Learning from incomplete episodes.
- Q-Learning - Off-policy value-based learning.
- SARSA - On-policy value-based learning.
Deep Reinforcement Learning
- Deep Reinforcement Learning - Scaling RL with Neural Networks.
- Deep Q-Networks - Experience Replay and Target Networks.
- Policy Gradient Methods - Directly optimizing the policy (PPO, TRPO).
- Actor-Critic Methods - Hybrid approaches.
Advanced Topics
- Reinforcement Learning from Human Feedback - Learning a reward from human preferences: reward model, PPO with KL penalty, DPO, and other human-in-the-loop forms.
- World Models - Causal dynamics simulation, joint embedding predictive architectures, and code-based environment models.
- Multi-Agent Reinforcement Learning
- Inverse Reinforcement Learning
- Hierarchical RL