Q-learning
Learn action values $q(s,a)$ from experience — without a full transition model (model-free).
Quality (deterministic sketch)
$$q(s,a)=r(s,a)+\gamma\\max_{{a'}} q(s',a')$$
$q$ scores “how good is it to take $a$ in $s$?”. Policy: pick $\\arg\\max_a q(s,a)$ (with exploration while learning).
Why model-free helps
In a physical robot you may not have a perfect $f(s,a)$. Q-learning updates $q$ from the transitions you actually observe: try $a$, see $r$ and $s'$, bootstrap with $\max_{{a'}} q(s',a')$.
Key takeaways
- $q$ scores state–action pairs.
- Model-free: learn from interaction.
- Basis for many deep RL methods (DQN, …).