Tabular Control

---
name: tabular-control
description: "Work with the repo's toy tabular RL workflows: Q-learning, Sarsa,
  and GridWorld."
metadata:
  disco-role: operating
disable-model-invocation: true
license: MIT
---

# Tabular Control

Use this sub-skill for the repo's small tabular-control workflows. The bundled demo keeps the 1D chain examples self-contained and avoids importing the original source files just to understand or run them.

## In scope
- Q-learning and Sarsa on the toy 1D chain environment
- comparing off-policy and on-policy tabular updates
- inspecting the GridWorld helper
- explaining terminal-state handling, epsilon-greedy selection, and step counts

## Not in scope
- DQN or any neural value-function approximation
- policy-gradient, actor-critic, A2C, PPO
- DDPG, SAC, TD3, or other continuous-control families
- repo-wide install, Gym-version, or plotting compatibility topics

## Bundle
- `scripts/tabular_control_demo.py` — safe runner for the toy chain example with an algorithm switch
- `scripts/gridworld.py` — fixture-friendly GridWorld helper adapted from the original class
- `references/algorithm-notes.md` — update-loop and comparison notes
- `references/troubleshooting.md` — execution and environment notes

## Typical requests
- Run the toy example: use `scripts/tabular_control_demo.py`
- Compare Q-learning vs Sarsa: pass `--algorithm both`
- Inspect the GridWorld helper: open `scripts/gridworld.py`
- Explain the tabular update loop: read `references/algorithm-notes.md`

## Routing
- For anything outside tabular control, return to the root router at `../../SKILL.md`.
- Hand off DQN, on-policy actor-critic, and off-policy continuous-control requests to the appropriate sibling sub-skill through the root router.
- If a request needs plotting or broader repo compatibility guidance, keep it out of this sub-skill and route upward.

## Notes
- The original source scripts use pandas and matplotlib, but the bundled helper keeps those optional and does not depend on them for the default run.
- The 1D chain examples and the GridWorld helper are separate toy workflows; do not assume they share the same state or action space.