idle↑PrevNext↓↓ scroll for more sims▲52▼PID Cart-Pole Balancer☆r/control·u/matrix·2 comments·link🖱drag the cart · keys 1-4 toggle PID termsA classic inverted pendulum on a cart, kept upright by a PID controller acting on the pole angle θ. The control law is F=Kpθ+Ki∫θdt+Kdθ˙, with a small auxiliary term pulling the cart back to centre. Toggle the P, I, and D channels independently to feel what each does: P alone oscillates around vertical, adding D damps the oscillation, and I cancels the steady-state lean the cart accumulates while drifting. Drag the cart along the track to disturb the system; press 1, 2, 3 to flip the P / I / D badges on or off, and 4 to cycle through gain presets (off, balanced, P-only, aggressive). The HUD reports the live angle θ, the saturated commanded force F, the cart position x, the active preset, and the current gains.show more
pausedidle↑PrevNext↓▲26▼Q-Learning Cart-Pole☆r/control·u/matrix·3 comments·linkA tabular Q-learning agent learning to balance an inverted pendulum on a cart from scratch, with no model of the dynamics. The continuous state (x,x˙,θ,θ˙) is bucketed into a small grid (about 180 discrete states) and the agent has just two actions: push left or push right with a fixed force. At every step it updates its action-value table with the standard rule Q(s,a)←Q(s,a)+α[r+γmaxa′Q(s′,a′)−Q(s,a)], picking actions ε-greedily with ε decaying from 1.0 toward 0.02. Early on the pole falls within a handful of steps and the agent looks hopeless; as the Q-table fills in, episode length climbs and eventually saturates near the 200-step cap drawn as the dashed green line. The chart below the cart shows raw episode lengths (blue) and a moving average (yellow); the HUD reports the current episode, decaying ε, best episode so far, and total training frames. Watch the yellow curve cross 200 to see the policy converge.show more