Lunar Lander — RL agent, live
A reinforcement-learning agent trained with Expected SARSA on a neural network built
from scratch in NumPy — running right here in your browser on a faithful JavaScript
port of the Gymnasium LunarLander-v3 physics.
What you're watching
The purple lander is controlled 50 times a second by a two-layer neural network (8 → 256 → 4) whose forward pass, backpropagation, Adam optimizer and replay buffer were all hand-written in NumPy — no PyTorch, no TensorFlow. It was trained with the Expected SARSA algorithm; the bars above show its live estimate of each action's value. The physics is a 1:1 port of Gymnasium's Box2D environment, so the agent flies the exact dynamics it was trained on (verified against the Python original to <10⁻⁶).
Try clicking the scene to shove the lander — the agent corrects and recovers. By default it replays episodes it lands cleanly; tick random episodes to see unfiltered behaviour (it sometimes parks in a cautious hover — nudge it!), or wind for conditions it was never trained on.