kits: Super Mario, a System One harness plays a platform game in a browser you watch - #27
Merged
Merged
Conversation
…owser you watch The fifth kit. The harness is a fork of the systemone base with this kit's instructions and one plugin: an MCP server (plugin/bin/mario-env) that opens Full Screen Mario in a headless Chromium inside the deployment, reads the game's own state each step and says it as literal sentences (where Mario is, whether he is on the ground, the nearest enemy, gap and wall in tiles, what one step reaches and what is within it), and acts by holding the keys the game listens to. The keys are sticky: a run continues between decisions and a jump keeps the run key down for the whole flight. Every step writes the browser's frame to frame.jpg in the session workspace, and the page shows it beside a panel that starts, streams, stops and continues the run. Measured on the live model before this commit: 37 actions in 69 s, one death, 47 game seconds played; earlier shapes of the state lost five to seven lives in the same budget (jumping one step late, then braking mid-air). The page-specific code is all in the plugin; the base knows nothing about the game. Credits in the kit's README and on the page: Full Screen Mario at supermarioplay.com (none of its files ship here), Browser Use (MIT), TypeSafe's Jev through OpenRouter. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…eps its last 150 items Verified on the public console at 1440 and 390 while a run streamed: the frame stays in view, the panel scrolls inside, Stop cancels the run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e is jumped The page is rebuilt as the other four kits are built: a Vite app with the shared app bar (with the way back to HarnessRouter), the stage on the left and the run's real conversation on the right in the shared chat column, every decision with the game's reply, reopenable from its link. Play starts a run with the default goal; the stage shows the run's counted figures and Stop. Before the first frame there is an empty state, not a broken image. The frame is Chrome's own screencast now, every third frame the game draws written to frame.jpg (about twenty a second, 7 KB each), read by the page with a few requests in flight and shown as each decodes, instead of one capture per step that played like slides. The environment measures the game instead of guessing: the wall detector keeps the pipe Mario is pressed against (a dx >= 0 filter dropped it, and a run stood at a 4-tile pipe for forty steps saying no wall was within 8 tiles); wall heights are taken from the ground, not from the feet mid-jump; the state says which hold clears the wall (short 3.1 tiles, medium 3.9, long 4.1, measured); and jump_right times the take-off, since a 4-tile pipe is cleared only from 1.5 to 3 tiles back and an enemy jumped from 2 tiles at a run is hit on take-off. On the model's own runs all four pipes of the level are cleared, three times in a row within 120 steps. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…vironment times the jumps Measured on the live game and written into the environment: an enemy walking at Mario is let under a standing jump taken when it is a tile away (six of six between 0.8 and 1.2 tiles); a run stops five tiles short of such an enemy, 3.5 short of a tall pipe (a jump with the run key held from against it embeds Mario in its side, "falling" in place) and at a gap's edge; a question block is hit by a jump started 1 to 1.5 tiles before it at a run; the hold an obstacle needs is taken when the model's is shorter, and said. Heights are measured from where Mario stands, so a stair step is one tile from the step below. Enemies behind Mario are in the state, since the pair pacing behind the fourth pipe took a life on every back-off while "no enemy within 12 tiles ahead" was true. The default goal asks for coins; the plugin is 0.2.0. The page compares the package version it was built with against the launched harness and relaunches it in place when they differ, so a kit updated in a new image reaches the harness without a visit to Starter Kits. The stage is sized from the height budget so the column fits a wide short window; the cards are shed below 1100px or 700px; the conversation column takes the height under the bar instead of hanging its composer below the fold. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Measured through the environment's own jump from against the fourth pipe: the back-off stops against the third pipe five tiles back, and overlapping its far edge by a tenth of a tile made the detector report it as a wall "0 tiles ahead", so the run-up jumped at once, 4.5 tiles early, with the apex 2.5 tiles short of the pipe. A wall whose far edge is at Mario's back now counts as behind him (a quarter tile of tolerance), the take-off is at 2.4 tiles, and the sampled jump crosses the pipe's top and lands on the far side. Held into a pipe's side in the air, Mario "falls" in place at terminal speed for as long as a key is held, and a let-under jump taken from against the pipe followed by the run key mid-air put him there every life. The let-under steps off the wall first and takes the run key back on the ground; after every action, and in the landing wait, a Mario held that way has every key let go, which drops him to the ground within a second (measured). The let-under wait is bounded (3.5 s, 5 s under a block row) and used for an enemy about to arrive or one under blocks; a farther one in the open is jumped at a run. A run stops three tiles short of a tall pipe and at a gap's edge as it stops short of an enemy. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… a negative distance The state said 'Question block overhead: -1 tiles ahead' once Mario was under one, and the model's confidence fell below the gate on that sentence four actions into a run (measured on the test box: stopped, no confident action). The distance is clamped at zero and said in words. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A run rewrites the URL to its own id without a navigation, so setting the hash back to the start changed nothing the router could see and the page kept its state (measured: New run after a finished run did nothing). The app now remounts the page outright and puts the URL back. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every attempt is now recorded (each screencast frame, timestamped) and every death becomes a contact sheet of the seconds before it; these changes come from those sheets, not from the trace text. An enemy met without speed is never jumped at (a jump from a standstill landed on it): one walking at Mario is let under, one walking away is waited for, since the pair turns at the next pipe. On a pipe's top nothing reaches him, so he waits for the pair below to be walking away before stepping off slowly. Beyond a gap the landing zone is waited clear for up to eight seconds, since the pair there walks into the gap. An enemy above is neither a target nor waited under: it drops off its ledge onto him. A run has no hold to judge: the model was sure of the run and unsure of its length, and the gate refused the weaker answer three times, ending a run with lives left. Enemies behind him are outrun, or let under when one is on him at a wall. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The TypeSafe demo (x.com/faadilhshaik/status/2100086301894881578) decides about ten times a second with momentary key states and bunny-hops the whole level at a run in 85 game seconds; the recordings of this kit showed the opposite design, long actions with multi-second waits beside enemies, is where the lives went. A run is now a moment of running (0.15 s), a jump is its hold and nothing more, the flight is the next decision's business, and every wait is gone. What stays is measured: the tall pipe's run-up to a take-off at 2.4 tiles (from every standing distance the apex is 3.8 and the pipe is never cleared), the gap's edge, the block's 1 to 1.5 tiles, the peel off a wall and the release that frees an embedded Mario. The kit asks for a step budget of 1200, which a launch now carries (HarnessRouter 8b48cf4). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ts are in the instructions First principles, at Richard's insistence: the environment is a faithful, fast interface to the game, the intelligence is the model's, and the measured facts of the game are instructions a person can read and edit in the harness. Each action sets the keys for a tick (0.1 s) and they stay set until the next action, the way a thumb does: run_right, jump_right, jump, walk_left, wait, no parameters. The state says which keys are held, that a held jump must be let go before another, what a decision covers at the speed there is, and what each wall and gap takes; the system prompt carries the jump heights, the run-up a 4-tile pipe needs, the let-under for an enemy walking at Mario, the block row and the coins. The run-ups, edge jumps, let-unders and waits the environment used to script are gone with the hold parameters; what stays is the release that frees a Mario embedded in a pipe's side, a glitch rather than play. The observation declares itself real-time for the harness (System One Harness 0.3.0). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
One sentence per state, computed from the measured facts and said outright: the enemy walking at Mario is a tile away, jump standing; the pipe's take-off point is here, jump_right and hold it three decisions; stopped at a tall pipe, walk_left two decisions first; nothing within reach, run. Also the block row overhead and when a walking enemy arrives. The environment executes none of it; the model reads it and chooses (a System One model acts on the state, not on a paragraph). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… enemy jump is measured A decision arrives about a quarter second after the state it reads; at a run that is 2.5 tiles, so every distance in the Now line is projected by that much. Measured on the first Goomba: a running jump started 3 to 4 tiles before it survives every time and stomps it for 300 at 4; from 5 to 9 tiles it hits the block row and drops onto it, every time. The wait-and-let-under advice there is gone; the prompt and the Now line say the running jump at 3 to 4 tiles. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…y outrank a pipe Recorded: after 'jump_right now' at a tall pipe the next line said the run frees the jump key, so the jump was let go at 0.3 s, peaked at 3.9 tiles and hit the 4-tile pipe; then 'stopped at a tall pipe: walk_left' walked Mario into the pair behind him. The line now says to keep the jump held while rising, treats an enemy within four tiles as the thing that matters, and asks for the enemy jump at the measured window. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d a ledge enemy is waited for Measured: from standing, jump_right as the enemy arrives survives eight of eight between 1 and 2.5 tiles, and it is the action the model follows (the standing jump tool it did not). Recorded five times near the first ledge: the Goomba on it walks off its edge and drops onto a Mario running under; the state now says to wait for it to come down and jump it at Mario's height. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A jump key still held from the last jump does nothing on the ground (the game wants a release first); recorded, four lives went to a jump that never happened. The action now lets the key go and presses it again within the tick, the way a thumb does. Measured: a running jump started 1.5 to 2.5 tiles before an enemy survives six of six (1.0 dies), so in the open the Now line asks for the jump from farther out; the 3-to-4 window is the block row's. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… under before backing off Recorded at the fourth pipe: a jump at a jog hit its side, and the back-off line then walked Mario into the pair behind him, four lives. The line now asks for full speed (4.5 and up, the measured clearing speed) before the take-off, sends him back to gain it when there is room, and when an enemy follows him to a pipe says to let it under a standing jump first. The gap line asks for speed the same way instead of a jump from a standstill at its edge. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Recorded six times: the jump across a gap lands nine tiles on, among the enemies waiting there, and a falling Mario can do nothing. Enemies walking toward the gap come to it and fall in, so the state says to stand at the edge until none is within ten tiles past it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s, and the prompt has the follower rule Recorded five times: the Goomba on the ledge past the first gap walks off its edge whichever way it faces when the jump is taken, so it counts in the landing zone whatever its direction. At the fourth pipe the model walked left into its follower with 0.5 to 0.7 probability against the state's standing-jump line: the prompt's own pipe advice said walk left. The prompt now carries the follower case. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ounts deaths again The Now line said 'now' one decision ahead by a constant quarter second; the loop decides 3.4 times a second here and 2.2 through the test box's relay, so the environment now measures the interval between its own actions and projects by that. The gap's landing zone counts a ledge enemy from the gap's edge; the pipe's back-off is four decisions and a jump too close to the pipe is refused for a fresh run-up. The page's observation parser expected the old sentence without coins and counted no deaths. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Three earlier edits to this prompt never landed (a heredoc escaping error in the edit script raised before the write), so the attempts since read the first facts prompt while the Now line carried the tactics. The prompt now states them all: the enemy take-off window, the standing jump_right as one arrives, the ledge enemy, the pipe's speed and back-off, the follower, the gap's landing zone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…enemy Measured: right kept through a running jump's descent lands 9.4 tiles on and slides 2 more; left held brakes it to 6.8 and lands Mario stopped. Recorded five times: the jump over the fourth pipe landed a tile before the first gap and the run carried him in before a decision could land. The in-air line now says to brake when a gap or an enemy sits where he would land. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
At half a second a decision Mario covers four tiles and a Goomba walks one, so an enemy first named at 12 tiles was 3.7 tiles away by the next state and the jump was already late (recorded on the test box, three lives). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…next state is past its window Recorded: at full speed one decision covered eight tiles, the gap went from 9.5 tiles ahead to 1.5 in one step, and a wait at that point slid Mario in. The gap line starts at 14 tiles, the stop starts while there is room, and a jump too close to stop is taken with the air brake. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…landing, and the nearest thing ahead decides Recorded on attempt 33 and reproduced nine times in a row by a player that obeys the state's own Now line: the jump over the fourth pipe comes down a tile before the first gap and no decision could arrive in time, a mid-air brake dropped him in from 0.8 tiles, the floor read from his feet vanished once he was below its top (the gap 12 tiles wide), and a gap five tiles on outranked the one-tile step he was pressed against until the clock ran out. Now the ground ahead is read from Mario's level as a profile (pits, drops, steps), a jump held in the air is pressed again the moment he lands (any later action cancels it), an action sets the keys and returns (no tick), the nearest thing ahead decides the Now line, a level change is the level cleared, and the first Goomba's take-off is taken in its window or given up for the stop and the standing jump. The Now-line player at a 0.2 s delay clears 1-1 in 50 game seconds with one death. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… README says what the environment is now The follower's one death on the level: the wait for the Goomba above to drop off its ledge let every key go a tile before the 3-tile gap. The nearest thing decides. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The fifth starter kit: Full Screen Mario played by a System One model (Jev through OpenRouter) in a headless browser inside the deployment, watched from the console.
What it is
systemonebase with one plugin,mario-env: an MCP server that opens the game in a Chromium it drives over the DevTools Protocol (Browser Use, MIT), reads the game's own state each step as a few literal sentences, and acts by holding the keys the game listens to. The base knows nothing about the game; all of it lives in the plugin, and the base declares no built-in tools (HarnessRouter 68be604).The frame
Chrome's own screencast, every third frame written whole to
frame.jpgin the session's workspace (about twenty a second, 7 KB each), read by the page through the live workspace with a few requests in flight. Measured from the console: first frame 8 s after Play, then 13 to 15 frames a second shown.What is measured, not tuned
The state is literal and the numbers come from the game: the ground ahead as a profile (the next gap, the next drop and what follows it, the next step or wall and its height), the nearest enemy ahead and behind, the block rows overhead, the next question block, how far one decision carries Mario at the loop's own measured pace, and a closing "Now:" line that says what the measured facts call for. The environment executes key states and times nothing: an action sets the keys and returns, and the model chooses every step. Two input-layer semantics, the way a thumb plays: a jump means a press (the game wants the release), and a jump held in the air goes again the moment Mario lands, so hops chain. A player that obeys the state's own "Now:" line with a model-like delay is the environment's test bench apart from any model; it clears 1-1 in 50 game seconds with one death at 0.2 s.
Level 1-1 is cleared (2026-09-20): by the model on its own run (Jev 1.13 through OpenRouter, 351 actions, 122 s, 6 deaths, 4 lives at the flag, score 18,850) and on the test box through the real page at 2.1 actions a second (160 actions, 86 s, lives 7, score 18,200). Every death class was found on a recording (a copy of each frame the page shows, cut into a contact sheet around each death), measured with a probe, then written into the state. The kit README records each measurement.
Credits are in the kit's README and on the page: the game at supermarioplay.com (Mario belongs to Nintendo; none of the game's files ship here), Browser Use, Jev by TypeSafe, and the open-source System One Harness.
Depends on HarnessRouter PR #223: the
systemonebase, its entrypoint installing the harness's browser extra with a Playwright Chromium on the data volume, Chromium's system libraries installed per container (cdcd738), and the base declaring no built-in tools (68be604). Verified end to end on the test box, and from a brand-new data volume installing the harness from the public repository.🤖 Generated with Claude Code