A prompt like “make a colorful boat racing game” leaves almost every interesting visual decision to the model. How red are the cliffs? How close is the camera? Does the water look painted, glassy or plastic? How much screen space does the boat occupy?
A good reference answers those questions immediately. I want to carry those decisions into something I can play.
Reference to Game is an open-source skill and toolkit for coding agents built around that idea. Start from an image, a video, a design sheet, existing assets or a mixture of references. Work out which decisions matter, build a small playable scene, then compare it with the source before expanding the game.
Give the agent something specific to preserve
Lovis's video-to-game helped make the value of a gameplay reference clear: a clip gives the coding agent a starting point for both appearance and behavior. I wanted to extend that approach so different references could define different parts of the game.
An illustration might govern the palette and character proportions. A video might govern acceleration, jump timing or camera movement. A full development sheet can describe objects, UI, levels and character poses that one screenshot never shows.
These sources do not always agree. The workflow asks the agent to assign authority separately for appearance, identity, mechanics, motion and UI. If a painting defines the character but a clip defines movement, the agent should preserve that distinction. Anything absent from the references has to be designed and identified as a new choice.
That makes the reference useful throughout development. It becomes something to check against when a material, camera or interface drifts.
Jungle Jetboat Jump: protect the color and motion
The boat scene has a very direct visual identity: red canyon walls, blue-green water, dense vegetation and a compact orange boat. The game needs to keep that balance while giving the player enough room to steer.
The implementation combines a 3D boat and crew, painted vegetation and procedural water. Ordered gates, pickups, boost, jumps and recovery turn the scene into a racing loop. A reference can suggest speed and danger, but the game still needs rules for what happens when a player misses a gate or leaves the course.
The useful question is how the playable camera and moving scene preserve the reference's energy. Exact placement of every leaf is less helpful than keeping the boat readable against the water and the canyon recognizable at speed.
Petal Pals: keep the painting in the game
Petal Pals starts from a softer world: a painted meadow, flower rows, small companions and cream-colored interface panels. Turning every painted element into smooth 3D geometry could make the scene feel like a different game.
This example uses directional painted characters in a 2.5D world, with 3D terrain and foliage arranged to support depth. The interactions follow the setting: observe, feed, photograph, befriend and explore. Save and restore behavior lets that progress persist.
The representation has a tradeoff. Those painted assets work best from constrained views, and the camera differs from the original painting. That is a useful limitation to make explicit. Full 3D is one option; layered art can be a better fit for the image you want to preserve.
Jellyworks: make the candy factory obey rules
Jellyworks has rounded toy machinery, wooden conveyors, translucent candy and a friendly cream-and-mint interface. The visual reference makes the factory appealing. The implementation has to make it understandable.
Its conveyor routing, ingredient mixing, molds and backpressure form a deterministic puzzle system. Edit and run modes let the player change the layout and then watch the consequences. Three puzzles provide concrete goals rather than an endless decorative conveyor animation.
Rule-driven candy spacing differs from the illustration, and translucent materials use an adaptive quality fallback. These are the kinds of differences a comparison should reveal, so a reader can judge whether they matter.
A fourth case, a different kind of world
The repository also includes Sunshine Train Battle, a first-person action example using cleaned generated meshes in a train-local world. Shooting, shielding, reloading, jumping, checkpoints and restart add a different set of interaction and collision requirements.
Across the four cases, the implementation changes with the reference. Sprites, procedural geometry, meshes and shaders can all earn a place. The common requirement is that the visual direction survives contact with gameplay.
Looking right and playing well need separate reviews
A running game can still miss its reference. A beautiful screenshot can still hide broken controls. Reference to Game keeps those questions separate.
The workflow starts with a small scene and fixes the largest visual mismatch before multiplying assets or levels. It then asks for real input-driven gameplay, state-driven UI, goals, recovery and restart. Comparison images expose visual differences; rule tests and playthrough evidence address behavior.
The CLI preserves and hashes source references, helps inspect common media, samples chosen palette regions, creates comparison sheets and ties review files to a source snapshot. It can detect stale evidence. It cannot decide whether the art direction is good, and a hash cannot prove that a claimed playthrough happened.
The four games were completed before the reusable skill was packaged. They are worked examples of the approach, not a controlled benchmark showing the skill's effect. The repository's evidence log distinguishes fresh build and rule checks from historical playthrough reports.
Run a game, then try your own reference
The examples include source and runtime assets. To run Petal Pals locally:
git clone https://github.com/gokayfem/reference-to-game.git
cd reference-to-game/examples/petal-pals/app
npm ci
npm run devTo install the reusable skill:
npx skills add gokayfem/reference-to-gameThe default implementation stack is Three.js, TypeScript and Vite. Existing examples run on local assets without a fal key. New asset generation is optional; the toolkit can record fal jobs and their estimated budgets when that is part of the brief. The repository credits distinguish the MIT toolkit code from demonstration artwork and media.
For a new game, begin with references you care about. Decide what each one should control. Build enough to move through the world and feel the rules. Then put the implementation beside the reference and look at what changed.
The more clearly you define the world first, the more useful the coding agent becomes at building the game you had in mind.
