Fly plays.
Giving a fly’s wiring something new to do.
Researchers mapped the nerve cells in a male fruit fly’s brain and nerve cord, along with the connections between them. This is called a connectome. Think of it as a wiring diagram: it tells us which cells connect to which. I took that map and built an experiment around it, with Space Invaders and Tetris as the testing ground.
Here’s what happens when you hit Start. A tiny picture of the game goes into simulated eye cells. Signals travel through 166,700 cells and 25.6 million connections, and a small controller uses some of those signals to choose a button press. It starts without knowing how to play. Good or bad results give it feedback to adjust its next choices.
The fly’s wiring stays fixed; the controller is the part that learns. Your browser runs the whole thing and saves each game’s practice separately. Bear in mind that learning doesn’t guarantee improvement. The early results are rough, which is part of what makes this worth experimenting with.
Explore the original MaleCNS mapLet’s try it.
Checking the download size…
It plays on its own. You can watch and pause.
Hit Start to watch the cells fire.The positions are measured. The activity is simulated.This is what the eyes get.
Trying: Waiting
Feedback adjusts the controller’s choices.
Each finished game adds a score here.
Give it a few attempts and see how they compare.
Each game keeps its own practice in this browser, when storage is available. Restart begins a new round with that practice. Reset learning starts that game from scratch.
Let’s follow a picture through the network.
First, where the map came from.
MaleCNS is a collaboration led by HHMI Janelia, with Google Research, Cambridge, and MRC LMB. Google’s AI helped trace cells through electron-microscope images. Experts checked and labeled the results. That gives researchers a way to study the paths between sensing something and doing something about it.
A wiring diagram is a useful starting point, but we still need rules for how signals move through it. Think of a road map: knowing where the roads go doesn’t tell you how much traffic is on each one.
Read about Google’s contribution ↗Then, the part I built.
I used that wiring as the basis for a TypeScript simulation, inspired by the ornata/fly project. I added simplified rules for the cells, a way to feed in a game screen, and a small controller that turns the resulting activity into moves.
Take Space Invaders for instance. An enemy moves across the screen, the picture going into the eye cells changes, and that changes signals entering the network. The controller samples activity from cells farther along the path and might choose to move left and fire. A later hit gives it feedback.
Picture in, signals through the wiring, button press out. Repeat, with feedback.
Give it a picture.
The playfield becomes a 64 by 48 pixel grayscale picture. Brightness and changes between pictures stimulate 6,006 simulated eye cells. The vision panel shows this input. Scores and interface buttons stay outside it. The mapping is approximate; this isn’t a reconstruction of exactly what a fly sees.
Let the cells pass signals around.
Picture a bucket with a small leak. Incoming signals fill it, some of that activity fades, and reaching a threshold makes the cell fire and reset. Its connections then affect other cells. All 166,700 retained cells participate, including ones you don’t see in the display.
Turn that activity into a choice.
The controller reads recent firing from 1,316 cells that carry signals away from the brain. Ten times per simulated second, it chooses a move. Feedback changes how likely different moves are next time. The bars show those probabilities, so the longest bar won’t always be the move it tries.
A few things you might be wondering +
What exactly is learning?
The controller has two parts: one chooses actions, and the other estimates how much reward might follow. When an outcome is better or worse than expected, it adjusts those estimates and choices. This is called actor–critic learning. The measured network stays fixed during play. A new browser starts with an untrained controller; returning to the same browser can restore its practice.
What counts as a good move?
In Space Invaders, hitting an enemy earns a reward, finishing a wave earns a bonus, and losing a life gets a penalty. Tetris rewards cleared lines and penalizes reaching the top. It also gives smaller hints based on changes in stack height and holes. Think of those as feedback along the way, since a beginner may go a long time without clearing a line.
The game can look at its board to calculate that feedback. The controller itself receives the reward number and neural activity. It doesn’t receive a list of enemy positions or a handy description of where a Tetris piece should go.
Will it actually get good?
I don’t have evidence of reliable improvement yet. In the first short comparisons, the trained controllers didn’t consistently beat frozen or random controllers, and none cleared a Tetris line. A score going up for one round could just be luck. The full write-up explains the comparisons and the results.
How much of this is a real fly?
The connections and cell annotations come from the research data. The signal dynamics, screen mapping, and button controller are modeling choices. N.B. a bright dot here is a simulated firing event, not a recording of a living cell. This experiment doesn’t establish that a fly is experiencing the game.
What am I looking at in the brain panel?
The dots are a sample of 4,096 measured cell locations, projected into two dimensions. Brighter dots fired more recently in the simulation. The full retained network still runs. Signal rate is the average firing rate of the cells the controller reads; Hz means events per second.
What happens when I leave?
The browser tries to save each game’s controller and recent scores. Reloading starts a new round using that saved practice. Restart also keeps it. Reset learning erases it for the selected game. Learning off keeps the controller’s settings fixed while it plays. Saves stay on this browser and device, so clearing browser storage clears them too.
Why the big download, and why might it run slowly?
The prepared wiring is about 139 MB (132 MiB). It downloads from Cloudflare Pages, with this site’s Vercel copy as a backup. Completed files are checked and cached when the browser has room, so you usually only need that big download once per browser.
After loading, your device does the simulation and learning. If it can’t keep up, simulated time slows down for both the brain and the game. For example, 0.5× actual means one second in the game takes about two seconds on your clock. Switching away pauses it until you hit Resume.
I wrote a longer walkthrough of the research, the implementation, and what the first tests actually showed. Read: Giving a fly’s wiring a game to play ↗
Inspired by ornata / fly ↗
Built with MaleCNS v1.0 data ↗ · CC BY 4.0
Data: FlyEM / HHMI Janelia, Cambridge,
MRC LMB and Google Research. Modified for this experiment.