Skip to content
jameskbbPublic

About

Neurons of a fly's brain taught to play golf

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Fly Golf

FLY GOLF title card: a trained golfer fly over a dark green diagonal texture, with the subtitle Can a fruit fly break 100?

CI License TypeScript Three.js Connectome

Can a fruit fly break 100?

Fly Golf connects simulated neural activity running over the reconstructed MaleCNS v1.0 fruit-fly connectome to a fully 3D golf environment. The fly plays an 18-hole course, chooses its own club from a full bag, sets up every shot, records its neural telemetry, and can use a trained readout of descending-neuron activity to make its golf decisions. It practised only on the front nine; the back nine is held out, so every shot it plays there is on holes it has never seen.

So, can it break 100? Sometimes. Over 18 real holes the trained fly broke 100 in 13 of its 46 real rounds: the ten rounds in the web demo average 99.6 (5 of 10 under 100), and 36 benchmark rounds on fresh seeds average 102.6 (8 of 36 under 100). The back nine, which it never practised on, costs it about one to two strokes more than the front. None of this shows that the real wiring helps: on the front nine a randomly rewired connectome and a readout of the raw senses both play better, and neither control has been run on the back nine yet (details).

▶ Watch the Web Demo · ▶ Watch the 25-second short · How It Works · Run It Locally · Scientific Caveats

Animated gameplay from Pond Hop: the trained fly addresses the ball, its readout picks the 7-iron, and the shot carries the pond and rolls up 4.7 feet from the flag, filmed with every panel hidden so the course fills the frame

One real shot on Pond Hop, the 164-yard third: the tee shot of the web demo's featured round (trained-eighteen-s07, seed 7), played by the current trained readout (20260917T165536Z). 400 ms of MaleCNS activity become a 7-iron that carries the pond and stops 4.7 ft (1.4 m) from the flag. It is the recorded shot replayed by the web demo in cinema mode (press F there), so no panel covers the course; watch it there or open the 1280×720 MP4.

What is real, and what is not

  • Real: the MaleCNS v1.0 anatomical wiring. 166,700 neurons and 25,582,938 connections from the brain and ventral nerve cord of one male fly (CC BY 4.0).
  • Modelled: the neural dynamics. A leaky integrate-and-fire engine implements the whole-brain model of Shiu et al. 2024 and reproduces its Brian2 reference spike for spike (LIF_ENGINE.md).
  • Engineered: the golf sensory mapping (which neurons receive golf information) and the golf motor mapping (which neurons become the swing), plus the golf physics. All are documented.
  • Trained: an optional readout that learns, offline, to turn descending-neuron activity into putt-or-swing, club, aim and power. The connectome never learns. This is not synaptic plasticity inside MaleCNS (TRAINING.md).
  • This is not a digital copy of a fly's mind, and no fly understands golf. It is simulated neural dynamics operating over a real wiring diagram, wired to golf by hand.

The 25-second short

The fly at address with its 7-iron, captioned 7-IRON

▶ Watch it here (25 s, sound on): one hole, cut for a phone: the fly's readout picks a 7-iron, carries the pond to 4.7 feet, and then takes three putts, including a lip-out from 14 inches.

Every frame is the web demo replaying the same recorded hole you can open yourself; the video adds captions, synthesized sound and speed changes, and nothing else. How it was cut, shot by shot, is in social/short-02, together with the capture and render tools that rebuild it from the recorded run.


The web demo

jameskbb.github.io/fly-golf is the same 3D app replaying recorded MaleCNS rounds. The neural simulation was computed beforehand; the replay runs entirely in your browser. There is no backend, and nothing is simulated or invented on the page. Pick a brain and watch 18 holes one shot at a time: each hole is drawn at random from that brain's recorded rounds, so every visit shows different play, and the scorecard says which round each hole came from. Pause, scrub through a swing, jump to any hole or stroke, orbit the camera, and read each shot's recorded spikes, active neurons, population rates, motor channels and club choice. Each strike plays a club-on-ball hit, synthesized in the browser rather than sampled: 🔊 Sound (or S) turns it off, and ⛶ Fullscreen (or F) hides every panel so the course fills the window. Press Watch the brain fire (or B) to see each shot's recorded 400 ms of activity light up a map of the fly's brain, slowed down while the fly lines up. The dots are the real MaleCNS v1.0 cell-body positions, binned; the light is only what each shot recorded: network spikes per 10 ms, each sensory and descending-neuron population's mean rate before and during the 150–400 ms readout window, and 18 named descending neurons' own rates. Which single neurons fired is not recorded, so none is shown firing alone, and Mock shots, which have no brain, show nothing. To watch the brain play new shots as they are computed, run it locally: the connectome download needs more than 1 GB of disk.

Each brain recorded ten complete 18-hole rounds, seeds 7–16 (the same seeds as the earlier front-nine demo), fixed before any round was played. Every one of them is in the demo; none was picked from several attempts:

Brain Out (front nine) In (back nine, never practised) Total, ten rounds Holes holed out Rounds under 100
Trained MaleCNS (featured) 49.3 50.3 99.6 (93–115) 176 of 180 5 of 10
Untrained MaleCNS 81.0 81.0 162 in every round 0 of 180 0 of 10
Mock (no neurons) 36.5 37.1 73.6 (68–75) 180 of 180 10 of 10

The trained fly takes drivers off the tee and wedges around the greens, read out of descending-neuron activity by a readout fitted from practice. The untrained fly's fixed rules hit a mid iron from everywhere and pick up every hole. The mock is not a fly and has no neurons: it reads the distance and lie directly, so it plays well, and it is the reference.

The web demo at the end of a mixed trained round: the 18-hole scorecard shows the Front Nine out in 49 and The Neuropil Nine in 53, a total of 102, with a ROUND row naming the recorded round each hole was drawn from, and the verdict Round complete: 102 (+30). The trained fly did not break 100. The back nine is ground it never practised on. The brain panel shows the recorded neural telemetry of the last putt

One random mix of the trained fly's recorded holes, captured as it came (every visit draws a new one): its 18 holes come from 9 of the ten recorded rounds, as the ROUND row shows. This mix totals 102, out in 49 and home in 53, so the verdict under the card says it did not break 100. Other mixes can land on either side of 100.

Does the fly break 100? Sometimes, and on average not quite. The ten demo rounds average 99.6: 5 of them are under 100 (93, 94, 94, 97, 98), one is exactly 100, which does not break 100, and the worst is 115. On 36 more rounds from fresh seeds (fly-golf bench, seeds 100–111, 200–211 and 300–311) the trained fly averages 102.6 and breaks 100 in 8; its best is 88 and its worst 119. Together: 13 rounds under 100 out of 46. The "98.6" this README used to quote was a front-nine round doubled, not an 18-hole round. The fly only got this close after the sensory injection was rebuilt so that it can actually perceive which way its target lies (see SENSORY_MAPPING.md).

The back nine, which it never practised on, costs it about one to two strokes: in the demo rounds it goes out in 49.3 and comes home in 50.3, and in the three bench blocks the back nine averages 51.8, 50.6 and 53.8 against 50.3, 51.4 and 49.8 on the front. So what the readout learnt on the front nine transfers to unseen holes almost fully. That is a statement about the sensory injection plus the readout, not evidence that the real MaleCNS wiring helps: on the front nine a degree-preserving shuffled connectome trains a better readout than the real one, and a readout of the raw senses with no brain at all beats both. Neither control has been run on the back nine yet (TRAINING.md).

All thirty demo rounds were recorded from a clean tree at commit 6eceef0 (readout 20260917T165536Z, course eighteen-v1), and every round re-simulates bit for bit (fly-golf replay). Holes 1-9 of each round are stroke for stroke the earlier front-nine round with the same seed. How the demo works and how to publish another run: GITHUB_PAGES.md.

Meet the three brains section header

Meet the three brains

The full-body 3D renders below are the same procedural models used on the course. They share one animated rig, but their bodies tell the scientific story: the wind-up toy has no brain, the plain fly uses the untrained connectome, and the dressed golfer uses that same connectome with a practised readout. The costumes are visual metaphors, not biological claims.

Mock: the wind-up

Full 3D model of the Mock controller persona: a clockwork tin fruit fly holding a putter, with a winding key and lamp-like eyes

No brain · test stand-in. Mock is a small, hand-written golf heuristic with seeded noise and a caddie's distance table. It reads the distance, slope and lie directly, chooses a club and hits the shot. No neurons or connectome are simulated. The clockwork body makes that impossible to mistake: it does exactly what its rules wind it up to do. Mock exists to test the course, physics, animation and recording, and to provide a strong no-brain reference.

MaleCNS: the wild fly

Full 3D model of the MaleCNS controller persona: the plain fruit fly holding a putter for the untrained real-connectome controller

Real connectome · untrained. Each shot runs 400 ms of simulated LIF dynamics across all 166,700 neurons and 25,582,938 connections in MaleCNS v1.0. Engineered sensory channels stimulate chosen visual and gravity-sensing populations; fixed, a-priori rules read descending-neuron activity into club, aim, power, tempo and contact. The wild fly has never had a lesson, and it plays like it: the current readout tends to reach for the same mid iron and does not finish holes.

Trained: the trained golfer

Full 3D model of the trained golfer persona holding a putter in a tartan tam, argyle vest, plus-fours and golf glove

Real connectome · practised readout. This is the same MaleCNS graph, neuron model, synapses and sensory input as the wild fly. Nothing inside the connectome learns. Offline practice fits linear weights that turn roughly 480 descending-neuron-type firing rates into a putt-or-swing decision, club, aim and power; the tam, argyle, plus-fours and glove make that external practice visible. It now finishes almost every hole and sometimes breaks 100 over 18, but the honest control result is that a readout of the raw senses, and a degree-preserving shuffled connectome, still play better. The full method and results are in TRAINING.md.

You can switch brains at any time, even mid-hole. The next shot uses the new controller, every shot record names it, and the scorecard marks which brain played each hole; a round involving more than one is labelled MIXED BRAINS. The picker headshots and the on-course characters are rendered from the same procedural 3D models.

Around the front nine section header

Around the front nine

Fly Golf National is 18 hand-built holes, par 72 and 6,831 yards. The front nine is the parkland half: 3,352 yards and par 36, straight holes and gentle doglegs with one kind of hazard at a time. It is also the only ground the trained fly has ever practised on. These north-up renders come from the course itself, so every stripe, bunker, water line, green and tree boundary matches the geometry used by the physics. The nine totals come from the holes' exact lengths, so the rounded yardages below sum to 3,351 on the front and 3,480 on the back rather than 3,352 and 3,479.

Open the full front-nine yardage book · bird's-eye renders and strategy for all nine holes
Bird's-eye render of hole 1, First Flight
1 · First Flight
Par 4 · 364 yd
Fairway bunkers pinch the landing area; two more guard the green.
Bird's-eye render of hole 2, The Dogleg
2 · The Dogleg
Par 4 · 412 yd
A hard turn left around the corner bunker rewards position over power.
Bird's-eye render of hole 3, Pond Hop
3 · Pond Hop
Par 3 · 164 yd
All carry over the pond to a green that tilts back toward the water.
Bird's-eye render of hole 4, Long Haul
4 · Long Haul
Par 5 · 522 yd
A creek crosses just past driving distance: lay up, or commit to the carry.
Bird's-eye render of hole 5, Lakeside
5 · Lakeside
Par 4 · 380 yd
The lake owns the entire left side, and the green leans toward it.
Bird's-eye render of hole 6, Little Sting
6 · Little Sting
Par 3 · 137 yd
Short on the card, but its small green is ringed by four bunkers.
Bird's-eye render of hole 7, Wingspan
7 · Wingspan
Par 4 · 423 yd
A long dogleg right with sand waiting on the inside corner.
Bird's-eye render of hole 8, Marsh Run
8 · Marsh Run
Par 5 · 553 yd
A marsh pond splits the fairway; a second water hazard guards short-right.
Bird's-eye render of hole 9, Home Stretch
9 · Home Stretch
Par 4 · 396 yd
A cross bunker divides the fairway before a big, fast finishing green.

Around the back nine section header

Around the back nine: The Neuropil Nine

Holes 10 to 18, 3,479 yards and par 36, played at dusk under a low golden sun. The back nine brings the hole types the front nine lacks: an island green, a drivable par 4, a double dogleg, a cape hole, a long par 3, a narrow chute and a wide-open hole of waste sand, with bunkers laid in patterns (a hexagonal cluster, church pews, curved bands) and greens that vary far more (radius 10 to 17 m, stimp 9.5 to 12.5). Each hole is named after a part of the fly's nervous system or body, its shape echoes that part, and it borrows its strategy from a famous golf hole, from the Redan at North Berwick to the island green at TPC Sawgrass. The holes, their names and the dusk theme are engineered design choices: nothing about a hole comes from the connectome, and naming a hole after a neuropil is decoration (that neuropil does not play it). Every fact below is from COURSE.md.

The trained fly has never practised a single shot here. Its readout was fitted on front-nine situations only, so every back-nine score is a test of how what it learnt transfers to holes it has never seen. fly-golf bench reports front- and back-nine splits for that reason. So far it plays them only a little worse than the front: in the ten demo rounds it comes home in 50.3 on average against 49.3 out, and in 36 bench rounds the back nine averages 52.1 against 50.5 (see the results; this says the readout transfers, not that the real wiring helps). As a check that every hole is fair to a player that aims where the environment points, the mock controller (a hand-written heuristic with no neurons, not the fly) holes out every back-nine hole over 22 rounds and averages within 0.6 strokes of par on each of them.

Dusk on hole 15, Doughnut Island: the trained fly's tee shot, a white trace high over the island green in its ring of water, heading past it
15 · Doughnut Island, stroke 1: a 6-iron from 139 yd carries 170 yd, over the island and into the water behind it.
Dusk on hole 18, The Way Home: the trained fly in its tam and argyle vest strikes a 6-iron from the fairway, the aim line running ahead of the ball
18 · The Way Home, stroke 3: a 6-iron from 166 yd finishes on the green, 50 ft from the pin.

Both stills are the trained fly's recorded round trained-eighteen-s07 (the web demo's featured round, seed 7), replayed by the web app in cinema mode: shots 84 and 108 of 111. On 15 that splash ends in the water 32.5 yd from the pin and costs a penalty stroke; the ball is dropped on the island, 19.9 yd from the pin, and the next shot, a lob wedge, finds the water too. The hole takes 8 strokes; 18 takes 6. That round is the worst of the ten, 115 (54 out, 61 in). Open the island-green shot.

Open the full back-nine yardage book · bird's-eye renders and strategy for holes 10 to 18
Bird's-eye render of hole 10, Fly's Eye, at dusk
10 · Fly's Eye
Par 4 · 408 yd
Seven pot bunkers packed in a hexagon, like the facets of the compound eye, wait just past the drive, in the manner of the Old Course and Royal Lytham. Stay left of the cluster.
Bird's-eye render of hole 11, The Ear, at dusk
11 · The Ear
Par 3 · 220 yd
A long par 3 after the Redan: one curved bunker sweeps across the front-left like an antenna's arista, and the big green, the steepest on the course, falls away to the back-left. Run it in from the right.
Bird's-eye render of hole 12, Balancing Act, at dusk
12 · Balancing Act
Par 5 · 561 yd
A double dogleg, right and then left, like the balance organs that beat opposite the wings. A burn crosses the line of play twice, as the Barry Burn does at Carnoustie.
Bird's-eye render of hole 13, The Bridge, at dusk
13 · The Bridge
Par 4 · 274 yd
Drivable on paper, after Riviera's 10th: Oakmont-style church pews guard the direct line to a small, fast green with water to its right. "Drivable" describes the geometry, not a choice the fly makes: the pin is 242 m from the tee, beyond the 225 m inside which the pin becomes its target, so the fly is shown the lay-up area on the left. Only a full-speed driver within about 2 degrees of the pin line would hold the green.
Bird's-eye render of hole 14, The Mushroom, at dusk
14 · The Mushroom
Par 4 · 425 yd
The tee sits in a cup of sand, the calyx. A narrow stalk of fairway runs out and splits into two lobes, one straight up to the green and one off to the right, a split fairway in the spirit of Riviera's 8th.
Bird's-eye render of hole 15, Doughnut Island, at dusk
15 · Doughnut Island
Par 3 · 139 yd
An island green inside a ring of water, the fly's compass neuropil played as TPC Sawgrass's 17th. Short means a drop on the tee side; long means a drop back on the island. A causeway at the back right is the only way on foot.
Bird's-eye render of hole 16, The Fan, at dusk
16 · The Fan
Par 4 · 418 yd
The widest hole on the course fans out toward the biggest, slowest green. Curved bands of waste sand, broken into columns, cross the fairway, with more waste on both sides.
Bird's-eye render of hole 17, Escape Route, at dusk
17 · Escape Route
Par 5 · 599 yd
The longest, straightest hole, after the fly's fastest escape pathway: a narrow chute through the trees, the tightest corridor on the course. Straight is the only strategy.
Bird's-eye render of hole 18, The Way Home, at dusk
18 · The Way Home
Par 4 · 436 yd
A cape hole after Macdonald's at the National Golf Links, around the clubhouse lake: the more water the drive carries, the shorter the way in. Water waits short-left of the green too.

Status and quick start section header

Status

Front nine: 9 hand-designed parkland holes, fairways, bunkers, water on holes 3/4/5/8, tree-lined out of bounds (COURSE.md) ✅
Back nine, The Neuropil Nine: 9 more holes played at dusk (island green, drivable par 4, double dogleg, cape, long par 3, narrow chute, waste sand), water on holes 12/13/15/18, a corridor width per hole. Held out from training (COURSE.md) ✅
18-hole rounds: fly-golf round plays all 18 by default (or --first/--last), the app's scorecard shows OUT, IN and TOT ✅
Full bag: driver, 3W, 5W, 4H, 5–9 irons, PW/GW/SW/LW, putter. The fly chooses via the club_reach motor channel ✅
Full-shot physics: drag + Magnus flight, spin, surface bounce and roll, penalties, lies ✅ tested, replayable
Trained readout v2 of MaleCNS descending-neuron activity (putter gate, club head, aim/power heads, calibrated by practice), with no-brain and shuffled controls. Refitted onto the v0.3 sensory injection: 50.5 strokes per front nine over 36 fresh rounds (was 66.5). On 18 holes: 102.6 over 36 bench rounds (8 under 100) and 99.6 over the ten demo rounds (5 under 100); the back nine, never practised on, costs about one to two strokes (TRAINING.md) ✅ experimental
Switch brains mid-round (Mock / MaleCNS / Trained); scorecard marks who played each hole, mixed rounds flagged ✅
Web demo: Showcase Mode replays recorded MaleCNS rounds on GitHub Pages, no backend (GITHUB_PAGES.md) ✅
fly-golf bench: complete 18-hole rounds per brain with front- and back-nine splits (holes finished, strokes, trees, club by distance); --nine front reproduces the earlier front-nine bench ✅
Deterministic, backend-authoritative putting physics (slope, stimp, cup capture, lip-outs) ✅ tested
Controller interface: SensoryEncoder → BrainController → MotorDecoder → MotorTarget ✅
MOCK CONTROLLER (hand-written heuristic, labelled everywhere) ✅
MaleCNS controller: full-graph LIF simulation (fly-golf-lif-v1, Brian2-exact), proxy sensory input, descending-neuron readout ✅ ~1.2 s of compute per 400 ms decision
3D course, procedural fruit fly golfer, swing animation driven by decoded motor output ✅
Live WebSocket telemetry, brain panel, technical panel, run history, deterministic replay ✅
A body per brain: Mock plays as a clockwork tin fly (no neurons inside), MaleCNS as the plain fly, Trained as the same fly in golf clothes ✅
Experiment records (JSONL) with git commit, seeds, versions, neural summary and trajectory ✅
Does the connectome play well? ❌ Not yet. The untrained readout hits a mid iron from everywhere and never finishes a hole. The v2 trained readout picks wedges around the green, putts every green shot, finishes 95 to 98 % of its holes and breaks 100 over 18 holes in a minority of rounds (13 of 46), but on the front nine a readout of the raw senses plays far better, and a degree-preserving shuffled connectome trains better than the real one. Neither control has been run on the back nine. See TRAINING.md

Run it locally

Requirements: uv, pnpm 10+, Node 22+. uv installs Python 3.12 for you.

make setup      # uv sync + pnpm install
make dev        # backend http://127.0.0.1:8000  +  app http://localhost:5173
make test       # all unit tests (no connectome needed)

Open http://localhost:5173 and press Hit (or Space). The header reads LIVE SIMULATION while the backend is connected. The app starts on the 18-hole course with the MOCK CONTROLLER. Switch to Practice green for seeded putts (putter only, recorded as putting-v2 on v0.2 sensing; the original v0.1 putts replay with fly-golf replay), click a hole number on the scorecard to jump to it, and press C to hide the card.

To let the real connectome play:

make data       # downloads ~1.1 GB (public, CC BY 4.0), verifies SHA-256, compiles in ~10 s

Then choose MaleCNS or Trained in the brain picker. Details are in data/README.md. The full graph needs about 1 GB of RAM while it runs.

The web demo, locally and without a backend:

pnpm showcase                        # http://localhost:5173/fly-golf/
pnpm build:showcase && pnpm preview  # the exact GitHub Pages build at http://localhost:4173/fly-golf/

Headless, from the command line:

uv --directory services/sim run fly-golf round --controller mock --seed 7      # all 18 holes
uv --directory services/sim run fly-golf round --controller malecns --seed 7 --first 10 --last 18   # the back nine only
uv --directory services/sim run fly-golf putt --controller malecns --seed 7 --traces
uv --directory services/sim run fly-golf train --scale 4 --seed 2 --jobs 12     # trained readout (docs/TRAINING.md)
uv --directory services/sim run fly-golf bench --controller malecns-trained    # 18-hole rounds, front/back splits (--nine front|back)
uv --directory services/sim run fly-golf runs
uv --directory services/sim run fly-golf replay <run_id>     # re-simulate; checks the trajectory is identical
uv --directory services/sim run fly-golf export-showcase <run_id> --slug <id> --title "…"   # docs/GITHUB_PAGES.md

Windows

Use the same commands through pnpm, so make is not required. In PowerShell:

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"   # uv
corepack enable; corepack prepare pnpm@latest --activate                            # pnpm
pnpm run setup
pnpm run dev
pnpm run test
pnpm run data        # optional: MaleCNS connectome

WSL2 also works exactly like Linux, and this project was developed on WSL2.

How it works section header

How it works

This diagram shows what the code does today. The colours show what is real data (green), modelled (blue), engineered by us (orange), trained (purple) and test infrastructure (grey).

flowchart LR
  G[GolfEnvironment<br/>deterministic physics] -->|Observation| S[SensoryEncoder<br/>proxy-v0.2]
  S -->|SensoryFrame<br/>14 bounded channels| B{BrainController}
  B --> M[MockBrainController<br/>MOCK: no neurons]
  B --> C[MaleCNSController]
  C --> E[LIF engine<br/>simulated dynamics]
  W1[(MaleCNS v1.0 wiring<br/>166,700 neurons · 25.6M connections)] --> E
  E -->|descending-neuron rates| R[fixed readout rules]
  E -->|descending-neuron rates| TR[trained readout<br/>optional, fitted offline]
  R --> D[MotorDecoder<br/>embodiment]
  TR --> D
  M --> D
  D -->|DecodedStroke| T[MotorTarget<br/>Simulation / future Hardware]
  T --> G
  G -.->|WebSocket state + shot records| W[3D web app]

  classDef real fill:#1f6b40,stroke:#39e08a,color:#fff
  classDef modeled fill:#1d4e7a,stroke:#6cc7ff,color:#fff
  classDef engineered fill:#7a4a12,stroke:#ffb938,color:#fff
  classDef trained fill:#4b2a78,stroke:#c38bff,color:#fff
  classDef mock fill:#444,stroke:#999,color:#fff
  class W1 real
  class E modeled
  class S,C,R,D,T,G engineered
  class TR trained
  class M mock
Loading

The loop runs once per stroke. The fly senses the scene, its brain simulates 400 ms, one stroke comes out, and the ball flies and rolls. There is no arrow back into the brain: the reward is recorded for analysis but never fed to the brain, and the neural state is reset to rest before every stroke. The only learning is the optional readout, fitted offline, outside the connectome.

The same app runs in two modes (ARCHITECTURE.md): live, driven by the FastAPI + WebSocket backend above, and showcase, which replays exported shot records as static files on GitHub Pages.

apps/web            React + three.js (R3F) app: course, fly, brain panel, replay; live and showcase builds
packages/protocol   Typed wire protocol and showcase format (zod) + shared JSON fixtures, checked from both sides
services/sim        Python backend (FastAPI, NumPy, Numba)
  fly_golf/golf         physics, course, clubs, environments
  fly_golf/brain        interfaces, sensory, mock, motor, registry, trained readout
  fly_golf/brain/malecns  engine.py (LIF), graph.py, populations.py, controller.py
  fly_golf/experiments  runner.py (sessions, JSONL records, replay), bench, showcase exporter
  fly_golf/training     the trained-readout pipeline
  fly_golf/data         prepare.py + compiler.py (make data)
  fly_golf/api          app.py (REST + WS), schemas.py
data/               lock file + README; downloads/compiled graph are git-ignored
docs/               architecture, provenance, mappings, training, course, build log
runs/               experiment records (git-ignored)

One stroke, step by step:

  1. The backend knows the hole, the ball, the lie and the target.
  2. The proxy encoder turns the golf state into perceptual channels: where the target is, how far away, which way the ground falls, how fast the green is, the lie and whether water is on the line. It never provides an aim angle, a club or a stroke power.
  3. In MaleCNS mode, those channels become constant current into documented sensory populations, such as LC10 visual-target neurons and Johnston's organ gravity neurons.
  4. The network runs for 400 ms of neural time. Every retained neuron and connection takes part.
  5. Descending-neuron activity is read out: by fixed a-priori rules, or by the optional trained readout. It becomes eight motor channels, including club_reach, the club the fly reaches for.
  6. The shared decoder turns those channels into a club, a start direction and a ball launch, and the physics flies and rolls the ball.
  7. The 3D fly pulls that club from its bag and animates the decoded stroke; the ball follows the backend trajectory.
  8. Everything is recorded. fly-golf replay <run> --controller re-simulates each trajectory and re-runs the controller on the recorded sensory frame, and both must match exactly.

Honest caveat about aim: the fly addresses the ball within ±10° of the target line, so most of a shot's direction comes from the embodiment. See MOTOR_MAPPING.md.

The honest model section header

Scientific caveats

Is this a digital fly? Short answer: no. We have a real fly's wiring diagram. We run a simplified simulation of electricity flowing through it. We choose where golf information goes in and where the stroke comes out.

An analogy: imagine you had a perfect map of every road in a city, including every lane and every intersection. That map is real and hard-won. Now you simulate traffic on it using a few simple rules: cars go, cars stop, cars turn. Traffic really does flow along the real roads, and the map shapes where it goes. But it is not the city. There are no drivers with destinations, no traffic lights with real timing, no weather. And you decide where cars enter the map and which exits you count as "the answer".

Fly Golf is that, for a fly's nervous system:

Layer What we use Status What it means
Wiring: which neuron connects to which, and how many synapses MaleCNS v1.0: 166,700 neurons and 25.6M connections, reconstructed from electron microscopy of one male fly ✅ Real data This is the scientifically valuable part. We keep every connection and prune nothing.
Excitatory or inhibitory Predicted from the neurotransmitter each neuron is likely to use (from the dataset) 🟦 Inferred A good estimate, but some cells are ambiguous; those default to excitatory.
Synapse strength Synapse count × 0.275 mV 🟦 Modelled A real synapse's strength depends on receptors, location and chemistry. We use one number for all.
How a neuron behaves "Leaky integrate-and-fire": charge builds up, the neuron fires, it resets. The same rules for every cell 🟦 Modelled Real neurons are far richer: ion channels, graded signals, neuromodulators such as dopamine, gap junctions. None of that is simulated.
Senses Golf facts ("target is 3° left, 150 m away, ground falls left, water on the line") turned into current injected into visual-target cells (LC10) and gravity-sensing antenna cells (Johnston's organ) 🟧 Engineered The fly does not see the course. We hand its brain a signal, in neurons we chose.
Body and muscles Firing of descending neurons (brain → body) is read out and turned into club, aim, power, tempo and strike 🟧 Engineered Real flies don't swing golf clubs. We chose which neurons count as "the stroke". The 3D swing is a visualisation of that readout.
Where the fly stands The fly is placed roughly facing the target (±10°) 🟧 Engineered Most of a shot's direction comes from this placement, not from neurons. See MOTOR_MAPPING.md.
Trained readout (optional) Linear weights fitted offline on descending-neuron rates from practice shots 🟪 Trained Learning happens outside the connectome. It is not synaptic plasticity: MaleCNS is never changed.
Memory and learning inside the brain None. Brain state resets before every stroke ❌ Not present The connectome cannot improve or remember the last shot.
Physics Deterministic ball flight, bounce and roll: slope, green speed, cup capture, lies, hazards 🟧 Engineered A simplified but calibrated golf model (see PROVENANCE.md and COURSE.md).
The web demo Shot records exported from real runs 📼 Recorded The browser replays what the simulator computed earlier. It simulates nothing.

So what is the connectome contributing? Everything between "current goes into the sensory neurons" and "descending neurons fire" is determined by the real wiring: which of the roughly 10,000–20,000 neurons that activate on each shot light up, how strongly, and in what order. That is the part nobody hand-wrote. It is also why the results are honest and currently unimpressive: with our first choice of inputs and outputs, the real wiring does not yet turn "the hole is close" into "hit it softly", and a shuffled connectome trains a better readout than the real one.

What would make it more fly-like? In rough order of scientific value:

  1. Controls. Lesions of the chosen populations, alongside the shuffled-wiring control that already exists (TRAINING.md).
  2. Real vision. Render the course through modelled fly eyes and drive the photoreceptors, instead of injecting summary numbers.
  3. Richer neurons. Graded signalling in the visual system, and neuromodulation.
  4. Learning. Plasticity rules, labelled experimental and tested against those controls.

Controller reference section header

Controller reference

In the app Controller id Neural simulation Readout
Mock (orange) mock none hand-written golf heuristic
MaleCNS (green) malecns full MaleCNS v1.0, 400 ms per shot fixed, a-priori rules
Trained (purple) malecns-trained the same MaleCNS simulation weights fitted offline from practice

The visual guide above explains what each mode does and why it has its particular body. What's the difference? in the app (key M) opens the same comparison alongside the installed readout's training and benchmark numbers.

If the connectome is not compiled, MaleCNS is disabled and the UI shows the fix command. The app never silently falls back to the mock.

API

GET /health · GET /api/status · GET /api/course · POST /api/session {controller, mode, seed?} · POST /api/controller {controller} (swap the brain mid-round) · POST /api/reset {seed?, hole?} · POST /api/next · POST /api/shot (alias /api/putt) · GET /api/runs · GET /api/runs/{id} · GET /api/runs/{id}/shots/{shot}/replay · WS /ws/simulation, which does a versioned hello (protocol v2) and then streams state, shot_phase, shot_result and controller_status.

Testing

  • Backend: make test runs about 300 tests without the connectome, and the real-data integration tests when it is compiled. They cover physics invariants (rest, friction, stimp calibration, determinism, cup capture, slope break), the course and full-shot physics, controllers (determinism, bounds, malformed input), the runner and replay, the LIF engine and its Brian2 parity, the MaleCNS adapter on a synthetic graph, training, the API and WebSocket, the showcase exporter, and the protocol fixtures.
  • Integration: make test-integration adds the real-connectome tests. They auto-skip without data.
  • Frontend: protocol contract tests, including that the version matches the backend; swing timeline and terrain tests; and showcase tests that validate every committed showcase file and rebuild each round's scorecard from its shots.
  • CI: .github/workflows/ci.yml runs lint, tests, both builds and a real server smoke test. It never downloads the connectome. .github/workflows/pages.yml deploys the web demo.

Roadmap and documentation section header

Roadmap

Done: putting, the full 18-hole course (the front nine, and the back nine held out from training), club selection, full swings, hazards, the trained readout with its controls, and the web demo. The back nine is a transfer test, not a practice ground: the trained fly plays it without ever having practised there. Next:

  1. Controls: the shuffled-wiring and no-brain controls on the back nine and over complete rounds, lesion experiments, and more rounds per brain on shared seeds.
  2. Showcase comparisons: first putt, before and after training, the shuffled-connectome control, and the best round, side by side on the web demo.
  3. Vision: render the course through modelled fly eyes and drive the photoreceptors (see SENSORY_MAPPING.md).
  4. Career: handicap, greens in regulation, putts and dispersion. Meet Gary, Drosophila melanogaster, 166,700 neurons.
  5. Physical world: a HardwareMotorTarget for a robotic putter.

Documentation

Attribution

  • Connectome: MaleCNS v1.0 © the MaleCNS collaboration (FlyEM/HHMI Janelia, Cambridge, MRC LMB, Google Research), CC BY 4.0. Cite https://doi.org/10.1016/j.cell.2026.08.015. No connectome data is stored in this repository; make data fetches it from the official release. The only derived file is the web app's small binned map of brain cell-body positions (apps/web/public/anatomy/, built by services/sim/scripts/build_soma_map.py), which keeps this attribution.
  • Neural model: the whole-brain LIF model of Shiu et al. 2024 (reference implementation MIT). Fly Golf's engine, data compiler and source lock are implemented in this repository.
  • History: early versions of Fly Golf used portions of the open-source DOOMFLY MaleCNS simulation (MIT). Its adapted LIF kernel is kept, unused for new shots, only to replay older records and run readouts trained on it (UPSTREAM.md).
  • Licenses: full notices are in THIRD_PARTY_NOTICES.md. Fly Golf's own code is MIT.

This repository is Fly Golf's public home and the place for new work. The project was incubated in a private repository; that history was not carried over, so this repository starts at the first public release.

About

Neurons of a fly's brain taught to play golf

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages