Skip to content
AI Primer
release

Fly-connectome controller traces drawings, text, and emojis with a simulated foreleg

A published fly-connectome controller moves a simulated foreleg to trace simple drawings, text, and emojis. The project includes a repository, while a related demo maps 20 simulated flies' paths into a guitar outline.

3 min read
Fly-connectome controller traces drawings, text, and emojis with a simulated foreleg
Fly-connectome controller traces drawings, text, and emojis with a simulated foreleg

TL;DR

The live Designer Fly build exposes the drawing interface and neural displays. The MaleCNS portal publishes the underlying male fly central-nervous-system connectivity, and Janelia's announcement describes it as a map of neurons and their connections.

The supplied-path workflow

Designer Fly takes UI shapes, text, emojis, and freehand marks. Its README says the editor supplies the pen geometry, then the controller moves the foreleg along those paths.

Its four input modes stay deliberately bounded:

  • Draw UI: rectangles, squares, circles, ellipses, triangles, and up to 128 pen strokes.
  • Text: up to 40 characters, converted from supplied vector lettering.
  • Emoji: 12 minimal, pre-drawn stroke designs.
  • Pen: mouse, stylus, or finger strokes, simplified and capped at 256 points each.

The repository routes every one of those inputs through the same motor and contact-based physics loop. A 1,024-neuron path network handles selected shape families; layout transforms and the travel, lowering, drawing, and lifting phases are conventional application logic.

The 2,048-neuron motor

The active motor reads the intended point, joint angles and velocities, tip error, and contact force, the implementation notes say. It produces three incremental rotary-actuator commands.

MuJoCo integrates the reduced foreleg and its attached stylus after each step. Ink appears only when the tip reaches the paper during a drawing phase, and the next body state becomes the controller's feedback.

The repository's reported check covered 34 strokes across eight layouts. Its worst stated tracking error was 0.01529 model millimeters against a 0.035 threshold, with more than 95% drawing contact per stroke.

One fixed foreleg

MaleCNS is connectivity data for a male fly's brain and ventral nerve cord, not a full behavioral model. Designer Fly starts from a selected MaleCNS v1.0 ventral-nerve-cord subgraph, then learns edge gains, neuron biases, and leaks, according to the source repository.

The project's own scope is specific:

  • The body remains fixed and one three-joint foreleg is physically controlled.
  • The stylus is attached, rather than grasped.
  • Other legs are posed visually.
  • Biological transmitter predictions and calibrated neuron dynamics are outside the simulation.

That split leaves measured wiring as a design constraint while the task sequencing, sensory interfaces, and physical scene remain engineered components.

Guitar paths

GlennHasABeard's separate experiment pairs real MaleCNS wiring with self-built dynamics, eyes, and joystick control. A moving light serves as the cue: wherever the simulated fly walks, it lays a block, gradually tracing a guitar.

The follow-up moves from a continuously guided route to 19 light anchors. It assigns 20 simulated flies to the outline, leaving each curve between anchors to the fly's own path.

Further reading

Discussion across the web

Where this story is being discussed, in original context.

On X· 2 threads
TL;DR1 post
The supplied-path workflow1 post
Share on X