Skip to content
AI Primer
workflow

AIandDesign shows Claude music-video workflow with Dreamina lip-sync handoffs

AIandDesign’s Claude Music Video skill prepares prompts and audio-only videos for Dreamina lip sync. Returned renders go into a working folder for automated Python-script editing, without MCP connections.

4 min read
AIandDesign shows Claude music-video workflow with Dreamina lip-sync handoffs
AIandDesign shows Claude music-video workflow with Dreamina lip-sync handoffs

TL;DR

  • Dreamina can render the performance clips without an MCP connection: AIandDesign's V3X workflow passes prompts, audio-carrying videos and reference images out, then brings finished clips back into a working folder.
  • Claude handles beat detection, video processing and effects with Python scripts, according to the creator's explanation.
  • Human feedback can change both the video and the tool: AIandDesign's revised demo incorporated notes into the edit and the underlying skill.

A “dark video” carries only audio into Dreamina's lip-sync step. The plugin's README also describes three concept proposals and still-frame previews before the full render.

Dreamina handoff

AIandDesign's V3X performance, made with fAIkout in the Rawkhaus Records universe, uses a pleasingly low-tech handoff between Claude and Dreamina:

  1. The Claude Music Video skill supplies generation prompts and “dark videos,” videos carrying only audio for lip-sync shots.
  2. AIandDesign feeds those into Dreamina with one reference image for the performers and another for the environment.
  3. Dreamina renders the clips outside Claude, without an MCP connection.
  4. AIandDesign places the finished renders in the working directory, where the skill handles the remaining assembly.

Dreamina's Seedance documentation describes combining image, video and audio references. AIandDesign also shared the K-pop video in icreatelife's creator showcase and pointed a follow-up reply back to the thread.

Beat maps and custom shaders

The creator says Claude writes Python scripts for beat detection, video processing and shader effects, using whatever is available in the environment. The skill's instructions spell out two timing cases:

  • Drifting tempo: the analyzer writes a beat map that follows changes across the song, which the other scripts use.
  • Weak beats: a warning calls for cuts on phrases, lyrics and energy changes instead of strict beat timing.

For “DJ FL-AI: The Future Is Now,” AIandDesign rendered the footage in Dreamina beforehand and then briefed the plugin. He says Claude added the typography, audio-reactive effects, abstract-to-photoreal transitions and overlays through custom shaders, none of which were baked into the source clips.

AIandDesign rated that first pass “95% perfect,” requiring one small revision. Alongside another demo, he has also described the skill's editing as better than his own.

Timestamp edits and human feedback

For “The Only Way Is Through,” the source material combined Runway-rendered character imagery, Hedra MCP lip-sync pieces and Archive.org footage. AIandDesign then asked dustinhollywood for specific feedback and used those notes to revise both the video and the plugin.

The follow-up posts describe three controls and results:

Installation and local rendering

The MIT-licensed plugin is available from the author's repository. Official Claude marketplace approval was still pending when AIandDesign published the DJ FL-AI demo, but the README's direct installation commands were already available:

The README documents /music-video-generator:music-video as the plugin command. Its rendering requirements are:

  • Claude Code running on the user's computer.
  • Python 3.10–3.13 and ffmpeg on the PATH.
  • A GPU supporting OpenGL 3.3, used through moderngl.
  • Optional video-generation MCP connections and provider credits for connected generation.

The project was developed and tested on Apple Silicon Macs. Linux and Windows have fallbacks but are untested; the README says the skill can load in Claude web chat or Cowork but cannot render there because those environments lack the required local Python, ffmpeg and GPU setup.

Each project keeps its scripts, data, clips and renders in an mv/ folder. For archive footage, the plugin reads license fields, flags missing licenses and adds source credits to the end card; the README leaves responsibility for footage rights with the user.

Further reading

Discussion across the web

Where this story is being discussed, in original context.

On X· 4 threads
TL;DR1 post
Dreamina handoff2 posts
Beat maps and custom shaders3 posts
Timestamp edits and human feedback5 posts
Share on X