Clawd video build plan

Project: opus5.5_cartoon_bear · Plan date: 2026-09-24 · Status: plan only. No video is made yet.

This document explains how a text prompt and a storyboard become a 15-second MP4. No person draws, and no animation program or video editor is used. The model writes code. The code draws 360 still images. FFmpeg joins the images into a video. The model looks at the images and corrects the code. The main goal of this document is to explain each part of that process. The video is the test case.

1. The process in one page

Most people think "AI video" means a model that generates video pixels directly, like Sora or Veo. This project does not use that method. Here, a language model writes a program. The program is a set of drawing instructions: "paint a beige wall", "put an orange block at x = 1250", "move it up 40 pixels at second 2.1". A browser runs the program and draws the pictures. The model never generates pixels itself.

The work has 5 parts:

1. Write. The model reads the storyboard and writes about 6 source files: the scene, the character, the timeline, the renderer, and the sound.
2. Draw. A copy of Chrome with no window ("headless") opens the scene. A Node script asks it for the picture at time 0.000 s, then at 0.042 s, then at 0.083 s, and so on. That gives 15 × 24 = 360 pictures.
3. Compose sound. A second Node script calculates the soundtrack as a list of numbers and saves it as a WAV file.
4. Encode. FFmpeg joins the 360 pictures and the WAV file into one MP4 file.
5. Check. The model opens sample frames as images, finds errors, and changes the code. Then steps 2 to 4 run again. This loop is where most of the time goes.

2. The tools: what each one is and how the build uses it

I checked each version below on your machine during preflight on 2026-09-24.

Not used: Pixabay (its API has no music or sound endpoint), Higgsfield (your README says "no higgs"), the hyperframes skill (you chose p5.js), any AI image or video generator, and any video editor.

3. What p5.js is, and what p5.brush adds

p5.js

p5.js is a free JavaScript library for "creative coding". It is the web version of Processing, a tool that artists and teachers have used since 2001. p5.js makes drawing in a browser easy. You do not manage the low-level canvas. You write short commands:

// runs one time at the start
function setup() {
  createCanvas(1920, 1080);
}
// runs again and again, about 60 times each second
function draw() {
  background('#f3e6cf');           // clear to paper colour
  fill('#D77757');                 // Clawd orange
  rect(900 + frameCount, 500, 200, 140);  // a block that moves right
}

Two ideas are important:

• setup() and draw(). setup() runs one time. draw() runs for each frame. Animation is only this: draw the picture again with slightly different numbers.
• Renderers. p5 can draw with the normal 2D canvas, or with WebGL. WebGL uses the graphics card (GPU). p5.brush needs WebGL.

This build changes one thing in the normal p5 method. It does not use the automatic draw() loop to make the video, because that loop runs on the real clock. The renderer calls renderFrame(i) itself, one frame at a time. Section 4 gives the reason.

p5.brush

Plain p5 draws clean computer shapes: exact edges and flat colour. p5.brush makes the shapes look hand-painted. It simulates pigment that bleeds at the edges, collects in darker rings, and shows paper texture. It also has pen, pencil, charcoal and marker lines that wobble like a real hand. The main commands:

brush.load();                      // start p5.brush on a WEBGL canvas
brush.fill('#e8a86b', 140);        // watercolour pigment, strength 140 of 255
brush.fillBleed(0.2);              // how far the colour spreads at the edge
brush.fillTexture(0.5, 0.35);      // paper grain inside, darker ring at the border
brush.rect(200, 200, 900, 500);    // the painted shape
brush.set('pen', '#3a2a24', 1.2);  // ink pen for outlines
brush.wash('#D77757', 255);        // flat opaque colour, no texture

4. How a video is made from frames

A video is a list of pictures

The output is 24 frames each second. 15 s × 24 = 360 frames, numbered 0 to 359. Frame i shows the moment t = i / 24 seconds. One frame lasts 1/24 s = 41.7 ms. Frame 72 is t = 3.000 s, the start of beat 2.

Each frame is a function of time only

This is the most important rule in the code. The picture at time t is calculated from t alone. It never depends on the previous frame. Example: Clawd's jump is not "move 5 px higher than the last frame". It is "at time t, height = 80 × sin(π × (t − 2.10) / 0.35)".

Why this matters:

• Speed does not matter. A frame can take 50 ms or 2 s to draw. The video is still correct, because the video clock and the real clock are separate.
• Any frame can be drawn alone. http://localhost:8765/?t=8.2 shows exactly the frame at 8.2 s. The model can check one moment with no need to play the video.
• The result repeats. The same code gives the same 360 frames each time. A fix changes only what it should change.

Random items, for example confetti, smoke puffs and paper grain, use a seeded random generator. p5.brush has its own seed command, brush.seed(n). I found it in the installed 2.2.3 code, so the watercolor also comes out the same in each render. A seed is a start number. The same seed always gives the same "random" sequence. A particle's position is calculated from its birth time, speed and gravity: y = y0 + v·age + ½·g·age². No state carries from frame to frame.

Why not record the screen?

A screen recorder captures what the browser shows on the real clock. If one frame takes 200 ms to draw, the recorder misses frames, and the motion stutters. Frame-by-frame capture has no such problem.

The capture loop

// render.mjs (outline)
start local server on port 8765
launch Chrome (headless, GPU on, window 1920×1080, deviceScaleFactor 1)
open http://localhost:8765/?render
wait until the page says "assets painted"          // the one-time p5.brush painting
for i = 0 … 359:
    page.evaluate(renderFrame(i))                  // draw frame i on the canvas
    screenshot the canvas → out/frames/f_0000.png … f_0359.png
close Chrome
run FFmpeg: PNGs → out/video.mp4
run score.mjs → out/score.wav
run FFmpeg: video.mp4 + score.wav → out/clawd-prose.mp4
run ffprobe: print frames, fps, size, duration

The FFmpeg commands

# 1. pictures to video
ffmpeg -framerate 24 -i out/frames/f_%04d.png \
  -c:v libx264 -preset slow -crf 16 -pix_fmt yuv420p \
  -vf "scale=out_color_matrix=bt709:out_range=tv" \
  -colorspace bt709 -color_primaries bt709 -color_trc bt709 \
  -movflags +faststart out/video.mp4

# 2. add the soundtrack; cut both to exactly 15 s
ffmpeg -i out/video.mp4 -i out/score.wav -c:v copy -c:a aac -b:a 192k -t 15 out/clawd-prose.mp4

# 3. measure the result
ffprobe -v error -count_frames -show_entries stream=codec_name,width,height,r_frame_rate,nb_read_frames,duration -of compact out/clawd-prose.mp4

What the options do, in plain words:

• libx264: the H.264 codec. Almost every player and website can play it.
• -crf 16: the quality level. A lower number gives higher quality and a larger file. 16 keeps the watercolor texture and small text sharp.
• -pix_fmt yuv420p: the colour format that common players need. It keeps colour at half resolution, so thin red text can get soft edges. The check step looks for this.
• The bt709 options tag the colours correctly. Without them, some players show Clawd's orange slightly wrong.
• +faststart: moves the file index to the start, so a browser can start to play the file before the download is complete.

5. What goes into one frame

Two canvases

1. The paint canvas (hidden, WebGL, p5 + p5.brush). At the start, it paints each item one time: paper, wall, desk, laptop, machine parts, vacuum, system card, buttons, avatar. After each item, the code copies the result into its own small 2D canvas. These copies are the sprites.
2. The stage canvas (visible, 1920×1080, Canvas 2D). For each frame, it clears and draws the sprites and text in order, with the positions, angles and sizes for time t. This is fast: copying an image takes about a millisecond (estimate, not measured).

The layer stack, back to front

Each frame draws these layers in this order. A later layer covers an earlier one, like cels in a hand-drawn cartoon.

Example: frame 132 (t = 5.500 s)

This is the list of drawing steps for one real frame in beat 2. Every other frame uses the same code with a different t.

1. Clear the stage. Apply the camera: the beam landed at 4.6 s, so the shake is almost finished. Offset = 2 px.
2. Draw the room sprite at (0, 0).
3. Draw the prose machine. It shakes: x offset = 6 × sin(2π × 11 × t) px. The chimney puffs 3 smoke sprites. Each puff grows and fades with its own age.
4. Draw the text ribbon. It comes out of the machine along a wave path. At 5.5 s, about 60% of the sentence is out. The code places each letter on the path one by one, because the 2D canvas cannot draw text along a curve.
5. Draw the LOAD-BEARING beam. It landed at 4.6 s and now rests across the pile.
6. Draw the SEAMS stitch line. It started at 5.0 s. At 5.5 s it is about 80% drawn.
7. Draw the WEDGE. The hammer hit at 5.3 s. The wedge is now between two lines of text. The two lines are pushed 12 px apart.
8. Draw two em dashes. Each flies on an ellipse path like a boomerang and spins.
9. Draw Clawd, typing at double speed: arms alternate each 2 frames, eyes are happy chevrons (^ ^), body bobs 4 px.
10. Draw the paper finish over everything.

How text is drawn

Most of the jokes are text, so text needs care:

• The stage uses ctx.font = "48px Georgia" and ctx.fillText(...). Georgia gives the formal Victorian prose look. The laptop screen uses a monospace font (Consolas).
• The 2D canvas does not wrap long text. The code measures each word with ctx.measureText and breaks the lines itself. The "gigantic paragraph" needs this.
• The CONCISE gag: the compressed paragraph uses the same string, laid out in a box 40% as wide, at 40% of the font size. It really does contain the same amount of prose.
• Typing: the number of visible letters = the time since typing started × letters each second. Deleting uses the same formula in reverse.

6. How Clawd is built and animated

The reference

Clawd is the small orange mascot in the Claude Code terminal. The terminal draws it with block characters:

 ▐▛███▜▌
▝▜█████▛▘
  ▘▘ ▝▝

Preflight found these facts in your installed ~/.local/bin/claude.exe:

• The body colour token: clawd_body: "rgb(215,119,87)" (4 matches). In hex, this is #D77757.
• The block pieces █████, ▘▘ ▝▝ and ▝▝ ▝▝, stored as UTF-16 at byte 99,320,476. The top row is stored in pieces, so the full shape above comes from the reference repo's decode. The local binary confirms the colour and the pieces.

From characters to a pixel grid

Each block character is one terminal cell. Each cell has 2 × 2 "quarter" squares that can be on or off. For example, ▛ has 3 quarters on and the lower-right quarter off. A terminal cell is about twice as tall as it is wide, so each quarter is a tall rectangle. When I decode the 3 lines above, the result is a grid 18 quarters wide and 6 quarters tall:

What the grid shows:

• Body: 12 quarters wide (columns 3 to 14) and 4 quarters tall.
• Eyes: two holes in row 1, at columns 5 and 12. They are black because the terminal background shows through. Each eye is 1 quarter: tall and thin.
• Arms: small nubs, 2 quarters wide on each side, in row 2.
• Legs: 4 legs, 1 quarter each, in row 4, at columns 4, 6, 11 and 13.
• No mouth, no fingers, no neck.
• Rows and columns count from 0, the same as the arrays in the code.

To verify before the build: the decode of the 3 lines is correct. But my search of your local binary did not find the head row ▐▛███▜▌ as one string. That row comes from the reference repo. Before phase 3, I will compare it with the logo that Claude Code shows in your terminal.

Size in the video: 1 quarter = 22 × 44 px. Clawd is then 264 px wide (body) and 220 px tall (with legs). That is about 20% of the frame height. The body is a fixed size, so it looks the same in all shots.

Drawing the pixel body in a watercolor world

A flat, sharp orange block looks strange in a soft painted room. The plan: fill the pixel shape with an orange p5.brush texture (painted one time, in Clawd's exact colour), keep the edges sharp, and add a soft painted shadow on the desk. The paper finish (layer 8) goes over Clawd too. Clawd stays a pixel character, and the texture joins it to the room.

How a pose works

The function pose(t) returns a few numbers for each moment: position x and y, squash, lean angle, arm angles, eye mode, eye offset. The drawing code turns these numbers into rectangles. The origin is at Clawd's feet, so squash keeps the feet on the desk.

function pose(t) {
  const p = { x: 1250, y: 760, sx: 1, sy: 1, lean: 0, armL: 0, armR: 0,
              eye: 'normal', eyeDx: -6, sweat: 0 };       // default: looks at laptop
  if (t >= 1.20 && t < 1.60) {                          // horrified freeze
    const k = easeOutBack(clamp01((t - 1.20) / 0.12));
    p.sy = 1 + 0.15 * k;  p.sx = 1 / Math.sqrt(p.sy);   // stretch up, keep volume
    p.eye = 'wide';  p.sweat = k;
  }
  // … one block like this for each action in the shot list
  return p;
}

The animation principles, as numbers

Acting with no face

Clawd has no mouth and no hands. The storyboard asks for "horrified", "cracks his fingers", "straightens an imaginary bow tie" and "raises one finger". The plan:

• Eyes do the acting. Eye modes: normal, wide (1.4× tall), squint (focus), happy chevrons (^ ^), shake (panic), stare (no blink, then one slow blink).
• Arm nubs do the gestures. For the finger crack, both nubs push forward, with 3 small impact stars and a crack sound. For the bow tie, both nubs move to the lower centre of the body and twitch, and a dotted ink bow tie shows for 0.3 s. It is imaginary, so it is drawn as a ghost. For "one finger", one nub grows 1 quarter taller.
• Extras: sweat drop, "!" mark, shake lines, a tilt of the whole body.
• Turning. A flat pixel sprite cannot turn in 3D. "Looks at the laptop" means both eyes shift left. "Turns to camera" means the eyes return to the centre, with a small hop.

7. The shot list: second by second

Your storyboard gives 5 beats and their times. Those times do not change. Inside each beat, I set the time of each action. Beat 2 had about 11 actions in 4 s (about 0.36 s each), so the small gags overlap there, as you decided.

Staging. The laptop is left of centre, and its screen faces the camera at a 3/4 angle, so you can read it. This is a normal cartoon cheat. Clawd sits right of the laptop and faces it. The user avatar floats at the upper left with its chat bubbles. The vacuum enters from the right, so it pulls Clawd out on the right side.

Beat 1: The innocent question (0.0–3.0 s, frames 0–71)

Beat 2: The Mannered Prose Machine awakens (3.0–7.0 s, frames 72–167)

Beat 3: Concision makes everything worse (7.0–10.0 s, frames 168–239)

Beat 4: Anthropic intervenes (10.0–13.0 s, frames 240–311)

Beat 5: Recovery… almost (13.0–15.0 s, frames 312–359)

8. How the music and sound effects are made in code

Sound is a list of numbers

A speaker cone moves in and out. Digital audio records its position many times each second. This build uses 48,000 positions each second (48 kHz). Each position is a number from −1 to +1. 15 s × 48,000 = 720,000 numbers. One video frame = 48,000 / 24 = 2,000 samples exactly, so frame i starts at sample i × 2,000. That exact match makes sync simple.

Node has no browser audio engine, so score.mjs does all the maths itself. It needs about 150 lines and no library (estimate).

The building blocks

const SR = 48000, N = SR * 15;
const mix = new Float32Array(N);                  // 720,000 samples of silence
const hz = (m) => 440 * 2 ** ((m - 69) / 12);

function note(t0, dur, midi, amp, wave = 'sine') {
  const s0 = Math.round(t0 * SR), n = Math.round(dur * SR);
  for (let i = 0; i < n; i++) {
    const t = i / SR, ph = (hz(midi) * t) % 1;
    const w = wave === 'saw' ? 2 * ph - 1 : Math.sin(2 * Math.PI * ph);
    const env = Math.min(1, t / 0.005) * Math.exp(-t * 6);   // 5 ms attack, decay
    mix[s0 + i] += amp * w * env;
  }
}
note(0.0, 0.4, 72, 0.3);   // C5 on the music box at t = 0.0 s

The musical grid

The tempo is 120 beats each minute. One beat = 0.5 s = 12 frames = 24,000 samples. The storyboard beats start at 0, 3, 7, 10 and 13 s. Those are musical beats 0, 6, 14, 20 and 26. Every scene change falls exactly on a beat, so the music and the pictures change together.

The score, section by section

Sync

The cue times live in one file, cues.json. The browser scene reads it, and score.mjs reads it. If the stamp moves from 8.60 s to 8.70 s, the picture and the thunk move together. Sound can start on any sample, but the picture changes only every 41.7 ms. So each hit sound starts on the first sample of its impact frame.

9. How the model checks its own work

The model can read images, but it cannot watch motion and it cannot hear. So each check turns the video into things it can read: still images and numbers.

Rendering time is not known yet. The preflight frame took 1777 ms, but that includes the page load and a small paint. I will measure the time for one frame in phase 1 and give you the total before the full render.

10. The build, step by step

File layout. All files go in a new subfolder, clawd-prose/:

clawd-prose/
  package.json       p5, p5.brush, puppeteer-core; scripts: render, stills, audio
  index.html         loads p5, p5.brush, then the scene files
  cues.json          every cue time, shared by picture and sound
  scene.js           painting (one time), layer stack, text, effects, renderFrame(i)
  clawd.js           the pixel grid, pose(t), eyes, arms
  render.mjs         server + Chrome + frames + FFmpeg + ffprobe
  score.mjs          synthesizer + score → out/score.wav
  README.md          how to run it
  out/               frames/, video.mp4, score.wav, clawd-prose.mp4, sheet.png

1. Phase 1: pipeline proof. Make the folder. Run npm install. Write index.html, a renderFrame(i) that only prints the frame number, and render.mjs. Render all 360 frames and encode them.
   Checkpoint: ffprobe shows 360 frames, 24 fps, 15.000 s. I measure the time for one frame.
2. Phase 2: paint the world. Write the p5.brush painting for the room, desk, laptop, machine parts, PROSE switch, CONCISE button, stamp, system card, vacuum, avatar, glasses and magnifier. Each one is painted one time with a fixed seed.
   Checkpoint: one "asset sheet" PNG with all items. I send it to you.
3. Phase 3: build Clawd. Write the grid, the textured fill, eye modes and arm nubs. Compare the grid with the terminal logo.
   Checkpoint: a "pose sheet" PNG with 8 poses: neutral, horrified, delighted typing, proud, panic, stare, astonished, pulled by the vacuum. I send it to you.
4. Phase 4: animate the 5 beats. Write cues.json and the timeline code, one beat at a time. After each beat, render and read 2 or 3 stills.
   Checkpoint: first silent draft MP4 and contact sheet.
5. Phase 5: sound. Write score.mjs: instruments, sound effects, score, mix, WAV.
   Checkpoint: you listen to score.wav. I also check the waveform picture and the levels.
6. Phase 6: full render. Render 360 frames, encode, add audio, measure with ffprobe.
7. Phase 7: check and fix. Contact sheet, key stills, motion check, audio check. Fix the errors and render again. Expect several passes.
8. Phase 8: deliver. The final MP4, the source, and a README with the 3 commands to run it. Delete out/frames/ (360 PNGs, about 0.3 MB to 3 MB each, so up to about 1 GB).

11. Unknown unknowns: what a newcomer does not know to ask

Each item below is a question that a person new to this process usually does not know to ask. Each answer says how this plan handles it.

About the process

• "Does the AI generate the video?" No. It generates code. Every pixel comes from a drawing command that you can read and change. If Clawd is 10 px too far left, the fix is one number.
• "Can the model watch the video?" Not in motion. It reads single frames and contact sheets as images, and motion as numbers. You must judge timing and smoothness.
• "Can the model hear the music?" No. It checks audio only as pictures (waveform) and numbers (levels, cue times). The first real listen is yours. Plan for at least one round of sound changes after you listen.
• "Is the preview in the browser the real video?" Not exactly. The browser preview plays on the real clock, so it can stutter, and it is silent. The MP4 is the real result.
• "Is the first render the final one?" Almost never. Most of the time goes into check and fix passes.

About the pictures

• Watercolor "boils" if you paint it again in each frame. p5.brush is random. Paint one time, keep the image, use a fixed seed. (Section 3.)
• p5.brush pigment is not transparent. According to the reference repo notes, pigment mixes with the paper colour, so items are painted on paper and then cut out along their outline. I will confirm this in phase 2.
• WebGL text is difficult. In p5's WebGL mode, text needs a font file loaded first. The plan draws all text on the 2D stage canvas, where system fonts work.
• Fonts come from the machine that renders. Georgia and Consolas are on your Windows machine. On another computer, the text can look different. The code waits for document.fonts.ready before frame 0, so the first frames do not use a fallback font.
• The canvas does not wrap text, and it cannot draw text along a curve. The code measures words and places letters itself.
• Headless Chrome can use software drawing instead of the GPU. Then WebGL is very slow. The preflight confirmed the GPU is in use.
• Screen scaling can double the screenshot size. Windows display scaling can make Chrome capture 3840×2160. The renderer sets deviceScaleFactor: 1.
• A flat pixel sprite cannot turn around. The eyes show the direction instead. (Section 6.)
• At 24 fps, fast motion looks like jumps. Real cameras add motion blur. Cartoons use smears and speed lines, and so does this plan.
• The storyboard is very full. About 15 separate text items and more than 30 actions in 15 s. Some text will pass faster than a person can read. (Section 7.)
• "No captions" and a text-heavy story. The first prompt in your README said "no captions or dialogue". This storyboard's text is part of the scene: on the laptop, on cards and on objects. That is not the same as captions, so the two do not conflict.

About the video file

• "Exactly 15 seconds" means 360 frames and 720,000 audio samples. If the audio is longer, some players show a black tail. The mux command cuts both to 15 s, and ffprobe confirms it.
• The colour format must be yuv420p, with even width and height. Otherwise the MP4 does not play in QuickTime, in some browsers, or on phones. 1920×1080 is even.
• Colour tags. Without the bt709 tags, some players show the orange slightly different. The encode command sets them.
• Temporary disk use. 360 PNG files at 1920×1080 can use up to about 1 GB. Your C: drive has 279 GB free.
• Sync tolerance. People notice sound that comes before the picture more easily than sound that comes late. The ITU-R BT.1359 figures are about +45 ms early and −125 ms late. I quote these from memory and did not check them. So each hit sound starts on its impact frame, not before it.

About Windows and the tools

• FFmpeg is installed but not on the Bash PATH. The scripts use the full path.
• Chrome blocks file:// loads for JSON and fonts. The local server solves this.
• Port 8765 can be in use. The script stops with a clear message if it is. It never stops other programs.
• The project path has a dot in it (opus5.5_cartoon_bear). The scripts use quoted absolute paths.

12. Decisions made, and open risks

Your decisions (2026-09-24)

• Audio: all-code synth, music and sound effects.
• Length: exactly 15 s. The small beat 2 gags overlap.
• Engine: written from scratch. The reference repo is reference only.

My defaults (change them if you want)

• Output: 1920×1080, 24 fps, H.264 + AAC, MP4.
• Fonts: Georgia for prose, Consolas for the laptop.
• Folder: opus5.5_cartoon_bear/clawd-prose/.
• Audio mono, 48 kHz.
• The user avatar is a small round face with dot eyes. It is not a real person.

Open risks