video_toolkit skill by calesthio

Video Toolkit

Create professional explainer videos from a text brief. The toolkit uses open-source AI models on cloud GPUs (Modal or RunPod) for voiceover, image generation, music, and talking head animation. Remotion (React) handles composition and rendering.

CRITICAL: Toolkit Path

The toolkit lives at a fixed path. ALWAYS cd here before running any tool command.

TOOLKIT=~/.openclaw/workspace/claude-code-video-toolkit
cd $TOOLKIT

NEVER run tool commands from inside a project directory. Tools resolve paths relative to the toolkit root.

Setup

Step 1: Check Current State

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/verify_setup.py

If everything shows [x], skip to "Quick Test" below. Otherwise continue setup.

Step 2: Install Python Dependencies

cd ~/.openclaw/workspace/claude-code-video-toolkit

Note: --break-system-packages is needed on Debian/Ubuntu with managed Python (PEP 668). Safe inside containers.

Step 3: Configure Cloud GPU Endpoints

The toolkit needs cloud GPU endpoint URLs in .env. Check if .env exists and has Modal endpoints:

If Modal endpoints are configured, you're ready. If not, ask the user to provide Modal endpoint URLs or set up Modal:

LTX-2 prerequisite: Before deploying LTX-2, create a HuggingFace secret and accept the Gemma 3 license:

Add each URL to .env:

Optional but recommended — Cloudflare R2 for reliable file transfer:

Step 4: Verify and Quick Test

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/verify_setup.py

All tools should show [x]. Then run a quick test to confirm the GPU pipeline works:

If you get a valid .mp3 file, setup is complete. If it fails, check:

.env has the correct MODAL_QWEN3_TTS_ENDPOINT_URL
Run python3 tools/verify_setup.py --json and check modal_tools for which endpoints are missing

Cost: Modal includes $30/month free compute. A typical 60s video costs $1-3.

Creating a Video

Step 1: Create Project

Templates: product-demo (marketing/explainer), sprint-review, sprint-review-v2 (composable scenes).

Step 2: Write Config

Edit projects/PROJECT_NAME/src/config/demo-config.ts:

Scene types: title, problem, solution, demo, feature, stats, cta.

Duration rule: Estimate durationSeconds as ceil(word_count / 2.5) + 2. You will adjust this after generating audio in Step 4.

Step 3: Write Voiceover Script

Create projects/PROJECT_NAME/VOICEOVER-SCRIPT.md:

Word budget per scene: (durationSeconds - 2) * 2.5 words. The -2 accounts for 1s audio delay + 1s padding.

Step 4: Generate Assets

CRITICAL: All commands below MUST be run from the toolkit root, not the project directory.

4a. Background Music

Presets: corporate-bg, upbeat-tech, ambient, dramatic, tension, hopeful, cta, lofi.

4b. Voiceover (per-scene)

Generate ONE .mp3 file PER SCENE. Do NOT generate a single voiceover file.

Speakers: Ryan, Aiden, Vivian, Serena, Uncle_Fu, Dylan, Eric, Ono_Anna, Sohee Tones: neutral, warm, professional, excited, calm, serious, storyteller, tutorial

For voice cloning (needs a reference recording):

4c. Scene Images

Image presets (use --preset instead of --prompt --width --height): title-bg, problem, solution, demo-bg, stats-bg, cta, thumbnail, portrait-bg

4d. Video Clips — B-Roll & Animated Backgrounds (optional)

Generate AI video clips for b-roll cutaways, animated slide backgrounds, or intro/outro sequences:

Use in Remotion compositions with <OffthreadVideo>:

LTX-2 rules:

Max ~8 seconds per clip (193 frames at 24fps). Default is ~5s (121 frames).
Width/height must be divisible by 64. Default: 768x512.
~$0.20-0.25 per clip, ~2.5 min generation time.
Cold start ~60-90s. Subsequent clips on warm GPU are faster.
Generated audio is ambient only — use voiceover/music tools for speech and music.
~30% of generations may have training data artifacts (logos/text). Re-run with --seed to vary.

4e. Talking Head Narrator (optional)

Generate a presenter portrait, then animate per-scene clips:

SadTalker rules — follow these exactly:

ALWAYS use --preprocess full (default crop outputs a square, wrong aspect ratio)
ALWAYS use --still (reduces head movement, looks professional)
ALWAYS generate per-scene clips (6-15s each), NEVER one long video
Processing: ~3-4 min per 10s of audio on Modal A10G
--expression-scale 0.8 keeps expressions subtle (range 0.0-1.5)

4e. Image Editing (optional)

Create scene variants from existing images:

4f. Upscaling (optional)

Step 5: Sync Timing

ALWAYS do this after generating voiceover. Audio duration differs from estimates.

Update each scene's durationSeconds in demo-config.ts to: ceil(actual_audio_duration + 2).

Example: if 01.mp3 is 6.8s, set scene 1 durationSeconds to 9 (ceil(6.8 + 2) = 9).

Step 6: Review Still Frames

Check: text truncation, animation timing, narrator PiP positioning, background contrast.

Step 7: Render

Output: out/ProductDemo.mp4

Composition Patterns

Per-Scene Audio

Use per-scene audio with a 1-second delay (from={30} = 30 frames = 1s at 30fps):

Per-Scene Narrator PiP

ALWAYS use <OffthreadVideo>, NEVER <video>. Remotion requires its own component for frame-accurate rendering.

Transitions

NEVER import from lib/transitions barrel — import custom transitions from lib/transitions/presentations/ directly.

Error Recovery

Problem	Solution
Tool command fails with "No module named..."	Run `pip3 install --break-system-packages -r tools/requirements.txt` from toolkit root
"MODAL_*_ENDPOINT_URL not configured"	Check `.env` has the endpoint URL. Run `python3 tools/verify_setup.py`
SadTalker output is square/cropped	You forgot `--preprocess full`. Re-run with that flag
Audio too short/long for scene	Re-run Step 5 (sync timing) and update config
`npm run render` fails	Make sure you're in the project dir, not toolkit root. Run `npm install` first
"Cannot find module" in Remotion	Check import paths. Custom components use `../../../lib/` relative paths
Cold start timeout on Modal	First call after idle takes 30-120s. Retry once — second call uses warm GPU

Tool	Typical Cost	Notes
Qwen3-TTS	~$0.01/scene	~20s per scene on warm GPU
FLUX.2	~$0.01/image	~3s warm, ~30s cold
ACE-Step	~$0.02-0.05	Depends on duration
SadTalker	~$0.05-0.20/scene	~3-4 min per 10s audio
Qwen-Edit	~$0.03-0.15	~8 min cold start (25GB model)
RealESRGAN	~$0.005/image	Very fast
LTX-2.3	~$0.20-0.25/clip	~2.5 min per 5s clip, A100-80GB

Total for a 60s video: ~$1-3 depending on scenes and narrator clips.

Modal Starter plan: $30/month free compute. Apps scale to zero when idle.

Video Toolkit

CRITICAL: Toolkit Path

The toolkit lives at a fixed path. ALWAYS cd here before running any tool command.

TOOLKIT=~/.openclaw/workspace/claude-code-video-toolkit
cd $TOOLKIT

NEVER run tool commands from inside a project directory. Tools resolve paths relative to the toolkit root.

Setup

Step 1: Check Current State

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/verify_setup.py

If everything shows [x], skip to "Quick Test" below. Otherwise continue setup.

Step 2: Install Python Dependencies

cd ~/.openclaw/workspace/claude-code-video-toolkit

Note: --break-system-packages is needed on Debian/Ubuntu with managed Python (PEP 668). Safe inside containers.

Step 3: Configure Cloud GPU Endpoints

The toolkit needs cloud GPU endpoint URLs in .env. Check if .env exists and has Modal endpoints:

If Modal endpoints are configured, you're ready. If not, ask the user to provide Modal endpoint URLs or set up Modal:

LTX-2 prerequisite: Before deploying LTX-2, create a HuggingFace secret and accept the Gemma 3 license:

Add each URL to .env:

Optional but recommended — Cloudflare R2 for reliable file transfer:

Step 4: Verify and Quick Test

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/verify_setup.py

All tools should show [x]. Then run a quick test to confirm the GPU pipeline works:

If you get a valid .mp3 file, setup is complete. If it fails, check:

.env has the correct MODAL_QWEN3_TTS_ENDPOINT_URL
Run python3 tools/verify_setup.py --json and check modal_tools for which endpoints are missing

Cost: Modal includes $30/month free compute. A typical 60s video costs $1-3.

Creating a Video

Step 1: Create Project

Templates: product-demo (marketing/explainer), sprint-review, sprint-review-v2 (composable scenes).

Step 2: Write Config

Edit projects/PROJECT_NAME/src/config/demo-config.ts:

Scene types: title, problem, solution, demo, feature, stats, cta.

Duration rule: Estimate durationSeconds as ceil(word_count / 2.5) + 2. You will adjust this after generating audio in Step 4.

Step 3: Write Voiceover Script

Create projects/PROJECT_NAME/VOICEOVER-SCRIPT.md:

Word budget per scene: (durationSeconds - 2) * 2.5 words. The -2 accounts for 1s audio delay + 1s padding.

Step 4: Generate Assets

CRITICAL: All commands below MUST be run from the toolkit root, not the project directory.

4a. Background Music

Presets: corporate-bg, upbeat-tech, ambient, dramatic, tension, hopeful, cta, lofi.

4b. Voiceover (per-scene)

Generate ONE .mp3 file PER SCENE. Do NOT generate a single voiceover file.

Speakers: Ryan, Aiden, Vivian, Serena, Uncle_Fu, Dylan, Eric, Ono_Anna, Sohee Tones: neutral, warm, professional, excited, calm, serious, storyteller, tutorial

For voice cloning (needs a reference recording):

4c. Scene Images

Image presets (use --preset instead of --prompt --width --height): title-bg, problem, solution, demo-bg, stats-bg, cta, thumbnail, portrait-bg

4d. Video Clips — B-Roll & Animated Backgrounds (optional)

Generate AI video clips for b-roll cutaways, animated slide backgrounds, or intro/outro sequences:

Use in Remotion compositions with <OffthreadVideo>:

LTX-2 rules:

Max ~8 seconds per clip (193 frames at 24fps). Default is ~5s (121 frames).
Width/height must be divisible by 64. Default: 768x512.
~$0.20-0.25 per clip, ~2.5 min generation time.
Cold start ~60-90s. Subsequent clips on warm GPU are faster.
Generated audio is ambient only — use voiceover/music tools for speech and music.
~30% of generations may have training data artifacts (logos/text). Re-run with --seed to vary.

4e. Talking Head Narrator (optional)

Generate a presenter portrait, then animate per-scene clips:

SadTalker rules — follow these exactly:

ALWAYS use --preprocess full (default crop outputs a square, wrong aspect ratio)
ALWAYS use --still (reduces head movement, looks professional)
ALWAYS generate per-scene clips (6-15s each), NEVER one long video
Processing: ~3-4 min per 10s of audio on Modal A10G
--expression-scale 0.8 keeps expressions subtle (range 0.0-1.5)

4e. Image Editing (optional)

Create scene variants from existing images:

4f. Upscaling (optional)

Step 5: Sync Timing

ALWAYS do this after generating voiceover. Audio duration differs from estimates.

Update each scene's durationSeconds in demo-config.ts to: ceil(actual_audio_duration + 2).

Example: if 01.mp3 is 6.8s, set scene 1 durationSeconds to 9 (ceil(6.8 + 2) = 9).

Step 6: Review Still Frames

Check: text truncation, animation timing, narrator PiP positioning, background contrast.

Step 7: Render

Output: out/ProductDemo.mp4

Composition Patterns

Per-Scene Audio

Use per-scene audio with a 1-second delay (from={30} = 30 frames = 1s at 30fps):

Per-Scene Narrator PiP

ALWAYS use <OffthreadVideo>, NEVER <video>. Remotion requires its own component for frame-accurate rendering.

Transitions

NEVER import from lib/transitions barrel — import custom transitions from lib/transitions/presentations/ directly.

Error Recovery

Problem	Solution
Tool command fails with "No module named..."	Run `pip3 install --break-system-packages -r tools/requirements.txt` from toolkit root
"MODAL_*_ENDPOINT_URL not configured"	Check `.env` has the endpoint URL. Run `python3 tools/verify_setup.py`
SadTalker output is square/cropped	You forgot `--preprocess full`. Re-run with that flag
Audio too short/long for scene	Re-run Step 5 (sync timing) and update config
`npm run render` fails	Make sure you're in the project dir, not toolkit root. Run `npm install` first
"Cannot find module" in Remotion	Check import paths. Custom components use `../../../lib/` relative paths
Cold start timeout on Modal	First call after idle takes 30-120s. Retry once — second call uses warm GPU

Tool	Typical Cost	Notes
Qwen3-TTS	~$0.01/scene	~20s per scene on warm GPU
FLUX.2	~$0.01/image	~3s warm, ~30s cold
ACE-Step	~$0.02-0.05	Depends on duration
SadTalker	~$0.05-0.20/scene	~3-4 min per 10s audio
Qwen-Edit	~$0.03-0.15	~8 min cold start (25GB model)
RealESRGAN	~$0.005/image	Very fast
LTX-2.3	~$0.20-0.25/clip	~2.5 min per 5s clip, A100-80GB

Total for a 60s video: ~$1-3 depending on scenes and narrator clips.

Modal Starter plan: $30/month free compute. Apps scale to zero when idle.

video_toolkit

Documentation

Documentation

Video Toolkit

CRITICAL: Toolkit Path

Setup

Step 1: Check Current State

Step 2: Install Python Dependencies

Step 3: Configure Cloud GPU Endpoints

Step 4: Verify and Quick Test

Creating a Video

Step 1: Create Project

Step 2: Write Config

Step 3: Write Voiceover Script

Step 4: Generate Assets

4a. Background Music

4b. Voiceover (per-scene)

4c. Scene Images

4d. Video Clips — B-Roll & Animated Backgrounds (optional)

4e. Talking Head Narrator (optional)

4e. Image Editing (optional)

4f. Upscaling (optional)

Step 5: Sync Timing

Step 6: Review Still Frames

Step 7: Render

Composition Patterns

Per-Scene Audio

Per-Scene Narrator PiP

Transitions

Error Recovery

Cost Estimates (Modal)

Documentation

video_toolkit

Documentation

Video Toolkit

CRITICAL: Toolkit Path

Setup

Step 1: Check Current State

Step 2: Install Python Dependencies

Step 3: Configure Cloud GPU Endpoints

Step 4: Verify and Quick Test

Creating a Video

Step 1: Create Project

Step 2: Write Config

Step 3: Write Voiceover Script

Step 4: Generate Assets

4a. Background Music

4b. Voiceover (per-scene)

4c. Scene Images

4d. Video Clips — B-Roll & Animated Backgrounds (optional)

4e. Talking Head Narrator (optional)

4e. Image Editing (optional)

4f. Upscaling (optional)

Step 5: Sync Timing

Step 6: Review Still Frames

Step 7: Render

Composition Patterns

Per-Scene Audio

Per-Scene Narrator PiP

Transitions

Error Recovery

Cost Estimates (Modal)