video-generation

Video Generation Skill

Overview

This skill generates high-quality videos using structured prompts and a Python script. The workflow includes creating JSON-formatted prompts and executing video generation with optional reference image.

Core Capabilities

• Create structured JSON prompts for AIGC video generation
• Support reference image as guidance or the first/last frame of the video
• Generate videos through automated Python script execution

Workflow

Step 1: Understand Requirements

When a user requests video generation, identify:

• Subject/content: What should be in the image
• Style preferences: Art style, mood, color palette
• Technical specs: Aspect ratio, composition, lighting
• Reference image: Any image to guide generation
• You don't need to check the folder under /mnt/user-data

Step 2: Create Structured Prompt

Generate a structured JSON file in /mnt/user-data/workspace/ with naming pattern: {descriptive-name}.json

Step 3: Create Reference Image (Optional when image-generation skill is available)

Generate reference image for the video generation.

• If only 1 image is provided, use it as the guided frame of the video

Step 3: Execute Generation

Call the Python script:

python /mnt/skills/public/video-generation/scripts/generate.py \
  --prompt-file /mnt/user-data/workspace/prompt-file.json \
  --reference-images /path/to/ref1.jpg \
  --output-file /mnt/user-data/outputs/generated-video.mp4 \
  --aspect-ratio 16:9

Parameters:

• --prompt-file: Absolute path to JSON prompt file (required)
• --reference-images: Absolute paths to reference image (optional)
• --output-file: Absolute path to output image file (required)
• --aspect-ratio: Aspect ratio of the generated image (optional, default: 16:9)

[!NOTE] Do NOT read the python file, instead just call it with the parameters.

Video Generation Example

User request: "Generate a short video clip depicting the opening scene from "The Chronicles of Narnia: The Lion, the Witch and the Wardrobe"

Step 1: Search for the opening scene of "The Chronicles of Narnia: The Lion, the Witch and the Wardrobe" online

Step 2: Create a JSON prompt file with the following content:

Step 3: Use the image-generation skill to generate the reference image

Load the image-generation skill and generate a single reference image narnia-farewell-scene-01.jpg according to the skill.

Step 4: Use the generate.py script to generate the video

Do NOT read the python file, just call it with the parameters.

Output Handling

After generation:

• Videos are typically saved in /mnt/user-data/outputs/
• Share generated videos (come first) with user as well as generated image if applicable, using present_files tool
• Provide brief description of the generation result
• Offer to iterate if adjustments needed

Notes

• Always use English for prompts regardless of user's language
• JSON format ensures structured, parsable prompts
• Reference image enhance generation quality significantly
• Iterative refinement is normal for optimal results

Providers (Gemini / MiniMax)

Auto-selected by environment variables (CLI unchanged):

• GEMINI_API_KEY set → Gemini Veo (default, unchanged).
• Only MINIMAX_API_KEY set → MiniMax video (/v1/video_generation, async 3-step poll/download).
• Force with VIDEO_GENERATION_PROVIDER=gemini|minimax.

MiniMax overrides: MINIMAX_API_HOST (default https://api.minimaxi.com), MINIMAX_VIDEO_MODEL (default MiniMax-Hailuo-2.3). The first reference image is used as MiniMax first_frame_image. MiniMax ignores --aspect-ratio (it uses resolution/duration).