benchmark-testing

Benchmark Testing

Create isolated test projects that exercise vercel-plugin skill injection with realistic, technology-agnostic prompts.

Workflow

1. Create test directories

BASE=~/dev/vercel-plugin-testing
mkdir -p "$BASE"/{01-slug,02-slug,...}

2. Install the plugin in each directory

for dir in "$BASE"/*/; do
  echo "=== $(basename "$dir") ==="

This creates .claude/settings.json with enabledPlugins in each directory.

3. Launch Claude Code in WezTerm panes

Critical details that must all be followed:

• Use --cwd <absolute-path> to set the working directory
• Use unset CLAUDECODE before x to avoid nested-session errors
• Use --settings .claude/settings.json (not --settings project) to load the plugin
• Use double quotes on the outer -ic string, single quotes around the prompt
• Wait 10 seconds between each launch to avoid overwhelming the system
• Always use spawn (new tabs) — split-pane runs out of space after ~4 panes

Working command template:

Prompt Guidelines

• Never name specific technologies (no "Next.js", "Stripe", "Vercel KV", etc.)
• Describe the product and features — let the plugin infer which skills to inject
• Make prompts ambitious and multi-featured to exercise multiple skill triggers
• Always append: "Link the project to my vercel-labs team so we can deploy it later."

Example prompts

Cleanup