benchmark-isaaclab

Isaac Lab Benchmarking

Parameter references may be outdated. Always verify with ./isaaclab.sh -p <script> --help. For profiling details (Tracy, Nsight), see the profiling skill. For installation, see the install-isaaclab skill.

Setup

See the install-isaaclab skill for installation (clone, conda env, Isaac Sim linking).

Before Running Any Benchmark

1. Use a WARM run for headline FPS/frametime — see the COLD/WARM/TRACY method in the profiling skill
2. Set CPU governor to performance — see perf-tuning skill
3. Do not patch Isaac Sim shutdown by default. If Tracy shutdown hangs after outputs are complete, use the scoped last-resort guidance in the profiling skill

Benchmark Scripts

All in scripts/benchmarks/. Run via ./isaaclab.sh -p scripts/benchmarks/<script>.py.

Note: benchmark_lazy_export.py, benchmark_view_comparison.py, and benchmark_xform_prim_view.py do NOT support --benchmark_backend or --output_path. They output results to stdout. Use --profile (where available) to save cProfile .prof files.

Common params: --device, --enable_cameras, --benchmark_backend, --output_path, --distributed

Note: --headless is deprecated. Omit --viz for headless mode, or use --viz none.

Passing Kit args (for profiling, output control, etc.):

./isaaclab.sh -p scripts/benchmarks/benchmark_non_rl.py \
    --task=Isaac-Ant-Direct-v0 --viz none --num_envs=4096 \
    --kit_args "--/app/profilerBackend=tracy --/log/file=/tmp/kit.log"

Batch suites

Critical Gotcha: Parameter Format

Isaac Lab uses UNDERSCORES (standard argparse). Isaac Sim uses HYPHENS.

Mixing them up is a common source of silent misconfiguration.

Common Tasks and Env Counts

Camera tasks (add --enable_cameras):

• Isaac-Cartpole-RGB-Camera-Direct-v0: 512-4096 envs

Classic physics (4096 / 8192 / 16384 envs):

• Isaac-Ant-Direct-v0, Isaac-Cartpole-Direct-v0, Isaac-Humanoid-Direct-v0

Locomotion (4096 envs):

• Isaac-Velocity-Rough-Anymal-C-v0, Isaac-Velocity-Rough-H1-v0, Isaac-Velocity-Rough-G1-v0

Manipulation (128-8192 envs):

• Isaac-Reach-Franka-v0, Isaac-Factory-GearMesh-Direct-v0

Output Files

• benchmark_<type>_<task>_<timestamp>.json — main results file
• kit.log — execution log (if --/log/file= is set via --kit_args)
• *.tracy / *.nsys-rep — profiling traces (only with profiling args)

JSON structure

Array of phase objects, each with phase_name, measurements (list of {name, data, type, unit}), and metadata (list of {name, data, type}). Phases: benchmark_info, startup, runtime, hardware_info, version_info. RL benchmarks add a train phase.