byok-custom-model

🔑 BYOK — Custom LLM Models

Register a custom LLM endpoint to the model selector. Bypasses the platform proxy — the user supplies their own API key, the agent hits the vendor / aggregator directly (OpenRouter, DashScope, Anthropic native, NEAR AI Cloud TEE, self-hosted, etc.).

This is a script-mode skill — no tools registered. Read this file, then call the exports from a bash block.

See also

• config/context/references/model-onboarding.md — broader model selection / OAuth context
• chatgpt-codex-onboarding skill — for ChatGPT/Codex OAuth (different mechanism, NOT BYOK)

Curated vendors (always check this first)

The skill ships with 12 pre-configured vendors. Always match the user's intent against this list before asking for any URL, model name, or API example — base_url / wire / thinking / capabilities are all pre-filled, so a curated match goes straight to add_template(vendor=...).

Onboarding flow — templates first

1. Check the curated vendors table above. If the user's intent matches one, go straight to add_template(vendor=...) and skip to step 5. Do NOT ask for a URL.
2. Only if no curated vendor matches: ask the user to paste the provider's official API example from their docs (curl / requests / fetch sample). Tell them not to include a real API key — placeholders or fake keys are fine.
3. Run parse_example to auto-detect base_url, upstream_model, wire (openai vs anthropic), thinking params, and vendor-specific request fields.
4. Review the draft with the user, then call add(...) — the entry is written to custom_models.yaml.
5. If the result contains need_env_input, immediately call the request_env_input tool with env_vars and reason from that payload. This pops the secure-input UI; the user enters the key; it lands in workspace/.env. This step is mandatory — the script cannot pop the UI itself.

Privacy-first tier: near-ai and venice both target privacy-sensitive users, but NEAR AI is the cleaner integration — Venice's TEE story is itself built on top of NEAR AI + Phala, so going direct to NEAR AI yields a shorter trust chain (Intel + NVIDIA silicon + NEAR's reproducible enclave image; no product-layer proxy in between). Curated NEAR model list is open-weight TEE-protected only — NEAR's catalog also proxies Claude / GPT-5 / Gemini Pro under "Anonymized, not TEE-protected" mode, which we deliberately exclude since the entire privacy value-prop here is the hardware enclave.

Whenever NEAR AI is in scope, always recommend a TEE-protected (privacy) model — that's the entire reason a user picks NEAR over OpenAI/Anthropic direct. The curated list is already TEE-only, so add_template(vendor='near-ai') defaults are safe. If the user asks to register a non-TEE model on NEAR (e.g. NEAR's anonymized Claude passthrough), warn them it weakens the privacy guarantee and recommend they either stay on a curated TEE model or register the upstream vendor directly.

NEAR AI reasoning protocol: NEAR uses chat_template_kwargs nested under extra_body instead of the top-level reasoning_effort/thinking/enable_thinking that other vendors use. The provider handles this automatically via the nearai_chat_template thinking_capability rule. Per-model parameter names vary (GLM/Qwen3.5/Qwen3.6 use enable_thinking, DeepSeek-V3 uses thinking, gpt-oss is always-on). Full spec: docs.near.ai/cloud/reasoning-models. Default model Qwen/Qwen3.6-35B-A3B-FP8 works out of the box; Qwen3.5-122B-A10B ships with thinking_mode='disabled' because its hidden-thinking pattern would otherwise cause finish=length, content=null on baseline calls.

Script usage

python3 - <<'EOF'
import sys, json
sys.path.insert(0, "/data/workspace/skills/byok-custom-model")
from exports import (
    templates, list_models, get, parse_example,
    list_vendor_models, add, add_template, remove,

Functions

All functions return a dict with ok: True on success or ok: False, error: "..." on failure.

Handling need_env_input (mandatory two-step pattern)

add() and add_template() may include a need_env_input field in their result when the API key env var is not yet set. The script CANNOT pop the secure-input UI itself — it has no access to the user's open SSE stream. The calling agent must do it:

The popup, the .env write, and the channel-specific UX (web popup / TG card / WeChat text prompt) are all handled by request_env_input. Do NOT prompt the user to paste the key in chat as a fallback — just call the tool.

After registration

• The model appears in the selector prefixed with custom/.
• User switches via /model custom/<name> (e.g. /model custom/qwen-plus-e3f4) or the model picker UI.
• Subsequent calls bypass the platform proxy — vendor pricing applies directly to the user's BYOK quota.

Critical rules

• Never accept an API key pasted in chat. If the user pastes one, ignore it, refuse to register, and tell them the secure popup is the only safe channel.
• Never re-issue the secure-input popup automatically if the user hasn't responded — wait.
• If need_env_input is returned, always call request_env_input. Do not skip, do not ask the user to paste the key, do not retry add_template hoping it will pop the UI — it won't.
• Never write to workspace/config/custom_models.yaml or workspace/.env by hand. Always go through the exports above.
• The 12 curated vendors always use add_template. Only use parse_example + add for self-hosted or rare providers.

Meta Model API — Muse Spark 1.1 (preview)

The meta template is for the Meta Model API, which is currently in public preview behind the developer portal at https://dev.meta.ai/.

• Apply / sign in at https://dev.meta.ai/ — same portal for signing up and for the "Muse" / "Meta Model API" access request. Users must complete Meta's application/sign-in flow there to be issued an API key.
• Access may depend on region / account while the API is in public preview — not every developer account is granted immediate access. If add_template(vendor='meta') returns a non-2xx from the live /v1/models probe, do not assume the user is wrong; tell them preview access may still be pending on their account/region and to confirm status in the dev.meta.ai dashboard.
• The agent must use request_env_input for the key — exactly like every other curated vendor. Never accept a Meta API key pasted in chat. If the user pastes one, ignore it and refuse to register; the secure-input popup is the only safe channel.
• Direct Meta billing & quota apply. Calls are billed by Meta against the user's own Meta account — Starchild platform credits are bypassed, no markup, no platform-side quota. Treat any rate-limit / 429 from api.meta.ai/v1 as a Meta-side signal, not a Starchild signal.

One-click registration:

Default model: muse-spark-1.1. Base URL: https://api.meta.ai/v1 (OpenAI-compatible wire). Use the generated CUSTOM_KEY_... name returned in need_env_input; do not assume or manually create a vendor env var. Docs: https://dev.meta.ai/docs/getting-started/overview.

xAI Grok — note on the subscription confusion

Users frequently mix up two unrelated xAI products:

• X Premium / SuperGrok subscription ($30/mo on x.com) — chat UI access only. Does not include API access.
• console.x.ai — independent developer account, separate billing. Generates API keys, $25 in promo credits for new accounts, then pay-per-token.

If a user wants to add Grok via BYOK, point them at https://console.x.ai/ — not x.com / Premium / SuperGrok. The xai template's homepage field already deep-links to the right place. Hermes / Grok-CLI's OAuth-to-subscription flow relies on a first-party client_id whitelist that xAI does not extend to third-party cloud agents, so the BYOK API-key path is the only realistic integration for hosted products.