npx skills add ...
npx skills add gocallum/nextjs16-agent-skills --skill ai-sdk-6-skills
AI SDK 6 Beta overview, agents, tool approval, Groq (Llama), and Vercel AI Gateway. Key breaking changes from v5 and new patterns.
npx skills add gocallum/nextjs16-agent-skills --skill ai-sdk-6-skills
Note: Pin versions during beta as breaking changes may occur in patch releases.
Unified interface for building agents with full control over execution flow, tool loops, and state management.
Request user confirmation before executing sensitive tools.
Client-side approval UI:
Combine tool calling with structured output generation:
Improve search relevance by reordering documents:
Minimal breaking changes expected. Most AI SDK 5 code will work with little modification.
Key differences:
ToolLoopAgent.generateText / streamText (requires stopWhen).@ai-sdk/* packages may have minor API adjustments during beta.Environment:
Popular Groq models for AI SDK 6:
llama-3.3-70b-versatile (Llama 3.3, 70B, balanced)llama-3.1-8b-instant (Llama 3.1, 8B, fast)mixtral-8x7b-32768 (Mixture of Experts)gemma2-9b-it (Google Gemma 2)qwen/qwen3-32b (Qwen 3)See Groq console for full list.
Groq offers reasoning models like qwen/qwen3-32b and deepseek-r1-distill-llama-70b:
A unified interface to access models from 20+ providers (OpenAI, Anthropic, Google, Groq, xAI, Mistral, etc.) through a single API. Requires Vercel account and credit card.
Get your key from Vercel Dashboard > AI Gateway.
⚠️ Note: Credit card required for Gateway usage. You will be billed for model calls routed through the gateway.
Set via environment variable or directly in code:
When deployed to Vercel, use OIDC tokens for automatic authentication (no API key needed):
Production/Preview: Automatic OIDC handling, no setup required.
Local Development:
vercel loginvercel env pullvercel dev to start dev server (handles token refresh automatically)Note: OIDC tokens expire after 12 hours; use vercel dev for automatic refresh, or run vercel env pull again manually.
Connect your own provider credentials to Gateway for private resource access:
Set up BYOK credentials in Vercel team's AI Gateway settings; no code changes needed after configuration.
Some providers offer tools executed server-side (e.g., OpenAI web search). Use through Gateway by importing the provider:
Note: Tools requiring account-specific configuration (e.g., Claude Agent Skills) may need direct provider access via BYOK.
Core Routing Options:
order: Try providers in sequence (fallback priority)only: Restrict to specific providers onlymodels: Fallback to alternative models if primary failsuser: Track usage per end-usertags: Categorize requests for analyticszeroDataRetention: Only use providers with zero data retentionbyok: Request-scoped BYOK credentialsRoute requests only to providers with zero data retention policies for sensitive data:
When zeroDataRetention: true, Gateway only routes to providers that don't retain your data. No enforcement applied if omitted or false.
Dynamically configure agents at runtime:
llama-3.3-70b-versatile for balanced performance and cost.llama-3.1-8b-instant for low-latency, lightweight tasks.parallelToolCalls: true (default) for faster multi-tool execution.serviceTier: 'flex' for 10x rate limits if you can tolerate occasional failures.only / order to control routing and costs.user and tags for spend tracking and debugging.zeroDataRetention for sensitive data.gateway.getCredits() regularly to monitor usage.ToolLoopAgent as a starting point; extend only if needed.stopWhen to control loop iterations (default: stepCountIs(20)).import { tool } from 'ai';
import { z } from 'zod';
const paymentTool = tool({
description: 'Process a payment',
inputSchema: z.object({
amount: z.number(),
recipient: z.string(),
}),
needsApproval: true, // Require approval
execute: async ({ amount, recipient }) => {
return { success: true, id: 'txn-123' };
},
});export function PaymentToolView({ invocation, addToolApprovalResponse }) {
if (invocation.state === 'approval-requested') {
return (
<div>
<p>Process payment of ${invocation.input.amount} to {invocation.input.recipient}?</p>
<button
onClick={() =>
addToolApprovalResponse({
id: invocation.approval.id,
approved: true,
})
}
>
Approve
</button>
<button
onClick={() =>
addToolApprovalResponse({
id: invocation.approval.id,
approved: false,
})
}
>
Deny
</button>
</div>
);
}
return null;
}import { ToolLoopAgent, Output } from 'ai';
import { z } from 'zod';
const agent = new ToolLoopAgent({
model: 'groq/llama-3.3-70b-versatile',
tools: { /* ... */ },
output: Output.object({
schema: z.object({
summary: z.string(),
temperature: z.number(),
recommendation: z.string(),
}),
}),
});
const { output } = await agent.generate({
prompt: 'What is the weather in San Francisco and what should I wear?',
});
console.log(output);
// { summary: '...', temperature: 72, recommendation: '...' }import { rerank } from 'ai';
import { cohere } from '@ai-sdk/cohere';
const { ranking } = await rerank({
model: cohere.reranking('rerank-v3.5'),
documents: [
'sunny day at the beach',
'rainy afternoon in the city',
'snowy night in the mountains',
],
query: 'talk about rain',
topN: 2,
});
console.log(ranking);
// [
// { originalIndex: 1, score: 0.9, document: 'rainy afternoon...' },
// { originalIndex: 0, score: 0.3, document: 'sunny day...' }
// ]pnpm add @ai-sdk/groqGROQ_API_KEY=your_groq_api_keyimport { groq } from '@ai-sdk/groq';
import { generateText } from 'ai';
const { text } = await generateText({
model: groq('llama-3.3-70b-versatile'),
prompt: 'Write a TypeScript function to compute Fibonacci.',
});
console.log(text);import { groq } from '@ai-sdk/groq';
import { generateObject } from 'ai';
import { z } from 'zod';
const result = await generateObject({
model: groq('llama-3.3-70b-versatile'),
schema: z.object({
recipe: z.object({
name: z.string(),
ingredients: z.array(z.string()),
instructions: z.array(z.string()),
}),
}),
prompt: 'Generate a simple pasta recipe.',
providerOptions: {
groq: {
structuredOutputs: true, // Enable for supported models
},
},
});
console.log(JSON.stringify(result.object, null, 2));import { groq } from '@ai-sdk/groq';
import { generateText, tool } from 'ai';
import { z } from 'zod';
const weatherTool = tool({
description: 'Get weather for a city',
inputSchema: z.object({ city: z.string() }),
execute: async ({ city }) => ({ temp: 72, condition: 'sunny' }),
});
const { text } = await generateText({
model: groq('llama-3.3-70b-versatile'),
prompt: 'What is the weather in NYC and LA?',
tools: { weather: weatherTool },
});
console.log(text);import { groq } from '@ai-sdk/groq';
import { generateText } from 'ai';
const { text } = await generateText({
model: groq('qwen/qwen3-32b'),
providerOptions: {
groq: {
reasoningFormat: 'parsed', // 'parsed', 'hidden', or 'raw'
reasoningEffort: 'default', // low, medium, high
},
},
prompt: 'How many "r"s are in the word "strawberry"?',
});
console.log(text);import { groq } from '@ai-sdk/groq';
import { generateText } from 'ai';
const { text } = await generateText({
model: groq('meta-llama/llama-4-scout-17b-16e-instruct'), // Multi-modal model
messages: [
{
role: 'user',
content: [
{ type: 'text', text: 'What is in this image?' },
{ type: 'image', image: 'https://example.com/image.jpg' },
],
},
],
});
console.log(text);AI_GATEWAY_API_KEY=your_gateway_api_keyimport { createGateway } from 'ai';
const gateway = createGateway({
apiKey: process.env.AI_GATEWAY_API_KEY,
});# Start dev with automatic token management
vercel devimport { generateText } from 'ai';
// Plain model string format: creator/model-name
const { text } = await generateText({
model: 'openai/gpt-5',
prompt: 'Explain quantum computing.',
});
console.log(text);import { createGateway } from 'ai';
const gateway = createGateway({
apiKey: process.env.AI_GATEWAY_API_KEY,
});
const { text } = await generateText({
model: gateway('anthropic/claude-sonnet-4'),
prompt: 'Write a haiku about AI.',
});
console.log(text);import { gateway } from 'ai';
const availableModels = await gateway.getAvailableModels();
availableModels.models.forEach((model) => {
console.log(`${model.id}: ${model.name}`);
if (model.pricing) {
console.log(` Input: $${model.pricing.input}/token`);
console.log(` Output: $${model.pricing.output}/token`);
}
});
// Use first model
const { text } = await generateText({
model: availableModels.models[0].id,
prompt: 'Hello world',
});import { gateway } from 'ai';
const credits = await gateway.getCredits();
console.log(`Balance: ${credits.balance} credits`);
console.log(`Total used: ${credits.total_used} credits`);import { streamText } from 'ai';
const { textStream } = await streamText({
model: 'openai/gpt-5',
prompt: 'Explain serverless architecture.',
});
for await (const chunk of textStream) {
process.stdout.write(chunk);
}import { generateText, tool } from 'ai';
import { z } from 'zod';
const weatherTool = tool({
description: 'Get weather',
inputSchema: z.object({ location: z.string() }),
execute: async ({ location }) => `Sunny in ${location}`,
});
const { text } = await generateText({
model: 'xai/grok-4', // Via Gateway
prompt: 'What is the weather in SF?',
tools: { getWeather: weatherTool },
});
console.log(text);import { generateText } from 'ai';
import type { GatewayProviderOptions } from '@ai-sdk/gateway';
const { text } = await generateText({
model: 'anthropic/claude-sonnet-4',
prompt: 'Use my Anthropic account',
providerOptions: {
gateway: {
byok: {
anthropic: [{ apiKey: 'sk-ant-...' }],
},
} satisfies GatewayProviderOptions,
},
});import { generateText, stepCountIs } from 'ai';
import { openai } from '@ai-sdk/openai';
const result = await generateText({
model: 'openai/gpt-5-mini',
prompt: 'What is the Vercel AI Gateway?',
stopWhen: stepCountIs(10),
tools: {
web_search: openai.tools.webSearch({}),
},
});
console.log(result.text);import { generateText } from 'ai';
import type { GatewayProviderOptions } from '@ai-sdk/gateway';
const { text } = await generateText({
model: 'openai/gpt-4o', // Primary model
prompt: 'Write a TypeScript haiku',
providerOptions: {
gateway: {
order: ['vertex', 'anthropic'], // Try Vertex AI first, then Anthropic
only: ['vertex', 'anthropic'], // Only allow these providers
models: ['openai/gpt-5-nano', 'gemini-2.0-flash'], // Fallback models
user: 'user-123',
tags: ['code-gen', 'v2'],
} satisfies GatewayProviderOptions,
},
});
// Fallback sequence:
// 1. Try vertex with openai/gpt-4o
// 2. Try anthropic with openai/gpt-4o
// 3. Try vertex with openai/gpt-5-nano
// 4. Try anthropic with openai/gpt-5-nano
// etc.import { generateText } from 'ai';
import type { GatewayProviderOptions } from '@ai-sdk/gateway';
const { text } = await generateText({
model: 'anthropic/claude-sonnet-4',
prompt: 'Summarize this document...',
providerOptions: {
gateway: {
user: 'user-abc-123', // Track per end-user
tags: ['document-summary', 'premium-feature'],
} satisfies GatewayProviderOptions,
},
});
// View analytics by user and feature in Vercel Dashboardimport { generateText } from 'ai';
import type { GatewayProviderOptions } from '@ai-sdk/gateway';
const { text } = await generateText({
model: 'anthropic/claude-sonnet-4',
prompt: 'Process sensitive document...',
providerOptions: {
gateway: {
zeroDataRetention: true, // Enforce zero data retention
} satisfies GatewayProviderOptions,
},
});import { ToolLoopAgent } from 'ai';
import { z } from 'zod';
const supportAgent = new ToolLoopAgent({
model: 'groq/llama-3.3-70b-versatile',
callOptionsSchema: z.object({
userId: z.string(),
accountType: z.enum(['free', 'pro', 'enterprise']),
}),
instructions: 'You are a support agent.',
prepareCall: ({ options, ...settings }) => ({
...settings,
instructions:
settings.instructions +
`\nUser: ${options.userId}, Account: ${options.accountType}`,
}),
});
const result = await supportAgent.generate({
prompt: 'How do I upgrade?',
options: {
userId: 'user-456',
accountType: 'free',
},
});import { createAgentUIStreamResponse } from 'ai';
import { useChat } from '@ai-sdk/react';
import { InferAgentUIMessage } from 'ai';
// Server-side
export async function POST(request: Request) {
const { messages } = await request.json();
return createAgentUIStreamResponse({
agent: weatherAgent,
messages,
});
}
// Client-side
type AgentMessage = InferAgentUIMessage<typeof weatherAgent>;
const { messages, sendMessage } = useChat<AgentMessage>();const ragAgent = new ToolLoopAgent({
model: 'groq/llama-3.3-70b-versatile',
tools: {
searchDocs: tool({
description: 'Search documentation',
inputSchema: z.object({ query: z.string() }),
execute: async ({ query }) => {
// Call vector DB (Upstash, Pinecone, etc.)
return { docs: [/* ... */] };
},
}),
},
instructions: 'Answer questions by searching docs.',
});const { text } = await generateText({
model: 'anthropic/claude-sonnet-4',
prompt: 'Complex task requiring reasoning',
providerOptions: {
gateway: {
models: ['openai/gpt-5', 'gemini-2.0-flash'],
},
},
});const isSensitive = userQuery.includes('payment');
const model = isSensitive
? 'anthropic/claude-sonnet-4'
: 'openai/gpt-5-nano';
const { text } = await generateText({
model,
prompt: userQuery,
});