langsmith-trace

LANGSMITH_API_KEY=lsv2_pt_your_api_key_here          # Required
LANGSMITH_PROJECT=your-project-name                   # Optional: default project
LANGSMITH_WORKSPACE_ID=your-workspace-id              # Optional: for org-scoped keys

IMPORTANT: Always check the environment variables or .env file for LANGSMITH_PROJECT before querying or interacting with LangSmith. This tells you which project contains the relevant traces and data. If the LangSmith project is not available, use your best judgement to identify the right one.

CLI Tool

curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh

<trace_langchain_oss> For LangChain/LangGraph apps, tracing is automatic. Just set environment variables:

Optional variables:

• LANGSMITH_PROJECT - specify project name (defaults to "default")
• LANGCHAIN_CALLBACKS_BACKGROUND=false - use for serverless to ensure traces complete before function exit (Python) </trace_langchain_oss>

<trace_other_frameworks> For non-LangChain apps, if the framework has native OpenTelemetry support, use LangSmith's OpenTelemetry integration.

If the app is NOT using a framework, or using one without automatic OTel support, use the traceable decorator/wrapper and wrap your LLM client.

client = wrap_openai(OpenAI())

@traceable def my_llm_pipeline(question: str) -> str: resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": question}], ) return resp.choices[0].message.content

Nested tracing example

@traceable def rag_pipeline(question: str) -> str: docs = retrieve_docs(question) return generate_answer(question, docs)

@traceable(name="retrieve_docs") def retrieve_docs(query: str) -> list[str]: return docs

@traceable(name="generate_answer") def generate_answer(question: str, docs: list[str]) -> str: return client.chat.completions.create(...)

</python>
 
<typescript>
Use traceable() wrapper and wrapOpenAI() for automatic tracing.
```typescript
import { traceable } from "langsmith/traceable";
import { wrapOpenAI } from "langsmith/wrappers";
import OpenAI from "openai";
 
const client = wrapOpenAI(new OpenAI());
 
const myLlmPipeline = traceable(async (question: string): Promise<string> => {
  const resp = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: question }],
  });
  return resp.choices[0].message.content || "";
}, { name: "my_llm_pipeline" });
 
// Nested tracing example
const retrieveDocs = traceable(async (query: string): Promise<string[]> => {
  return docs;
}, { name: "retrieve_docs" });
 
const generateAnswer = traceable(async (question: string, docs: string[]): Promise<string> => {
  const resp = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: `${question}\nContext: ${docs.join("\n")}` }],
  });
  return resp.choices[0].message.content || "";
}, { name: "generate_answer" });
 
const ragPipeline = traceable(async (question: string): Promise<string> => {
  const docs = await retrieveDocs(question);
  return await generateAnswer(question, docs);
}, { name: "rag_pipeline" });

Best Practices:

• Apply traceable to all nested functions you want visible in LangSmith
• Wrapped clients auto-trace all calls — wrap_openai()/wrapOpenAI() records every LLM call
• Name your traces for easier filtering
• Add metadata for searchability </trace_other_frameworks>

<traces_vs_runs> Use the langsmith CLI to query trace data.

Understanding the difference is critical:

• Trace = A complete execution tree (root run + all child runs). A trace represents one full agent invocation with all its LLM calls, tool calls, and nested operations.
• Run = A single node in the tree (one LLM call, one tool call, etc.)

Generally, query traces first — they provide complete context and preserve hierarchy needed for trajectory analysis and dataset generation. </traces_vs_runs>

<command_structure> Two command groups with consistent behavior:

Key differences:

<querying_traces> Query traces using the langsmith CLI. Commands are language-agnostic.

</querying_traces>

Basic filters:

• --trace-ids abc,def - Filter to specific traces
• --limit N - Max results
• --project NAME - Project name
• --last-n-minutes N - Time filter
• --since TIMESTAMP - Time filter (ISO format)
• --error / --no-error - Error status
• --name PATTERN - Name contains (case-insensitive)

Performance filters:

• --min-latency SECONDS - Minimum latency (e.g., 5 for >= 5s)
• --max-latency SECONDS - Maximum latency
• --min-tokens N - Minimum total tokens
• --tags tag1,tag2 - Has any of these tags

Advanced filter:

• --filter QUERY - Raw LangSmith filter query for complex cases (feedback, metadata, etc.)

<export_format> Export creates .jsonl files (one run per line) with these fields:

Use --include-io or --full to include inputs/outputs (required for dataset generation). </export_format>