• October 22, 2026
• 6 min read

LangSmith — Prompt Management¶

Prompts live in the Prompt & Context Hub (the Prompts section of a workspace): versioned templates you edit in the Playground and pull into code with Client.pull_prompt(). The Hub also stores contexts (skills and agent instructions) with the same commit / environment model; this page covers prompts.

Concepts¶

Prompt identifiers used by the SDK:

Push¶

from langchain_core.prompts import ChatPromptTemplate
from langsmith import Client

client = Client()

v1 = ChatPromptTemplate.from_messages([
    ("system", "You triage support tickets for {product}."),
    ("human", "{ticket}"),
])
url = client.push_prompt("ticket-triage", object=v1, commit_tags=["v1"], description="Ticket triage")

v2 = ChatPromptTemplate.from_messages([
    ("system", "You triage support tickets for {product}. Answer with JSON: priority and team."),
    ("human", "{ticket}"),
])
client.push_prompt("ticket-triage", object=v2, commit_tags=["v2"], commit_description="Force JSON output")

• push_prompt creates the prompt if it does not exist, then adds a commit on top of parent_commit_hash="latest". is_public, description, readme and tags (prompt-level tags) update metadata.
• Push prompt | model (a RunnableSequence) to store the model configuration with the prompt; the provider must be supported by the Playground.
• StructuredPrompt.from_messages_and_schema(messages, schema=Model.model_json_schema()) stores a template plus output schema.
• Reserved staging / production are assigned with Promote in the UI; keep immutable release tags (v1, v2) for the SDK.
• Other calls: list_prompts(query="triage", is_public=False), list_prompt_commits(name), get_prompt, update_prompt, delete_prompt, like_prompt.

Pull¶

prompt = client.pull_prompt("ticket-triage:production")
messages = prompt.invoke({"product": "WebShop", "ticket": "Checkout returns 500"}).to_messages()

chain = client.pull_prompt("ticket-triage-with-model:production", include_model=True)   # prompt | model
chain.invoke({"product": "WebShop", "ticket": "Checkout returns 500"})

With LangChain the result is a runnable: prompt | ChatOpenAI(model="gpt-4o-mini"). Without LangChain, convert the rendered prompt into a provider payload:

from openai import OpenAI
from langsmith.client import convert_prompt_to_openai_format     # needs langchain-openai installed

payload = convert_prompt_to_openai_format(prompt.invoke({"product": "WebShop", "ticket": "500 on checkout"}))
resp = OpenAI().chat.completions.create(**{**payload, "model": "gpt-4o-mini"})   # payload carries a placeholder model

convert_prompt_to_anthropic_format is the Anthropic counterpart. Prompts pulled with include_model=True or from public owners can instantiate models and carry configuration: review them before use.

Versions vs Environments¶

Promoting a commit to Production does not remove it from Staging. Typical flow:

1. Edit in the Playground (or push from code), commit.
2. Run a dataset experiment against the commit (04); compare with the current production commit.
3. Promote to Staging; CI and pre-production pull ticket-triage:staging.
4. Promote to Production after the gate passes; the app pulls ticket-triage:production.
5. Roll back from the environment's rollback history if online evaluators or alerts regress.

Code should reference the environment, never a hash, in production paths.

Caching¶

pull_prompt uses a process-wide in-memory cache with stale-while-revalidate: callers always get a fast answer and the entry refreshes in the background.

from langsmith import Client
from langsmith.prompt_cache import configure_global_prompt_cache, prompt_cache_singleton

configure_global_prompt_cache(max_size=200, ttl_seconds=7200, refresh_interval_seconds=600)   # before creating clients

client = Client()
client.pull_prompt("ticket-triage:production")                     # network
client.pull_prompt("ticket-triage:production")                     # cache hit
client.pull_prompt("ticket-triage:production", skip_cache=True)    # force a fresh fetch
print(prompt_cache_singleton.metrics.hits, prompt_cache_singleton.metrics.misses)

Client(disable_prompt_cache=True)                                  # no caching for this client

• Tests that edit a prompt and expect the change immediately must use skip_cache=True or prompt_cache_singleton.invalidate("ticket-triage:staging").
• Offline / air-gapped runs: prompt_cache_singleton.dump("prompts_cache.json") while online; later configure_global_prompt_cache(ttl_seconds=None) + prompt_cache_singleton.load("prompts_cache.json") serves pulls without network access.
• prompt_cache_singleton.clear() and .reset_metrics() reset state between tests.

Playground and Prompt Canvas¶

Commit the prompt before starting an experiment so the run references a stable version.

Testing Prompts in Code¶

# tests/test_prompt_contract.py
import pytest
from langsmith import Client


@pytest.fixture(scope="module")
def prompt():
    return Client().pull_prompt("ticket-triage:staging", skip_cache=True)


def test_prompt_variables(prompt):
    assert set(prompt.input_variables) == {"product", "ticket"}          # callers pass exactly these


def test_prompt_renders(prompt):
    messages = prompt.invoke({"product": "WebShop", "ticket": "Checkout returns 500"}).to_messages()
    assert messages[-1].content == "Checkout returns 500"
    assert "JSON" in messages[0].content                                  # output-format instruction survives edits

Behavior testing runs the pulled prompt through the model over a dataset and records which prompt version produced the experiment:

from langsmith import Client, wrappers
from openai import OpenAI

client = Client()
oai = wrappers.wrap_openai(OpenAI())
PROMPT_ID = "ticket-triage:staging"


def target(inputs: dict) -> dict:
    prompt = client.pull_prompt(PROMPT_ID, skip_cache=True)
    messages = prompt.invoke({"product": "WebShop", **inputs}).to_messages()
    resp = oai.chat.completions.create(
        model="gpt-4o-mini", temperature=0, response_format={"type": "json_object"},
        messages=[{"role": "system" if m.type == "system" else "user", "content": m.content} for m in messages],
    )
    return {"answer": resp.choices[0].message.content}


def valid_json(outputs: dict) -> bool:
    import json
    try:
        json.loads(outputs["answer"])
        return True
    except ValueError:
        return False


client.evaluate(target, data="triage-regression", evaluators=[valid_json],
                experiment_prefix="prompt-staging", metadata={"prompts": [PROMPT_ID]})

The metadata["prompts"] list fills the prompt column of the experiment table. Datasets and evaluators are covered in 04.

Automation Around Commits¶

• Webhook (Prompts page → + Webhook, one per workspace): POSTs prompt_id, prompt_name, commit_hash, created_at, created_by and manifest on every commit. Use it to trigger a CI evaluation job or to sync manifests into Git; the GitHub sync tutorial uses a small intermediary server that holds a GitHub token.
• API commits can pass skip_webhooks (true, or a list of webhook IDs) to avoid triggering them; the Playground asks which webhooks to skip.
• Per-prompt Access and Permissions → Owners only restricts who can promote to Production.
• Resource tags (key-value labels for workspace resources) are unrelated to commit tags.

Pitfalls¶

See also¶

• LangSmith — LLM Tracing, Prompts & Evals
• LangSmith — Datasets & Evaluations
• LangSmith — Testing, CI & Production
• LangChain — LLM Application Framework
• Langfuse — Prompt Management
• MLflow — Evaluation & Prompts