• October 19, 2026
• 10 min read

Pipecat — Conversation: Turns, Context & Tools¶

How a voice agent actually holds a conversation: the services in the cascade, the shared context, VAD and turn detection (who is speaking, when to answer), interruptions, and function calling. Verified on pipecat-ai 1.13.0; code that needs provider keys is marked as not run.

The Cascade¶

from pipecat.services.deepgram.stt import DeepgramSTTService          # extra: deepgram
from pipecat.services.openai.llm import OpenAILLMService               # extra: openai
from pipecat.services.cartesia.tts import CartesiaTTSService           # no extra needed

stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
llm = OpenAILLMService(
    api_key=os.getenv("OPENAI_API_KEY"),
    settings=OpenAILLMService.Settings(
        model="gpt-5-mini",
        system_instruction="You are a friendly support agent. Keep answers under two sentences.",
    ),
)
tts = CartesiaTTSService(api_key=os.getenv("CARTESIA_API_KEY"), settings=CartesiaTTSService.Settings(voice="..."))

(Not run: needs keys. The OpenAILLMService construction and its Settings(model=..., system_instruction=...) were run with a placeholder key.)

Settings classes are nested on the service (OpenAILLMService.Settings) and read back from the read-only service.settings. Any OpenAI-compatible server works with OpenAILLMService(base_url=...), which is also how the tests in 04 run without network access.

Context and Aggregators¶

The conversation lives in an LLMContext: messages, tools and tool_choice, in an OpenAI-compatible format that adapters translate for other providers.

from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import (
    LLMContextAggregatorPair,
    LLMUserAggregatorParams,
)

context = LLMContext(messages=[{"role": "developer", "content": "Keep answers short."}], tools=[lookup_order])
pair = LLMContextAggregatorPair(
    context,
    user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
)
# pipeline: [..., stt, pair.user(), llm, tts, transport.output(), pair.assistant()]

Changing the context at runtime¶

Push a frame into the pipeline (await worker.queue_frame(...)):

run_llm=True triggers a reply; the default (None) updates silently.

Long calls¶

Context grows with every turn. Turn on automatic summarization on the assistant aggregator before long sessions hit token or latency limits:

from pipecat.processors.aggregators.llm_response_universal import LLMAssistantAggregatorParams

pair = LLMContextAggregatorPair(
    context,
    user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
    assistant_params=LLMAssistantAggregatorParams(enable_auto_context_summarization=True),
)

LLMAssistantAggregatorParams (verified) has enable_auto_context_summarization=False and auto_context_summarization_config= (LLMAutoContextSummarizationConfig); the summarizer itself is LLMContextSummarizer in pipecat.processors.aggregators.llm_context_summarizer. Not run end to end (needs a real model to write summaries).

Voice Activity Detection (VAD)¶

VAD tells the pipeline when someone is speaking. Pipecat ships Silero VAD (it runs locally on the CPU and needs no extra download):

from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.audio.vad.vad_analyzer import VADParams

vad = SileroVADAnalyzer(params=VADParams(stop_secs=0.2))

Verified VADParams defaults:

Leave confidence and min_volume alone unless you have profiled your audio; use an upstream noise filter (RNNoise, Krisp Viva, ai-coustics) for noisy input. If you change stop_secs, re-measure your STT latency and pass it to the STT service as ttfs_p99_latency so turn timing stays correct. Attach VAD to the user aggregator: LLMUserAggregatorParams(vad_analyzer=vad).

User Turns¶

VAD hears speech; turn strategies decide when a turn starts and ends, which is when the bot interrupts or answers.

from pipecat.turns.user_start import MinWordsUserTurnStartStrategy, TranscriptionUserTurnStartStrategy, VADUserTurnStartStrategy
from pipecat.turns.user_stop import SpeechTimeoutUserTurnStopStrategy, TurnAnalyzerUserTurnStopStrategy
from pipecat.turns.user_turn_strategies import UserTurnStrategies

params = LLMUserAggregatorParams(
    vad_analyzer=SileroVADAnalyzer(),
    user_turn_strategies=UserTurnStrategies(
        start=[VADUserTurnStartStrategy(), TranscriptionUserTurnStartStrategy()],
        stop=[SpeechTimeoutUserTurnStopStrategy(user_speech_timeout=0.6)],
    ),
)

All start strategies take enable_interruptions=True by default. LocalSmartTurnAnalyzerV3 is an audio model that judges whether the user is done (it builds offline in 1.13.0; the local-smart-turn extra covers its dependencies). Defaults in 1.13.0: Smart Turn for turn end, SileroVADAnalyzer with stop_secs=0.2 for speech.

Choosing a stop strategy:

Other LLMUserAggregatorParams fields¶

Verified in 1.13.0: user_turn_stop_timeout=5.0, user_idle_timeout=0 (off), audio_idle_timeout=1.0, user_mute_strategies=[...], filter_incomplete_user_turns=False, add_tool_change_messages=False, empty_user_turn=EmptyUserTurnConfig(...).

• Empty turns (1.12.0): when the user spoke but nothing was recognized, the aggregator can prompt recovery. EmptyUserTurnConfig has interrupted_prompt (asks the user to repeat when the bot was cut off), idle_prompt and max_consecutive_recoveries=1. It is on by default.
• Mute strategies stop the user from interrupting at the wrong moment: FirstSpeechUserMuteStrategy (during the first bot speech), MuteUntilFirstBotCompleteUserMuteStrategy, FunctionCallUserMuteStrategy (while a tool runs), AlwaysUserMuteStrategy.
• filter_incomplete_user_turns=True drops turns the model judges unfinished.

Interruptions¶

When a user turn starts, the bot stops speaking and in-flight work is cancelled (InterruptionFrame; system frames survive, queued speech is dropped). They are on by default through enable_interruptions on the start strategies. Keep them on for natural conversations; if the bot is cut off by background noise, raise MinWordsUserTurnStartStrategy(min_words=...) or use a noise filter, rather than turning interruptions off. The assistant aggregator records only the part the user actually heard.

Function Calling¶

A tool is an async def whose first parameter is params: FunctionCallParams. Type hints and a Google-style docstring become the schema:

from pipecat.services.llm_service import FunctionCallParams


async def lookup_order(params: FunctionCallParams, order_id: str):
    """Look up an order's status.

    Args:
        order_id: The order number the customer provided.
    """
    await params.result_callback({"order_id": order_id, "status": "shipped"})


context = LLMContext(tools=[lookup_order])        # "direct function": schema derived from the signature

The schema the OpenAI adapter produced from that function (run on 1.13.0):

{
  "type": "function",
  "function": {
    "name": "lookup_order",
    "description": "Look up an order's status.",
    "parameters": {
      "type": "object",
      "properties": {"order_id": {"type": "string", "description": "The order number the customer provided."}},
      "required": ["order_id"]
    }
  }
}

Ways to register tools¶

FunctionCallParams¶

function_name, tool_call_id, arguments, llm, pipeline_worker, context, result_callback, app_resources (shared objects passed to PipelineWorker(app_resources=...): database pool, HTTP client) and worker_runner.

await params.result_callback(result, properties=FunctionCallResultProperties(...)): run_llm (run the model after this result; default yes), on_context_updated (callback after the result is in context), is_final=False (intermediate update on an async call).

Synchronous vs asynchronous tools¶

from pipecat.adapters.schemas.direct_function import tool_options


@tool_options(cancel_on_interruption=False, timeout_secs=30, cancellable_by_llm=True)
async def generate_report(params: FunctionCallParams, topic: str):
    """Generate a report on a topic.

    Args:
        topic: The subject of the report.
    """
    ...

tool_options(fn=None, *, cancel_on_interruption=True, timeout_secs=None, cancellable_by_llm=False) (signature verified).

LLM-service options (LLMService.__init__): run_in_parallel=True (several tool calls in one turn run concurrently), group_parallel_tools=True (the model runs once after the whole batch; turn it off and the bot may answer once per tool), function_call_timeout_secs, enable_async_tool_cancellation=False.

Telling the user something is happening¶

A slow tool leaves silence. Say something first, then call the tool. Run offline, the TTSSpeakFrame appeared downstream of the LLM between FunctionCallInProgressFrame and FunctionCallResultFrame, so the caller hears it while the tool works:

from pipecat.frames.frames import TTSSpeakFrame


async def lookup_order(params: FunctionCallParams, order_id: str):
    """Look up an order's status.

    Args:
        order_id: The order number.
    """
    await params.llm.push_frame(TTSSpeakFrame("One moment while I check that."))
    status = await orders_api.get(order_id)
    await params.result_callback(status)

What a full tool turn does (run offline)¶

Against a local fake OpenAI server that answered the first request with a tool call and the second with text, Pipeline([pair.user(), llm, pair.assistant()]) fed an LLMRunFrame made two requests. The second request's roles were ['system', 'user', 'assistant', 'tool'], the system message was the configured system_instruction, the tool list was ['lookup_order'], and the context ended as:

roles:  ['user', 'assistant', 'tool', 'assistant']
tool:   {'role': 'tool', 'content': '{"order_id": "ord-42", "status": "shipped"}', 'tool_call_id': 'call_1'}
final:  {'role': 'assistant', 'content': 'Your order ord-42 has shipped.'}

The full test is in 04.

MCP tools¶

MCPClient (extra mcp) registers the tools of an MCP server as Pipecat functions. See Model Context Protocol (MCP) for the protocol.

Multiple Agents (Alpha)¶

Pipecat 1.x lets several workers coordinate over a shared bus (AsyncQueueBus in process, RedisBus or PgmqBus across machines): hand-off between agents, parallel fan-out, jobs, sidecar workers and distributed agents. LLMWorker adds an LLM pipeline and automatic @tool registration; LLMContextWorker adds a context; UIWorker drives a client GUI. The multi-agent API is still marked alpha in the 1.13.0 release notes (LLMWithBackend), so pin your version and read the Agent Handoff and Multiple LLM Agents guides before relying on it.

Common Mistakes¶

See also¶

• Pipecat — Open-Source Framework for Voice Agents
• Pipecat — Pipelines, Frames & Processors
• Pipecat — Transports, Runner & Production
• Pipecat — Testing & Evals
• Model Context Protocol (MCP)
• Pydantic AI — Agents, Tools & Structured Output