- Chat providers: These are first-class integrations with third-party chat providers.
- Vercel AI SDK chat provider: This managed provider exposes your agent through a Vercel AI SDK-compatible endpoint for custom clients.
- Signals: These are events from third-party providers that Tilde delivers to your agent.
team resources or private user_team resources. A private resource remains in its execution team and has one owner. Private sessions may also include selected team users as conversation members: members can see the session, read its history, send messages, and receive its live events, while the owner and team, organization, or system administrators manage membership and session administration. Messages, attachments, events, tasks, replies, and turn queues inherit the session’s audience.
Agents, sessions, and routines also expose independent visibility and ownership modes. Visibility controls discovery, history, messaging, and the realtime audience. Ownership controls settings, membership, grants, and deletion. Private user/group grants are tenant-scoped, and ownership or administrator authority alone does not reveal a private conversation. Session membership remains the conversation-specific way to admit additional team users.
Session members are Tilde authorization principals, not ChatKit delivery participants. Adding an agent, Slack channel, or other inbox participant determines where messages flow; adding a Tilde user as a session member determines who may open the private conversation. Removing a member revokes subsequent history and event access, and leaving the execution team invalidates the grant.
Each ChatKit realtime WebSocket retains its authenticated effective user and current Identity groups. Team agent lifecycle changes are sent to every connected team member. Private agent and session events follow their current visibility grants and conversation membership. Authorization changes emit an access.changed invalidation so clients refresh their personalized workspace projection.
Realtime events use a closed, client-facing union: agent.*, session.*, participant.*, message.*, queue_item.*, turn.*, activity.*, task.*, and chat.error. Streaming content arrives through sequenced message.delta events. Internal event-bus payloads and audience identifiers are never forwarded.
Read state belongs to the user-session relationship. PUT /api/v1/team/{team_id}/chatkit/workspace/sessions/{session_id}/read-state accepts { "unread": false } when a user opens a session or { "unread": true } to mark it manually. Other users sharing the session keep independent state.
Signal provider instances and rules may also be personal. A personal rule names the team where its agent action runs. Sessions created by that rule are user_team sessions for the same owner, preventing the rule from turning personal activity into a team-visible conversation.
Signal providers and rules use the same two planes. Visibility controls discovery and delivery inspection; ownership controls configuration, target/session policy, grants, state-changing retries, and deletion. Deliveries and sessions inherit the rule and target authorization rather than defining independent grants.
Build a customer-hosted integration with custom ChatKit providers. Definitions are reusable within a team, and each connection has independent credentials and configuration.
Set up ChatKit
1
Create the endpoint
Wrap your route with Keep the API key and webhook signing key in server-side environment variables.
chatKitEndpoint. It verifies Tilde’s signature and gives the handler the current session, new messages, provider metadata, and ChatKit client.app/api/agent/route.ts
2
Register the agent
Open Tilde, select your workspace, and go to ChatKit → Agents. Choose Team when every team member should discover the agent, or Private when discovery and lifecycle updates should be limited to its visibility grantees. Register the endpoint and copy the one-time API key and webhook signing key into your app’s environment.For local development, enable Local running endpoint and run the app through a Dev Tunnel. For production, enter the deployed HTTPS endpoint.Registration also creates a credentialless Message agent tool provider bound to this agent. Add its
message and wait_for_response tools to any Tilde MCP server when another agent should invoke it.3
Connect a channel and test
Go to ChatKit → Configure Chat Providers and choose where people will talk to the agent. Link the provider to your registered agent, then open ChatKit workspace to start a test session.
Choose how the endpoint responds
With AI SDK 7, keep system context in the model call’sinstructions field. Conversation messages must not contain system messages; recalled memory remains explicitly untrusted context.
chatKitEndpoint requires a top-level responseMode:
agentLoopstreams returned assistant text as the visible reply, preserving the existing endpoint pattern.tooltreats returned assistant text as private reasoning. The agent must callcontext.session.tools.sendMessage(also available throughcontext.$provider.tools) for visible replies.
to, cc, bcc, subject, HTML, and reply-all.
Pass the same session context to context.session.createMCPClient({ serverId: process.env.TILDE_MCP_SERVER_ID! }) when the agent consumes tools through MCP. Session tools are added without asking the model for routing identifiers. In a private owner-workspace session, Tilde also derives the authenticated human’s personal Memory and Wiki tools from the authorized session. The agent keeps its own identity, and unrelated personal connections are not exposed.
For a shared agent whose MCP server enables speaker-bound personal-tool federation, use context.mcp.connect({ serverId }). ChatKit forwards an invocation-scoped capability for the verified speaker outside model input. Tilde resolves only that speaker’s eligible personal accounts; an unmapped external speaker receives no personal tools. The capability, user ID, account IDs, and credentials are never copied into messages or room state.
ChatKit assigns compact participant handles and emits durable participant.joined and participant.left events when a participant becomes visible to or leaves the conversation. Conversation snapshots expose them in participant_events, separate from messages, so clients can render session activity without adding synthetic chat messages. Tilde still includes the lifecycle context in agent history, and these changes do not start an agent turn.
Confirm an outbound message
After creating a message, readGET /api/v1/team/{team_id}/chatkit/session/{session_id}/message/{message_id}/deliveries with the same organization scope. It requires visibility of the message and its session. Each receipt identifies the channel, provider, status, optional provider message ID, last error, and delivery timestamp.
provider_status reports pending, delivered, or failed when the adapter can query final delivery status, as with Telnyx WhatsApp. That result takes precedence over initial queue acceptance: a later carrier rejection returns dead_letter and its error. Without a queryable provider status, delivered means acceptance. Neither means the recipient read the message. Keep the original message ID for an uncertain send; only a confirmed provider rejection establishes that a new attempt is safe.
Share a room with people and agents
A ChatKit session can be a private or team room with a durable human and agent roster. Room owners can invite a canonical Tilde user, select anadmin or member collaboration role, add visible agents, inspect pending invitations, revoke an invitation, and remove a participant. The invited user accepts or declines before receiving private room history.
Room roles do not replace resource permissions. Session visibility controls messages, attachments, replay, and live events. Session ownership controls invitations and roster administration. Removing a participant stops future delivery and speaker-bound personal-tool federation.
client.chatkit.rooms exposes the typed roster and invitation contract. runBoundedRoomGroup deduplicates agents, caps participants and rounds, and stops when a round makes no progress. OpenBot currently keeps the owner room UI dormant until canonical human identity discovery can replace raw user identifiers; the underlying API and SDK contract are available.
Keep substantial work durable
Owner-only WhatsApp and Linq channels can opt into private conversations withexternal_participant_policy: { join: "linked_only", agent_invocation: "linked_only", personal_tools: "linked_participants", session_scope: "personal" } in their provider configuration. The sender must have a verified identity link routed to that channel and current team membership. Messages retain the verified human actor and the original channel’s reply target. Existing shared conversations stay separate; a subsequent real inbound message starts or reuses the owner’s private conversation. Other channels retain the default session_scope: "team" behavior.
Goals, tasks, background jobs, routines, and AgentRuns let an agent continue substantial work without keeping correctness in one process.
- A goal records the outcome for one agent and session. A task records an independently deliverable piece of that goal, including dependencies and progress.
- A background job delegates one bounded objective to an explicit child agent. Jobs persist an optional
model_id, hard duration/token/cost budgets, child session, transcript message IDs, artifacts, and terminal result. - An AgentRun persists the model-loop objective, generation-fenced lease, steps, token/cost totals, loop counters, wake state, and tool-effect receipts.
- A routine stores recurring intent and its schedule. Prefer an event trigger when the same work can start from a Signal.
client.chatkit.work({ agentId, sessionId }) for goals, tasks, and jobs; client.chatkit.routines(agentId) for recurring work; and client.chatkit.runs for a durable host. Job actions are steer, stop, resume, and collect-result. Only the bound agent can mutate its work. Authorized owners can inspect and control the corresponding conversation work.
Tilde supplies child job ID, generation, model, and budget in the signed request’s typed execution context. Hidden continuations carry the run ID, worker ID, generation, and hidden flag there. A continuation for a bound job also carries that job’s context in execution.job, preserving its selected model and budget. Agent hosts must not accept caller-provided model, budget, run, or worker headers as authority.
Bind a delegated model loop to its job by passing job: { id: job.jobId, generation: job.generation } to client.chatkit.runs.create. Use the direct agent_job execution context or the nested job context of an agent_run continuation. Tilde validates the child agent, session lineage, and current job generation, then creates or returns the one Run bound to that generation. Do not reuse an arbitrary active Run from the same session. chatKitAgentRunIdempotencyKey(triggerId, context.execution) still supplies retry identity; the typed binding supplies job authority. Ordinary human runs omit job.
Once bound, an HTTP turn finishing does not complete the job. An active, waiting, paused, or stalled Run keeps the job running. The worker polls the same durable Run without redispatching the accepted HTTP invocation. A completed Run completes the job; a failed or canceled Run reports failure. Only that terminal result produces the parent’s completion wake. Hosts without a bound Run retain the HTTP-terminal contract.
Pass expectedGeneration: run.generation when recording a step with client.chatkit.runs.appendStep; the SDK sends expected_generation. Bound runs require that fence in addition to the current worker lease, so accounting from an older generation cannot land after steering. It remains optional for legacy unbound runs.
Steering a bound job queues the instruction until the Run reaches an idle continuation boundary with no active lease or unresolved tool effect. The next continuation waits for pending steering. Tilde updates the existing Run and acknowledges the command atomically. Stopping the job cancels its bound Run; resuming creates a fresh job generation and a separate Run binding.
A hidden continuation records its completed step before requesting a run transition. Tilde atomically acknowledges the matching planned continuation and applies that transition only for the current generation and unexpired worker lease, with a durable step recorded after the continuation started. Owner control cannot acknowledge a pending continuation, and an uncertain receipt still requires reconciliation or cancellation.
Compact model context without rewriting history
Context compaction changes model input, not ChatKit history. An agent reportsstarted, ended, or failed lifecycle events to /chatkit/sessions/{session_id}/compaction-events. A successful checkpoint stores the exact summary, agent, transcript boundary, compacted and retained message IDs, and token counts.
Load /chatkit/sessions/{session_id}/messages/from-last-compaction?agent_id=... to receive the latest successful checkpoint separately from retained and newer messages. In @trytilde/sdk-vercel-ai-node, createChatKitCompactionController supplies a bounded prepareStep loop and composeChatKitCompactionPrepareStep preserves provider preparation before applying compaction.
Human-created private workspace sessions belong to the authenticated human, including when a deployment service owns the selected agent. Tilde binds the workspace participant to that human for personal tool access. Delegated child sessions inherit the parent session’s ownership.
Test your agent in ChatKit workspace
Use ChatKit workspace to invoke your agent directly and test conversations. Select the correct workspace and agent, then start a session and send a message. When creating a private session, the creator becomes its owner automatically. The create request may include additional team user IDs, and the session membership endpoints can list, add, or remove non-owner members afterward. Members receive the same live message and agent-turn stream for that session; they do not gain permission to change ownership or manage other members.ChatKit workspace requires the Vercel AI Endpoint ChatKit provider to be enabled for your agent.
Approve agent-requested capability changes
An agent may create a durable self-extension proposal when it needs a connector, MCP server, skill registry, custom tool, agent bundle, memory bank, or wiki that it cannot already use. Tilde validates the requested category, rejects secret-shaped fields, and authors the permission, credential, cost, audience, egress, security, and undo preview. The active conversation renders that proposal as a secure Yes or No Human Approval card. The decision is bound to the proposal ID, immutable proposal hash, proposal generation, authenticated human principal, requesting agent, and originating session. A typed button decision is required; free-text confirmation in chat does not approve a proposal. Only the requesting agent’s human owner, a human team administrator, or a system administrator can decide the proposal. The requesting human cannot approve their own request, and an agent credential cannot approve, reject, cancel, roll back, or claim one-time outputs. Yes atomically completes the linked Human Approval and queues leased execution. No atomically cancels the approval and records proposal rejection. Credential values, credential references, approval tokens, OAuth state, and generated signing keys are never included in model-visible proposal text. If execution needs provider setup, the server returns a secret-free setup-item reference only after approval. The approving human continues through the normal server-authored credential or OAuth card. The original agent can then resume its task from the durable decision and proposal status. Execution records whether each resource was created by the proposal or reused. Rollback removes only proposal-created receipts. One-time generated values are encrypted and can be consumed once through the human-only outputs endpoint.Search conversations
Use consolidated ChatKit search to find session titles, agents participating in sessions, and message content that you can view in a workspace. Results include the session context needed to open the matching conversation.kind of session_title, agent, or message. Message results include the canonical ChatKit message. Use the opaque next_page_token to continue in relevance order.
To search messages inside one conversation, add its session_id. Session-scoped search returns message results only and returns 404 if the session is not visible to you or does not belong to the selected workspace.
Audit coding-agent sessions
Connect Codex, Claude Code, Cursor, OpenCode, or Gemini CLI withopenbot plugin to record coding
sessions in ChatKit while installing the Tilde MCP servers and managed skills
you select.
Provider-specific message metadata
Supported chat providers add validated metadata to the endpoint context. Use the provider-specific property inside yourchatKitEndpoint handler.
- GitHub
- Slack
GitHub messages expose repository, issue, pull request, comment, and event metadata through
context.github.app/api/code-review/route.ts
Work with session context
context.messages contains the new input for the current turn. Load context.session.history() when the model needs the earlier conversation, then convert both collections together.
sessionId, team and organization IDs, the invoking user when known, and typed metadata for supported providers such as context.slack and context.github.
For agent-to-agent delegation, context.body.session.parentAgentId identifies the authenticated agent that opened the child session. Direct conversations omit it. A specialist that operates caller-owned runtime state—such as a browser display—can use this server-authored ID to continue the caller’s state without accepting a routing identity from model input or client parameters.
context.agent identifies the agent receiving the current turn. It includes the canonical id, displayName, providerId, status, optional principalUserId, optional authenticated avatar.url, and createdAt / updatedAt timestamps from Tilde. Use this context instead of duplicating an agent name or avatar in application configuration.
agent field is optional on the wire so an updated SDK remains compatible with requests from an older Tilde deployment.
Handle unprocessed content
convertToAiSdkMessages automatically converts standard text and reasoning parts. It also caches transformed message parts for later model requests, improving prompt caching and agent performance. Use onUnprocessed for content that needs application-specific handling before it can be sent to a model.
fileUploadreceives each unprocessed file part and its parent message.firecrawlmaps page-monitoring and completed-check signals to typed message converters.githubmaps GitHub issue, pull request, and CI signal types to typed message converters.sentrymaps a signal type, such assentry.issue.created, to a typed message converter.- Return an AI SDK message or part to include it. Return
nullto omit it. - Handlers can be asynchronous. If a handler throws, message conversion fails.
onUnprocessed once for each unprocessed message, then caches the result. Subsequent conversions reuse the cached value instead of invoking the handler again.
- File uploads
- GitHub signals
- Firecrawl signals
- Sentry signals
Use Supported media is downloaded and passed to the model as an inline file. Unsupported stored attachments become a text part containing the file name, media type, and download URL.
createChatKitAttachmentFilePartHandler to download ChatKit attachments with Tilde authentication and convert them into model-safe AI SDK file parts.app/api/agent/route.ts
Choose how new turns are handled
Each chat provider can control what happens when another message arrives while the agent is still working. When a message comes from the agent’s generated MCP provider, ChatKit uses the policy configured on the target agent.Let agents invoke other agents through MCP
Every registered agent has a generated Message agent tool provider. Add that provider to an existing MCP server just like any other credentialless provider. Its routing fields are fixed to the target agent, so callers only supply the message and, optionally, a ChatKit session ID.messageaccepts Vercel AI SDK-compatible UI message parts, persists the inbound ChatKit message, and immediately returns a ticket and session ID.wait_for_responseaccepts that ticket, subscribes to the live ChatKit session, streams response deltas through MCP progress notifications (or logging notifications when the caller did not supply a progress token), and returns the final canonical ChatKit message.- Reuse the returned session ID to continue the same child-agent conversation. Omit it to create a new session.
- Queue status notifications include the target agent’s concurrency policy and whether multiple messages were batched into the turn.
Trigger work from events via Signals
Signals turn supported external events into ChatKit messages. A rule selects an event type, maps it to an agent, and determines whether related events reuse the same session. Use a stable session key when repeated events belong to the same body of work. For example, route every update for one Sentry issue into the same remediation session so the agent can continue from its existing history. Map typed signal messages while converting ChatKit history. This example turns a Sentryissue.created signal into the user message sent to the model.
app/api/sentry-remediation/route.ts
ChatKit and memory
ChatKit preserves the messages inside a session. Memory stores selected knowledge that should be available across sessions, channels, or agents. Use both when an agent needs conversational continuity and longer-lived organizational context.Configure realtime voice
Voice settings belong to your Tilde agent. In agent registration, select a voice profile and its models, voice, and maximum conversation duration. You can change these settings in the agent editor later.- Text agent with speech: Tilde transcribes incoming audio, invokes your normal
chatKitEndpointcallback, and speaks its streamed text response. - Telnyx Conversation Relay: Telnyx handles recognition and speech synthesis for incoming phone calls. Tilde receives caller text, invokes your normal callback, and streams its response text to Telnyx. This profile is phone-only.
- OpenAI Realtime: the realtime model generates spoken responses directly. Tilde records the final transcripts without invoking your text endpoint again.
context.audio for transcribed speech turns and
context.telnyx for Telnyx calls. Provider facts are supplied in the signed
request. A subsequent typed message does not become a speech turn merely because
its session previously contained a call.
Each call creates a normal ChatKit session. Browser media admission is one-time,
expires after five minutes, and remains subject to current session membership.
Call duration is limited by the configured maximum. Call recordings are not
retained by this initial realtime implementation.
For a manual test, use the SDK repository’s examples/realtime-voice example. It
registers three agents. Its microphone page supports the two OpenAI modes; a
dedicated relay endpoint handles phone calls. To receive calls, configure a
Telnyx Voice chat provider with an existing encrypted Telnyx Voice credential,
Voice API application, phone number, and default agent. This creates a real
ChatKit channel that owns the caller’s participant route.
Choose self-managed webhook setup to copy Tilde’s returned webhook URL into
Telnyx yourself. Choose managed webhook setup to let Tilde update the existing
application’s webhook URL using your credential. Managed setup does not provision
or fund a Telnyx account, purchase a number, or assign numbers to applications.
Use a dedicated test application for the manual example. The Tilde API must be publicly
reachable through HTTPS and WSS. Hookdeck webhook replay alone cannot carry live
bidirectional calls.
For relay, select telnyx_relay, transcription model deepgram/nova-3, voice
Telnyx.Ultra.Callie, language en-US, and interruption enabled. The Telnyx
route uses its own credential; relay does not require an OpenAI speech key.
Your callback continues to use its own text model and tools. The SDK example
accepts TELNYX_AGENT_MODE=telnyx_relay and keeps the browser demos available.
Partial relay transcripts do not start agent turns. When speech interrupts a
response, Tilde retains generated text and separately records the spoken prefix
reported by the carrier. The SDK annotates both text and UI history so the next
turn can distinguish generated words from that reported prefix. Caller ID does
not authorize access to a Tilde human’s personal tools.
The Telnyx Conversation Relay guide
describes the text WebSocket protocol used between Telnyx and Tilde.
The initial browser implementation streams audio through the Rust API. Direct
browser-to-provider WebRTC and direct SIP routing are separate transport options
and are not implied by the OpenAI Realtime profile. Native Realtime uses its
configured instructions and does not inherit endpoint tools. Browser voice
identifies its caller but does not yet establish personal-tool federation.
Change resources through native tools
Agents use native Tilde API/MCP operations under their existing permissions. The capability proposal API has been retired. Chain dependent operations using returned resource IDs, reconcile partial failures before retrying, and read back the resulting resource. Do not widen permissions or switch credentials after an authorization failure. Before enabling a connector, read the managed enable-connections skill. Discover existing user and agent access and verify the correct account first. Choose personal/user or bot ownership explicitly; when unclear, ask whether other bots should be able to use the account. Native brokering returns aconnector_setup_required descriptor for the pending
resource. API clients render an enable-provider event outside message bubbles
and open secure configuration modals. In external channels, invoke sendMessage
with the server-returned hosted setup URL. Credentials stay in native secure
setup operations, outside chat and persisted client workflow snapshots.
Recover missing conversation context
Session-scoped MCP connections providechatkit_search_history. The query
searches the current conversation by default. Set include_related_sessions to
search other conversations that the authenticated agent actively participates
in with the current session’s verified human owner. Ordinary search permissions
also apply. Models cannot supply a different agent, tenant or user identity to
this tool. Follow next_page_token, even after an empty filtered page.