> ## Documentation Index
> Fetch the complete documentation index at: https://trytilde.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Anatomy of an agent

> What a Tilde agent is made of: your framework code, the Tilde SDK, and a container.

A Tilde agent has three parts:

* **Your agent code**, written in the framework you already use.
* **The Tilde SDK**, which connects your code to the gateway and gives it an invocation context.
* **A container or Lambda function** that packages the two.

Your code contains no chat provider integration, no provider API keys, and no HTTP server. The gateway supplies all three at runtime.

## Example agent

This agent uses the Vercel AI SDK. It is two small files: one connects to Tilde, and one defines the agent and handles a single invocation.

<CodeGroup>
  ```typescript src/index.ts theme={"system"}
  import { connectAgent } from "@trytilde/sdk";
  import { respond } from "./agent.js";

  // `tilde deploy` loads this module to find the agent's prompts,
  // tools, and skills. Export what it should register.
  export { agent } from "./agent.js";

  // Dial out to Tilde. `connectAgent` reads TILDE_GATEWAY_URL and
  // TILDE_DEPLOYMENT_TOKEN, opens an outbound HTTP/2 stream, and
  // sends ready heartbeats. This process listens on no port.
  // Under `tilde deploy`, it starts nothing.
  const connection = connectAgent({
    // Tilde calls `run` once for each invocation, with a fresh
    // context.
    run: respond,

    onRegistered: (registration) =>
      console.log("Agent is ready", registration.deploymentId),
  });

  // On shutdown, stop taking work and let active invocations
  // finish and report. Your orchestrator sends SIGTERM during a
  // rollout.
  for (const signal of ["SIGINT", "SIGTERM"] as const)
    process.once(signal, () => void connection.close());
  ```

  ```typescript src/agent.ts theme={"system"}
  import { inference, type AgentContext } from "@trytilde/sdk";
  import {
    convertToAiSdkMessages,
    tildeAiSdk,
    tildeCallOptions,
  } from "@trytilde/sdk-vercel-ai-node";
  import { createOpenAI } from "@ai-sdk/openai";
  import {
    APICallError,
    ToolLoopAgent,
    convertToModelMessages,
    stepCountIs,
  } from "ai";

  // `inference` returns a base URL for Tilde's inference gateway
  // and a placeholder API key. The gateway swaps in the real
  // provider credential, so this process holds no key. Each
  // request resolves to the invocation that makes it.
  const openai = createOpenAI(inference("openai/production"));

  // A plain AI SDK agent, defined once. `tilde deploy` registers
  // its instructions as a versioned prompt.
  export const agent = new ToolLoopAgent({
    id: "support-agent",
    instructions: [
      "You are a support assistant.",
      "Use the current channel tools to respond to the latest message.",
      "Model text is private and is not delivered to the user.",
    ].join(" "),
    model: openai.responses("gpt-4o-mini"),
    // Let the model call tools, read results, and call again.
    stopWhen: stepCountIs(8),
    maxOutputTokens: 600,
    // Record model calls in the Tracing tab.
    experimental_telemetry: { isEnabled: true, functionId: "support-agent" },
    // Lets each call carry Tilde's tools, skills, and new input.
    ...tildeCallOptions,
  });

  // Handle one invocation.
  export async function respond(ctx: AgentContext) {
    try {
      // 1. Load the conversation from Tilde, and convert it into
      //    AI SDK messages. Attachments are downloaded for you.
      const history = await ctx.message.history();
      const messages = await convertToAiSdkMessages({
        messages: history.items,
        context: ctx,
      });

      // 2. Call the model. `tildeAiSdk` adds the channel's tools,
      //    such as sendMessage, the agent's Tilde tools, and its
      //    skills. Each tool call runs through the gateway, which
      //    checks the agent's capabilities and records it. It also
      //    stops the call when a user interrupts or an operator
      //    stops the run.
      await agent.generate({
        messages: await convertToModelMessages(messages),
        ...tildeAiSdk(ctx),
      });
    } catch (error) {
      // Provider error bodies can contain sensitive details.
      // Log only the status.
      const status = APICallError.isInstance(error)
        ? error.statusCode
        : undefined;
      if (ctx.signal.aborted) console.warn("Invocation cancelled");
      else console.error("Invocation failed", { status });
      const suffix = status ? ` (${status})` : "";
      throw new Error(`Agent run failed${suffix}`);
    }
  }
  ```

  ```dockerfile Dockerfile theme={"system"}
  FROM node:22-slim AS build
  WORKDIR /app
  COPY package.json pnpm-lock.yaml ./
  RUN corepack enable && pnpm install --frozen-lockfile
  COPY . .
  RUN pnpm build

  FROM node:22-slim
  WORKDIR /app
  ENV NODE_ENV=production
  COPY --from=build /app/node_modules ./node_modules
  COPY --from=build /app/dist ./dist
  USER node
  # No EXPOSE: the agent only dials out.
  CMD ["node", "dist/index.js"]
  ```
</CodeGroup>

The container needs two environment variables, `TILDE_GATEWAY_URL` and `TILDE_DEPLOYMENT_TOKEN`. See [Deploy direct agents](/docs/deployment/direct-agents).

### The invocation context

Everything your agent does during an invocation goes through `ctx`. It is bound to one invocation, and carries that invocation's short-lived token.

| Member | Purpose |
| - | - |
| `ctx.message.history()` | The conversation so far, paginated, with attachments. |
| `ctx.channel.current` | The tools of the channel the message arrived on, such as `sendMessage`. |
| `ctx.inference(name)` | Client settings for an assigned [inference connection](/docs/inference-providers). The module-level `inference(name)` resolves to the running invocation. |
| `ctx.tools` | The agent's [tools](/docs/tools/overview), with their schemas. |
| `ctx.skills` | The agent's [skills](/docs/skills/overview), as a folder on disk or as tools. |
| `ctx.prompt(definition)` | Renders a [prompt](/docs/prompts/overview) you declared, and links it to the invocation's model calls. |
| `ctx.goals`, `ctx.tasks` | The agent's own plan, shown in the [session](/docs/sessions). |
| `ctx.reason(text)` | Records a working note. It is never sent to the user. |
| `ctx.signal` | Aborts when a user interrupts, or an operator stops the run. Pass it to model and tool calls. |
| `ctx.stop()` | Ends the invocation. |

The adapter package connects Tilde to the framework's own objects. `convertToAiSdkMessages` turns Tilde history into AI SDK messages, and `tildeAiSdk` adds Tilde's tools and skills to a call. You still call `generate`, `stream`, `generateText`, or `streamText` yourself. To use a framework with no adapter, read `ctx` directly inside `run`. See [Framework integrations](/docs/frameworks/overview).

## Lifecycle of a request

<img src="https://mintcdn.com/tilde/UIZH9VuZaVUMv6QU/images/request-lifecycle.drawio.svg?fit=max&auto=format&n=UIZH9VuZaVUMv6QU&q=85&s=0916acd8694abb4cd0ea93c5994fdd62" alt="Sequence of one request: a message reaches the gateway, passes access checks and request guards, wakes the agent with a short-lived token, the agent calls the LLM through the gateway, replies through a channel tool, and the gateway applies response guards before delivery" width="1442" height="862" data-path="images/request-lifecycle.drawio.svg" />

<div className="zoom-hint"><Icon icon="magnifying-glass-plus" size={13} /> <em>Click to zoom</em></div>

<Steps>
  <Step title="A message arrives">
    A chat provider's webhook, or your own app through native chat, delivers the message to the gateway.
  </Step>

  <Step title="The gateway checks and routes it">
    The gateway checks that the sender can message this agent, runs [request guards](/docs/security/middleware) in Tilde Enterprise, and picks a ready deployment. See [Routing](/docs/routing).
  </Step>

  <Step title="The gateway wakes the agent">
    The gateway sends the invocation down the agent's open HTTP/2 stream, with a short-lived token. The SDK calls your `run` function.
  </Step>

  <Step title="The agent loads its context">
    The agent reads the conversation history and the channel's tools through the gateway.
  </Step>

  <Step title="The agent calls the model">
    The LLM request goes to the gateway with a placeholder key. The gateway injects the real credential, forwards the request, streams the response back, and records the usage.
  </Step>

  <Step title="The agent replies">
    The agent calls the channel's send tool.
  </Step>

  <Step title="The gateway screens the reply">
    The gateway checks the agent's [capabilities](/docs/iam#agent-capabilities), runs response guards in Tilde Enterprise, and records the tool call.
  </Step>

  <Step title="The reply is delivered">
    The gateway sends the reply through the provider's API, with the connection's credential.
  </Step>

  <Step title="The agent reports">
    The SDK uploads logs and traces, and reports that the invocation ended.
  </Step>
</Steps>

For a sidecar agent, the sidecar takes the gateway's place in steps 3 to 7, on loopback.

## Lifecycle of an agent

An agent has many deployments over its life. Each deployment is one release of your container, registered in Tilde and tied to a commit.

<img src="https://mintcdn.com/tilde/UIZH9VuZaVUMv6QU/images/agent-deployment-lifecycle.drawio.svg?fit=max&auto=format&n=UIZH9VuZaVUMv6QU&q=85&s=97649b38f1092716e01c0823394425a3" alt="Two rows of steps. First deployment: build, register the deployment, roll out, connect, and serve. Deploying an update: build version two, register it, roll it out beside version one, route new conversations to it, drain version one, and retire it" width="1573" height="552" data-path="images/agent-deployment-lifecycle.drawio.svg" />

<div className="zoom-hint"><Icon icon="magnifying-glass-plus" size={13} /> <em>Click to zoom</em></div>

### First deployment

1. **Build** the agent image from your Dockerfile.
2. **Register a deployment** with `tilde deploy`. Tilde registers the agent's prompts, tools, and skills with it, and returns a deployment token once. The deployment is registered but offline.
3. **Roll out** your replicas with the token.
4. **Connect.** Each replica dials the gateway and sends ready heartbeats.
5. **Serve.** Once a replica is ready, new conversations route to the release.

### Deploying an update

An update is a new deployment of the same agent. You never replace a deployment in place.

1. **Build** the new image.
2. **Register** a second deployment. It has its own token.
3. **Roll out** the new release beside the old one. Both stay connected.
4. **Route.** In latest mode, new conversations go to the new release as soon as it is ready. In weighted mode, you send it a percentage first.
5. **Drain.** Conversations that started on the old release stay pinned to it, so keep it running until they end.
6. **Retire** the old deployment. Its token stops working.

To roll back, promote the old deployment. It becomes the serving deployment again, and conversations pinned to the new release stay there.

Automate these steps in your pipeline. See [Release agents from CI](/docs/deployment/ci-cd).