Skip to main content
Data reaches an ontology in two steps. Pipelines bring raw records in, one table per stream. Models are SQL that turns those records into the tables your objects read.

Declare pipelines

Declare pipelines once, in ducktail.toml at the root of the project repository. Each [[pipeline]] has a name, a kind, and the streams it writes.
ducktail.toml
Set connection = "<slot>" on a pipeline to use a Tilde connection. The pipeline then gets the account’s current credentials, and refreshed OAuth tokens, from Tilde. A webhook pipeline with a connection receives that provider’s verified webhooks. Every slot a pipeline names must be declared in the ontology. See Connection slots. Write any other secret as ${NAME}, and set it on the environment. See Secrets.

Connection slots

A connection slot is a third-party account the project needs. Declare it in the ontology, and bind it to a connection in each environment.
ontology/connections.ts
List the slots in ontology({ connections: [linear, github, slack] }). Handlers use the same slots to call the provider. See Call third parties.

Custom streams

A custom stream takes records from your own code. Each record is validated against a zod schema, and keyed, so a newer record of a key replaces the older one.
ontology/sources.ts
List streams in ontology({ streams: [feedback] }), and write records with the management API. Use a management API key with write_streams on the project. At most 500 records go in one request.

Reference data

A stream can also ship static records with the project, such as configuration or lookup tables. Tilde validates them at build time, and loads them on every deploy. A redeploy replaces changed records. A record you remove from the list is not deleted.

Tilde chat events

An ontology can read your organisation’s own Tilde data: chat threads, messages, participants, channel identities, runs, and inference usage. Declare a tilde webhook pipeline with the streams you want. Message bodies reach the ontology only if you list public_chat_messages.
ducktail.toml
Then turn on Registry events in the environment’s settings. Tilde delivers existing rows first, then every change. Agents’ conversations become data the ontology can join with everything else, for example to link a WhatsApp thread to a customer.

Models

A model is SQL over sources or other models, in dbt’s vocabulary. Read sources with source(pipeline, stream), or use a custom stream directly.
ontology/models.ts
You can also keep SQL in a file with sql: sqlFile("./customers.sql"), using {{ ref('model') }}, {{ src('pipeline', 'stream') }}, and {% if is_incremental() %}. Models read upstream tables only through ref, src, and changes, so Tilde always knows a model’s dependencies. The build rejects SQL that names a raw table directly.

SQL dialect

Models and handler SQL use DuckDB’s SQL dialect. JSON payloads read with ->>, including paths such as payload->>'$.address.country'.
Ontologies can also use PostgreSQL storage, with storage: "postgres" in ontology(). All SQL is then PostgreSQL, and models can declare indexes. An environment runs only ontologies of its own storage, so talk to us before you switch.

When a deploy rebuilds a model

Some changes make a deployed model rebuild from scratch: new inputs to changes(), a new schema, or a new dependency. A deploy that would rebuild a model is refused, and names each model. Deploy again with --accept-rebuild model:<name> when the rebuild is what you want. See Deploy.