AI tooling

This page explains how Refract wires Anthropic, OpenAI, and Google Gemini through a single tools.ai client with shared validation, timeouts, and metrics.

📖 See also: AI Development covers Cursor rules and agent workflows — not LLM API tooling.

How it works

Refract treats AI like other pluggable tools: the contract lives in apps/shared, vendor SDKs live in apps/tools/ai/<provider>/, and the backend loader returns one instrumented client on tools.ai. Unlike mailer or queue, you can enable more than one provider at once. Each provider block in config lists an allowlist of model ids. The loader routes calls by params.model; when the model is omitted, the primary provider is the first one loaded (anthropic → openai → google).

Configuration shape

tools.ai is optional. If you omit it from development.ts / production.ts, buildTools leaves tools.ai undefined.

Field	Location	Purpose
`request`	Top-level on `tools.ai`	Shared timeout, retry, and wall-clock cap for all providers
`anthropic`	Optional provider block	`apiKey`, `models[]`, optional `defaultModel`
`openai`	Optional provider block	Same shape as anthropic
`google`	Optional provider block	Same shape as anthropic

At least one provider block is required when tools.ai is present. Zod enforces defaultModel ∈ models when set.

Request policy (`tools.ai.request`)

Option	Default	Meaning
`timeoutMs`	`60000`	Per-attempt timeout (each retry can use up to this, within the total budget)
`maxAttempts`	`3`	Retry count for retryable errors on `complete` and `countTokens`
`retryDelayMs`	`500`	Base delay; backoff doubles per attempt
`totalTimeoutMs`	`timeoutMs × maxAttempts`	Wall-clock cap for the full operation (all attempts or an entire stream)

Retries on complete are not idempotent. A timeout after the provider already finished can produce a second billed generation on retry. Tune totalTimeoutMs and maxAttempts for your cost tolerance.

Model routing

`params.model`	Behavior
Omitted or empty	Uses the first configured provider’s `defaultModel` or first entry in `models`
Known id in config allowlist	Routes to the provider that owns that id
Unknown id	Throws `AiValidationError` before any SDK call

getAvailableModels(tools) projects the union of configured allowlists — use it to populate admin UI or validate user-facing model pickers.

Calling `tools.ai`

Every method takes (params, tools). Pass the same tools object the app already builds at startup so instrumentation can read tools.logger and tools.metrics.

if (tools.ai === undefined) {
  throw new Error('AI is not configured');
}

const result = await tools.ai.complete(
  {
    prompts: [{ role: 'user', content: 'Summarize this invoice.' }],
    model: 'claude-sonnet-4-20250514',
  },
  tools,
);

Streaming uses the same params shape; chunks include cumulative content and may report token usage on the final chunk.

Observability

Instrumentation records StatsD-style metrics and feature logger ai entries:

Metric	Tags
`ai.request.duration_ms`	`provider`, `operation`, `outcome`
`ai.request.count`	`provider`, `operation`, `outcome`
`ai.request.tokens.input` / `.output`	`provider`, `operation` (histogram values, not tags)

Operations: complete, stream, count_tokens, list_models. OpenAI countTokens logs tokenEstimate: true because the adapter uses a character heuristic, not the provider tokenizer API. Provider setup guides: Anthropic, OpenAI, Google Gemini.

Extending this system

Here’s the usual path from “enable AI” to a production call site.

Ensure tooling-ai-anthropic, tooling-ai-openai, and/or tooling-ai-google are listed in apps/backend/package.json (they ship with the boilerplate).
Add a tools.ai block to apps/backend/src/configuration/development.ts (and production.ts when you go live). Example:

ai: {
  request: {
    timeoutMs: 60_000,
    maxAttempts: 3,
    retryDelayMs: 500,
    totalTimeoutMs: 180_000,
  },
  anthropic: {
    apiKey: process.env.ANTHROPIC_API_KEY,
    models: ['claude-sonnet-4-20250514'],
    defaultModel: 'claude-sonnet-4-20250514',
  },
},

Set API keys in .env.development (see provider pages for variable names).
Implement your feature in apps/backend/src/core/ and call tools.ai.complete with an allowlisted model or omit model for the primary provider default.
Run make test module=shared path=apps/shared/src/ai/__tests__ and make test module=backend path=apps/backend/src/tools/ai/__tests__/instrumentation.spec.ts.

What not to do

❌ Never import vendor LLM SDKs in resolvers, utilities outside the tooling packages, or the portal. Use tools.ai only.

❌ Never duplicate retry or metrics in apps/tools/ai/*/src/client.ts. Instrumentation in apps/backend/src/tools/ai/instrumentation.ts is the single owner.

❌ Never assume an arbitrary model string will work. Unknown ids throw AiValidationError; user-facing model pickers should use getAvailableModels or your own allowlist derived from config.

❌ Never confuse this page with AI Development. That doc is for Cursor agents, not tools.ai.

File conventions

New AI-related code belongs in these trees:

Purpose	Path
Contract, Zod schema, validation helpers	`apps/shared/src/ai/`
Loader, instrumentation, tests	`apps/backend/src/tools/ai/`
Provider SDK implementations	`apps/tools/ai/<provider>/` (package name `tooling-ai-<provider>`)
Environment config	`apps/backend/src/configuration/<env>.ts` under `tools.ai`

Provider packages export buildAnthropicClient, buildOpenaiClient, or buildGoogleClient from src/index.ts. The loader dynamic-imports them and never re-exports SDK types to the rest of the backend.

What’s next?

Anthropic — Claude setup and env vars
OpenAI — GPT setup and token estimate behavior
Google Gemini — Gemini setup and system prompts
Configuration — env files and pluggable tools table
Tooling system — how loaders and workspace packages fit together

Getting Started

Architecture

Frontend

Deployment

Quality

Reference

How it works

Configuration shape

Request policy (`tools.ai.request`)

Model routing

Calling `tools.ai`

Observability

Extending this system

What not to do

File conventions

What’s next?

​How it works

​Configuration shape

​Request policy (tools.ai.request)

​Model routing

​Calling tools.ai

​Observability

​Extending this system

​What not to do

​File conventions

​What’s next?

How it works

Configuration shape

Request policy (`tools.ai.request`)

Model routing

Calling `tools.ai`

Observability

Extending this system

What not to do

File conventions

What’s next?