Queue

Queue moves work off the request path: producers call tools.queue.sendToQueue; the consumer process runs handlers from consumerRegistry.

​

Why

Queues give you retries, DLQs, and async processing. Refract keeps a single consumer service that hosts all queue workers (and the scheduler tick worker).

​

Independent from scheduler

tools.queue and tools.scheduler are separate config keys. Valid combinations: The scheduler only enqueues; consumers still run via createConsumer for each configured queue.

​

Scaling

Run exactly one consumer replica. Vertical scaling only. See Scheduler overview for the same rule (duplicate ticks if scaled horizontally).

​

Adding a new queue

1. Add QueueName (+ DLQ if needed) in apps/backend/src/utilities/queue.ts and REGISTERED_QUEUE_NAMES.
2. Add config under tools.queue.queues in each env file.
3. Register consumer loader in consumerRegistry.ts.
4. Implement consumer under apps/backend/src/tools/queue/consumers/.
5. Restart consumer; run make test module=backend.

​

Deferred enqueue (sendToQueue)

One-off delays use tools.queue.sendToQueue — not the scheduler. Validated on every send via sendToQueueParamsSchema in apps/shared/src/queue/schema.ts. SQS caps delay at 900 seconds; see SQS. Deferred jobId / replace / removeQueuedJob on SQS require Redis (tools.cache) on both API and consumer processes.

​

Monitoring

• Logs per consumer feature logger.
• Bull Board (BullMQ queue client): http://localhost:3998.
• DLQ queues in config — inspect failed messages after max retries.

​

Gotchas

• API (backend) does not run queue consumers.
• Switching queue provider requires aligned config on API and consumer.
• Recurring wall-clock work belongs on tools.scheduler, not repeated delayed sends.

​

What’s next?

• BullMQ — Redis queue client setup.
• SQS — AWS queue client setup.
• Scheduler overview — recurring schedules.
• Configuration