Scheduler — BullMQ - Refract Documentation

BullMQ job schedulers are the production scheduler client: durable cron patterns in Redis, ticks on a dedicated queue, handlers invoked on each tick — business work still flows through tools.queue when needed.

Why

BullMQ job schedulers (v5.16+) replace the deprecated repeatable-jobs API. Cron metadata lives in Redis, aligned with the existing BullMQ stack. Each tick runs the schedule handler with tools; handlers enqueue via tools.queue when needed. Retries and DLQs remain on business queues (BullMQ or SQS).

Setup

Ensure "tooling-scheduler-bullmq": "workspace:*" in apps/backend/package.json.
Run make deps-install.
Set tools.scheduler in env configs (see table and example below).
Optional dedicated Redis: SCHEDULER_REDIS_URL=redis://scheduler:6379 in .env.development (falls back to REDIS_URL via resolveSchedulerRedisUrl in apps/backend/src/configuration/resolveSchedulerRedisUrl.ts).
Start one consumer: make start (Compose consumer service).
Run make test module=tooling-scheduler-bullmq.

Config options (`tools.scheduler`, BullMQ)

Field	Required	Description
`client`	Yes	`SchedulerClientType.BULLMQ`
`connection`	Yes	`{ url }` or `{ host, port, password? }` — scheduler Redis, not inferred from `tools.queue`
`schedulerQueueName`	Yes	BullMQ queue for ticks (default constant: `SCHEDULER_TICK_QUEUE_NAME` → `scheduler_ticks_v1`)
`schedulerTickJobName`	Yes	BullMQ job `name` on each tick template (default constant: `SCHEDULER_TICK_JOB_NAME` → `tick`)
`schedules`	Yes (array)	`{ name, pattern, tz? }[]` — may be empty when no cron work is configured yet

Each schedule name must match a handler in scheduleRegistry.ts (see Scheduler overview). Omit a name from the array to stop it; reconcile removes orphans on consumer restart. Example (apps/backend/src/configuration/development.ts):

import {
  SCHEDULER_TICK_JOB_NAME,
  SCHEDULER_TICK_QUEUE_NAME,
  SchedulerClientType,
} from 'shared';
import { ScheduleName } from '../tools/scheduler/scheduleName';
import { resolveSchedulerRedisUrl } from './resolveSchedulerRedisUrl';

scheduler: {
  client: SchedulerClientType.BULLMQ,
  connection: { url: resolveSchedulerRedisUrl() },
  schedulerQueueName: SCHEDULER_TICK_QUEUE_NAME,
  schedulerTickJobName: SCHEDULER_TICK_JOB_NAME,
  schedules: [
    {
      name: ScheduleName.MY_SCHEDULE,
      pattern: '0 0 3 * * *',
    },
  ],
},

How the client is created

buildScheduler in apps/tools/scheduler/bullmq/ opens a Redis connection from required connection. On registerSchedules:

Lists job schedulers on schedulerQueueName via getJobSchedulers.
Removes schedulers not listed in tools.scheduler.schedules (orphans and stale keys).
For each config entry, calls upsertJobScheduler when missing or when pattern / tz changed (stable scheduler id: scheduler:${scheduleName}).

Tick jobs use schedulerTickJobName as the BullMQ job template name; payload includes { scheduleName } so the worker resolves the handler by schedule name, not by job name. A Worker on the same queue processes ticks: handler → optional tools.queue.sendToQueue. Official reference: BullMQ job schedulers.

Mixed stack with SQS queue

tools.queue.client may be sqs while tools.scheduler.client is bullmq. Scheduler Redis is separate from SQS; ticks still call tools.queue.sendToQueue, which delivers to SQS (including DelaySeconds cap for deferred sends). See Scheduler overview and Queue overview.

Observability

Implementation: apps/tools/scheduler/bullmq/src/observability.ts. Each registerSchedules run logs a reconcile summary (removed, registered, updated, unchanged) and emits scheduler.reconcile.* metrics. Each tick logs start/completion with durationMs and emits scheduler.tick.* metrics. Failures use categorical reason tags — not raw error text. Consumer bootstrap logs via scheduler-consumer when apps/backend/src/consumer.ts calls registerSchedules. See the monitoring table in Scheduler overview.

Local observability

If scheduler Redis is the same instance as the queue (REDIS_URL), Bull Board may show scheduler_ticks_v1 alongside business queues. Ticks are lightweight; monitor downstream queue depth for real work.

Production verification

Production and staging use schedules: [] until real cron work is added. After deploy:

Restart one consumer replica.
Confirm scheduler-bullmq-reconcile logs show no errors; removed may be 1 on the first restart if a prior scheduler entry existed in Redis, then 0 on later restarts.
Confirm registerSchedules returns success in consumer bootstrap logs.
Optional: Bull Board — scheduler_ticks_v1 has no delayed jobs when schedules: [].

Caveats and limitations

Single consumer replica — mandatory; no horizontal scaling.
Redis required — schedules do not run without scheduler Redis.
No backfill — long outages do not replay every missed cron window automatically.
6-field patterns only — standard subset documented in cronPatternSchema; no L/W/# in v1.
Deploy/restart — pattern changes apply on next registerSchedules (consumer restart).
Empty schedules[] — valid; reconcile clears all job schedulers from Redis until entries are added. Production and staging currently use an empty array.

What’s next?

Scheduler overview — handlers, registry, adding schedules.
Test scheduler client — runScheduleNow for Jest only.
Queue BullMQ — business queue client.
Configuration

Getting Started

Architecture

Frontend

Deployment

Quality

Reference

Scheduler — BullMQ

Why

Setup

Config options (`tools.scheduler`, BullMQ)

How the client is created

Mixed stack with SQS queue

Observability

Local observability

Production verification

Caveats and limitations

What’s next?

​Why

​Setup

​Config options (tools.scheduler, BullMQ)

​How the client is created

​Mixed stack with SQS queue

​Observability

​Local observability

​Production verification

​Caveats and limitations

​What’s next?

Why

Setup

Config options (`tools.scheduler`, BullMQ)

How the client is created

Mixed stack with SQS queue

Observability

Local observability

Production verification

Caveats and limitations

What’s next?