Skip to content

Runbook: Add last_sent_at to notification_rules

Context

The operations email delivery feature (operations-notifications-delivery change) adds a last_sent_at timestamptz NULL column to operations.notification_rules. This column acts as an idempotency guard: once a notification is sent, lastSentAt is set so retries skip the already-delivered rule.

This migration MUST run before the new API version is deployed. If the column is missing at runtime, TypeORM maps it to undefined and the listener treats every rule as not-yet-sent — which means BullMQ retries can trigger duplicate sends.

Pre-deployment checklist

  • [ ] A database backup exists or the deployment platform (Dokploy) has an automated backup for this window.
  • [ ] The API is not serving traffic to the OperationsModule during the migration window, OR the migration is applied before the new binary is started (rolling deploy with zero-downtime requires that the old binary tolerate the new column — it does, because the old code never reads last_sent_at).
  • [ ] You have confirmed that the migration file has been generated (see step 1 below).

Step 1 — Generate the migration file

You must generate the migration file yourself. Never let the AI run typeorm migration:generate.

bash
pnpm --filter api typeorm migration:generate \
  apps/api/src/modules/operations/infrastructure/migrations/AddNotificationRuleLastSentAt

This produces a timestamped migration file under apps/api/src/modules/operations/infrastructure/migrations/. Verify that the generated up method contains:

sql
ALTER TABLE "operations"."notification_rules"
  ADD "last_sent_at" TIMESTAMP WITH TIME ZONE NULL;

And the down method contains:

sql
ALTER TABLE "operations"."notification_rules"
  DROP COLUMN "last_sent_at";

Step 2 — Apply the migration

Run pending migrations:

bash
pnpm --filter api typeorm migration:run

Verify the column was added:

sql
SELECT column_name, data_type, is_nullable
FROM information_schema.columns
WHERE table_schema = 'operations'
  AND table_name   = 'notification_rules'
  AND column_name  = 'last_sent_at';

Expected result:

column_namedata_typeis_nullable
last_sent_attimestamp with time zoneYES

Step 3 — Deploy the new API version

After the migration succeeds, proceed with the normal Dokploy deployment. The new binary will start reading and writing last_sent_at correctly.

Rollback

If you need to roll back the code but keep the column (safe — old code ignores unknown columns):

  1. Deploy the previous API version.
  2. Leave the column in place (it defaults to NULL, causing no side-effects with the old code).

If you need to remove the column entirely:

bash
pnpm --filter api typeorm migration:revert

This runs the down migration which drops last_sent_at.