Skip to content

Messaging and Notifications Architecture

Overview

This document describes how FarmCove's messaging system, abuse prevention mechanisms, and notification infrastructure work together to provide a secure, scalable, and user-friendly communication platform. The system enables bidirectional conversations across multiple channels (WhatsApp, SMS, web, in-app) while preventing abuse and maintaining conversation context.

Table of Contents

  1. System Components
  2. Architecture Overview
  3. Data Flow and Integration
  4. Storage Strategy
  5. Abuse Prevention Integration
  6. Outbound WhatsApp outbox & dispatcher
  7. Real-time Capabilities
  8. Key Design Decisions

System Components

Core Tables

  1. Notifications System
  2. notification - Stores all system notifications
  3. notification_template - Reusable notification templates
  4. notification_delivery - Tracks delivery status per channel

  5. notification_preference - User communication preferences

  6. Messaging System

  7. conversation - Permanent container for user interactions
  8. conversation_instance - The message grouping a conversation is currently living on; ends by message-count rollover or idle-closing
  9. message - Individual messages within instances
  10. message_attachment - Media/file attachments

  11. conversation_summary - AI-generated summaries of closed instances

  12. Abuse Prevention

  13. rate_limit_config - Configurable rate limits per channel
  14. abuse_pattern - Detected abuse patterns and actions
  15. abuse_action_history - History of penalties applied

Architecture Overview

Bidirectional Communication Flow

┌─────────────────┐     ┌──────────────────┐     ┌─────────────────┐
│                 │     │                  │     │                 │
│  Notification   │────▶│    Messaging     │◀────│     Abuse       │
│    System       │     │     System       │     │   Prevention    │
│                 │◀────│                  │────▶│                 │
└─────────────────┘     └──────────────────┘     └─────────────────┘
         │                       │                         │
         └───────────────────────┴─────────────────────────┘
                                 │
                          ┌──────▼──────┐
                          │   Supabase  │
                          │  (Storage)  │
                          └─────────────┘

Key Integration Points

  1. Notification → Conversation: Notifications can initiate conversations
  2. Message → Notification: Important messages can trigger in-app alerts
  3. Abuse Prevention: Monitors all messaging activity in real-time
  4. Shared Context: All systems reference the same user records

Data Flow and Integration

Scenario 1: Notification-Initiated Conversation

When a notification needs user response (e.g., transaction alert):

  1. Create Notification
  2. System creates notification in notification table
  3. Marks WhatsApp channel for delivery

  4. Creates delivery record in notification_delivery

  5. Start Conversation

  6. Sana (WhatsApp bot) sends notification
  7. The service layer reuses the conversation's active instance, creating one only when none exists (getOrCreateConversationInstance, services/conversation/sana.ts); the database has no get-or-create function for this
  8. Creates/reuses conversation in conversation table

  9. Links notification via conversation_instance_id

  10. Track Message

  11. Creates outbound message in message table
  12. Links to notification via notification_id

  13. Updates delivery status

  14. Handle Response

  15. User reply creates inbound message
  16. Abuse prevention checks rate limits
  17. Conversation continues in same instance

Scenario 2: User-Initiated Conversation

When user starts conversation directly:

  1. Receive Message
  2. WhatsApp webhook receives message
  3. The webhook only ACKs 2xx once the message is durably owned (queued), and returns 503 otherwise so Meta redelivers the same wamid — the full status-code contract and its duplicate-tolerance guarantees live in SANA.md § Webhook ACK policy
  4. System checks rate limits via check_rate_limits_and_abuse()

  5. If allowed, creates conversation structure

  6. Optional Alert

  7. Can create in-app notification for visibility
  8. Notification links to conversation instance

  9. User sees activity across all channels

  10. Continue Dialog

  11. Messages join whichever instance is currently active — there is no clock that moves a live conversation onto a new instance mid-dialogue
  12. The instance ends one of two ways: it reaches the per-instance message cap and the webhook rolls it over, or it goes idle long enough for the hourly sweep to close it (see Instance Lifecycle)
  13. Once closed, the instance is summarized by AI

Storage Strategy

Database Tables

All data is stored in Supabase (PostgreSQL) with Row-Level Security:

  • conversation: Stores channel type, user association, status
  • conversation_instance: Groups the messages of one stretch of a conversation
  • message: Stores content, direction, sender, timestamps
  • notification: Stores templates, content, delivery preferences
  • rate_limit_config: Defines limits per channel/user type
  • abuse_pattern: Logs detected patterns with confidence scores

Instance Lifecycle

A conversation is permanent; the instance is the stretch of it currently accepting messages. At most one instance per conversation is active — enforced by the partial unique index conversation_instances_one_active_uidx. Statuses move one way: active → closed → summarized.

Conversation (permanent)
    └── Instance 1 (closed: message_limit)
    │     ├── Message 1
    │     ├── Message 2
    │     └── ... up to the per-instance cap (default 500)
    │
    └── Instance 2 (active — the successor)
          ├── Message 501
          └── ...

Two mechanisms end an instance:

  • Message-count rollover. check_rate_limits_and_abuse compares the per-instance cap against the ACTIVE instance's message_count alone (never the sum over the conversation's history — that sum only grows, so the cap would be permanent). On reaching it the checker returns allowed = false, limit_type = 'instance', reset_at = NULL. That verdict is an instruction, not a rejection: the webhook calls the rollover_conversation_instance RPC, which in one transaction closes the capped instance (auto_closed_reason = 'message_limit') and inserts its successor, then processes the message. The RPC serializes on the parent conversations row lock, so concurrent inbound messages produce exactly one successor (losers see already_current); it refuses outright while the user has an active abuse action.
  • Idle closing. An hourly Vercel Cron (/api/cron/close-stale-conversation-instances) closes active instances whose last activity — COALESCE(last_message_at, started_at) — is 8 hours or more in the past, stamping auto_closed_reason = 'time_limit'. The same sweep retries AI summaries for instances left in closed, so a summarization failure is never terminal.

Summarization is decoupled from the close: the instance sits in closed until a summary exists, then moves to summarized. conversation_summaries_instance_uidx makes this idempotent — one summary per instance, so a retried summarization conflicts instead of appending a second, divergent one.

Benefits:

  • Prevents unbounded growth
  • Enables efficient AI summarization
  • Maintains conversation context
  • Supports quick message retrieval

Abuse Prevention Integration

Real-Time Protection

  1. Rate Limiting (Before Message Creation)

Defaults for the all user type; every limit is configurable per channel and user type in rate_limit_config (the seeded premium tier doubles them).

Per-Minute: 5 messages
Per-Hour: 60 messages
Per-Day: 200 messages
Per-Instance: 500 messages

The per-minute/hour/day counters live in the conversation's rate_limit_data and reset on the clock; exceeding one applies a throttle and rejects the message. The per-instance cap is different in kind — it counts the active instance's messages and triggers a rollover rather than a rejection (see Instance Lifecycle).

Outbound messages count too. message_count is incremented by an AFTER INSERT trigger on every message row regardless of direction, so a heavily-notified user can reach the per-instance cap without ever sending anything.

  1. Pattern Detection (Every 5th Message)
  2. Spam: Identical repeated messages
  3. Flooding: <2 seconds between messages

  4. Gibberish: Short meaningless content

  5. Progressive Penalties (by offenses recorded in the last 30 days)

  6. 1st offense: Warning
  7. 2nd offense: 15-minute throttle
  8. 3rd offense: 1-hour throttle
  9. 4th offense: 24-hour throttle
  10. 5th+ offense: 7-day block

When the Check Itself Fails

Rate limiting is an availability control, not an access control, so a failure of the check must not cost the user their message. When check_rate_limits_and_abuse errors, the webhook logs the SQLSTATE and details, then queries abuse_actions directly for an active block or suspension:

  • An active block/suspend is found → deny, and reply with the access-restricted message. This backstop is the access control, so it may not fail open: an error reading it aborts processing rather than admitting the message.
  • Nothing in force → fail open and process the message normally.

Deny verdicts are unaffected by this policy — a check that successfully returns allowed = false still denies (or, for the instance verdict, rolls over).

Storage in conversation Table

Rate limit data stored as JSONB in conversation:

{
  "daily_message_count": 45,
  "daily_reset_at": "2024-01-09T00:00:00Z",
  "hourly_message_count": 12,
  "hourly_reset_at": "2024-01-08T15:00:00Z",
  "is_throttled": false,
  "throttled_until": null
}

Outbound WhatsApp outbox & dispatcher

Every outbound WhatsApp send — Sana replies, error notices, read receipts, and notification deliveries alike — is written to a durable outbox BEFORE any wire call is attempted, and delivered later by a dispatcher that paces sends per recipient and owns all retries.

Enqueue before wire

whatsAppClient.sendX(...) no longer touches the network. It validates and writes an outbox row, then returns; the caller's request can end immediately. This inverts the old failure mode: a send can no longer be lost because the serverless function was frozen mid-request, and a slow provider can no longer hold an inbound webhook past Meta's ACK window.

Two failures therefore exist where there used to be one, and they are deliberately distinct:

Failure Meaning Where it surfaces
Enqueue failure Nothing was queued; the message will NOT send Throws at the call site; the site's catch guard
Delivery failure Queued, but undeliverable after all attempts Emitted by the dispatcher from the row's context

The call-site guards (sendGenericError, the confirmation guards in mediaResponse / flowResponse / interactiveListResponse / processQuickReplyResponse / unsupportedResponse) now guard the ENQUEUE, and log sana.reply_enqueue_failed. The dispatcher logs sana.reply_delivery_failed for the terminal-delivery case. Keeping the two event names separate is what lets a log query distinguish "never queued" from "queued but undeliverable" — they demand different operator responses.

Tables

message_outbox (the entries), message_outbox_pairs (per-(sender, recipient) pacing and claim state), and message_outbox_attempts (one row per wire attempt, for triage). Full schema, indexes, and RLS: docs/architecture/database/COMMUNICATION.md.

Pacing: per-pair FIFO, measured at attempt start

A pair is the (business number, recipient number) tuple. The dispatcher claims ONE pair at a time and sends its entries in FIFO order, leaving at least WHATSAPP_PAIR_MIN_SEND_INTERVAL_MS (6500ms) between two sends to the same pair. Meta's published per-pair limit is 1 message / 6 seconds; the extra 500ms is scheduling margin so clock skew cannot land two sends inside one 6-second window. Verified 2026-08-03 against https://developers.facebook.com/documentation/business-messaging/whatsapp/throughput.

The interval is measured from ATTEMPT START, not completion — a send that takes 4 seconds has already consumed most of its own window, and pacing from completion would silently halve throughput. Read receipts ride the recipient's pair for the same reason: a receipt consumes the same per-recipient budget as any other message, so giving receipts their own pair would defeat the pacing.

The pair's recipient key is stored digits-only, canonicalised at the single enqueue chokepoint (enqueueOutboxMessage in services/messageOutbox/base.ts, via normalizeOutboxRecipientKey). The same human otherwise arrives in two notations — Meta's wa_id (digits) on a webhook reply, a +-prefixed stored contact on a notification send — and would split into two pair rows, each with its own send interval, halving the guarantee the pair exists to provide. Only the pacing key is normalised; the payload keeps the caller's original address, which is what the wire call must use.

A claim-less drain is not necessarily an empty channel. When every pending pair is still inside its pacing interval, claim_message_outbox_pair returns outcome = 'none' together with has_more = true and the earliest absolute next_due_at at which a FIFO head becomes workable. The dispatcher publishes a delayed wake-up for the remainder of that already-running interval. This keeps read receipts followed shortly by processing confirmations moving without depending on the every-minute recovery sweep; the sweep remains a crash and publish-failure backstop.

Dispatch order: prepare, then open the attempt, then send

Everything variable-length — payload validation and the template-media re-derivation, which uploads to WhatsApp — runs BEFORE the attempt is opened. start_message_outbox_attempt re-validates the lease, so it is the final pre-wire gate and the POST is issued immediately after it commits. Three properties follow: a claim lost during a slow preparation costs an unsent message rather than a duplicate one; the pacing stamp measures the interval from the send rather than from the start of preparation; and a crash during preparation leaves no open attempt, so the lease sweep requeues it instead of parking it ambiguous. The instant between the attempt's commit and the request leaving the process is irreducible — that residual window is precisely why an interrupted open attempt is recovered as ambiguous.

A preparation failure has no attempt to record against, so one is opened solely to requeue or dead-letter it, which also stamps the pacing gate — a repeatedly failing entry can never spin the drain loop faster than a real send.

Error classification

Every failed attempt is classified from the provider's error code. The classification decides whether the entry is retried, parked, or abandoned — never the HTTP status alone. Codes verified 2026-08-03 against https://developers.facebook.com/documentation/business-messaging/whatsapp/support/error-codes.

Code(s) Meaning Classification
131056 Per-pair throughput exceeded Retryable — backoff, honouring Retry-After
130429 Cloud API throughput limit Retryable — backoff, honouring Retry-After
80007 WABA rate limit Retryable — backoff, honouring Retry-After
2, 4, 131000, 131016, 131057, 133004 Transient Graph/API faults Retryable — ordinary backoff
131064 Quality messaging limit Retryable, but only after a LONG cooldown
131048 Spam/quality account restriction Permanent + sana.whatsapp_account_restricted
368, 130497, 131031 Account-level restrictions Permanent + sana.whatsapp_account_restricted
131026 Recipient undeliverable Permanent
other 4xx Malformed/refused request Permanent
timeout / network Outcome unknown Ambiguous — see below
(no WhatsAppApiError) Local executor fault Permanent — dead-lettered, never ambiguous

The last row is an invariant, not a code: the transport wraps every wire-path failure in a WhatsAppApiError, so a plain Error/TypeError reaching the dispatcher was thrown by local executor code before a request existed. Such a failure provably never reached Meta, so classifying it ambiguous — which asserts the message MAY have been delivered and blocks retries on that basis — would be a lie. It is dead-lettered as permanent with a synthetic { source: 'local_execution', message } body.

Recording a failure is itself retried once: a lost fail_message_outbox_attempt write leaves the row sending until the lease sweep requeues it AND loses the provider's error code. If both attempts fail, the code, HTTP status, classification, outboxId and attemptId are logged at ERROR so the log becomes the durable record the row no longer carries.

Retry backoff is exponential from WHATSAPP_OUTBOX_BACKOFF_BASE_MS (10s), capped at WHATSAPP_OUTBOX_BACKOFF_CAP_MS (15min), over WHATSAPP_OUTBOX_MAX_ATTEMPTS (5) total attempts. Retry-After and the quality cooldown are FLOORS, not alternatives: whichever is longer wins, because retrying before either has elapsed can only burn an attempt.

Ambiguity is resolved by correlation, never by a blind retry

A timed-out or network-failed send has an unknowable outcome: the message may have been delivered. Retrying it could send the user a duplicate, so the entry is parked in the ambiguous state and is NEVER retried automatically.

Resolution comes from the provider instead. Every send stamps the outbox row's UUID into biz_opaque_callback_data, which Meta echoes back on its status webhooks. When a status webhook arrives carrying that token, the webhook handler calls resolveAmbiguousOutboxSend with the token, the wamid, AND the reported status. The STATUS decides the resolution, because correlation alone proves only that Meta knows the send, not that it succeeded:

Reported status Resolution Effect
sent/delivered/read accepted Row settles accepted; the caller owes persistence
failed rejected Row becomes terminally failed

The accepting set is an allow-list (MESSAGE_OUTBOX_ACCEPTING_DELIVERY_STATUSES), so an unknown future status can never be mistaken for acceptance evidence.

resolve_message_outbox_ambiguous reports six outcomes:

  • resolved — settled as accepted. The entry never wrote its messages row, so resolution runs the persistence step, and then re-applies the triggering status to the row it just created: the webhook's own status write ran before that row existed and therefore matched nothing, so without the replay the message would permanently lack the state that caused its own creation.
  • resolved_failed — settled as terminally failed. The sana.reply_delivery_failed event is emitted from the row's stored log_context (the dispatcher that would normally hold those correlation fields is long gone), and any linked notification delivery is marked failed.
  • evidence_recorded — the row was still sending, so the correlation is stored as evidence WITHOUT a state transition. The dispatcher holding the claim remains the only writer of its own outcome; lease recovery reads the evidence if that dispatcher never returns, settling the row accepted rather than parking it ambiguous (counted as resolvedByEvidenceCount).
  • not_ambiguous, not_found, invalid_resolution — nothing to settle, an unknown token, and a caller-contract violation respectively.

An ambiguous entry that is never correlated stays for operator triage and is never pruned.

Dispatch is woken by QStash, drained by cron

The QStash publish after an enqueue is a HINT, not the delivery mechanism: it is best-effort, and a failed publish is logged without failing the enqueue. The outbox row is the source of truth.

For that hint to be genuinely optional, the every-minute sweep cron must be able to drain without QStash — otherwise a QStash outage takes the repair path down alongside the enqueue path it exists to repair, and nothing drains at all. So the sweep does both: it publishes a wake-up (the low-latency path, which gives an idle dispatcher a full MESSAGE_OUTBOX_DISPATCH_TIME_BUDGET_MS run), and it then runs its own bounded in-process drain for MESSAGE_OUTBOX_SWEEP_DRAIN_TIME_BUDGET_MS as the availability backstop. Running both concurrently is safe: the claim RPC takes each pair under SKIP LOCKED with a lease CAS, so two dispatchers can never claim the same pair, and the loser simply finds nothing to do.

The sweep also performs the recovery work no single dispatcher run can: requeueing entries whose claim lease expired (a dispatcher that died mid-send), retrying the message-row persistence step for entries accepted but not yet persisted (after MESSAGE_OUTBOX_PERSISTENCE_GRACE_MS), repairing notification deliveries left pending by a terminally failed entry (the dispatcher marks them inline, so this catches only the runs whose failure write was lost), warning when the oldest due entry exceeds MESSAGE_OUTBOX_PENDING_AGE_ALERT_MS, and pruning terminal entries — accepted after 30 days, failed after 90 (kept longer for triage), along with the pair rows no entry references any more (prunedPairsCount). Each step is independently error-isolated, so a failing step never costs the others their run — including the drain, which still executes when the due-work probe itself failed.

A dispatcher run also owes a follow-up wake-up in two cases no claim-time has_more describes: a requeued send schedules due-soon work on a pair the claim already reported drained, and a run that stopped on its time budget after doing work never made the claim that would have proved the queue empty. Both set workRemains, so neither waits for the next sweep cycle.

Read receipts

Read receipts go through the outbox like everything else, but with maxAttempts: 1: a receipt that failed to land has no lasting value by the time a retry would run. Their key is read:{wamid}, so repeat acknowledgement of the same inbound message collapses onto one entry.

Notification delivery status

A notification delivery stays pending until the dispatcher produces an outcome — the enqueue is not the delivery. On terminal failure the dispatcher marks the delivery failed (best-effort: the outbox row already records the failure, so a failed status write must not mask it).

Idempotency keys

An enqueue is deduplicated on its key: two enqueues with the same key resolve to ONE row. A key derived from the triggering event is what makes a re-driven caller — a Meta webhook redelivery, a re-run QStash job — collapse onto the send it already queued instead of sending the user a second copy.

Send Key
Sana reply reply:{inboundWamid}:{purpose}
Read receipt read:{wamid}
Notification nd:{notificationDeliveryId}

The reply correlate is the triggering inbound message's WhatsApp id, which reaches each call site as CreateMessageData.parent_external_id. It is the provider's id rather than our own row id because it exists before the inbound row is written, and it is exactly what Meta replays on a redelivery. purpose is a slug from MessageOutboxKeyPurpose naming WHICH of the event's sends this is — one inbound message can legitimately produce several distinct replies, so the correlate alone does not identify a send. Keys are built by buildReplyOutboxOptions (utils/messageOutbox.ts), which also stamps the purpose and correlate into the row's log_context.

Where no stable correlate exists in scope, the client falls back to a random UUID: the send is still durable, just not replay-deduplicated. Fabricating a key from something unstable would be worse — it would either fail to collapse a genuine replay or collapse two sends that are legitimately distinct.

Real-time Capabilities

Enabled Tables

  • conversation - Live conversation updates
  • conversation_instance - Instance status changes
  • message - Real-time message delivery
  • notification - Instant notification alerts

Use Cases

  • Live chat in web interface
  • Message status updates (sent → delivered → read)
  • Instant abuse detection alerts
  • Real-time conversation handoff

Key Design Decisions

1. Unified User Context

  • Single user table referenced by all systems
  • Consistent preferences across channels
  • Unified permission and RLS policies

2. Channel-Agnostic Design

  • Generic fields support any channel
  • Easy to add new channels (Telegram, Email)
  • Channel-specific data in JSONB metadata

3. Notification-Conversation Bridge

  • Notifications can start conversations seamlessly
  • Messages can trigger follow-up notifications
  • Bidirectional flow with clear ownership

4. Bounded Instances

  • A conversation lives on one active instance; a message cap and an idle timeout bound how large it grows
  • Automatic summarization preserves history
  • Prevents infinite conversation growth

5. Proactive Abuse Prevention

  • Checks happen before resource consumption
  • Progressive penalties educate users
  • Configurable per channel and user type

6. AI-Ready Architecture

  • Summaries provide historical context
  • Pattern detection improves over time
  • Ready for future AI enhancements

Benefits

  1. Seamless Integration: Notifications and messages work together naturally
  2. Scalable Design: Bounded instances keep per-conversation message volumes from degrading reads and summarization
  3. User Protection: Robust abuse prevention maintains platform quality
  4. Multi-Channel: Single architecture supports all communication channels
  5. Complete Audit Trail: Every interaction is logged and traceable
  6. Real-Time Experience: Instant updates across all interfaces
  7. Flexible Configuration: Rate limits and rules can be adjusted per channel

Future Enhancements

  1. Cross-Channel Continuity: Start on WhatsApp, continue in-app
  2. Smart Routing: AI decides best channel for each notification
  3. Behavioral Learning: Personalized rate limits based on user history
  4. Rich Media: Enhanced support for videos, documents, voice notes
  5. Conversation Intelligence: AI-powered insights and suggestions