Designing Scalable Real-Time CRM Integrations

This guide is for technical decision-makers and developers building integrations on top of real estate CRMs — specifically Follow Up Boss, which uses a webhook-based event model and a REST API for reads and writes. The patterns here are not FUB-specific: they apply to any CRM with a webhook endpoint and a REST or GraphQL write API. But the examples draw from FUB's concrete capabilities.

If you are looking for a no-code setup guide, see How to Connect Zapier to Follow Up Boss instead. This article assumes you are building or evaluating a custom integration, not a Zapier Zap.

Layer 1: Event ingestion — receiving webhooks reliably

Why does the ingestion layer need to be its own concern?

The worst thing a webhook endpoint can do is take too long to respond. If your endpoint spends 5 seconds calling an AI API or writing to a database before returning a 200, you risk the CRM timing out and retrying — creating a duplicate processing event. The ingestion layer should do exactly two things: validate the payload and enqueue it for async processing. Everything else happens downstream.

Ingestion layer design checklist

• Return 200 within 500ms. Accept the payload immediately; do not block on downstream calls.
• Verify the source. If the CRM provides a signature header or shared secret, validate it before acknowledging. Reject unsigned payloads with a 401 — do not enqueue them.
• Assign a unique event ID. Extract or generate a deterministic ID for each event (the CRM's event ID, or a hash of the payload body). This is your idempotency key for the processing layer.
• Write to a durable queue. Redis with BullMQ, AWS SQS, or Google Pub/Sub all work. The queue must survive a process restart without losing events.
• Log the raw payload. Before enqueueing, write the raw JSON to cold storage (a database table, S3, Cloud Storage). This is your audit trail for debugging and replay.

Layer 2: Async processing — the event worker

What should the event worker do?

The worker picks jobs off the queue and processes them. For a CRM integration, processing typically means:

1. Check idempotency. Has this event ID been processed successfully already? If yes, skip it. This guard prevents duplicate processing on retries.
2. Fetch full context from the CRM API if needed. Webhook payloads are often partial. The FUB webhook for "contact updated" does not include the full contact record — just the changed fields and the contact ID. Fetch the current contact state from the REST API to enrich your processing context.
3. Run business logic. Score the contact, enrich with third-party data, call an AI API, run a compliance scan, compute a derived field.
4. Prepare the write-back payload. Construct the update you want to push back to the CRM.
5. Write back to the CRM. Call the CRM's write API with the result.
6. Mark the event as processed. Update your idempotency store with a success marker and timestamp.

Retry and dead-letter handling

Processing will fail sometimes — the downstream AI API is slow, the CRM write returns a 429 rate limit, your database is momentarily overloaded. Design for this explicitly:

• Use exponential backoff retries: attempt 1 immediately, attempt 2 after 30 seconds, attempt 3 after 2 minutes, attempt 4 after 10 minutes.
• Set a maximum retry count (typically 4–5 attempts). After exhausting retries, move the job to a dead-letter queue.
• Alert on dead-letter queue growth. A rising DLQ is an early warning of a systemic problem before it becomes visible to users.
• Build a replay mechanism. You should be able to take any raw event from your audit log and reprocess it manually without code changes.

Layer 3: Write-back — updating the CRM without conflicts

What makes write-back hard at scale?

The typical pattern is: receive a CRM event → compute a new value → write it back to the CRM. The problem at scale is that events can arrive and be processed out of order. If you receive an "email opened" event and a "call logged" event for the same contact, and they both trigger an Ace Score recomputation, you might write an older score on top of a newer one depending on which worker finishes first.

Conflict resolution patterns

• Last-writer-wins with timestamp comparison. Before writing a field, read the current value and its modification timestamp. Only write if your computed value's timestamp is more recent than what is already in the CRM.
• Optimistic locking. Some APIs support conditional writes (only update if the current version matches a version number you provide). FUB's REST API uses standard HTTP; you can implement a read-then-compare pattern before writing.
• Serializing updates per contact. For contact-level computations, use a per-contact queue (keyed by contact ID) so only one worker processes a given contact at a time. This eliminates race conditions entirely for that contact's writes, at the cost of increased latency under high per-contact event volume.

Schema evolution: handling CRM API changes

What breaks when the CRM adds or renames a field?

CRM vendors periodically rename fields, deprecate endpoints, or change payload structures. Integrations that parse fields by hardcoded string name break silently. Build defensively:

• Access JSON fields with null-safe getters rather than direct property access. A missing field should return a default, not throw an exception.
• Log any field access that returns null when a value was expected. This surfaces schema drift early.
• Version your event processing logic. If the CRM introduces a new payload version, you can run both parsers side-by-side during a migration window.
• Subscribe to the vendor's API changelog or release notes. FUB, like most SaaS CRMs, announces breaking changes with advance notice.

Backpressure: what happens when volume spikes

A portal blast (50 new Zillow leads in 30 seconds), a team migration (10,000 contacts imported in bulk), or a new integration going live can all spike webhook volume by 10–100x in seconds. Your ingestion layer will receive these fine because it just enqueues. The risk is your processing workers falling behind.

Design for backpressure by:

• Monitoring queue depth in real time. Set an alert when queue depth exceeds a threshold (e.g., more than 1,000 pending jobs).
• Scaling workers horizontally. Worker processes should be stateless so you can spin up additional workers when queue depth rises.
• Prioritizing by event type. New lead events (time-sensitive) should be in a high-priority queue ahead of bulk field updates (not time-sensitive).
• Setting a maximum concurrency per CRM API key. Follow Up Boss and most CRMs have rate limits. Your workers should respect these limits at the worker pool level, not just per-request.

Observability: knowing your integration is healthy

What metrics should a real-time CRM integration emit?

Where AI layers fit in this architecture

AI enrichment — scoring, summarization, next-action generation — is naturally a processing layer concern. An AI call is slow (500ms to several seconds), non-deterministic, and should not block event ingestion. It belongs in the async worker, with its own retry budget and timeout, and its output written back to the CRM only after the AI call succeeds.

Follow Up Ace is architected this way. Incoming FUB activity events queue into a processing pipeline that computes Ace Score and Ace Tier fields, then writes them back to the contact record. Agents see updated scores in FUB within seconds of activity — but the ingestion endpoint never blocks waiting for AI results. The write-back is decoupled.

For AI-intensive operations (Ace Lead Summary, Ace Next Action, pipeline health analysis), these run in a separate worker pool with lower priority than the real-time scoring pipeline, ensuring that bulk analysis work does not starve the latency-sensitive scoring path.

See the Ace Trove overview for how these tiers map to the paid Ace Trove fields, or the agentic interface for how the same pipeline powers on-demand conversational queries against CRM data.

Summary: the three-layer architecture

• Ingestion layer — validates, acknowledges within 500ms, enqueues, logs raw payload. Stateless and horizontally scalable.
• Processing layer — async workers with idempotency, retry/DLQ, and prioritized queues. Fetches full context from CRM API when needed.
• Write-back layer — conflict-resistant CRM updates using timestamp comparison or per-contact serialization. Rate-limit-aware at the pool level.

For the operational playbook on top of this architecture — webhook configuration, Zapier augmentation, and alert routing — see How Event-Driven Integration Works in Follow Up Boss and the real-time alerts checklist.