The month the books quietly doubled

A case study in CRM-to-ERP integration — and why "every system said success" is not the same as "the books are right." The pattern is real; the specifics are not: the client is an international nonprofit, and names, dates, tools, and amounts have been changed, rounded, or removed.

The first sign was not an alert. It was a finance reviewer with a spreadsheet, doing the same reconciliation she does after every sync: pull the new receipts from the ERP, line them up against the deposits, VLOOKUP the dollars. This time the dollars came back way higher than expected.

Nothing was red anywhere. Salesforce said every batch had synced. The integration platform showed green runs. The ERP had accepted every single write with a success response. And the books were inflated by a mid-six-figure sum.

We work on the data engineering side of this integration, and this incident became the best masterclass in finance-systems integration we've ever had. Here's the story — and what any team syncing a CRM to an ERP can take from it. (If you want the full technical teardown, that's a separate deep-dive: Anatomy of a Salesforce–ERP sync.)

The setup

The client runs donor management on Salesforce (NPSP) and their finance books on a cloud ERP. Donations arrive through a range of giving platforms and processors, get recorded as payments in Salesforce, and are bundled by bank payout into batches. An integration layer — automation workflows riding on event plumbing — picks up each batch and posts it into the ERP as a receipt: one payout, one batch, one receipt.

Two design details matter for this story:

1. There is no Run button. An operator sets a status field on a batch rule to Sync, and that field flip is the trigger. Everything downstream — the event relay, the workflow, the ERP write, the ID stamped back onto each Salesforce payment — is invisible to the person who started it.
2. Delivery is "at least once." The event pipe had no queue, no deduplication, and no dead-letter handling. If the workflow engine acknowledged slowly, the same event was redelivered tens of seconds later. Under normal load, that never happened. Under normal load.

The gap nobody could see

A handful of the batch rules — the ones covering a major bank's ACH and wire files — were manual. No owner on paper, no calendar entry, no alert if they didn't run. One month, they simply weren't run for a few weeks.

Here's the structural trap: batches only exist after a rule runs. So no "stuck batch" report could ever show batches that were never created. Weeks of donations sat unposted, and the only symptom was that revenue looked quietly lower than usual.

The storm

The team's only deadline lived in habit: last month's transactions posted by the monthly close. Two days before that deadline, an operator did the obvious thing — pushed the whole backlog through in one afternoon. Entirely per procedure, because no procedure said anything about volume.

Several hundred batches fired in one afternoon. The workflow engine slowed under the burst, acknowledgements lagged, and the redelivery clock started: those batches produced roughly half again as many executions. About a quarter of the payouts were written two to four times.

And this is where a second, dormant defect turned duplicates into something worse. The workflow was built as "probe, then create or update": ask the ERP whether a receipt already exists for this payout — create if not, update if so. But the update path added lines instead of replacing them. So when a redelivered event found the receipt its twin had just created, it appended the entire line set again. Amounts doubled in place. No error. No new record. Business dates untouched.

The result: well over a hundred damaged receipts, a mid-six-figure inflation on a mid-seven-figure run of legitimate value — in one afternoon.

View full

The two silences

What makes this incident worth writing about isn't the failure. It's why it was silent, in two distinct ways.

A doubled write is a successful write. Every status check in the chain asked "did the API call fail?" — none asked "is the result right?"

The pipeline checked for failure, not correctness. The ERP accepted every create and every additive update. Green, everywhere, the whole time.

The review process watched new records, not changes. Finance's controls were built around reviewing what newly appeared. An in-place update creates no new record and doesn't touch business dates, so the damage was invisible to the entire review method. The books changed, and no process looked at changes.

Either silence alone is survivable. Together, the only detector left was a human with a spreadsheet — which is exactly what caught it.

The fix: rules first, architecture second

The forensic work matched every damaged receipt one-to-one to a captured execution before the platform's log retention could erase the evidence. Then came two layers of fix.

Interim governance — four rules, each born from a specific failure:

1. Work only from an explicit resync list with a yes/no flag — never infer intent from a prose column in someone else's workbook. (Learned the hard way: during remediation, a handful of payouts were resynced that shouldn't have been, because the signal lived in a column heading only its author understood. Caught within a couple of hours — negligible damage — by rule 3.)
2. Resync a payout only when the ERP is empty for it. A surviving receipt plus a resync equals the additive-update trap all over again.
3. Check the stamp is new. The receipt ID written back to Salesforce must be a brand-new record number. An old number means the run updated something — stop immediately.
4. Pace bulk runs. Small groups, each reaching a terminal state before the next begins. The same integration that broke on hundreds of batches in an afternoon later handled a few dozen batches in under a minute, cleanly — pacing is the difference.

Then the re-architecture: the sync is being rebuilt on a platform with a real queue in front (buffering and dead-lettering instead of blind redelivery), a per-batch concurrency lock, an update path that replaces instead of appends, and a hard refusal to re-run an already-stamped batch. The full before/after is in the architecture deep-dive.

Load-bearing knowledge must live in the artifact that drives the action. Every rule that lived in habit, in a chat thread, or in a spreadsheet column eventually failed exactly where it lived.

What we'd tell any Salesforce–ERP team

• A successful write is not a correct write. If your only checks are API-level, you are blind to the most expensive class of failure. Reconcile amounts across systems, not just statuses.
• Risk scales with concurrency, not dollars. A later, larger wave processed perfectly clean; the storm broke on a smaller run. What mattered was hundreds of events in an afternoon versus a steady trickle.
• Load-bearing knowledge must live in the artifact that drives the action. The deadline lived in habit. The delete-first rule lived in a chat ritual. The resync flag lived in a prose column. Every one of them eventually failed exactly where it lived.

The reviewer's spreadsheet caught what three systems' worth of green checkmarks missed. The goal of good integration architecture isn't to replace her judgment — it's to make sure she's never the only line of defense.

Want the compressed version? The seven hard lessons from this incident are in dgtl Bytes. And if this story rhymes uncomfortably with your own stack, that's what the Integration Spine™ exists for — a governed backbone instead of a truce between systems.

We write about Salesforce, finance systems, and the unglamorous plumbing between them. If you're wrestling with a CRM-to-ERP sync, we'd love to compare notes.