Almost every organization I've worked with, regardless of industry, has some version of this problem: the same person or account shows up as several different records across different systems. A CRM has one entry, billing has another, support has a third, and none of them agree on basic details like name spelling or contact info. Ask which one is correct and the honest answer is usually "depends who you ask."

It's tempting to treat this as a data entry problem and move on. It's rarely that simple, and the fixes that only address entry rarely hold.

Why it actually happens

A few causes show up over and over:

  • Systems built independently. Sales, support and billing tools usually get adopted at different times by different teams, with no shared identifier connecting a record in one to a record in another.
  • No canonical key. Without a consistent unique identifier, systems fall back to matching on name, email or phone, all of which vary in format and change over time.
  • Manual entry variance. "Jon Smith," "Jonathan Smith" and "J. Smith" are three strings and one person. Every manual entry point is a chance for that kind of drift.
  • Mergers, migrations and new tools. Every time a new system gets bolted onto the stack, its data arrives with its own identifiers and its own history, disconnected from everything already in place.

Why a one-off matching script doesn't fix it

The instinct is to write a script that fuzzy-matches records across systems and merges the duplicates. It works, once. Then a new record comes in next week through the same broken process that created the original duplicates, and you're right back where you started. A one-time cleanup treats the symptom. The systems that keep producing new duplicates are still doing exactly what they were doing before.

What actually holds up

The fixes that stick tend to have a few things in common:

  • A defined source of truth. One system (or one dataset) is designated as canonical for a given entity, and everything else references it rather than maintaining a competing copy.
  • A real matching strategy, not just a script. Deterministic matching where a reliable shared key exists, probabilistic or fuzzy matching where it doesn't, with a clear, documented threshold for what counts as a match.
  • Validation at the point of entry. Catching a likely duplicate when a record is created is far cheaper than reconciling it after it's propagated into five downstream reports.
  • Ongoing monitoring, not a one-time project. New duplicates should be flagged automatically as they appear, not discovered months later when someone can't reconcile a customer count.
  • An owner. Someone has to be accountable for the matching rules and for what happens when they don't agree, or the whole process quietly decays the same way it did before.

This is integration and governance working together, not one or the other. The technical matching work only holds if there's ownership behind it, and the ownership only means something if there's an actual process enforcing it.

Data integration and identity resolution across systems is core to what I do.

See What I Do