Record Matching: Deciding Whether Two Rows Are the Same Person

Record Matching: Deciding Whether Two Rows Are the Same Person

Record matching is the process of deciding whether two rows held in two different systems refer to the same real-world person or company, so they can be merged into a single record rather than kept as duplicates. It is the operation underneath every CRM merge, every list deduplication, every enrichment write-back and every suppression check. The output is a decision with a confidence attached to it, and the entire discipline consists of choosing what evidence counts and where the confidence threshold sits.

The term exists because systems disagree about identity by default. Every application that stores people invents its own primary key, and none of those keys mean anything outside the system that issued them. Bring a CRM export and a sending platform's contact list together and nothing in either file tells you which rows are the same human being. Before teams named this problem they treated it as a spreadsheet chore, and the results were what you would expect: the same buyer contacted twice under two spellings of their company, an enriched job title written onto a namesake, a suppression list that failed to suppress because it was joined on a field that did not line up.

How record matching actually works

There are two families of method, and mature setups run both.

Deterministic matching compares a unique key exactly. An email address, a company domain, a government registration number, an id one system already knows about the other. The answer is binary, it is auditable months later, and it costs nothing to compute. The limitation is that it needs the key to be present and correct in both systems, and the moment it is missing the method has nothing to say.

Probabilistic matching, often called fuzzy matching, scores similarity across several weaker fields at once: first and last name, company string, location, job title, sometimes a normalised phone number. It produces a score rather than a verdict, which means somebody has to decide what score is good enough. In practice that means two numbers rather than one, because a single threshold forces every borderline pair into an automatic decision. Setting an upper bound for automatic merges and a lower bound for automatic rejects leaves a review band in the middle, and the size of that band is a design choice about how much human attention the problem deserves.

DeterministicExact match on a unique key

• Email address, company domain, or a shared system id
• Binary answer, auditable long after the merge
• Costs nothing to compute and needs no tuning
• Silent when the key is missing from either side
• Fails badly on shared mailboxes, where the key is real but the identity is not

ProbabilisticScored similarity across weak fields

• Name, company string, location, title, normalised phone
• Produces a score, so somebody must choose thresholds
• Finds pairs a key-based join structurally cannot
• Needs an upper bound, a lower bound, and a human review band
• Confidence is a property of the model, not of the truth

Which key you match on should be decided by the system you are matching against, not by which field looks richest. Against a sending platform, email is the only reliable key, because those platforms store one record per address and treat the address as the identity. A join built on name plus company string against the same platform will miss on the order of one row in thirty, and the reason is mundane: the same person is stored under a slightly different company string in the two systems, because one came from a form the person filled in, one came from a provider, and neither normalised the value. Those misses are invisible in the output, since a failed join produces no row and no error.

The last mechanical piece is survivorship, meaning the rules that decide which value wins when two records carrying different values for the same field are merged. Common rules are most recently updated, most trusted source, first non-empty, and human-entered beats machine-written. They need to be written down per field rather than set once globally, because the right rule for a phone number and the right rule for a job title are different. The clearest publicly documented worked example of these rules in a live integration is Apollo and HubSpot deduplication, which is worth reading before configuring any two-way link between systems that both believe they own the contact.

How it is measured

Three numbers describe a matching setup, and most teams report only the third.

Precision is the share of automatic merges that were correct, and it can only be established by pulling a sample of merged records and checking them by hand against a source outside both systems. Recall is the share of genuine duplicates the setup actually found, which is harder, because the pairs it missed are by definition not in any list it produced. The usual approximation is to hand-label a small slice of the data completely and treat that slice as ground truth for both figures.

The third number is the duplicate rate in the destination system, which is the one that appears on dashboards because it is trivial to compute. On its own it is misleading in a specific direction: a setup that merges aggressively drives the duplicate rate toward zero while quietly producing exactly the error you least want. A falling duplicate rate with no precision measurement beside it is not evidence of health.

Worth watching alongside those: the size of the review band, and how much of it is ever worked. A band that grows every week is a backlog of unresolved identity that will eventually be cleared by somebody in a hurry, using a bulk action, on rows nobody sampled.

Where the textbook definition breaks

The tidy definition assumes the answer is knowable and that a good enough algorithm will find it. There are two ordinary situations where no threshold resolves anything, because the question itself has no single answer.

The first is one person at two employers. Somebody left a company eight months ago and is now somewhere else, and both rows are correct as records of a moment. Merging them produces a record with the right name, the wrong current employer for half its history, and an email address that may or may not still receive mail. Keeping them apart produces two records for one human, which is also wrong, and which will send that person a message at an address they abandoned. Neither answer is available; only a choice about which failure you would rather have.

The second is two people sharing a mailbox. A generic address such as an office or enquiries mailbox is a real, deliverable, unique key that belongs to a business rather than to a person. Deterministic matching on email, which is the strongest method available, will happily merge two different humans into one record because they both surfaced with the same address. The strongest key is exactly the one that fails here, which is why role mailboxes should be flagged and excluded from person-level matching before any join runs rather than handled afterwards.

The operational point sits underneath both of these, and it is an asymmetry that should shape every threshold you set.

A false positive merges two strangers and is nearly unrecoverable. Once two records collapse into one, the field-level provenance is gone: you cannot tell later which title belonged to which person, the activity histories are interleaved, and the merged row looks perfectly normal. Nothing flags it. It is usually discovered when somebody receives a message addressed to a person they have never heard of.

A false negative creates a duplicate and is cheap to fix later. The two rows sit there, visibly similar, waiting to be found by the next pass. The worst outcome is a person contacted twice, which is a real cost and a recoverable one.

Those two errors are treated as symmetric by most default configurations, and they are not. Bias the thresholds toward not merging, put the borderline band in front of a human, and accept that you will carry more duplicates than a tidier setup would. That is the cheaper error by a wide margin.

Before a matching job is allowed to write anything

• ✓Yes: The match key is the one the destination system actually stores identity under
• ✓Yes: Role and shared mailboxes are flagged and excluded from person-level matching
• ✓Yes: Two thresholds are set, leaving a review band rather than a single cut point
• ✓Yes: Survivorship rules are written per field, not applied globally
• ✓Yes: The pre-merge rows are retained so a bad merge can be unwound
• ✓Yes: The match key and score are stored on the merged record for later audit
• ✕No: A name-plus-company join is used against a system that keys on email
• ✕No: Auto-merge is tuned to minimise duplicates rather than to avoid false merges

What to do with it

Pick the key from the destination. If the system you are reconciling against stores one record per email address, match on email and treat everything else as supporting evidence. If it stores companies, match on domain and never on company name alone, because company name is the field most likely to differ between two systems describing the same organisation.

Keep the source rows. A merge that cannot be unwound is a permanent decision made by a threshold somebody set once, and storing the pre-merge versions turns an unrecoverable error into an annoying one. Store the key and the score that produced each merge alongside it, so a later audit can ask why rather than only what.

Then normalise before you compare, rather than asking the matching logic to absorb formatting differences it was never designed for. Lowercasing, stripping legal suffixes from company names, reducing a domain to its registrable form and dropping mail subaddressing are all cheap, deterministic steps that turn a large share of would-be fuzzy matches into exact ones. Every pair resolved by normalisation is a pair no threshold has to guess about, which is the least glamorous and most reliable improvement available here.

Related terms and guides

Match rate measures how often a provider returns a value at all, which is a different question from whether the value belongs to the row you asked about, and it has its own entry here. CRM enrichment is the workflow where most matching decisions get made in practice, and data hygiene covers the maintenance that stops duplicates accumulating faster than you resolve them.

For the mechanics of resolving a person to an address before any of this can happen, best email finder tools covers the sources and email verification tools covers turning a returned address into a usable one. Waterfall enrichment explains why several sources chained together produce more disagreements to resolve, which is the hidden cost of a deeper chain. And because most matching problems begin with a segment definition too loose to identify anyone precisely, ideal customer profile is a reasonable place to start upstream of all of it.

If you would rather see a campaign run against a list where identity was resolved before anything was sent, see what a first campaign looks like.