Data Hygiene: The Maintenance That Keeps a CRM Worth Querying

Data Hygiene: The Maintenance That Keeps a CRM Worth Querying

Data hygiene is the ongoing practice of keeping the records in a CRM accurate, deduplicated and consistently formatted enough that a segment query returns what you meant it to return. It covers duplicates, stale values, malformed and junk entries, inconsistent field formats, and the question of who owns a record and is therefore responsible for it. Email-list hygiene, meaning verification, bounce handling, suppression lists and unsubscribes, is a separate discipline with its own entries in this glossary; everything below is about the records themselves.

The term exists because nothing else in a sales stack owns the problem. Enrichment fills fields, matching resolves identity, reporting reads whatever is there, and every one of those processes assumes somebody upstream made the values usable. Nobody did. Hygiene is the name for the work that no tool is naturally responsible for, and giving it a name is most of what makes it get scheduled rather than performed in a panic before a quarterly review.

How data hygiene actually works

Four defect classes are worth naming separately, because each needs a different detection method and a different fix.

Duplicates. The same person or company stored twice, usually because two systems wrote in without a shared key, or because a form submission created a new record rather than updating an existing one. Detection is the record matching problem, which has its own entry here, and the fix is a merge governed by survivorship rules.

Stale values. A field that was true when it was written and is no longer true. Detection requires an external source, because nothing internal knows. Why this happens, and how fast, is covered by data decay, also in this glossary.

Malformed and junk values. A company field holding an employment status such as "Freelance" or "Self-employed" rather than a company. A first-name field holding a role token like "Info" or "Sales", pulled from a shared mailbox. A truncated value with an unclosed bracket. Test entries somebody created and never removed. These are the most dangerous class in an outbound context, because they interpolate directly into copy: a merge field holding an employment status renders a sentence addressed to a company that does not exist, and the recipient sees it. Detection is pattern-based and cheap. The reason these survive is that they are individually rare, so no percentage-based quality check ever notices them.

Inconsistent enumerations. One industry stored under six spellings, a country field carrying four representations of the same nation, job levels that mean different things depending on who typed them. Nothing here is wrong exactly, which is why it persists, and every one of them breaks a filter. A segment query for one spelling silently excludes the rows carrying the other five.

1. Step 1Define the field contract

   For the fields your real queries filter on, state the allowed shape and values

2. Step 2Constrain at entry

   Picklists rather than free text, validation on write, no silent acceptance of anything

3. Step 3Detect periodically

   Pattern checks for junk, matching passes for duplicates, external checks for staleness

4. Step 4Assign an owner

   A record nobody owns is a record nobody fixes, whatever the report says

5. Step 5Report with denominators

   Every defect count carries the population it was measured over

The structural choice underneath all four classes is where the work happens. Validation at the point of entry is cheap and preventive: a picklist instead of a free-text field costs one configuration decision and eliminates an entire enumeration problem permanently. Periodic cleanup is expensive and merely detective: it finds defects after they have been queried against, reported on and acted upon.

The catch is that entry validation is only as good as the field contract behind it, and a contract that is too strict gets routed around. Required fields with no legitimate value produce records where somebody typed a full stop to get past the form. That is worse than an empty field, because an empty field is honest.

Ownership is the last piece and the one most often left implicit. Every record needs a named owner, because a defect report that lands in a shared queue with no assignee is a document rather than a task. Where two systems both believe they own the contact, the rules that decide precedence matter enormously, and Apollo and HubSpot deduplication is the clearest worked example of what those rules look like when they are written down properly.

How it is measured

There is no single hygiene score worth reporting, and attempts to build one usually produce a number that moves for reasons nobody can trace. What works better is a small set of counts, each tied to a defect class and each reported against the population it was measured over.

Duplicate rate is the share of records that have at least one probable match elsewhere in the table, which requires a matching pass rather than a query. Completeness is the share of records holding a value in each field your live segments filter on, reported per field rather than averaged, because an average across forty fields hides the two that matter. Conformance is the share of values in a constrained field that sit inside the allowed set, which is the number that catches enumeration drift before a filter starts silently excluding rows.

None of those three says anything about whether a value is true. Accuracy still needs an external source and a manual sample, and no internal report can substitute for it. A hygiene dashboard showing perfect completeness and conformance on a table full of stale values is entirely possible and entirely uninformative about whether the segment is real.

Where the textbook definition breaks

Hygiene gets written about as a tidiness virtue, something between good housekeeping and professional pride. It is a query-integrity problem, and framing it as tidiness is why it loses every prioritisation argument it ever enters.

The failure mode is what makes it serious. Dirty data does not crash anything. A segment query runs against a table with inconsistent industry spellings and it succeeds: no error, no warning, a result set of a plausible size that somebody exports and sends to. The rows that were excluded are invisible, because a filter that excludes a row produces nothing to look at. Everyone downstream treats the output as the segment, and the only signal that anything went wrong is a campaign performing oddly against a population nobody can reconstruct afterwards.

Records in the CRM

The population you believe you are querying

Records the filter returned

Excludes anything stored under a different spelling of the same value

Records where the filter field is correct

Stale and junk values pass the filter without being true

Records genuinely in the segment

The set you meant, which nothing in the export tells you the size of

The second break is sharper and catches experienced teams. A hygiene metric computed over already-filtered rows reports on the survivors, so a batch where every row was rejected upstream can score a perfect quality rate.

Follow the arithmetic. Quality flags are usually raised against the rows that passed the earlier gates, because those are the rows still in play. If an upstream step rejects everything, for instance because a required column arrived blank and every row failed validation on it, then the set carrying flags is empty and the set being measured is empty. Zero flags out of zero rows is a flawless score. The report reads green. Every dashboard reads green. The batch is worthless, and the number that exists specifically to catch a worthless batch has certified it.

The same shape appears anywhere a rate is computed over an already-filtered population, which in practice is almost everywhere: match rates, quality rates, pass rates, coverage figures. Assert the denominator before reading any quality percentage. Print the input row count next to the accepted row count next to the flag count, and treat a rate whose denominator is zero, or whose denominator you cannot name, as a failure rather than as a pass. A check that cannot see anything must report that it could not see, because a guard that returns "fine" when it has nothing to look at is not a guard.

What to do with it

Start from the queries you actually run. Hygiene applied evenly across every field in a CRM is unbounded work with no completion condition, and it competes with everything else on the roadmap forever. Hygiene applied to the six fields your live segment definitions filter on is a finite job with a visible payoff, and it is usually a small fraction of the table.

Move as much as possible to the point of entry for exactly those fields. Picklists over free text, validation on write, and a deliberate decision about what happens when a value does not fit rather than silent acceptance.

Add one pattern check specifically for the junk class, and run it at the boundary where records leave the CRM for a campaign rather than only inside the CRM. Employment statuses sitting in a company field, role tokens sitting in a first-name field, unclosed brackets from a truncated value: each of these is individually rare enough to be invisible in any rate, and each one renders into copy a real person reads. A check for them costs almost nothing and is the only defence, because no threshold set on a percentage will ever fire on a handful of rows.

Then report every defect count with its denominator attached, permanently, in the report format itself rather than as something a careful reader works out. That single habit is what separates a quality number you can act on from one that flatters whatever population happened to survive the step before it.

Related terms and guides

Record matching handles the duplicates, data decay explains the stale values, and CRM enrichment is the process that both repairs fields and, run carelessly, introduces new defects at scale. All three have their own entries here.

Upstream, the quality bar you need is set by what you query, so ideal customer profile is where the field contract really originates: a segment defined in fields your CRM does not hold cleanly is a hygiene problem disguised as a targeting problem. On the inbound side, LinkedIn CRM covers what actually flows between a professional network and your records, which is a common source of enumeration drift. Waterfall enrichment explains why chaining several sources produces more disagreements to reconcile, and therefore more hygiene work rather than less. And for a smaller organisation deciding how much of this to build before it is needed, startup lead generation is a reasonable calibration of what is worth doing early.

If you would rather see a campaign built against a segment defined in fields that were checked before anything sent, see what a first campaign looks like.