On this page
Key takeaways
- Deduplicate people by stable identity, not by name and company.
- Preserve batch provenance instead of deleting rows blindly.
- A missing row is not proof that a person left a company.
What is cross-export deduplication?
Cross-export deduplication identifies the same lead in multiple Sales Navigator exports while retaining where and when that lead appeared.
Within one search, adjacent pages or split segments can overlap. Across days, the same person may appear in a saved list, a fresh search and a teammate's territory. Deleting every repeated row removes context: a prospect can belong to several relevant segments. Keep one person record and a separate membership or observation record for each source.
Normalize public LinkedIn URLs by removing tracking parameters and trailing slashes before matching. When a source-specific stable ID is available, retain it too. Do not key on fullName plus company: names collide and people change jobs. If a URL is missing, flag the row for review instead of silently merging it into someone else.
Three kinds of repeat rows
The right action depends on why the duplicate exists.
- Same run, same person twice: one row should be retained, with the duplicate count logged.
- Different lists, same person: keep one identity plus both list memberships for routing and reporting.
- Different dates, changed title or company: keep the observation history and update the current view only after review.
A small data model that survives repeat runs
Create a `person_key` from the normalized profile URL or stable ID. Store `batch_id`, `source_url`, `source_type`, `exported_at`, `title` and `company` on every observation. A current-leads table may choose the latest observation, but keep the older ones for audit. Your outreach queue should reference `person_key`, not row number, because row numbers change whenever a CSV is sorted.
Scrapigator's 'Only new' mode can avoid rows already exported by the account when the goal is a fresh prospecting list. A full export is still useful for periodic snapshots, list membership checks and changed-role detection. Use a distinct decision for each job instead of treating one toggle as a universal deduplication policy.
| Goal | Export mode | Keep |
|---|---|---|
| Fresh outreach list | Only new | Person key, first-seen date |
| Detect changed roles | Full snapshot | Current and prior title/company |
| Audit list membership | Full snapshot | Every source list and batch ID |
Two tables are better than deleting rows
Create a `people` table with one current row per person key and an `observations` table with one row per appearance in an export. An observation stores the batch, source search or list, timestamp, title and company as seen then. If the same person appears in two regions, you want one person in the outreach queue and two source memberships for audit. Deleting the second raw row throws away that information.
Use a stable LinkedIn person ID when your exporter supplies one. Otherwise, normalize the public profile URL by removing query parameters and a trailing slash. Keep a review bucket for missing URLs and for URLs that point only to a Sales Navigator lead page. Name + company is a weak fallback: names collide, companies rename, and people change jobs. Never silently merge two weak matches.
| Input A | Input B | Decision |
|---|---|---|
| Same stable person ID | Different titles | One person, two observations; review current role |
| Same normalized public URL | Different list names | One person, keep both memberships |
| Same name and company | Different profile URLs | Do not auto-merge; inspect |
| Missing URL | Same name only | Manual review or leave separate |
Download the lead observation template (CSV template).
Reconcile a weekly refresh without repeat outreach
Imagine Monday's export has 500 observations and Friday's has 530. Join both by person key. You might find 40 newly seen people, 20 missing from the latest snapshot, 15 changed titles and 490 unchanged people. Those categories are more useful than the raw difference of 30 rows. `Missing from latest` is not proof that a person left a company: filters, visibility and source membership may have changed.
Your action queue should contain an `already_contacted` or CRM record key. Only a new person who is qualified and unsuppressed should enter outreach. If you need a full membership snapshot, run a full export; an `Only new` exporter setting intentionally omits previous people, so it cannot prove who disappeared.
Use this checklist
- Normalize and validate keys before counting unique people.
- Preserve every source membership and export date.
- Keep uncertain matches for manual review.
- Compare full snapshots only with other full snapshots.
Deduplicate two exports safely
A 100-row sample is enough to test the key before processing a larger history.
- 1
Preserve the two raw files
Name each file with the source and export date. Never edit the only copy of a raw export.
- 2
Normalize identifiers
Prefer profile ID or public URL. Strip query parameters and normalize case where appropriate.
- 3
Create a membership table
Record which search or list produced each key, including batch date. Do not collapse this into one source column.
- 4
Build a current view
Select one row per person and mark changed title or company for review.
- 5
Gate outreach
Join the current view to CRM contacts and prior sends before assigning a lead to a sequence.
Dedupe rules that backfire
- Deduplicating on name alone or name plus company.
- Deleting all but the latest row and losing which lists the person belonged to.
- Treating absence from an incomplete later export as a deletion or a job change.
The next step
The output should answer two different questions: 'Who is this person now?' and 'Where did we see them?' If your Sheet or CRM can answer both, repeat exports become useful research instead of clutter.
Continue with the Google Sheets workflow, the saved search versus list explainer, the search-limit guide.
Questions
What is the best key for deduplicating Sales Navigator leads?
Use a stable profile ID or normalized public profile URL when available. Name and company alone are unreliable.
Should I remove a lead found in two lists?
Keep one person identity but preserve both list memberships. They may explain different campaigns or owners.
What does 'Only new' mean?
It suppresses rows already seen in prior exports for the relevant account or workflow. Use a full snapshot when you need to compare changes over time.
Can I infer that a missing lead left the company?
No. A changed filter, partial run or search result cap can also explain the absence.
How do I stop duplicate outreach?
Match normalized lead identity against CRM records and prior sends before a new sequence enrollment, not only when importing the CSV.
Export your next Sales Navigator search
500 rows a month free. Pro is $10/month with unlimited exports and a 7-day money-back guarantee.
