Skip to main content

​
What It Does

  • Removes duplicate rows based on one or more selected columns
  • Keeps only the first occurrence of each match
  • Works across text, numeric, and ID fields
  • Outputs a clean dataset — ready for routing, enrichment, or export

​
🏁 Getting Started

Remove Duplicates node config
1

Add the Node

Drag in the Remove Duplicates node to your play.
2

Choose Columns to Deduplicate On

Pick one or more columns — rows with matching values in all these fields will be grouped.
3

Run the Node

The output will include just the first unique row for each match group.

​
Inputs

​
🛠️ Required Field

  • Deduplication Columns (✅)
    Choose one or more columns to identify duplicates.
    Tip: You’ll need to select at least one column — otherwise the node won’t run.

​
Output

Returns a clean version of your data:
  • One row per match group
  • Original column order is preserved
  • All outputs are downloadable in CSV and Parquet formats
Remove Duplicates node output

​
How It Works

  1. Loads the dataset
  2. Treats all selected columns as strings to avoid type mismatch issues
  3. Identifies duplicates using those fields
  4. Keeps only the first row in each group
  5. Outputs a deduplicated dataset

​
🚀 Use Cases

Use CaseDeduplication Columns
Clean up lead list from Nrev AIemail
Remove duplicate form entriesemail, timestamp
Prep for company-level enrichmentcompany_id, domain
Deduplicate Slack exportsuser_id, created_at

​
✨ Pro Tips

Deduplicate on a stable key — like email, deal_id, or company_id — to keep the cleanest version of each record.
Try different column combinations to see what works best for your data — especially if some values are missing or inconsistent.

​
⚠️ Important Notes

Only the first row in each duplicate group is kept — make sure input row order is what you want.
No columns selected = no action. Pick at least one field before running the node.
Be sure your selected columns exist in the input — mismatches will cause errors.

​
🛠 Troubleshooting

SymptomLikely CauseFix
All rows were droppedAll rows matched — try more columnsAdd more fields to deduplication
Node errors on runNo column selectedSelect at least one column
Wrong rows keptInput ordering was unexpectedReorder your data upstream if needed

​
📝 FAQ

Absolutely. A row is only removed if all selected fields match another row.
Yes — the node treats all selected fields as strings when checking for duplicates.
The very first one — based on the original input order.

​
💰 Pricing

The Remove Duplicates node is free — no credits required.

Clean lists, clean plays. Deduplicate with confidence and send only the best rows downstream.