> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nrev.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Remove Duplicates

> Keep only unique rows based on selected columns. Ideal for cleaning up exports, deduping lead lists, or prepping for enrichment — all without losing structure.

## What It Does

* Removes duplicate rows based on one or more selected columns
* Keeps only the **first occurrence** of each match
* Works across text, numeric, and ID fields
* Outputs a clean dataset — ready for routing, enrichment, or export

***

## 🏁 Getting Started

<Frame>
  <img src="https://mintcdn.com/nurturev/mHCimB8YAP--VMGj/images/Remove%20Duplicates%20node%20config.png?fit=max&auto=format&n=mHCimB8YAP--VMGj&q=85&s=6d39ccf17f2704506c93352b321f8ca9" alt="Remove Duplicates node config" style={{ borderRadius: '0.5rem', width: '100%', margin: '1.5rem 0' }} width="1178" height="1664" data-path="images/Remove Duplicates node config.png" />
</Frame>

<Steps>
  <Step title="Add the Node">
    Drag in the **Remove Duplicates** node to your play.
  </Step>

  <Step title="Choose Columns to Deduplicate On">
    Pick one or more columns — rows with matching values in **all** these fields will be grouped.
  </Step>

  <Step title="Run the Node">
    The output will include just the first unique row for each match group.
  </Step>
</Steps>

***

## Inputs

### 🛠️ Required Field

* **Deduplication Columns (✅)**\
  Choose one or more columns to identify duplicates.\
  *Tip: You’ll need to select at least one column — otherwise the node won’t run.*

***

## Output

Returns a clean version of your data:

* One row per match group
* Original column order is preserved
* All outputs are downloadable in CSV and Parquet formats

***

## How It Works

1. Loads the dataset
2. Treats all selected columns as **strings** to avoid type mismatch issues
3. Identifies duplicates using those fields
4. Keeps only the **first** row in each group
5. Outputs a deduplicated dataset

***

## 🚀 Use Cases

| Use Case                          | Deduplication Columns   |
| --------------------------------- | ----------------------- |
| Clean up lead list from Nrev AI   | `email`                 |
| Remove duplicate form entries     | `email`, `timestamp`    |
| Prep for company-level enrichment | `company_id`, `domain`  |
| Deduplicate Slack exports         | `user_id`, `created_at` |

***

## ✨ Pro Tips

<Tip>
  Deduplicate on a stable key — like `email`, `deal_id`, or `company_id` — to keep the cleanest version of each record.
</Tip>

<Tip>
  Try different column combinations to see what works best for your data — especially if some values are missing or inconsistent.
</Tip>

***

## ⚠️ Important Notes

<Warning>
  Only the **first row** in each duplicate group is kept — make sure input row order is what you want.
</Warning>

<Warning>
  No columns selected = no action. Pick at least one field before running the node.
</Warning>

<Warning>
  Be sure your selected columns exist in the input — mismatches will cause errors.
</Warning>

***

## 🛠 Troubleshooting

| Symptom               | Likely Cause                        | Fix                                  |
| --------------------- | ----------------------------------- | ------------------------------------ |
| All rows were dropped | All rows matched — try more columns | Add more fields to deduplication     |
| Node errors on run    | No column selected                  | Select at least one column           |
| Wrong rows kept       | Input ordering was unexpected       | Reorder your data upstream if needed |

***

## 📝 FAQ

<AccordionGroup>
  <Accordion title="Can I use multiple columns?">
    Absolutely. A row is only removed if **all** selected fields match another row.
  </Accordion>

  <Accordion title="Are data types normalized?">
    Yes — the node treats all selected fields as strings when checking for duplicates.
  </Accordion>

  <Accordion title="Which row is kept in duplicates?">
    The very first one — based on the original input order.
  </Accordion>
</AccordionGroup>

***

## 💰 Pricing

> The Remove Duplicates node is **free** — no credits required.

***

<p style={{ fontSize: '1rem', fontWeight: 'bold', marginTop: '1.5rem' }}>
  Clean lists, clean plays. Deduplicate with confidence and send only the best rows downstream.
</p>