# Duplicate Image Finder (`plainpipe/image-duplicate-finder`) Actor

Find duplicate and near-duplicate images in image URLs or an upload by perceptual hash. Use when you need to dedupe an image set or spot resized copies.

- **URL**: https://apify.com/plainpipe/image-duplicate-finder.md
- **Developed by:** [Plain Pipe](https://apify.com/plainpipe) (community)
- **Categories:** Developer tools, Automation
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$0.40 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Duplicate Image Finder

Find duplicate and near-duplicate images among your image links or an upload with perceptual hashes (dHash and aHash): one row per image with its hashes, group and closest match.

**Use this when:** you have a list of image links and need to know which ones show the same picture, even at a different size or JPEG quality.

### Why this one

- $0.40 per 1,000 results, one result per image; links that fail or are not images are not charged.
- Each row carries both 64-bit hashes (`dHash`, `aHash`), the per-hash Hamming distances, a `groupId`/`groupSize` cluster and the closest earlier image, so you can re-threshold without re-running.
- Pure JavaScript hashing: no browser, no proxy, no external service. The prefilled example (3 images) finishes in under 4 seconds in our test runs.
- Checked before each release against the prefilled input plus 7 edge-case inputs (redirects, SVG and 404 links, bad thresholds).

### What it does

Give it a list of image links (and optionally one uploaded file). It fetches each image, computes two 64-bit perceptual hashes with jimp, a pure JavaScript image library, and compares them. Images that look the same, such as one picture saved at different sizes or re-compressed as JPEG, get the same group ID even when their files differ byte for byte.

The two hashes are computed by the Actor itself:

- **dHash**: the image is scaled to 9x8 grayscale and each bit records whether a pixel is darker than its right neighbour.
- **aHash**: the image is scaled to 8x8 grayscale and each bit records whether a pixel is brighter than the mean.

Two images match when both hashes differ in at most `threshold` bits (Hamming distance).

### Use cases

- Clean up a media library: find the same picture stored at several sizes or re-saved as JPEG.
- Check a product catalog for repeated photos across different items.
- Flag re-uploads of an image before they go live.
- Find all copies of one reference image in a list (`againstFirst` mode).
- Deduplicate an image set before you use it for testing or training.

### How to use it

1. Paste image links into **Image URLs**, one per line, or upload a file in **Upload an image**. You need at least two images.
2. Under Matching settings, optionally set **Threshold** (default 6; lower is stricter) and **Compare mode**.
3. Run the Actor. Results land in the dataset; export them as JSON, CSV or Excel, or read them through the API.
4. Sort or filter by `groupId` to see each cluster of near-duplicates, or filter on `isDuplicate: true` to list the copies only.

### Input

| Field | What it does | Default | Example |
|---|---|---|---|
| `imageUrls` | Direct links to PNG, JPEG, GIF, BMP or TIFF files, one per line (2 to 500). Entries that are not http(s) links fail the run before any image is fetched. | none | `["https://www.w3.org/Graphics/PNG/nurbcup2si.png", "..."]` |
| `imageFile` | One uploaded image, listed after the URLs. | none | upload in the form |
| `threshold` | Maximum Hamming distance (0-64 bits) for two images to count as near-duplicates. | `6` | `3` |
| `compareMode` | `all-pairs` compares each image with every earlier image and groups chains of matches; `againstFirst` compares each image only with the first one. | `all-pairs` | `"againstFirst"` |
| `maxItems` | Stop after this many result rows. | `100` | `500` |

### Output

One row per image that could be fetched and decoded, in input order. Example from a real run (prefill input: the same photo at 250 px and 500 px wide, plus an unrelated PNG); this is the 500 px copy:

```json
{
  "source": "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Example.jpg/500px-Example.jpg",
  "width": 500,
  "height": 517,
  "format": "jpeg",
  "sizeBytes": 20241,
  "dHash": "40682c1b0f070101",
  "aHash": "ff2f87c3e1f1f9ff",
  "groupId": 1,
  "groupSize": 2,
  "isDuplicate": true,
  "duplicateOf": "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Example.jpg/250px-Example.jpg",
  "distance": 0,
  "dHashDistance": 0,
  "aHashDistance": 0,
  "closestEarlier": "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Example.jpg/250px-Example.jpg"
}
```

| Field | Meaning |
|---|---|
| `dHash`, `aHash` | 16-character hex strings (64 bits each). |
| `groupId` / `groupSize` | Images in the same cluster of near-duplicates share this number; groups are numbered in order of first appearance. `groupSize` is how many images the group holds. |
| `closestEarlier` / `distance` | The earlier image (among those compared) with the smallest distance, and that distance: the larger of `dHashDistance` and `aHashDistance`. `null` for the first image. |
| `duplicateOf` / `isDuplicate` | Set to `closestEarlier` only when `distance` is within the threshold, otherwise `null`; `isDuplicate` says the same as a boolean. |

### Pricing

Pay per result: **$0.40 per 1,000 results** ($0.0004 per result). One result is one image row. You only pay for results the Actor returns. Set a maximum total charge on the run to cap your spend.

### Limits

- At most 500 images per run and 20 MB per file. Links must point straight at the image; the file signature must be PNG, JPEG, GIF, BMP or TIFF. WebP, AVIF, HEIC and SVG are not supported.
- Perceptual hashes catch resized, re-compressed and lightly edited copies. They do not reliably catch crops, rotations, mirrored copies, heavy filters or large overlays, and very simple images (flat colours, plain logos) can match each other by chance. Lower the threshold if you see false matches.
- Transparent pixels are treated as white. Animated GIFs are compared by their first frame.
- In `all-pairs` mode groups are chained: if A matches B and B matches C, all three share a group even when A and C are further apart than the threshold.
- Files that cannot be fetched or decoded are skipped with a warning and not charged. If fewer than two images can be decoded, the run fails with no results and no charge.

### FAQ

**How much does it cost?** $0.0004 per image row, so 1,000 images cost $0.40. Links that fail or are not images are skipped and not charged.

**Which threshold should I use?** Start with the default 6. If unrelated images match, lower it (3 or less); if resized copies are missed, raise it a little (8-10). 0 matches identical hashes only.

**Why was an image skipped?** The run log gives the reason: an HTTP error, a link to a web page instead of the file, an unsupported format (WebP, SVG) or a file above 20 MB.

**Does it find byte-for-byte identical files only?** No. It compares how the images look, so a resized or re-compressed copy matches too. Identical files always have distance 0.

**Can I use it from my own code?** Yes. Start the Actor through the Apify API with a JSON input and read the dataset; group by `groupId`.

### Legal and data

This Actor processes only what you give it or public, non-personal technical data. It does not log in anywhere, does not collect personal data, and does not bypass paywalls, CAPTCHAs or access controls. You are responsible for having the right to process the inputs you provide.

### Support

Open an issue on the Issues tab. Issues are answered within 14 days.

Built and maintained by Plain Pipe.

# Actor input Schema

## `imageUrls` (type: `array`):

Direct links to image files (PNG, JPEG, GIF, BMP or TIFF), one per line, for example https://www.w3.org/Graphics/PNG/nurbcup2si.png. At least two, up to 500 images per run, 20 MB each. Each image that can be decoded gives one result row; broken links are skipped and not charged.

## `imageFile` (type: `string`):

Or upload one image file here (PNG, JPEG, GIF, BMP or TIFF), for example photo.jpg. It is compared together with the URLs above and listed after them. Optional.

## `threshold` (type: `integer`):

Two images count as near-duplicates when both their dHash and aHash differ in at most this many of 64 bits. 0 = identical hashes only. Default 6; for example 3 for stricter matching or 10 for looser.

## `compareMode` (type: `string`):

all-pairs (default) compares every image with every earlier image and groups chains of matches. againstFirst compares each image only with the first one, for example to find copies of one reference image.

## `maxItems` (type: `integer`):

Stop after this many result rows (one row per image, one billable result each). Default 100; for example 500 for a full run.

## Actor input object example

```json
{
  "imageUrls": [
    "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Example.jpg/250px-Example.jpg",
    "https://www.w3.org/Graphics/PNG/nurbcup2si.png",
    "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Example.jpg/500px-Example.jpg"
  ],
  "threshold": 6,
  "compareMode": "all-pairs",
  "maxItems": 10
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "imageUrls": [
        "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Example.jpg/250px-Example.jpg",
        "https://www.w3.org/Graphics/PNG/nurbcup2si.png",
        "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Example.jpg/500px-Example.jpg"
    ],
    "maxItems": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("plainpipe/image-duplicate-finder").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "imageUrls": [
        "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Example.jpg/250px-Example.jpg",
        "https://www.w3.org/Graphics/PNG/nurbcup2si.png",
        "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Example.jpg/500px-Example.jpg",
    ],
    "maxItems": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("plainpipe/image-duplicate-finder").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "imageUrls": [
    "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Example.jpg/250px-Example.jpg",
    "https://www.w3.org/Graphics/PNG/nurbcup2si.png",
    "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Example.jpg/500px-Example.jpg"
  ],
  "maxItems": 10
}' |
apify call plainpipe/image-duplicate-finder --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,plainpipe/image-duplicate-finder"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/TDBl83nKTVPdQp71j/builds/Zoso1DeJFJ9VeIo8y/openapi.json