# Duplicate Image Finder: exact and near duplicate images (`examsens.ai/duplicate-image-finder`) Actor

Groups exact and near duplicate images among the image URLs you list, using perceptual hashing (pHash, dHash, aHash) and file checksums. Finds resized, recompressed and grayscale copies and marks the largest copy in each group to keep.

- **URL**: https://apify.com/examsens.ai/duplicate-image-finder.md
- **Developed by:** [Ed Rayyis](https://apify.com/examsens.ai) (community)
- **Categories:** Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.45 / 1,000 image processeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Duplicate Image Finder: exact and near duplicate images

Give it a list of image URLs and it tells you which ones are the same picture. Exact copies are caught by file checksum. Near copies, such as the same photo saved at another size, recompressed, or turned to grayscale, are caught by perceptual hashing, which compares what the image looks like and ignores the bytes.

Matching images are put in numbered groups. In each group, the copy with the most pixels is marked as the one to keep, so you can delete or skip the rest.

No outside service or AI model is used, and there's no key to supply.

### How matching works

Each image is downloaded once, turned upright from its EXIF orientation, flattened onto white where it's transparent, and reduced to a 64 bit pHash: the image is shrunk to 32 by 32 gray pixels and the low frequencies of its discrete cosine transform are compared with their median. Two images are grouped when their pHash values differ in no more than the number of bits you set (10 by default) or when their files are byte for byte identical. Groups chain, so if A matches B and B matches C, all three share a group.

In testing, the same photo at 400 by 300 and at 800 by 600, a grayscale copy, a blurred copy and a second URL serving the identical file all hashed to a distance of 0 from each other. Four unrelated photos sat 28 to 36 bits apart. A square crop of the 400 by 300 photo fell outside the default distance and was not grouped, so heavy crops need a higher setting or won't match at all.

The dHash and aHash of every image are returned too, for anyone who wants to run their own comparison.

### Input

- Image URLs (required). Direct links to JPEG, PNG, WebP, GIF, BMP, TIFF and other formats Pillow reads, up to 20,000 in one run. Repeated URLs are read once. For an animated image, the first frame is compared.
- Maximum hash distance, 10 by default, from 0 to 32.
- Only return images that have a duplicate, off by default.
- Maximum file size, 25 MB by default.
- Parallel downloads, 5 by default.

### Output

One row per image:

- `url`, `finalUrl` after redirects, `ok`
- `groupId` and `groupSize`, or null and 1 when the image has no match
- `isDuplicate`, `keep` and `keepUrl`, the copy with the most pixels in the group (the larger file breaks a tie)
- `closestMatchUrl` and `closestMatchDistance`
- `matches`: up to 20 matching images, closest first, each with `url`, `distance`, `similarity` from 0 to 1 and `sameFile`
- `exactDuplicate`, true when another image in the run is the same file
- `width`, `height`, `megapixels`, `format`, `fileSizeBytes`, `sha256`
- `phash`, `dhash` and `ahash` as 16 digit hex strings

A link that fails to download, answers with an error status, or isn't an image gets a row with `ok` false and the reason in `error`, and the rest of the run carries on. The `OUTPUT` record in the key value store lists every group with its URLs, plus counts of images processed and failed.

### Pricing

Pay per event: one event per image processed. Images that fail aren't charged. Images left out by the "only duplicates" option were still compared and are still charged. Before an image is downloaded, the run checks that the maximum charge you set can still pay for it; when it can't, the run stops there, and every image row in the dataset is a paid one.

### Support

Questions and problems: the Issues tab on this page. Replies are written by an AI assistant on behalf of the developer and reviewed by the developer.

# Actor input Schema

## `imageUrls` (type: `array`):

Direct links to image files (JPEG, PNG, WebP, GIF, BMP, TIFF and other formats Pillow reads). Every image is compared with every other image in the same run. Up to 20,000 per run.

## `maxDistance` (type: `integer`):

Two images are grouped when their 64 bit pHash values differ in this many bits or fewer. 0 groups only images that hash the same; 10 catches resized, recompressed and grayscale copies. Raise it to catch looser matches, with more false matches.

## `onlyDuplicates` (type: `boolean`):

Leave images with no match out of the dataset. They are still compared and still count as processed.

## `maxFileSizeMb` (type: `integer`):

Skip images larger than this, with an error item.

## `maxConcurrency` (type: `integer`):

How many images to download at the same time.

## Actor input object example

```json
{
  "imageUrls": [
    {
      "url": "https://picsum.photos/id/237/400/300"
    },
    {
      "url": "https://picsum.photos/id/237/800/600"
    },
    {
      "url": "https://picsum.photos/id/237/400/300?grayscale"
    },
    {
      "url": "https://picsum.photos/id/1025/400/300"
    }
  ],
  "maxDistance": 10,
  "onlyDuplicates": false,
  "maxFileSizeMb": 25,
  "maxConcurrency": 5
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "imageUrls": [
        {
            "url": "https://picsum.photos/id/237/400/300"
        },
        {
            "url": "https://picsum.photos/id/237/800/600"
        },
        {
            "url": "https://picsum.photos/id/237/400/300?grayscale"
        },
        {
            "url": "https://picsum.photos/id/1025/400/300"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("examsens.ai/duplicate-image-finder").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "imageUrls": [
        { "url": "https://picsum.photos/id/237/400/300" },
        { "url": "https://picsum.photos/id/237/800/600" },
        { "url": "https://picsum.photos/id/237/400/300?grayscale" },
        { "url": "https://picsum.photos/id/1025/400/300" },
    ] }

# Run the Actor and wait for it to finish
run = client.actor("examsens.ai/duplicate-image-finder").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "imageUrls": [
    {
      "url": "https://picsum.photos/id/237/400/300"
    },
    {
      "url": "https://picsum.photos/id/237/800/600"
    },
    {
      "url": "https://picsum.photos/id/237/400/300?grayscale"
    },
    {
      "url": "https://picsum.photos/id/1025/400/300"
    }
  ]
}' |
apify call examsens.ai/duplicate-image-finder --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,examsens.ai/duplicate-image-finder"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/wICpmnP60Se5uauHP/builds/PTcDtIxdbpuCvgyQu/openapi.json