# Bulk Image Scraper — Real Dimension & File Size Filter (`codeclouds/bulk-image-scraper`) Actor

Crawl a website (or list of URLs) and collect every image, filtered by the REAL decoded pixel width/height and file size — not just the declared HTML attribute. Detects true format via magic bytes and dedupes by content hash. Optional full download to the key-value store.

- **URL**: https://apify.com/codeclouds/bulk-image-scraper.md
- **Developed by:** [Dennis](https://apify.com/codeclouds) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 image-results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Bulk Image Scraper — Real Dimension & File Size Filter

Crawl a website (or a list of URLs) and collect every image it links to — but unlike most bulk
image scrapers, this Actor filters on the **real, decoded pixel width/height and file size** of
each image, not on whatever the page's HTML claims. It also detects the true image format from
its magic bytes (catches a `.jpg` URL that is secretly serving WebP) and deduplicates by content
hash, so the same picture served through two different CDN URLs is only counted once.

### Why this Actor is different

Most bulk image scrapers on the Store either crawl a site for image URLs (no download, no real
dimensions) or download everything into a ZIP (no filtering, no crawling). Neither downloads an
image just to verify its **actual** resolution or file size — they trust the `width`/`height`
HTML attributes, which are frequently missing, wrong, or stale (lazy-loading placeholders,
CDN-resized `srcset` variants, responsive images). This Actor combines site-crawling, optional
full download, and a lightweight header-probe step (reads only the first bytes of an image, no
full download needed just to learn its resolution/format) so you can reliably ask for "only
images at least 1200px wide" or "only files under 500KB" and get results that are actually true.

### When should an AI agent use this?

- "Find every product photo on this webshop category page that's at least 800x800 pixels."
- "Download all the hero/banner images from this website, deduplicated, into a key-value store."
- "List every image on this page along with its real resolution and file size."
- "Collect CSS background images from this landing page above 1200px wide."
- "Check whether any image on this page is actually a different format than its URL extension suggests."
- "Build an image dataset from this site, but skip anything smaller than 400x300 or under 10KB."

### What this Actor does

1. Fetches each start URL and, if `maxCrawlDepth` > 0, follows same-host links up to that depth
   (breadth-first), respecting `robots.txt` for every internally discovered link.
2. Extracts image candidates from each page: `<img src>`, `<picture><source srcset>`,
   `<img srcset>` (responsive variants, optional), and CSS `background-image: url(...)` inline
   styles (optional) — resolving every relative/protocol-relative URL against the page it was
   found on.
3. Filters candidates by URL extension first (cheap), then probes the remaining header bytes of
   each image with a lightweight library to learn its **real** width, height, true format, and —
   when the server sends a `Content-Length` header — its exact file size, all without downloading
   the full image.
4. Applies your width/height/file-size range filters using those real, verified values (never the
   HTML attribute).
5. Downloads the full image bytes only when you set `downloadFiles: true`, or when a file-size
   filter is active and the server didn't send a reliable `Content-Length` — the common
   metadata-only case never needs a full download.
6. Computes a SHA-256 content hash whenever the full bytes were fetched, and deduplicates results
   by that hash across the whole run (catches the same image served from two different URLs), in
   addition to always deduplicating by exact URL.
7. Pushes one dataset record per surviving image, and — with `downloadFiles: true` — saves the
   actual file to the run's key-value store.

### Input

| Field | Type | Description |
|---|---|---|
| `startUrls` | array of strings | One or more pages to scan for images. Example: `["https://example.com/gallery"]` |
| `maxCrawlDepth` | integer | Link-hops to follow from each start URL. `0` = only the start URL itself. Default `1`. |
| `sameHostOnly` | boolean | Only follow links on the same hostname as the start URL. Default `true`. |
| `imageExtensions` | array of strings | Only collect images whose URL ends in one of these extensions. Default: all common formats. |
| `minWidth` / `maxWidth` | integer | Real decoded pixel width range. Optional. |
| `minHeight` / `maxHeight` | integer | Real decoded pixel height range. Optional. |
| `minFileSizeKb` / `maxFileSizeKb` | integer | Real file size range in kilobytes. Optional. |
| `includeBackgroundImages` | boolean | Also collect CSS `background-image: url(...)` references. Default `false`. |
| `includeSrcset` | boolean | Also collect every URL in a `srcset` attribute, not just the main `src`. Default `true`. |
| `downloadFiles` | boolean | Download the full image bytes into the key-value store, not just metadata. Default `false`. |
| `maxImagesPerPage` | integer | Safety cap per page. Default `200`. |
| `maxTotalImages` | integer | Safety cap for the whole run. Default `1000`. |

### Output

One dataset item per surviving image:

```json
{
  "imageUrl": "https://example.com/photos/product-42.jpg",
  "sourcePageUrl": "https://example.com/catalog/widgets",
  "discoveryMethod": "img",
  "extension": "jpg",
  "detectedFormat": "jpg",
  "width": 1600,
  "height": 1200,
  "fileSizeBytes": 284113,
  "contentHash": "1f3a9c...e02b",
  "altText": "Blue widget, front view",
  "downloadedFileKey": null
}
```

- `discoveryMethod` is one of `img`, `picture`, `srcset`, `css-background`.
- `detectedFormat` comes from the image's actual header bytes and can differ from `extension`
  when a URL's extension doesn't match what the server actually serves.
- `contentHash` (SHA-256) is only populated when the full bytes were fetched (see step 5/6 above)
  — a pure metadata run without a file-size filter leaves it `null` and dedups by URL only.
- `downloadedFileKey` is the key-value store key for the saved file, only set when
  `downloadFiles: true` and the image passed every filter.

### Use cases

- AI/ML teams curating an image training dataset filtered to a minimum real resolution.
- Content marketers collecting usable (large enough) product or stock photos from a site without
  pulling in thousands of tracking pixels and icons.
- Competitive analysis: collecting a competitor's product photography at scale for comparison.
- Deduplicating a gallery that serves the same image through several CDN URLs.

### Pricing

This Actor uses Apify's Pay-Per-Event (PPE) pricing model.

- **Actor Start:** $0.00005 (Apify default)
- **Image result:** $0.003 per verified image record (real dimensions, format, hash)
- **Image download:** $0.01 per image, only charged in addition to the result event when
  `downloadFiles: true` and the file was actually saved

### Legal

- This Actor only reads and, optionally, downloads publicly reachable image files and the HTML/CSS
  that references them — no login or paywalled content is accessed.
- `robots.txt` is respected for every internally discovered link while crawling
  (`maxCrawlDepth > 0`); URLs you supply directly in `startUrls` are always fetched, the same
  convention used by other scraping tools.
- This Actor does not verify copyright status or usage rights for any image it finds. Downloading
  or reusing images you don't own or have permission to use is your responsibility as the operator
  of this Actor, not the Actor's.
- No personal data is extracted deliberately. If an image happens to depict an identifiable
  person, that is a property of the source content, not something this Actor detects or filters.
- A `403 Forbidden` response from a page or image is treated as a normal per-item failure and
  skipped — this Actor does not attempt to bypass bot protection.

### FAQ

**Q: Does this render JavaScript-generated image galleries?**
A: No — this is an HTTP-only crawler (no headless browser). Images that only appear after
client-side JavaScript runs (e.g. a React/Vue gallery with no server-rendered `<img>` tags) will
be missed. Most sites still serve `<img>`/`<picture>` tags directly in the HTML.

**Q: Why is `contentHash` sometimes `null`?**
A: Computing a content hash requires the full image bytes. If you didn't set `downloadFiles: true`
and no file-size filter forced a full download, this Actor only reads the image's header bytes
(cheaper and faster), so no hash is available for that record. Deduplication still happens by
exact URL in that case.

**Q: How is this different from just checking the `width`/`height` HTML attributes?**
A: Those attributes are frequently absent, wrong, or describe a placeholder/lazy-loaded version
of the image. This Actor probes the image's own header bytes to get the resolution the browser
would actually render, regardless of what the HTML says.

**Q: Can I get only the largest/best photo when a page has multiple `srcset` sizes of the same image?**
A: Set a `minWidth`/`minHeight` filter and this Actor will keep only the variants meeting it; you
can also compare `contentHash` values (when downloading) to spot exact duplicates across variants
that decode to the same underlying file.

**Q: Will this download images I don't have rights to?**
A: This Actor collects/downloads whatever is publicly reachable on the pages you point it at — see
Legal above. Always confirm you have the right to use any image before republishing it.

### Related Actors

No other Actor in this portfolio currently covers generic website scraping — this is the first.

***

*Keywords: bulk image scraper, image downloader, website image crawler, image dimension filter,
real resolution filter, file size filter, magic bytes format detection, content hash deduplication,
srcset scraper, CSS background image scraper, image dataset builder.*

### Keywords

image scraper, bulk image downloader, website image crawler, image dimensions filter, srcset,
background image scraper, content hash dedup, magic bytes, image dataset, web crawler

### Changelog

#### 0.1.0

- Initial release: HTTP-only site crawl, real dimension/file-size filtering via header probing,
  magic-bytes format detection, content-hash dedup, optional full download to key-value store.

# Actor input Schema

## `startUrls` (type: `array`):

One or more web pages to scan for images. Example: https://example.com/gallery

## `maxCrawlDepth` (type: `integer`):

How many link-hops to follow from each start URL. 0 = only scan the start URL itself, no link-following. Example: 1

## `sameHostOnly` (type: `boolean`):

Only follow links that stay on the same hostname as the start URL. Turn off to allow crawling across subdomains/hosts.

## `imageExtensions` (type: `array`):

Only collect images whose URL ends with one of these extensions. Example: jpg, png, webp

## `minWidth` (type: `integer`):

Skip images narrower than this, based on the REAL decoded pixel width (not the HTML width attribute). Leave empty for no minimum. Example: 400

## `minHeight` (type: `integer`):

Skip images shorter than this, based on the REAL decoded pixel height. Leave empty for no minimum. Example: 300

## `maxWidth` (type: `integer`):

Skip images wider than this, based on the REAL decoded pixel width. Leave empty for no maximum. Example: 4000

## `maxHeight` (type: `integer`):

Skip images taller than this, based on the REAL decoded pixel height. Leave empty for no maximum. Example: 4000

## `minFileSizeKb` (type: `integer`):

Skip images smaller than this file size in kilobytes. Leave empty for no minimum. Example: 10

## `maxFileSizeKb` (type: `integer`):

Skip images larger than this file size in kilobytes. Leave empty for no maximum. Example: 5000

## `includeBackgroundImages` (type: `boolean`):

Also collect images referenced via a CSS "background-image: url(...)" inline style, not just <img>/<picture> tags.

## `includeSrcset` (type: `boolean`):

Also collect every image URL listed in an <img>/<source> "srcset" attribute (responsive images), not just the main "src".

## `downloadFiles` (type: `boolean`):

Download the complete image bytes into the run's key-value store (not only metadata). Increases run time/cost. When off, only metadata is collected and file size/dimensions are read from response headers where possible.

## `maxImagesPerPage` (type: `integer`):

Stop collecting candidate images on a single page after this many are found (safety cap for very large galleries). Example: 200

## `maxTotalImages` (type: `integer`):

Stop the whole run after this many images have been returned across all pages combined. Example: 1000

## Actor input object example

```json
{
  "startUrls": [
    "https://example.com"
  ],
  "maxCrawlDepth": 1,
  "sameHostOnly": true,
  "imageExtensions": [
    "jpg",
    "jpeg",
    "png",
    "gif",
    "webp",
    "svg",
    "avif",
    "bmp",
    "ico"
  ],
  "includeBackgroundImages": false,
  "includeSrcset": true,
  "downloadFiles": false,
  "maxImagesPerPage": 200,
  "maxTotalImages": 1000
}
```

# Actor output Schema

## `results` (type: `string`):

All discovered images in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://example.com"
    ],
    "imageExtensions": [
        "jpg",
        "jpeg",
        "png",
        "gif",
        "webp",
        "svg",
        "avif",
        "bmp",
        "ico"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("codeclouds/bulk-image-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": ["https://example.com"],
    "imageExtensions": [
        "jpg",
        "jpeg",
        "png",
        "gif",
        "webp",
        "svg",
        "avif",
        "bmp",
        "ico",
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("codeclouds/bulk-image-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://example.com"
  ],
  "imageExtensions": [
    "jpg",
    "jpeg",
    "png",
    "gif",
    "webp",
    "svg",
    "avif",
    "bmp",
    "ico"
  ]
}' |
apify call codeclouds/bulk-image-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,codeclouds/bulk-image-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/QKg1ckk35g8HC4YOR/builds/lkrPYWeG5CdlcmZZR/openapi.json