# Link Extractor: Extract All Links, Internal & External, Bulk (`data-done-right/link-extractor`) Actor

Link extractor for bulk URLs: extract all links from website pages as one row per link, with absolute URL, anchor text, internal or external type, nofollow, sponsored, UGC, file type and domain. Optional link status check and same-domain crawl.

- **URL**: https://apify.com/data-done-right/link-extractor.md
- **Developed by:** [Data Done Right](https://apify.com/data-done-right) (community)
- **Categories:** SEO tools, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.72 / 1,000 link extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Link Extractor: Extract All Links, Internal & External, Bulk

This **link extractor** reads a list of web pages and returns **one row per link**: the absolute URL, anchor text, **internal or external** type, **nofollow / sponsored / ugc** flags, file extension, domain and the page it was found on. Use it to **extract links from a website**, build a **URL extractor** list, audit outbound links, or collect every PDF a site links to. Bulk input, flat output, no browser, no login, no API key.

Optional extras: **check the HTTP status** of every unique link (with the final URL after redirects), **crawl 1-2 levels** of the same site, and filter links with **regular expressions**.

### What does this link extractor return

One dataset row per link:

- **Where:** `sourcePage` (the page), `depth` (0 = a page you listed, 1-2 = crawled).
- **Link:** `href` (absolute URL, relative links and `<base href>` resolved), `text` (anchor text, falling back to aria-label, title or image alt), `domain`, `extension`.
- **Type:** `type` is `internal`, `external`, `mailto`, `tel`, `anchor`, `file` (pdf, zip, docx, images, media and similar), `image` or `other`. `isInternal` is a plain true/false (null for mailto and tel).
- **Rel flags:** `nofollow`, `sponsored`, `ugc` and the full `rel` string.
- **Images:** `isImageLink` and `imageSrc` for links that wrap an image. Optional extra rows for every `<img src>`.
- **Repeats:** `occurrences` (how many times the link appears on the page when duplicates are removed).
- **Status check (optional):** `statusCode`, `finalUrl`, `redirectCount`, `ok`, `checkError`.

A per-page summary with counts by type is saved in the key-value store record `SUMMARY` (`pages[]`), failed pages in `FAILED`, skipped duplicates and caps in `NOTES`.

### How to extract all links from a website page

1. Paste page URLs in **Page URLs**, one per line (a bare domain works too).
2. Pick **Which links**: internal, external or both. Optional: regex filters, anchors, image sources.
3. Run it and download the dataset as JSON, CSV or Excel. The **Links** table view shows the key columns.

### How to extract only external links (or only PDFs)

Set **Which links** to `external` to get outbound links only. To collect files, set **Only links matching (regex)** to `\.(pdf|docx?|xlsx?|zip)$`. To drop noise use **Skip links matching** such as `/(login|logout|cart)`.

### How to check if links are broken

Turn on **Check link status**. Each unique http(s) link is requested once (HEAD first, GET if the server refuses HEAD) and every row gets `statusCode`, `finalUrl`, `redirectCount` and `ok`. Filter on `ok = false` or `statusCode >= 400` to find dead links. Links that cannot be reached at all (DNS, timeout) get `checkError`. Use **Max links to check** to cap cost.

### How to crawl a site for links

Set **Crawl depth** to 1 or 2 and **Max pages**. The crawler follows internal HTML links only (same site, `www` ignored; subdomains optional), fetches at most **Parallel page requests** pages at a time, and by default honours `robots.txt` for pages it discovers. Pages you list yourself are always fetched.

### Input example

```json
{
  "urls": ["https://www.python.org", "https://www.iana.org/domains/reserved"],
  "linkScope": "both",
  "dedupePerPage": true,
  "excludePatterns": ["/(login|logout)"],
  "checkStatus": false,
  "crawlDepth": 0,
  "maxLinks": 100
}
```

### Output example

```json
{
  "sourcePage": "https://www.python.org/",
  "href": "https://docs.python.org/3/",
  "text": "Documentation",
  "type": "external",
  "isInternal": false,
  "domain": "docs.python.org",
  "extension": null,
  "nofollow": false,
  "sponsored": false,
  "ugc": false,
  "rel": null,
  "isImageLink": false,
  "imageSrc": null,
  "source": "a",
  "occurrences": 1,
  "depth": 0,
  "statusCode": 200,
  "finalUrl": "https://docs.python.org/3/",
  "redirectCount": 0,
  "ok": true,
  "checked": true,
  "checkError": null
}
```

Without `checkStatus` the status fields are `null` and `checked` is `false`.

### How much does it cost to extract links

Pay per event, no start fee:

- **`link-extracted`: $0.0009 per link row** ($0.90 per 1,000 links).
- **`link-checked`: $0.0008 per link whose status was obtained** ($0.80 per 1,000), only when **Check link status** is on. Links that do not answer at all are free.

Plus small Apify platform usage. Pages that fail (404, timeout, not HTML) produce no rows and cost nothing. Use **Max links** and **Max links to check** as spending caps; a maximum spending limit on the run also stops it cleanly.

### Use cases

- SEO audits: outbound links, nofollow and sponsored usage, broken links.
- Link building and backlink research: who does a page link to?
- Content inventory: every PDF, document or download linked from a site.
- Migration checks: find links to old domains or paths with a regex.
- Lead lists: mailto and tel links from contact pages.

### Limits (honest)

- **No JavaScript rendering.** It reads the raw HTML the server sends. Links added by JavaScript after load (single-page apps, lazy menus) are not seen.
- Only the first 5 MB of each page is parsed. Pages that block normal requests (403, bot walls, CAPTCHAs) are reported in `FAILED`; there is no bot-protection evasion.
- Crawling is shallow (depth 0-2), same site only, polite and low in concurrency; it is not a full-site spider. Use **Max pages** to bound it.
- Status checks use HEAD/GET from the cloud: some sites answer differently to bots, so a 403 or 999 may still work in a browser. Literal private addresses (127.0.0.1, 10.x, localhost) are never requested.
- `javascript:` and `data:` links are ignored; `#anchor` links are only returned if you ask for them.

### FAQ

**Does it execute JavaScript?** No. It is fast and cheap because it reads HTML only.

**What counts as internal?** Same host as the page, ignoring `www.`. Turn on **Treat subdomains as internal** to include `blog.example.com`.

**Are duplicates removed?** Per page, yes by default (`occurrences` counts the repeats). Turn **Remove duplicates per page** off for every occurrence.

**Does it respect robots.txt?** For pages found by crawling, yes by default. Pages you list yourself are fetched as requested.

**Are failed pages charged?** No. Only link rows (and, if enabled, status checks that returned an HTTP status).

**How many links per run?** Up to 200,000 rows (default 10,000), from up to 5,000 listed URLs.

### Related tools

- **HTTP Status Code Checker: Bulk URL, Website & Redirect Chains** for full redirect chains of a URL list.
- **SEO Audit Tool & Meta Tag Checker: Crawler, Broken Links** for scoring every page of a site.
- **Sitemap Parser & Extractor: Sitemap to CSV, URLs & New Pages** to list a site's pages from its sitemap.

# Actor input Schema

## `urls` (type: `array`):

Pages to extract links from, one per line. A bare domain such as example.com also works. Example: https://www.python.org.

## `linkScope` (type: `string`):

Keep internal links (same site, www ignored), external links, or both. Mailto and tel links are only returned with 'both'. Default both.

## `dedupePerPage` (type: `boolean`):

Return each link once per page and count repeats in the occurrences field. Turn off to get every link occurrence. Default on.

## `includePatterns` (type: `array`):

Optional. Keep a link only if its absolute URL matches at least one of these regular expressions (case-insensitive). Example: .pdf$

## `excludePatterns` (type: `array`):

Optional. Drop links whose absolute URL matches any of these regular expressions. Example: /(login|logout|cart)

## `includeAnchors` (type: `boolean`):

Also return links that jump to a section of the same page (type anchor). Default off.

## `includeMailtoTel` (type: `boolean`):

Return email and phone links (type mailto, tel). Only applies when Which links is 'both'. Default on.

## `includeImageSources` (type: `boolean`):

Also return one row per img src (type image, source img). Links that wrap an image are always flagged with isImageLink and imageSrc. Default off.

## `subdomainsAreInternal` (type: `boolean`):

If on, blog.example.com counts as internal for example.com. Default off.

## `checkStatus` (type: `boolean`):

Request each unique http(s) link (HEAD, then GET if HEAD is refused) and add statusCode, finalUrl (after redirects), redirectCount and ok to every row. Charged separately per link that answered. Default off.

## `maxChecks` (type: `integer`):

Most unique links checked in one run; the rest get no status. Caps the cost of Check link status. Default 1000.

## `checkConcurrency` (type: `integer`):

How many links are checked at the same time. Default 10.

## `crawlDepth` (type: `integer`):

0 = only the pages you list. 1 or 2 = also follow internal links to pages on the same site, up to this many levels. Default 0.

## `maxPages` (type: `integer`):

Most pages fetched in the whole run, listed pages included. Applies to crawling. Default 50.

## `respectRobotsTxt` (type: `boolean`):

Skip crawled pages that robots.txt disallows. Pages you list yourself are always fetched. Default on.

## `maxUrls` (type: `integer`):

Most listed URLs processed. Extra ones are skipped. Default 100.

## `maxLinks` (type: `integer`):

Most link rows returned (and charged) in this run. Default 10000.

## `concurrency` (type: `integer`):

How many pages are fetched at the same time. Keep low for polite crawling. Default 5.

## `retries` (type: `integer`):

Tries per page after timeouts, 429 or 5xx errors. Default 3.

## `timeoutSecs` (type: `integer`):

Give up on one request after this many seconds. Default 30.

## `proxyConfiguration` (type: `object`):

Optional. Fetch pages through Apify Proxy or your own proxies. Usually not needed for public pages.

## Actor input object example

```json
{
  "urls": [
    "https://www.python.org",
    "https://www.iana.org/domains/reserved"
  ],
  "linkScope": "both",
  "dedupePerPage": true,
  "includeAnchors": false,
  "includeMailtoTel": true,
  "includeImageSources": false,
  "subdomainsAreInternal": false,
  "checkStatus": false,
  "maxChecks": 1000,
  "checkConcurrency": 10,
  "crawlDepth": 0,
  "maxPages": 50,
  "respectRobotsTxt": true,
  "maxUrls": 100,
  "maxLinks": 100,
  "concurrency": 5,
  "retries": 3,
  "timeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

All result items of this run (the default dataset), as JSON, CSV, Excel and more.

## `records` (type: `string`):

Key-value store records: the run summary, notes on any problems, and any files.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://www.python.org",
        "https://www.iana.org/domains/reserved"
    ],
    "maxLinks": 100
};

// Run the Actor and wait for it to finish
const run = await client.actor("data-done-right/link-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": [
        "https://www.python.org",
        "https://www.iana.org/domains/reserved",
    ],
    "maxLinks": 100,
}

# Run the Actor and wait for it to finish
run = client.actor("data-done-right/link-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://www.python.org",
    "https://www.iana.org/domains/reserved"
  ],
  "maxLinks": 100
}' |
apify call data-done-right/link-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,data-done-right/link-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/GqRq7SAAkXf0JhRpC/builds/WB6grTHXney2cH13h/openapi.json