Website PDF Finder & Text Extractor

Give it a website, a page or a PDF link. It finds every PDF linked from the site and turns each one into clean text, with metadata and optional RAG-ready chunks. Other PDF extractors need you to collect the PDF URLs yourself. This one crawls the site for you, so https://www.example.com/investors is all you need to get the annual reports, presentations and policies linked there.

Each PDF is one dataset row:

• Text: markdown with detected headings and lists, or plain text. Running headers, footers and page numbers are removed.
• Basics: title, page count, word count, file size and SHA-256.
• Document properties: author, dates, producer, language and form fields.
• Provenance: the page the PDF was found on and its link text.

Turn on change tracking and a scheduled run tells you which PDFs are new, changed or removed since the previous run. Scanned PDFs with no text layer, broken files and links that turn out not to be PDFs are free.

What you can use it for

• RAG and AI knowledge bases. Load a company's manuals, datasheets, policies or research into a vector database. Chunks split at headings, and each one carries its page range and heading path, so answers can cite the page.
• Investor relations and competitor research. Pull annual reports, sustainability reports and presentations from IR pages and read them, search them or summarise them with an LLM.
• Document monitoring. Run weekly on tender portals, regulator pages, price lists, terms and conditions or safety data sheet libraries, and get only the PDFs that are new or changed.
• Document inventories and audits. Switch text off to list every PDF on a site with its size, source page and last-modified header, at a tenth of the price.
• Lead generation and sales research. Extract the text of brochures, catalogues and case studies to find products, partners and contacts.
• Direct PDF text. Paste PDF links, or Google Drive and Dropbox share links, and get the text back with no crawling.

Input

Field	Default	What it does
startUrls	required	One per line. Web pages and websites are crawled for PDF links. Direct PDF links are extracted as they are. Google Drive and Dropbox share links are converted to direct downloads. Duplicates are removed. Invalid lines get a free error row.
extractText	true	Off = listing mode. PDFs are found and listed with their URL, link text, source page, size and SHA-256, but not read.
textFormat	markdown	markdown keeps headings (#, ##, ###, detected from font size) and bullet lists. plain gives the same paragraphs without markup.
chunkSize	0	Set to 200-20000 to also get a chunks list (see Output). 1000-2000 suits most embedding models.
includePages	false	Also return the text of each page in a pages list.
removeHeadersFooters	true	Drop lines that repeat at the top or bottom of most pages, and page numbers.
crawlDepth	1	How many clicks from the start page to look for PDFs. 0 = only the start page. Pages that look like reports, downloads, publications or investors are visited first.
maxPagesPerSite	25	Max HTML pages fetched per start URL. Crawling is free.
maxPdfsPerSite	50	Max PDFs taken from one start URL.
includeKeywords	empty	Keep only PDFs whose URL or link text contains one of these words, e.g. annual, 2025, datasheet.
excludeKeywords	empty	Skip PDFs whose URL or link text contains any of these words.
allowOtherHosts	true	Include PDFs hosted on another domain, such as a CDN or document host. Pages are always crawled on the same site only.
maxPagesPerPdf	500	Stop reading a PDF after this many pages.
maxPdfSizeMb	50	Skip larger files (free too_large row).
trackChanges	false	Remember every PDF by SHA-256 and mark it new, changed or unchanged on the next run. Report PDFs that disappeared.
onlyNewOrChanged	false	With change tracking: skip the text of unchanged PDFs and charge only the listing price for them.
stateKey	default	Separate change history per schedule or list.
concurrency	5	Start URLs processed in parallel.

Example: annual and sustainability reports from an investor page, as markdown chunks.

{
    "startUrls": ["https://www.example.com/investors/reports"],
    "includeKeywords": ["annual", "sustainability"],
    "crawlDepth": 1,
    "chunkSize": 1500
}

Output

One row per PDF. A start URL that failed or had no PDFs also gets one row, which is free.

{
    "input": "https://www.irs.gov/forms-pubs/about-form-w-9",
    "status": "extracted",
    "pdfUrl": "https://www.irs.gov/pub/irs-pdf/fw9.pdf",
    "sourcePage": "https://www.irs.gov/forms-pubs/about-form-w-9",
    "linkText": "Form W-9 PDF",
    "fileName": "fw9.pdf",
    "title": "Form W-9 (Rev. March 2024)",
    "titleSource": "metadata",
    "pageCount": 6,
    "pagesExtracted": 6,
    "extractionStoppedEarly": null,
    "charCount": 37828,
    "wordCount": 6268,
    "pagesWithoutText": 0,
    "likelyScanned": false,
    "headerFooterLinesRemoved": 5,
    "textFormat": "markdown",
    "text": "Form W-9(Rev. March 2024)\n\n## Request for Taxpayer Identification Number and Certification\n\n...",
    "chunkCount": 33,
    "chunks": [
        { "index": 0, "text": "...", "pageStart": 1, "pageEnd": 1, "heading": "Request for Taxpayer Identification Number and Certification", "charCount": 1480 }
    ],
    "metadata": {
        "title": "Form W-9 (Rev. March 2024)",
        "author": "SE:W:CAR:MP",
        "subject": "Request for Taxpayer Identification Number and Certification",
        "keywords": "Fillable",
        "creator": "Designer 6.5",
        "producer": "Designer 6.5",
        "language": "en-US",
        "createdAt": "2024-03-06T13:18:13.000Z",
        "modifiedAt": "2024-03-06T13:18:13.000Z",
        "pdfVersion": "1.7",
        "hasForm": true
    },
    "fileSizeBytes": 140815,
    "sha256": "2d420cbb4123dcf1fb82595b2359cfbb5d81f00b9df9d359fcc7af361d093f53",
    "contentType": "application/pdf",
    "lastModified": null,
    "changeStatus": null,
    "httpStatus": 200,
    "errorCode": null,
    "checkedAt": "2026-10-30T12:00:00.000Z"
}

• title comes from the PDF's own title. Junk titles such as "Microsoft Word - draft3.docx" are skipped. After that, the actor tries the first large heading, then the link text, then the file name. titleSource says which one was used.
• chunks are split at headings and paragraphs, never mid-sentence unless a single sentence is longer than the chunk size. A new chunk starts at a heading once the current one is reasonably full. heading is the path of headings above the chunk, joined with >.
• likelyScanned is true when there is almost no text per page, which usually means a scanned document. Rows with no text at all get status no_text and are free. OCR is not included.
• extractionStoppedEarly is one of:
  • page_limit: hit maxPagesPerPdf;
  • time_limit: very slow files are cut off after about two minutes of parsing;
  • charge_limit: the run's max charge was reached;
  • size_limit: the text was cut to fit the 9 MB dataset row limit, after per-page text and chunks were dropped.
• With change tracking on:
  • changeStatus is new, changed or unchanged;
  • previousSha256, firstSeenAt and previousSeenAt show the history;
  • PDFs that disappeared from a fully crawled site get a free row with status removed.

The dataset has views for PDFs, Text, Metadata, Changes and Problems. The RUN_SUMMARY record in the key-value store counts:

• pages crawled;
• PDFs found, extracted and listed;
• failures;
• changes;
• charged events.

Status values

Status	Meaning	Charged
extracted	Text and metadata extracted.	yes
listed	Listing mode: found and checked, not read.	listing price
unchanged	Change tracking with onlyNewOrChanged: same file as last run, text skipped.	listing price
no_text	No text layer (scan or images only).	no
encrypted	Password protected.	no
invalid_pdf	Broken or unreadable file.	no
not_pdf	The link did not return a PDF.	no
too_large	Larger than maxPdfSizeMb.	no
http_error, unreachable, blocked	The server returned an error, did not answer, or refused automated access.	no
blocked_by_robots	The site's robots.txt asks crawlers to stay out.	no
no_pdfs_found	The start URL was crawled but had no matching PDF links.	no
removed	Change tracking: a PDF from the previous run is no longer linked.	no
invalid_input	The line is not a usable URL.	no

Links that only hint at a PDF, such as a "Download (PDF)" button pointing at /download?id=7, are checked by reading the file header. If one turns out to be something else, it is skipped silently. The RUN_SUMMARY counts these skipped links.

Pricing

Pay per result. No subscription, and crawling is free.

Event	Price	When
PDF extracted	$0.003	One PDF read, including its first 50 pages.
Extra PDF page	$0.00005	Each page beyond 50, for example $0.0075 for a 200-page report.
PDF listed	$0.0003	Listing mode, or an unchanged PDF skipped with onlyNewOrChanged.

Examples:

• 1,000 short PDFs cost about $3.
• One 300-page annual report costs about $0.016.
• A weekly check of a 200-PDF library where 5 files changed costs about $0.07.

Nothing is charged for scans without text, errors, non-PDF links, robots.txt opt-outs, removed rows or invalid input. The actor stops cleanly when the run's maximum charge is reached and lists what it did not process in RUN_SUMMARY.

How it works and what it respects

• The actor fetches only public pages and files from the URLs you give it, and the pages they link to on the same site, up to your limits. It never logs in and never fills in forms.
• robots.txt is honoured for every page and every PDF host, with the user agent WebsitePdfExtractor. If a site's robots.txt answers with a server error, the site is treated as closed.
• Requests to one host are spaced out, and only a few PDFs download at a time.
• PDFs are parsed with PDF.js. Script execution and font loading are switched off.

Limits

• No OCR. Scanned PDFs are detected and returned free as no_text. Run them through an OCR tool if you need their text.
• Pages rendered with JavaScript. PDF links on these pages are found when they appear in the page's HTML or embedded JSON data, which covers most Next.js and React sites. Links that only appear after a click or a search form are not found. Paste the PDF URLs directly in that case.
• Layout. Headings are detected from font size, so documents set in one font size come out as paragraphs only. Tables come out as text rows, and multi-column layouts are read in the PDF's own text order.
• Change tracking. Changes are detected from file contents (SHA-256). A PDF re-exported with identical text but new internal timestamps counts as changed.
• Bot protection. Sites behind Cloudflare challenges or similar return a free blocked row.

Support

Open an issue on the actor's Issues tab with the start URL and what you expected. Runs, input and output stay in your Apify account.