# Markdownee (`markdownee/crawler`) Actor
Crawl any website and extract clean main content as Markdown, plain text, readable HTML, minified HTML, or raw original HTML — ready for LLMs, RAG pipelines, and vector databases. Built on the Trafilatura Core engine and an adaptive Crawlee + Playwright crawler.
- **URL**: https://apify.com/markdownee/crawler.md
- **Developed by:** [Markdownee](https://apify.com/markdownee) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet
## Pricing
Pay per usage
This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.
Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage
## What's an Apify Actor?
An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.
For examples already wired to this Actor's own input schema, see the [API](#api) section below.
Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).
# README
Run hosted crawls that collect page content as text, Markdown, or HTML. Configure
which links to follow, what extraction should retain, and where each format is
stored. These outputs can feed research collections, retrieval systems, and
dataset preparation. [Token savings](https://www.markdownee.com/about/#token-efficient-output-for-llms)
depend on the pages, settings, and downstream tokenizer.
- Boilerplate removal is powered by **[Trafilatura Core](https://www.trafilaturacore.com/)**, our
**open-source pure-TypeScript port** of
[Trafilatura](https://www.markdownee.com/trafilatura/). Its **extraction core** is a
direct port of Python Trafilatura — with
[go-trafilatura](https://github.com/markusmobius/go-trafilatura) used only as a DOM translation
aid — and applies **Trafilatura's own heuristics** to strip navigation, sidebars, footers, and
similar clutter
- Fetch rendered pages with [Crawlee](https://crawlee.dev/) and
[Playwright](https://playwright.dev/), or choose HTTP-only Cheerio.
- Run the extraction engine without a Python runtime or GPU.
- Use the hosted Actor or self-host through the
[npm CLI](https://www.markdownee.com/help/npm/) or
[npm library](https://www.markdownee.com/help/npm-lib/); the open-source code
is on [GitHub](https://github.com/markdownee/markdownee).
- Enable image downloading for stored content that needs image files.
### Configure a run in Console
Add starting URLs and choose **Save** destinations. Each token combines a format
with `dataset` or `kvs`; choose both to store the same format in both places.
Use a link selector, include/exclude patterns, sitemaps, and page/depth limits to
define the crawl. Then select **Start** and inspect its records and content.
Dataset downloads support JSON, CSV, and Excel. KVS content can be downloaded
separately or fetched through the Apify API.
### Input recipes
Starting URLs are required. The
[Input tab](https://apify.com/markdownee/crawler/input-schema?fpr=glueo) lists the
complete contract and defaults.
Collect a blog section with Markdown storage:
```json
{
"startUrls": [{ "url": "https://blog.example.com/" }],
"selector": "a[href]",
"globs": [{ "glob": "https://blog.example.com/**" }],
"maxCrawlDepth": 2,
"save": ["markdown-kvs"]
}
```
Request several formats for one starting page:
```json
{
"startUrls": [{ "url": "https://example.com/article" }],
"maxRequestsPerCrawl": 1,
"save": ["markdown-kvs", "minified-html-dataset", "original-kvs"]
}
```
Store readable and compact HTML independently:
```json
{
"startUrls": [{ "url": "https://example.com/article" }],
"save": ["html-kvs", "minified-html-dataset", "markdown-kvs"]
}
```
`outputLayout` defaults to `minimal` for body text and HTML fragments.
`standard` adds ordinary metadata and complete HTML documents; `enhanced`
includes additional allowlisted metadata and crawl information. Layout neither
adds destinations nor modifies the captured original.
Look for Markdown published by a documentation site:
```json
{
"startUrls": [{ "url": "https://docs.example.com/" }],
"globs": [{ "glob": "https://docs.example.com/**" }],
"markdownDiscovery": "alternate",
"save": ["markdown-kvs"]
}
```
`markdownDiscovery` affects the source of all output formats. `off` leaves
HTML fetching unchanged. `alternate` follows advertised same-origin links;
`negotiate` also requests Markdown through Accept; `probe` also tries a .md
sibling. Up to three alternates, one refetch, and one sibling may be attempted;
robots.txt can add an origin-level request. Crawler-path capabilities and
per-origin budgets restrict attempts. Rejected representations fall back to HTML.
Configure proxies and a persistent session pool:
```json
{
"startUrls": [{ "url": "https://shop.example.com/" }],
"proxyConfiguration": {
"useApifyProxy": true,
"apifyProxyGroups": ["RESIDENTIAL"]
},
"proxyRotation": "recommended",
"sessionPoolName": "shop-example",
"waitUntil": "networkidle"
}
```
These controls affect requests and session reuse; they do not assure access to
a site. URL filtering also needs a `selector` to enable link following, unless
the full URL list or sitemap supplies the crawl frontier.
### Choose fetching and extraction settings
- **Crawler type:** adaptive Playwright renders by default. A positive
`renderingTypeDetectionRatio` enables sampling to decide when HTTP can be used.
Explicit Firefox, Chromium, and HTTP-only Cheerio are also available.
- **Extraction mode:** `precision` favors less noise, `recall` retains more
borderline content, `balanced` is the default, and `keep` cleans the document
without main-content selection.
- **Content handling:** images accept `exclude`, `alt-text`, `resolved-url`,
or `save`. Links, tables, and detected user-comment sections accept `include`
or `exclude`.
- **Waits and consent:** selectors, dynamic-content waits, scrolling, cookies,
and headers help control capture. Enabled consent handling attempts recovery
and removes residual consent containers before extraction.
- **Deduplication:** `minimal` uses Crawlee URL deduplication; `standard` also
checks canonical URLs; `aggressive` also checks extracted-content hashes.
`startUrls`, `globs`, and `exclude` use arrays of objects containing `url` or
`glob`, respectively. `save` uses strings such as `markdown-kvs` or
`txt-dataset`. Supported formats are `txt`, `markdown`, `html`,
`minified-html`, and `original`.
### Interpret results
Successful pages produce `success` records. Requests that exhaust retries
produce `failed` records; skips are recorded when `storeSkippedUrls` is enabled.
| Field | Meaning |
| ----------------------------------------- | -------------------------------------------------------------------------------------- |
| `url` | Requested page address |
| `status` | `success`, `failed`, or `skipped` |
| `metadata` | Available title, author, date, description, site, language, and extended fields |
| `crawl` | Available final `loadedUrl`, `scrapedAt`, `httpStatusCode`, `depth`, and `referrerUrl` |
| `original` | Crawler-captured HTML's hash and byte count, with stored content when requested |
| `txt`, `markdown`, `html`, `minifiedHtml` | Selected format nodes containing hashes, byte counts, and content or KVS references |
| `markdownSource` | Discovery mechanism, source URL, and whether served Markdown supplied the output body |
| `errors`, `retryCount`, `crawledTime` | Failed-request details |
| `skipReason` | `robotsTxt`, `limit`, `enqueueLimit`, `filters`, `redirect`, or `depth` |
This illustrative record uses sample metadata, hashes, and byte counts:
```json
{
"url": "https://blog.example.com/why-rag-matters",
"status": "success",
"metadata": {
"title": "Why RAG Matters",
"author": "Jane Doe",
"date": "2026-01-15",
"description": "A practical look at retrieval-augmented generation.",
"siteName": "Example Blog",
"languageCode": "en"
},
"crawl": {
"loadedUrl": "https://blog.example.com/why-rag-matters",
"scrapedAt": "2026-05-31T10:00:00.000Z",
"httpStatusCode": 200,
"depth": 1,
"referrerUrl": "https://blog.example.com/"
},
"original": {
"hash": "f8e6bd335e04d03e1be6798c2c72349c",
"bytes": 89898
},
"markdown": {
"hash": "43f204bfbee5dbe6862cb38620f257b5",
"bytes": 5234,
"key": "markdown-c485356090a92c6a45e8c1155c14d8ee.md",
"url": "https://api.apify.com/v2/key-value-stores//records/"
}
}
```
`markdownSource` appears when enabled discovery supplied an origin-published
representation. Its mechanism is `response`, `alternate`, `negotiated`, or
`sibling`. `verbatim` is true for a served Markdown body after source front-matter
handling and the selected layout; it is false for a cleaned HTML round trip or
when no Markdown output was saved. These records derive `original` from the
served representation.
Ordinary original HTML is a capture before extraction. It can already reflect
browser rendering, serialization, and consent handling, and is not a complete
web archive.
### Select storage destinations
**KVS routes**, the default, put each format in a separate file keyed by
`{format}-{md5(url)}.{ext}`. The record includes its key and public URL when
available. **Dataset routes** place content inline for a combined export.
Large inline HTML increases record size and memory use; KVS is an alternative.
Choose one or both destinations for each format. Markdown is convenient for
text-based document structure, TXT for text-only consumers, readable HTML for
inspection, and Minified HTML for compact markup storage. Keep original HTML
when the crawler capture is useful for later diagnosis or reprocessing.
### Estimate run costs
Apify charges depend on compute, storage, proxy traffic, and the selected plan.
Browser work, page size, concurrency, waits, and crawl limits affect resource
use. Start with a representative sample to estimate a larger collection.
Apify offers [$5 of free usage monthly](https://apify.com/pricing?fpr=glueo);
consult that page for current rates.
### Integrations and recurring collection
The [API tab](https://apify.com/markdownee/crawler/api?fpr=glueo) provides
JavaScript/Python client examples, OpenAPI information, and MCP setup.
Use Apify scheduling for recurring runs and its integrations for destinations
such as Make, Zapier, n8n, Google Drive, or Slack.
Your application can use the returned content for summarization, translation,
classification, search, or training-data preparation. Frameworks such as
LangChain and LlamaIndex, and stores such as Pinecone, Qdrant, Weaviate, or Chroma,
consume the records through your ingestion code.
### FAQ
#### Is it legal to scrape website content?
Scraping publicly available, non-personal data is generally legal in most
jurisdictions. Markdownee can honor each site's `robots.txt` (enable **Respect
robots.txt**), and you remain responsible for complying with each site's Terms of
Service and for how you use extracted content — especially copyrighted material you
intend to republish.
#### Why is content missing or mixed with page furniture?
Compare `precision`, `balanced`, and `recall` on the affected pages. If content
appears after JavaScript execution, use a browser crawler and review selector
waits, dynamic-content waits, and scroll limits. Examine the captured original
to distinguish fetching problems from extraction decisions.
#### How do I configure a larger crawl?
Supply `selector`, for example `a[href]`, and bound matching links with globs,
depth, and request limits. Enable sitemaps when appropriate. Review proxy,
session-pool, and rotation settings for the target site; failures remain
possible and are recorded.
#### How do I start a run from code?
Use the [API tab](https://apify.com/markdownee/crawler/api?fpr=glueo) for client
examples or MCP configuration. The Console also supports scheduling and run
monitoring.
#### Where can I report a problem?
Open the Actor's **Issues** tab with the settings and outcome needed to
understand the problem.
# Actor input Schema
## `startUrls` (type: `array`):
URLs to extract content from
## `crawlerType` (type: `string`):
Select how pages are fetched. playwright-adaptive uses browser rendering by default; a positive renderingTypeDetectionRatio enables detection for HTTP-only fetching. Choose an explicit Playwright browser or cheerio for HTTP requests without JavaScript execution.
## `renderingTypeDetectionRatio` (type: `number`):
For the adaptive crawler, set the fraction of pages sampled to determine whether browser rendering is needed (0–1). The default 0 disables detection and uses the browser. With detection enabled, an HTTP fetch may miss content populated by JavaScript.
## `markdownDiscovery` (type: `string`):
Choose how to find a Markdown representation published by the origin. An accepted representation supplies content for all requested formats. off (default) leaves HTML fetching unchanged. alternate follows same-origin Markdown links advertised in a element or Link response header. negotiate also requests text/markdown through Accept, with a refetch when needed. probe additionally tries a .md sibling URL. Discovery can attempt three alternates, one negotiated refetch, and one sibling; robots.txt checks can add a request per origin. Per-origin budgets limit unsuccessful attempts. Available steps depend on the crawler path. Rejected or unavailable representations fall back to HTML extraction.
## `globs` (type: `array`):
Glob patterns matching URLs of pages that will be included in crawling. Setting this option allows you to customize the crawling scope. For example `https://{store,docs}.example.com/**` lets the crawler access all URLs starting with `https://store.example.com/` or `https://docs.example.com/`.
## `exclude` (type: `array`):
Glob patterns matching URLs of pages that will be excluded from crawling. Note that this affects only links found on pages, but not Start URLs, which are always crawled.
## `selector` (type: `string`):
CSS selector for links to enqueue. Leave empty to disable link enqueueing.
## `keepUrlFragment` (type: `boolean`):
URL fragments (the parts of URL after a #) are not considered when the scraper determines whether a URL has already been visited. Turn this on to treat URLs with different fragments as different pages.
## `useSitemaps` (type: `boolean`):
If enabled, the crawler looks for sitemap.xml at the root of each start URL domain and enqueues matching URLs from it in addition to link-following.
## `deduplication` (type: `string`):
Deduplication level applied on top of Crawlee's built-in URL deduplication. standard (default): skip pages whose was already extracted, across all handler types. aggressive: also skip pages whose extracted text content matches a previously extracted page. minimal: disable additional deduplication — only Crawlee's built-in URL dedup remains active.
## `respectRobotsTxtFile` (type: `boolean`):
If enabled, the crawler will consult the robots.txt file for each domain before crawling pages.
## `initialCookies` (type: `array`):
Cookies that will be pre-set to all pages the scraper opens. This is useful for pages that require login. The value is expected to be a JSON array of objects with `name` and `value` properties. For example:
```json
[
{
"name": "cookieName",
"value": "cookieValue",
"path": "/",
"domain": ".example.com"
}
]
```
You can use the [EditThisCookie](https://docs.apify.com/academy/tools/edit-this-cookie) browser extension to copy browser cookies in this format, and paste it here.
Note that the value is secret and encrypted to protect your login cookies.
## `customHttpHeaders` (type: `object`):
HTTP headers that will be added to all requests made by the crawler. This is useful for setting custom authentication headers or other headers required by the target website. The value is expected to be a JSON object with header names as keys and header values as values. For example: `{ "Authorization": "Bearer token123", "X-Custom-Header": "value" }`.
## `maxRequestsPerCrawl` (type: `integer`):
Maximum number of requests the crawler will handle. Counts handled page outcomes (successes and final failures), including start URLs and pagination pages. The crawler automatically finishes after reaching this number. 0 means unlimited.
## `maxResultsPerCrawl` (type: `integer`):
Maximum number of results that will be saved to dataset. The scraper will terminate after reaching this number. 0 means unlimited.
## `maxCrawlDepth` (type: `integer`):
Maximum link depth from Start URLs. Pages discovered further from start URLs than this limit will not be crawled. 0 means unlimited.
## `initialConcurrency` (type: `integer`):
Initial number of browser pages or HTTP clients running in parallel. Crawlee auto-scales up to maxConcurrency. 0 lets Crawlee pick the default.
## `maxConcurrency` (type: `integer`):
Maximum number of browser pages running in parallel. Kept low by default because the browser crawler cannot abort in-flight pages, so concurrency is the only hard cap on peak memory — large pages can exhaust memory at higher values. Raise it for lightweight pages or the HTTP (cheerio) crawler. This setting also avoids overloading target websites and getting blocked.
## `maxRequestRetries` (type: `integer`):
Maximum number of retries for failed requests on network, proxy, or server errors.
## `mode` (type: `string`):
Choose the content-selection policy. precision favors less boilerplate at the risk of omitted content; recall retains more content and may include clutter. balanced is the default. keep cleans the whole document without main-content selection.
## `imageHandling` (type: `string`):
What becomes of images in the extracted content. exclude (default) removes image structures entirely, including their captions. alt-text replaces each image with its textual stand-in (alt text, falling back to figcaption, aria-label, or title) — no URL, no bytes. resolved-url keeps each image as one clean tag whose src is the resolved absolute image URL (lazy-load and srcset variants collapsed); nothing is downloaded. The save MODE additionally downloads the image bytes, re-encodes them, and stores them in the key-value store, referencing them from the results — not to be confused with the save TOKENS option, which routes output formats to destinations.
## `maxImageEdge` (type: `integer`):
Long-edge pixel cap for images stored by the save image-handling mode: larger images are downscaled so their longer side is at most this many pixels (never upscaled). 0 disables the cap. Ignored unless imageHandling is save.
## `rasterizeSvg` (type: `boolean`):
For SVG images stored by the save image-handling mode: store a rasterized PNG instead of the sanitized SVG source (vision models accept no SVG). Disable to store the sanitized SVG source. Ignored unless imageHandling is save.
## `linkHandling` (type: `string`):
What becomes of hyperlinks in the extracted content. include (default) renders links inline. exclude unwraps each link — the anchor text survives, the URL is dropped.
## `tableHandling` (type: `string`):
What becomes of tables in the extracted content. include (default) renders tables. exclude discards each table subtree entirely, including its cell text.
## `commentHandling` (type: `string`):
What becomes of user-comment sections (forum or blog comments — NOT HTML markup) in the extracted content. include keeps detected comment containers; exclude removes them before output serialization.
## `languageCode` (type: `string`):
Filter extracted content by language code (e.g. "en"). Leave empty to accept any language.
## `outputLayout` (type: `string`):
Choose the structure surrounding generated content: minimal provides TXT/Markdown bodies and HTML fragments; standard adds ordinary metadata, front matter, and complete HTML documents; enhanced includes additional allowlisted metadata and crawl information. Readable HTML and Minified HTML use the same layout.
## `save` (type: `array`):
What to save and where, as `format-destination` tokens. Format is one of `txt`, `markdown`, `html`, `minified-html`, `original` (raw page HTML before extraction); destination is `dataset` (inline in the dataset record) or `kvs` (a blob in the key-value store). List a format twice to save it to both, e.g. `markdown-dataset markdown-kvs`. Saving generated or original HTML (large content) to the dataset is not recommended — it risks out-of-memory on large pages; prefer `kvs`.
## `datasetName` (type: `string`):
Name or ID of the dataset for storing results. Leave empty to use the default run dataset.
## `keyValueStoreName` (type: `string`):
Name or ID of the key-value store for content files. Leave empty to use the default store.
## `requestQueueName` (type: `string`):
Name of the request queue for pending URLs. Leave empty to use the default queue.
## `storeSkippedUrls` (type: `boolean`):
If enabled, pushes a dataset record for each URL skipped during crawling (excluded by globs, robots.txt, depth limit, or concurrency cap). Can produce high record volume — enable for auditing only.
## `proxyConfiguration` (type: `object`):
Enables loading websites from IP addresses in specific geographies and to circumvent blocking.
## `proxyRotation` (type: `string`):
Proxy rotation strategy. recommended automatically picks the best proxies. per-request uses a new proxy for each request. until-failure uses one proxy until it fails.
## `sessionPoolName` (type: `string`):
Name for a persistent, shared session pool. Sessions (IP + cookies) are saved under this key and reused across Actor runs. Useful when proxies are frequently blocked — previously working sessions are preferred over random ones.
## `maxSessionRotations` (type: `integer`):
Maximum number of session (IP + browser fingerprint) rotations per request on block detection. Independent of maxRequestRetries. Set to 0 to disable session rotation.
## `navigationTimeoutSecs` (type: `integer`):
Maximum time to wait for page navigation in seconds
## `blockMedia` (type: `boolean`):
Block loading of images, stylesheets, fonts (.woff), PDFs, and ZIPs. On by default: it cuts browser memory and bandwidth substantially, which helps avoid out-of-memory on large pages. Disable it (set to false) if a page needs media to render its content (e.g. image- or CSS-driven lazy loading). Has no effect when using the raw HTTP crawler type or non-Chromium browsers (Chromium only).
## `waitForSelector` (type: `string`):
Wait for this CSS selector to appear before extracting content. The request fails and is retried if the selector does not appear within the timeout. Leave empty to disable.
## `softWaitForSelector` (type: `string`):
Wait for this CSS selector to appear before extracting content. Unlike waitForSelector, the request continues even if the selector does not appear within the timeout. Leave empty to disable.
## `waitForDynamicContentSecs` (type: `integer`):
Maximum seconds to wait for dynamic page content to load after navigation. The crawler continues when the network goes idle or this timeout elapses, whichever comes first. 0 disables this wait. Also used as the timeout for waitForSelector and softWaitForSelector.
## `waitUntil` (type: `string`):
When to consider navigation finished. networkidle waits for 500ms of network silence (best for JS-heavy SPAs, slower); load waits for the load event (default, good for most articles); domcontentloaded is fastest but may fire before client-side rendering completes; commit fires when network response is received and the document has started loading.
## `headless` (type: `boolean`):
Run browser in headless mode
## `ignoreCorsAndCsp` (type: `boolean`):
Ignore Content Security Policy and Cross-Origin Resource Sharing restrictions. Enables free XHR/Fetch requests from pages.
## `closeCookieModals` (type: `boolean`):
Automatically handle cookie consent: Ghostery-based ad/tracker blocking, accepting consent walls that replace the page (e.g. consent-or-pay) via the site’s own consent manager and re-fetching the article, and removing residual consent/CMP containers before extraction.
## `maxScrollHeight` (type: `integer`):
Maximum pixels (px) to scroll down the page until all content is loaded. Setting to 0 disables scrolling.
## `userAgent` (type: `string`):
Custom User-Agent string for the browser. Leave empty to use the default browser User-Agent.
## `ignoreHttpsErrors` (type: `boolean`):
Ignore HTTPS certificate errors. Use at your own risk.
## Actor input object example
```json
{
"startUrls": [
{
"url": "https://blog.apify.com/what-is-web-scraping/"
}
],
"crawlerType": "playwright-adaptive",
"renderingTypeDetectionRatio": 0,
"markdownDiscovery": "off",
"globs": [],
"exclude": [],
"selector": "",
"keepUrlFragment": false,
"useSitemaps": false,
"deduplication": "standard",
"respectRobotsTxtFile": false,
"initialCookies": [],
"customHttpHeaders": {},
"maxRequestsPerCrawl": 0,
"maxResultsPerCrawl": 0,
"maxCrawlDepth": 0,
"initialConcurrency": 0,
"maxConcurrency": 3,
"maxRequestRetries": 3,
"mode": "balanced",
"imageHandling": "exclude",
"maxImageEdge": 1568,
"rasterizeSvg": true,
"linkHandling": "include",
"tableHandling": "include",
"commentHandling": "include",
"languageCode": "",
"outputLayout": "minimal",
"save": [
"markdown-kvs"
],
"storeSkippedUrls": false,
"proxyRotation": "recommended",
"maxSessionRotations": 10,
"navigationTimeoutSecs": 60,
"blockMedia": true,
"waitForSelector": "",
"softWaitForSelector": "",
"waitForDynamicContentSecs": 10,
"waitUntil": "load",
"headless": true,
"ignoreCorsAndCsp": false,
"closeCookieModals": true,
"maxScrollHeight": 5000,
"userAgent": "",
"ignoreHttpsErrors": false
}
```
# Actor output Schema
## `overview` (type: `string`):
No description
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"startUrls": [
{
"url": "https://blog.apify.com/what-is-web-scraping/"
}
],
"initialCookies": [],
"customHttpHeaders": {}
};
// Run the Actor and wait for it to finish
const run = await client.actor("markdownee/crawler").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {
"startUrls": [{ "url": "https://blog.apify.com/what-is-web-scraping/" }],
"initialCookies": [],
"customHttpHeaders": {},
}
# Run the Actor and wait for it to finish
run = client.actor("markdownee/crawler").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"startUrls": [
{
"url": "https://blog.apify.com/what-is-web-scraping/"
}
],
"initialCookies": [],
"customHttpHeaders": {}
}' |
apify call markdownee/crawler --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"type": "http",
"url": "https://mcp.apify.com/?tools=fetch-actor-details,markdownee/crawler"
}
}
}
```
The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).
## OpenAPI specification
Download the OpenAPI definition: https://api.apify.com/v2/actors/40m9bPPlI58mdR5CD/builds/DuPfS18SRJLQs2JWP/openapi.json
