# Blog Content Crawler and Summarizer

**Use case:** 

Crawl all blog posts on a website. Get the content as Markdown and AI summary. If the summary is relevant, feed the content to your AI.

## Input

```json
{
  "startUrls": [
    {
      "url": "https://www.erste.pl/"
    }
  ],
  "crawlerType": "playwright:adaptive",
  "includeUrlGlobs": [],
  "excludeUrlGlobs": [
    {
      "glob": "https://www.erste.pl/kariera/**"
    },
    {
      "glob": "https://www.erste.pl/kariera*"
    },
    {
      "glob": "https://www.erste.pl/oferty-pracy*"
    },
    {
      "glob": "https://www.erste.pl/en/**"
    },
    {
      "glob": "https://www.erste.pl/en*"
    },
    {
      "glob": "https://www.erste.pl/uk/**"
    },
    {
      "glob": "https://www.erste.pl/uk*"
    },
    {
      "glob": "https://www.erste.pl/przydatne-informacje/**"
    },
    {
      "glob": "https://www.erste.pl/przydatne-informacje*"
    },
    {
      "glob": "https://www.erste.pl/polityka-cookies/**"
    },
    {
      "glob": "https://www.erste.pl/polityka-cookies*"
    },
    {
      "glob": "https://www.centrum24.pl/**"
    },
    {
      "glob": "https://www.erste.pl/wyniki-wyszukiwania/**"
    },
    {
      "glob": "https://www.erste.pl/wyniki-wyszukiwania*"
    },
    {
      "glob": "https://www.erste.pl/bank-porad/**"
    },
    {
      "glob": "https://www.erste.pl/bank-porad*"
    },
    {
      "glob": "https://www.erste.pl/rodo/**"
    },
    {
      "glob": "https://www.erste.pl/rodo*"
    },
    {
      "glob": "https://www.erste.pl/relacje-inwestorskie/**"
    },
    {
      "glob": "https://www.erste.pl/relacje-inwestorskie*"
    },
    {
      "glob": "https://www.erste.pl/tfi/o-nas/**"
    },
    {
      "glob": "https://www.erste.pl/tfi/o-nas*"
    },
    {
      "glob": "https://www.erste.pl/tfi/notowania/**"
    },
    {
      "glob": "https://www.erste.pl/tfi/notowania*"
    },
    {
      "glob": "https://www.erste.pl/tfi/nasi-eksperci-komentuja*"
    },
    {
      "glob": "https://www.erste.pl/serwis-ekonomiczny/**"
    },
    {
      "glob": "https://www.erste.pl/serwis-ekonomiczny*"
    },
    {
      "glob": "https://www.erste.pl/inwestor/**"
    },
    {
      "glob": "https://www.erste.pl/inwestor*"
    },
    {
      "glob": "https://www.erste.pl/aktualnosci/**"
    },
    {
      "glob": "https://www.erste.pl/aktualnosci*"
    },
    {
      "glob": "https://www.erste.pl/tfi/aktualnosci/**"
    },
    {
      "glob": "https://www.erste.pl/tfi/aktualnosci*"
    },
    {
      "glob": "https://www.erste.pl/factoring/aktualnosci/**"
    },
    {
      "glob": "https://www.erste.pl/factoring/aktualnosci*"
    },
    {
      "glob": "https://www.erste.pl/erste-leasing/komunikaty-prasowe/**"
    },
    {
      "glob": "https://www.erste.pl/erste-leasing/komunikaty-prasowe*"
    },
    {
      "glob": "https://www.erste.pl/**/komunikaty-prasowe/**"
    },
    {
      "glob": "https://www.erste.pl/**/komunikaty-prasowe*"
    },
    {
      "glob": "https://www.erste.pl/finansiaki/**"
    },
    {
      "glob": "https://www.erste.pl/finansiaki*"
    },
    {
      "glob": "https://www.erste.pl/erste-education/**"
    },
    {
      "glob": "https://www.erste.pl/erste-education*"
    },
    {
      "glob": "https://www.erste.pl/tfi/edukacja/**"
    },
    {
      "glob": "https://www.erste.pl/tfi/edukacja*"
    },
    {
      "glob": "https://www.erste.pl/tfi/kontakt/**"
    },
    {
      "glob": "https://www.erste.pl/tfi/kontakt*"
    },
    {
      "glob": "https://www.erste.pl/klient-indywidualny/kontakt/**"
    },
    {
      "glob": "https://www.erste.pl/klient-indywidualny/kontakt*"
    },
    {
      "glob": "https://www.erste.pl/private-banking/kontakt*"
    },
    {
      "glob": "https://www.erste.pl/firmy/kontakt/reklamacje*"
    },
    {
      "glob": "https://www.erste.pl/obsluga-bez-barier/**"
    },
    {
      "glob": "https://www.erste.pl/obsluga-bez-barier*"
    },
    {
      "glob": "https://www.erste.pl/inwestor/**/produkty-strukturyzowane-private-banking/**"
    },
    {
      "glob": "https://www.erste.pl/pad/**"
    },
    {
      "glob": "https://www.erste.pl/pad*"
    },
    {
      "glob": "https://www.erste.pl/ppk*"
    },
    {
      "glob": "https://www.erste.pl/klient-indywidualny/loteria/**"
    },
    {
      "glob": "https://www.erste.pl/klient-indywidualny/loteria*"
    },
    {
      "glob": "https://www.erste.pl/**/bezcenne-chwile/**"
    },
    {
      "glob": "https://www.erste.pl/**/bezcenne-chwile*"
    },
    {
      "glob": "https://www.erste.pl/**/apple-pay/**"
    },
    {
      "glob": "https://www.erste.pl/**/apple-pay*"
    },
    {
      "glob": "https://www.erste.pl/**/google-pay/**"
    },
    {
      "glob": "https://www.erste.pl/**/google-pay*"
    },
    {
      "glob": "https://www.erste.pl/placowka-partnerska*"
    },
    {
      "glob": "https://www.erste.pl/fatca-crs*"
    },
    {
      "glob": "https://www.erste.pl/firmy/placowki-z-doradca-firmowym*"
    },
    {
      "glob": "https://www.erste.pl/factoring/o-nas*"
    },
    {
      "glob": "https://www.erste.pl/korporacje/kontakt/wyszukiwarka-wrzutni-online*"
    },
    {
      "glob": "https://www.erste.pl/erste-f24/kontakt*"
    },
    {
      "glob": "https://www.erste.pl/erste-f24/przydatne-informacje/reklamacje*"
    },
    {
      "glob": "https://www.erste.pl/klient-indywidualny/bankowosc-internetowa/bezpieczne-bankowanie*"
    },
    {
      "glob": "https://www.erste.pl/klient-indywidualny/bankowosc-internetowa/bezpieczne-bankowanie/**"
    },
    {
      "glob": "https://www.erste.pl/erste-f24/przydatne-informacje/rodo*"
    }
  ],
  "maxCrawlDepth": 20,
  "maxCrawlPages": 9999999,
  "useSitemaps": false,
  "useLlmsTxt": false,
  "respectRobotsTxtFile": true,
  "keepUrlFragments": false,
  "ignoreCanonicalUrl": true,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": []
  },
  "initialCookies": "***",
  "customHttpHeaders": {},
  "signHttpRequests": false,
  "initialConcurrency": 0,
  "maxConcurrency": 200,
  "requestTimeoutSecs": 90,
  "minFileDownloadSpeedKBps": 128,
  "maxRequestRetries": 5,
  "maxSessionRotations": 10,
  "ignoreHttpsErrors": false,
  "dynamicContentWaitSecs": 20,
  "waitForSelector": "",
  "softWaitForSelector": "",
  "maxScrollHeightPixels": 15000,
  "removeCookieWarnings": true,
  "blockMedia": true,
  "expandIframes": true,
  "clickElementsCssSelector": "main button[aria-expanded=\"false\"]",
  "pageFunction": "",
  "keepElementsCssSelector": "main,\n.banner_whitebox,\n.banner__whitebox",
  "removeElementsCssSelector": "script,\nstyle,\nnoscript,\nsvg,\nimg[src^='data:'],\n.header__search,\n.header__search-layer,\n.js-search-header,\n.search-header__autocompleter,\n.header__search-preloader,\n.header__search-frequently,\n.frequently_searched_container,\n.search_loader,\n.search-header__results-title,\n.header__search-show-all-container,\n[role=\"region\"][aria-label*=\"skip\" i],\n[role=\"dialog\"],\n[role=\"alertdialog\"],\n[aria-modal=\"true\"]",
  "htmlTransformer": "none",
  "readableTextCharThreshold": 100,
  "aggressivePrune": false,
  "debugMode": false,
  "debugLog": true,
  "storeSkippedUrls": false,
  "saveHtml": false,
  "saveHtmlAsFile": true,
  "saveMarkdown": true,
  "summarize": true,
  "saveFiles": false,
  "saveScreenshots": false,
  "maxResults": 9999999,
  "clientSideMinChangePercentage": 15,
  "renderingTypeDetectionPercentage": 10,
  "reuseStoredDetectionResults": false
}
```

## Output

```json
{
  "url": {
    "label": "URL",
    "format": "string"
  },
  "markdown": {
    "label": "Markdown",
    "format": "string"
  }
}
```

## About this Actor

This example demonstrates how to use [Website Content Crawler](https://apify.com/apify/website-content-crawler) with a specific input configuration. Visit the [Actor detail page](https://apify.com/apify/website-content-crawler) to learn more, explore other use cases, and run it yourself.


## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
This Task's input is already configured above — use it as-is rather than inventing a new one.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For full API examples (JavaScript, Python, CLI, MCP, OpenAPI), see this Task's Actor page: https://apify.com/apify/website-content-crawler.md

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).