# Changelog of Website Content Crawler: Markdown for AI and RAG (`automa-flow/website-content-crawler`) Actor

- **URL**: https://apify.com/automa-flow/website-content-crawler/changelog.md
- **Full Actor documentation**: https://apify.com/automa-flow/website-content-crawler.md

## Changelog

### 2026-09-29: Secure-for-AI mode for RAG pipelines (opt-in)

#### Features

- New `securityMode` input: `off` (default), `detect` or `sanitize`. With `off`, crawling, Markdown, hashes and billing stay exactly as before.
- `detect` adds a `security` object with findings to each page and keeps the original Markdown. Do not ingest a row while `security.ai_ingestion_allowed` is false.
- `sanitize` replaces supported secrets, emails, phone numbers and payment-card numbers with placeholders, including values assigned to names such as `DB_PASSWORD` or `AWS_SECRET_ACCESS_KEY`.
- In `sanitize`, a page that speaks to the model reading it, tells it to ignore its instructions or tries to replace its role is quarantined as `CONTENT_QUARANTINED` and is not billed. If every page is quarantined, the run is empty with that stop reason, not a source failure.
- Looser wording such as "ignore the above" marks a page for review (`ai_ingestion_allowed` false) instead of quarantining it.
- Documentation that describes tools, function calls or API keys is delivered normally.

#### Notes

- Pattern check for common attacks, mainly English, with Russian, German, Spanish and French "ignore previous instructions". A reworded attack can pass. Not DLP, GDPR compliance or a safety guarantee.

### 2026-09-26: More reliable requests and spending checks

#### Fixes

- Spending checks and billing receipts use the effective event prices supplied for your run, including discounts and zero-priced events.
- Compressed HTTP responses are limited while they are decoded, so a small compressed download cannot expand past the response limit in memory.