# GOV.UK Content Search Scraper (`parseforge/govuk-content-search-scraper`) Actor Scrape GOV.UK: search the entire UK government publications catalogue (policies, guidance, news, statistics). Filter by query, organisation, format or date. Returns titles, descriptions, URLs, organisations and publication dates. - **URL**: https://apify.com/parseforge/govuk-content-search-scraper.md - **Developed by:** [ParseForge](https://apify.com/parseforge) (community) - **Categories:** Business, Automation, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $22.87 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ![ParseForge Banner](https://github.com/ParseForge/apify-assets/blob/main/banners/banner-default.jpg?raw=true) ## ๐Ÿ‡ฌ๐Ÿ‡ง GOV.UK Content Search Scraper > ๐Ÿš€ **Search the entire UK government catalogue in seconds.** Pull **600,000+ pages, publications, news stories, statistics, consultations, and services** straight from GOV.UK. Filter by query, organisation, format, or date. No sign-up, no manual paging, no parser to maintain. > ๐Ÿ•’ **Last updated:** 2026-05-15 ยท **๐Ÿ“Š 11 fields** per record ยท **๐Ÿ“š 600,000+ pages** ยท **๐Ÿ›๏ธ 1,400+ government bodies** ยท **๐Ÿ“‚ 130+ document formats** The **GOV.UK Content Search Scraper** queries the official UK government content index and returns up to **11 structured fields per record**, including titles, descriptions, URLs, document formats, publication dates, and the publishing organisations and world locations. GOV.UK is the canonical home for almost every UK government publication, news story, statistical release, consultation, and citizen service. The catalogue covers **the entire central UK government estate**, from the Cabinet Office and HM Treasury to HMRC, the Ministry of Defence, the Department for Transport, and over a thousand executive agencies, arms-length bodies, and tribunals. This Actor makes that data downloadable as CSV, Excel, JSON, or XML in under five minutes. Filters run server-side, so you skip the parser engineering entirely. | ๐ŸŽฏ Target Audience | ๐Ÿ’ก Primary Use Cases | |---|---| | Policy analysts, regulatory and compliance teams, journalists, lobbyists, GovTech vendors, academic researchers, market-intelligence firms | Policy monitoring, regulatory horizon-scanning, consultation tracking, FOI release feeds, ministerial speech analysis, statistics release pipelines, press monitoring | --- ### ๐Ÿ“‹ What the GOV.UK Content Search Scraper does Six filtering workflows in a single run: - ๐Ÿ”Ž **Free-text search.** Query any keyword or phrase across titles, descriptions, and body text of every GOV.UK page. - ๐Ÿ“‚ **Format filter.** Restrict to a single GOV.UK document format from a list of 130+ (news_story, press_release, guidance, official_statistics, consultation_outcome, statutory_guidance, FOI release, and many more). - ๐Ÿ›๏ธ **Organisation filter.** Restrict to a single department or agency by slug (e.g. `cabinet-office`, `hm-revenue-customs`, `department-for-transport`). - ๐Ÿ“… **Date range filter.** `publishedAfter` and `publishedBefore` scope to any window on the public timestamp. - ๐Ÿ”ข **Sort order.** Relevance (default), newest first, oldest first, title A-Z, or most popular. - ๐ŸŒ **World locations and topical events.** Returned per record so you can pivot by country or government event. Each record includes the document title, description, canonical GOV.UK URL, content ID, format and document type, publication date, every publishing organisation (with slug and acronym), associated world locations, and topical events. > ๐Ÿ’ก **Why it matters:** UK government publications drive regulation, market opportunities, public-sector procurement, and the news cycle. Building your own GOV.UK pipeline means writing a paginated search client, mapping 130+ formats, joining organisation slugs, and refreshing daily. This Actor skips all of that and gives you a clean refreshed snapshot on every run. --- ### ๐ŸŽฌ Full Demo _๐Ÿšง Coming soon: a 3-minute walkthrough showing how to go from sign-up to a downloaded GOV.UK dataset._ --- ### โš™๏ธ Input
InputTypeDefaultBehavior
querystring"vehicle tax"Free-text search across the full GOV.UK catalogue. Empty = browse all pages of the chosen format / organisation.
formatstring""One of 130+ GOV.UK document formats. Empty = all formats.
organisationstring""Organisation slug (e.g. cabinet-office, hm-revenue-customs). Empty = all organisations.
publishedAfterstring""Earliest publication date (YYYY-MM-DD).
publishedBeforestring""Latest publication date (YYYY-MM-DD).
orderBystring""Sort order: relevance, newest, oldest, title A-Z, or most popular.
maxItemsinteger10Records to return. Free plan caps at 10, paid plan at 1,000,000.
**Example: 50 most recent HMRC press releases.** ```json { "maxItems": 50, "format": "press_release", "organisation": "hm-revenue-customs", "orderBy": "-public_timestamp" } ```` **Example: every Department for Transport publication mentioning "low traffic neighbourhood" since 2024.** ```json { "maxItems": 200, "query": "low traffic neighbourhood", "organisation": "department-for-transport", "publishedAfter": "2024-01-01" } ``` > โš ๏ธ **Good to Know:** GOV.UK formats evolve over time. Older content may carry a legacy `format` value while newer content uses `documentType`. Both are emitted per record so you can pivot on whichever is most useful for your analysis. The `url` field always returns the canonical absolute GOV.UK link. *** ### ๐Ÿ“Š Output Each record carries up to **11 fields**. Download the dataset as CSV, Excel, JSON, or XML. #### ๐Ÿงพ Schema | Field | Type | Example | |---|---|---| | ๐Ÿ“Œ `title` | string | `"Tax your vehicle"` | | ๐Ÿ“ `description` | string | `"Renew or tax your vehicle for the first time using a reminder letter..."` | | ๐Ÿ”— `url` | string | `"https://www.gov.uk/vehicle-tax"` | | ๐Ÿ†” `contentId` | string | `"fa748fae-3de4-4266-ae85-0797ada3f40c"` | | ๐Ÿ“‚ `format` | string | `"transaction"` | | ๐Ÿ“‚ `documentType` | string | `"transaction"` | | ๐Ÿ“… `publishedAt` | ISO 8601 | `"2017-12-07T12:54:39Z"` | | ๐Ÿ›๏ธ `organisations` | array | `[{"title": "Driver and Vehicle Licensing Agency", "slug": "driver-and-vehicle-licensing-agency", "acronym": "DVLA"}]` | | ๐ŸŒ `worldLocations` | array | `[{"title": "France", "slug": "france"}]` | | ๐Ÿท๏ธ `topicalEvents` | array | `[{"title": "Spring Budget 2024", "slug": "spring-budget-2024"}]` | | ๐Ÿ•’ `scrapedAt` | ISO 8601 | `"2026-05-15T18:29:40.375Z"` | #### ๐Ÿ“ฆ Sample record
๐Ÿš— Citizen service: Tax your vehicle (DVLA) ```json { "title": "Tax your vehicle", "description": "Renew or tax your vehicle for the first time using a reminder letter, your log book or the green 'new keeper' slip - and how to tax if you do not have any documents", "url": "https://www.gov.uk/vehicle-tax", "contentId": "fa748fae-3de4-4266-ae85-0797ada3f40c", "format": "transaction", "documentType": "transaction", "publishedAt": "2017-12-07T12:54:39Z", "organisations": [ { "title": "Driver and Vehicle Licensing Agency", "slug": "driver-and-vehicle-licensing-agency", "acronym": "DVLA", "type": "executive_agency" } ], "scrapedAt": "2026-05-15T18:29:40.375Z" } ```
*** ### โœจ Why choose this Actor | | Capability | |---|---| | ๐Ÿ“š | **Whole-of-government coverage.** 600,000+ pages from 1,400+ central government bodies, agencies, and tribunals. | | ๐ŸŽฏ | **Multi-dimensional filters.** Query, format, organisation, date range, and sort order combine freely. | | ๐Ÿ›๏ธ | **Organisation joins.** Each record names every publishing body with slug and acronym for clean joins to your CRM. | | โšก | **Fast.** 50 pages in seconds, 10,000 records in a few minutes. | | ๐ŸŒ | **Authoritative source.** Cited by policy researchers, lobbyists, and regulatory teams across the UK. | | ๐Ÿ” | **Always fresh.** Every run hits the live catalogue, so your dataset reflects current publications. | | ๐Ÿšซ | **No authentication.** Works with public open-government data. No login needed. | > ๐Ÿ“Š Searchable government publications are the foundation of every regulatory horizon-scanning tool, policy newsletter, and procurement dashboard in the UK. *** ### ๐Ÿ“ˆ How it compares to alternatives | Approach | Cost | Coverage | Refresh | Filters | Setup | |---|---|---|---|---|---| | **โญ GOV.UK Content Search Scraper** *(this Actor)* | $5 free credit, then pay-per-use | **600,000+ pages, 130+ formats** | **Live per run** | query, format, organisation, date range, sort | โšก 2 min | | Commercial policy-monitoring platforms | $10k - $100k/year | Comparable + summaries | Daily | Many | ๐Ÿข Weeks (procurement) | | RSS feeds per organisation | Free | Limited per feed | Hourly | Few | ๐Ÿ•’ Hours (one feed at a time) | | Manual GOV.UK browsing | Free | Whole site | Live | Same as the website | โณ Forever (no automation) | Pick this Actor when you want server-side cross-organisation search, structured records, and zero pipeline maintenance. *** ### ๐Ÿš€ How to use 1. ๐Ÿ“ **Sign up.** [Create a free account w/ $5 credit](https://console.apify.com/sign-up?fpr=vmoqkp) (takes 2 minutes). 2. ๐ŸŒ **Open the Actor.** Go to the GOV.UK Content Search Scraper page on the Apify Store. 3. ๐ŸŽฏ **Set input.** Type a query (or leave empty), pick an organisation or format, set a date window if you need one, and set `maxItems`. 4. ๐Ÿš€ **Run it.** Click **Start** and let the Actor collect your results. 5. ๐Ÿ“ฅ **Download.** Grab your dataset in the **Dataset** tab as CSV, Excel, JSON, or XML. > โฑ๏ธ Total time from signup to a downloaded GOV.UK dataset: **3-5 minutes.** No coding required. *** ### ๐Ÿ’ผ Business use cases
#### ๐Ÿ›๏ธ Policy & Regulatory Monitoring - Daily horizon-scanning across every UK department - Filter by `consultation_outcome` to track ended consultations - Pivot on `topicalEvents` for budget and statement coverage - Build a structured policy-change feed for compliance teams #### ๐Ÿ“ฐ News, PR & Communications - Press monitoring for ministerial speeches and statements - Track press releases by department for media briefings - Catch FOI releases the moment they go public - Power newsroom dashboards with structured GOV.UK feeds
#### ๐Ÿ“Š Statistics & Open Data - Pull every `official_statistics` release for analytics pipelines - Track `statistics_announcement` for upcoming publications - Cross-join with publishing organisation for sector views - Replace brittle RSS parsing with structured records #### ๐Ÿ›’ Public Procurement & GovTech - Monitor `procurement` and `contract` notices by organisation - Spot upcoming consultations that signal policy direction - Build vendor dashboards from `corporate_report` releases - Feed CRM enrichment with department slugs and acronyms
*** ### ๐Ÿ”Œ Automating GOV.UK Scraper Control the scraper programmatically for scheduled runs and pipeline integrations: - ๐ŸŸข **Node.js.** Install the `apify-client` NPM package. - ๐Ÿ **Python.** Use the `apify-client` PyPI package. - ๐Ÿ“š See the [Apify API documentation](https://docs.apify.com/api/v2) for full details. The [Apify Schedules feature](https://docs.apify.com/platform/schedules) lets you trigger this Actor on any cron interval. Hourly or daily refreshes keep downstream policy monitors and dashboards in sync automatically. *** ### ๐ŸŒŸ Beyond business use cases Open government data powers more than commercial workflows. The same structured records support research, education, civic projects, and personal initiatives.
#### ๐ŸŽ“ Research and academia - Quantitative studies of policy publication patterns - Public-administration coursework on government output - Historical archives of consultations and outcomes - Reproducible datasets for political-science research #### ๐ŸŽจ Personal and creative - Side projects mapping departmental publication volumes - Newsletter generators that summarise weekly releases - Personal RSS-style feeds across multiple organisations - Visualisations of policy themes over time
#### ๐Ÿค Non-profit and civic - Civic-tech tools that surface relevant consultations to citizens - Investigative journalism on department-level disclosure patterns - Watchdog dashboards tracking ministerial communications - Accessibility projects that reformat GOV.UK content #### ๐Ÿงช Experimentation - Train classifiers that auto-tag policy areas - Build agent pipelines that summarise daily releases - Prototype recommender systems for citizen services - Stress-test search infrastructure with real volume data
*** ### ๐Ÿค– Ask an AI assistant about this scraper Open a ready-to-send prompt about this ParseForge actor in the AI of your choice: - ๐Ÿ’ฌ [**ChatGPT**](https://chat.openai.com/?q=How%20do%20I%20use%20the%20GOV.UK%20Content%20Search%20Scraper%20by%20ParseForge%20on%20Apify%3F%20Show%20me%20input%20examples%2C%20output%20fields%2C%20common%20use%20cases%2C%20and%20how%20to%20integrate%20it%20into%20a%20workflow.) - ๐Ÿง  [**Claude**](https://claude.ai/new?q=How%20do%20I%20use%20the%20GOV.UK%20Content%20Search%20Scraper%20by%20ParseForge%20on%20Apify%3F%20Show%20me%20input%20examples%2C%20output%20fields%2C%20common%20use%20cases%2C%20and%20how%20to%20integrate%20it%20into%20a%20workflow.) - ๐Ÿ” [**Perplexity**](https://perplexity.ai/search?q=How%20do%20I%20use%20the%20GOV.UK%20Content%20Search%20Scraper%20by%20ParseForge%20on%20Apify%3F%20Show%20me%20input%20examples%2C%20output%20fields%2C%20common%20use%20cases%2C%20and%20how%20to%20integrate%20it%20into%20a%20workflow.) - ๐Ÿ…’ [**Copilot**](https://copilot.microsoft.com/?q=How%20do%20I%20use%20the%20GOV.UK%20Content%20Search%20Scraper%20by%20ParseForge%20on%20Apify%3F%20Show%20me%20input%20examples%2C%20output%20fields%2C%20common%20use%20cases%2C%20and%20how%20to%20integrate%20it%20into%20a%20workflow.) *** ### โ“ Frequently Asked Questions #### ๐Ÿงฉ How does it work? Type a query, optionally pick an organisation or format, click Start, and the Actor pages through the official GOV.UK content index, applies your filters, and emits a clean structured record per page. No browser automation, no captchas, no setup. #### ๐Ÿ“ How accurate is the data? Every record comes from the canonical GOV.UK content index used by the gov.uk website itself, so titles, descriptions, dates, and organisation references match what you see on the page. #### ๐Ÿ” How often is the dataset refreshed? GOV.UK is updated continuously as departments publish new pages. Every run of this Actor hits the live catalogue. #### ๐Ÿ“‚ Which document formats are supported? 130+ GOV.UK formats including news\_story, press\_release, guidance, official\_statistics, consultation\_outcome, statutory\_guidance, FOI release, statistical\_data\_set, transparency, corporate\_report, decision, and many more. Use the `format` filter to scope to one type. #### ๐Ÿ›๏ธ How do I find an organisation slug? The slug is the last segment of the organisation page URL. For example, the Driver and Vehicle Licensing Agency lives at `https://www.gov.uk/government/organisations/driver-and-vehicle-licensing-agency`, so its slug is `driver-and-vehicle-licensing-agency`. #### ๐Ÿ“… Can I scope to a date range? Yes. Use `publishedAfter` and `publishedBefore` (YYYY-MM-DD). Both are inclusive and run against the document's public timestamp. #### โฐ Can I schedule regular runs? Yes. Use Apify Schedules to run this Actor on any cron interval (hourly, daily, weekly) and keep downstream policy monitors in sync. #### โš–๏ธ Is this data legal to use? GOV.UK content is published under the Open Government Licence v3.0, which permits commercial reuse with attribution. Review the licence terms for your specific application. #### ๐Ÿ’ผ Can I use this data commercially? Yes. Open Government Licence v3.0 explicitly allows commercial reuse with attribution. You remain responsible for following the licence terms in your product. #### ๐Ÿ’ณ Do I need a paid Apify plan to use this Actor? No. The free Apify plan is enough for testing and small runs (10 records per run). A paid plan lifts the limit and gives you scheduling, higher concurrency, and larger datasets. #### ๐Ÿ” What happens if a run fails or gets interrupted? Apify automatically retries transient errors. If a run still fails, you can inspect the log in the Runs tab, fix the input, and re-run. Partial datasets from failed runs are preserved so you never lose progress. #### ๐Ÿ†˜ What if I need help? Our support team is here to help. Contact us through the Apify platform or use the Tally form linked below. *** ### ๐Ÿ”Œ Integrate with any app GOV.UK Content Search Scraper connects to any cloud service via [Apify integrations](https://apify.com/integrations): - [**Make**](https://docs.apify.com/platform/integrations/make) - Automate multi-step monitoring workflows - [**Zapier**](https://docs.apify.com/platform/integrations/zapier) - Connect with 5,000+ apps - [**Slack**](https://docs.apify.com/platform/integrations/slack) - Get new-publication alerts in your channels - [**Airbyte**](https://docs.apify.com/platform/integrations/airbyte) - Pipe GOV.UK pages into your warehouse - [**GitHub**](https://docs.apify.com/platform/integrations/github) - Trigger runs from commits and releases - [**Google Drive**](https://docs.apify.com/platform/integrations/drive) - Export datasets straight to Sheets You can also use webhooks to trigger downstream actions when a run finishes. Push fresh policy publications into your CRM, or alert your communications team in Slack. *** ### ๐Ÿ”— Recommended Actors - [**๐Ÿ›๏ธ UK Parliament Members Scraper**](https://apify.com/parseforge/members-uk-parliament-scraper) - MPs and Lords with biographies, committees, and contact details - [**๐Ÿ—ฃ๏ธ Hansard UK Parliament Debates Scraper**](https://apify.com/parseforge/hansard-uk-debates-scraper) - Full transcripts of Commons and Lords debates - [**๐Ÿ›ก๏ธ OpenSanctions Sanctions & PEP Scraper**](https://apify.com/parseforge/opensanctions-entities-scraper) - 280k+ sanctioned entities and PEPs - [**โšก Carbon Intensity UK Scraper**](https://apify.com/parseforge/carbon-intensity-uk-scraper) - National Grid carbon intensity forecasts - [**๐Ÿš‚ OurAirports Global Airport Database Scraper**](https://apify.com/parseforge/ourairports-scraper) - 85,000+ airports worldwide > ๐Ÿ’ก **Pro Tip:** browse the complete [ParseForge collection](https://apify.com/parseforge) for more reference-data scrapers. *** **๐Ÿ†˜ Need Help?** [**Open our contact form**](https://tally.so/r/BzdKgA) to request a new scraper, propose a custom data project, or report an issue. *** > **โš ๏ธ Disclaimer:** this Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by the UK Government, the Government Digital Service, or any UK department. All trademarks mentioned are the property of their respective owners. Only publicly available open-government content is collected, under the Open Government Licence. # Actor input Schema ## `query` (type: `string`): Free-text search across the full GOV.UK catalogue (titles, descriptions, body). Leave empty to browse all pages of the chosen format / organisation. ## `format` (type: `string`): Restrict to a single GOV.UK document format. The full GOV.UK list is available; pick one or leave empty for all formats. ## `organisation` (type: `string`): Restrict to a single UK government organisation by slug. Pick from the list (covers ministerial departments, non-ministerial departments, agencies and other public bodies). Leave empty for all organisations. ## `topicalEvent` (type: `string`): Restrict to a single GOV.UK topical event by slug (e.g. 'spring-budget-2024'). Find slugs at gov.uk/government/topical-events. Leave empty to skip. ## `worldLocation` (type: `string`): Restrict to a single foreign country / world location by slug (e.g. 'france', 'ukraine'). Find slugs at gov.uk/world. Leave empty to skip. ## `person` (type: `string`): Restrict to pages tagged with a specific person (e.g. minister) by slug (e.g. 'rishi-sunak'). Leave empty to skip. ## `taxon` (type: `string`): Restrict to a single GOV.UK taxonomy topic by slug (e.g. 'business-and-industry'). Leave empty to skip. ## `partOfTaxonomyTree` (type: `string`): Restrict to all descendants of a taxonomy node by content\_id. Leave empty to skip. ## `manual` (type: `string`): Restrict to sections of a single GOV.UK manual by slug (e.g. 'hmrc-internal-manuals/vat-supply-and-consideration'). Leave empty to skip. ## `publishedAfter` (type: `string`): Earliest publication date (inclusive). Format: YYYY-MM-DD. ## `publishedBefore` (type: `string`): Latest publication date (inclusive). Format: YYYY-MM-DD. ## `orderBy` (type: `string`): Sort order. Default is GOV.UK relevance. ## `maxItems` (type: `integer`): Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000 ## Actor input object example ```json { "query": "", "format": "", "organisation": "", "topicalEvent": "", "worldLocation": "", "person": "", "taxon": "", "partOfTaxonomyTree": "", "manual": "", "orderBy": "", "maxItems": 10 } ``` # Actor output Schema ## `overview` (type: `string`): Overview of scraped data ## `fullData` (type: `string`): Complete dataset # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "query": "", "topicalEvent": "", "worldLocation": "", "person": "", "taxon": "", "partOfTaxonomyTree": "", "manual": "", "maxItems": 10 }; // Run the Actor and wait for it to finish const run = await client.actor("parseforge/govuk-content-search-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "query": "", "topicalEvent": "", "worldLocation": "", "person": "", "taxon": "", "partOfTaxonomyTree": "", "manual": "", "maxItems": 10, } # Run the Actor and wait for it to finish run = client.actor("parseforge/govuk-content-search-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "query": "", "topicalEvent": "", "worldLocation": "", "person": "", "taxon": "", "partOfTaxonomyTree": "", "manual": "", "maxItems": 10 }' | apify call parseforge/govuk-content-search-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=parseforge/govuk-content-search-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "GOV.UK Content Search Scraper", "description": "Scrape GOV.UK: search the entire UK government publications catalogue (policies, guidance, news, statistics). Filter by query, organisation, format or date. Returns titles, descriptions, URLs, organisations and publication dates.", "version": "0.0", "x-build-id": "w9KCwM0UG14UQgM5g" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/parseforge~govuk-content-search-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-parseforge-govuk-content-search-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/parseforge~govuk-content-search-scraper/runs": { "post": { "operationId": "runs-sync-parseforge-govuk-content-search-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/parseforge~govuk-content-search-scraper/run-sync": { "post": { "operationId": "run-sync-parseforge-govuk-content-search-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "query": { "title": "Search Query", "type": "string", "description": "Free-text search across the full GOV.UK catalogue (titles, descriptions, body). Leave empty to browse all pages of the chosen format / organisation.", "default": "" }, "format": { "title": "Document Format", "enum": [ "", "employment_tribunal_decision", "hmrc_manual_section", "news_story", "press_release", "research_for_development_output", "guidance", "official_statistics", "transparency", "statistics_announcement", "national_statistics", "research", "residential_property_tribunal_decision", "world_news_story", "corporate_report", "policy_paper", "armed_forces_covenant_business", "speech", "decision", "foi_release", "aaib_report", "detailed_guide", "correspondence", "consultation_outcome", "document_collection", "notice", "person", "protected_food_drink_name", "form", "independent_report", "product_safety_alert_report_recall", "manual_section", "cma_case", "employment_appeal_tribunal_decision", "case_study", "written_statement", "export_health_certificate", "utaac_decision", "impact_assessment", "international_treaty", "european_structural_investment_fund", "medical_safety_alert", "tax_tribunal_decision", "world_location_news_article", "statutory_guidance", "organisation", "statistical_data_set", "about", "closed_consultation", "maib_report", "ministerial_role", "oral_statement", "promotional", "policy_group", "drug_safety_update", "guide", "regulation", "answer", "government_response", "raib_report", "authored_article", "fatality_notice", "service_standard_report", "recommended-link", "call_for_evidence_outcome", "licence_transaction", "map", "worldwide_organisation", "flood_and_coastal_erosion_risk_management_research_report", "statutory_instrument", "animal_disease_case", "complaints_procedure", "recruitment", "hmrc_manual", "transaction", "world_location_news", "travel_advice", "taxon", "closed_call_for_evidence", "service_manual_guide", "manual", "traffic_commissioner_regulatory_decision", "mainstream_browse_page", "countryside_stewardship_grant", "topical_event", "our_governance", "personal_information_charter", "international_development_fund", "farming_grant", "algorithmic_transparency_record", "drcf_digital_markets_research", "veterans_support_organisation", "design_decision", "hmrc_contact", "local_transaction", "accessible_documents_policy", "marine_equipment_approved_recommendation", "publication_scheme", "asylum_support_decision", "business_finance_support_scheme", "ai_assurance_portfolio_technique", "media_enquiries", "procurement", "open_consultation", "equality_and_diversity", "standard", "finder", "social_media_use", "membership", "terms_of_reference", "step_by_step_nav", "life_saving_maritime_appliance_service_station", "simple_smart_answer", "statistics", "about_our_services", "data_ethics_guidance_document", "smart_answer", "sfo_case", "welsh_language_scheme", "modern_slavery_statement", "place", "access_and_opening", "our_energy_use", "open_call_for_evidence", "ukhsa_data_access_approval", "staff_update", "help_page", "inside-government-link", "service_manual_topic", "history", "operational_field", "world_location_news_page", "special_route", "calendar", "digital_markets_measure", "edition", "coronavirus_landing_page", "service_manual_service_toolkit", "travel_advice_index" ], "type": "string", "description": "Restrict to a single GOV.UK document format. The full GOV.UK list is available; pick one or leave empty for all formats.", "default": "" }, "organisation": { "title": "Organisation Slug", "enum": [ "", "cabinet-office", "hm-revenue-customs", "hm-treasury", "home-office", "ministry-of-defence", "ministry-of-justice", "department-for-transport", "department-for-education", "department-for-business-and-trade", "department-for-energy-security-and-net-zero", "department-for-environment-food-rural-affairs", "department-for-science-innovation-and-technology", "department-of-health-and-social-care", "department-for-work-pensions", "department-for-culture-media-and-sport", "department-for-levelling-up-housing-and-communities", "foreign-commonwealth-development-office", "attorney-generals-office", "office-of-the-secretary-of-state-for-scotland", "office-of-the-secretary-of-state-for-wales", "northern-ireland-office", "the-office-of-the-leader-of-the-house-of-commons", "the-office-of-the-leader-of-the-house-of-lords", "uk-export-finance", "office-for-national-statistics", "ofgem", "ofcom", "ofqual", "ofsted", "office-of-rail-and-road", "competition-and-markets-authority", "national-crime-agency", "environment-agency", "natural-england", "uk-health-security-agency", "medicines-and-healthcare-products-regulatory-agency", "nhs-business-services-authority", "uk-statistics-authority", "driver-and-vehicle-licensing-agency", "driver-and-vehicle-standards-agency", "highways-england", "national-highways", "hm-courts-and-tribunals-service", "hm-prison-and-probation-service", "land-registry", "companies-house", "charity-commission", "intellectual-property-office", "uk-space-agency", "civil-aviation-authority", "maritime-and-coastguard-agency", "rural-payments-agency", "valuation-office-agency", "the-pensions-regulator", "financial-conduct-authority", "bank-of-england", "office-for-budget-responsibility", "government-actuarys-department", "crown-prosecution-service", "serious-fraud-office", "uk-debt-management-office", "national-audit-office", "government-equalities-office", "geospatial-commission", "ofs", "uk-research-and-innovation", "innovate-uk", "british-business-bank" ], "type": "string", "description": "Restrict to a single UK government organisation by slug. Pick from the list (covers ministerial departments, non-ministerial departments, agencies and other public bodies). Leave empty for all organisations.", "default": "" }, "topicalEvent": { "title": "Topical Event Slug", "type": "string", "description": "Restrict to a single GOV.UK topical event by slug (e.g. 'spring-budget-2024'). Find slugs at gov.uk/government/topical-events. Leave empty to skip.", "default": "" }, "worldLocation": { "title": "World Location Slug", "type": "string", "description": "Restrict to a single foreign country / world location by slug (e.g. 'france', 'ukraine'). Find slugs at gov.uk/world. Leave empty to skip.", "default": "" }, "person": { "title": "Person Slug", "type": "string", "description": "Restrict to pages tagged with a specific person (e.g. minister) by slug (e.g. 'rishi-sunak'). Leave empty to skip.", "default": "" }, "taxon": { "title": "Taxon (Topic) Slug", "type": "string", "description": "Restrict to a single GOV.UK taxonomy topic by slug (e.g. 'business-and-industry'). Leave empty to skip.", "default": "" }, "partOfTaxonomyTree": { "title": "Part of Taxonomy Tree", "type": "string", "description": "Restrict to all descendants of a taxonomy node by content_id. Leave empty to skip.", "default": "" }, "manual": { "title": "Manual Slug", "type": "string", "description": "Restrict to sections of a single GOV.UK manual by slug (e.g. 'hmrc-internal-manuals/vat-supply-and-consideration'). Leave empty to skip.", "default": "" }, "publishedAfter": { "title": "Published After", "type": "string", "description": "Earliest publication date (inclusive). Format: YYYY-MM-DD." }, "publishedBefore": { "title": "Published Before", "type": "string", "description": "Latest publication date (inclusive). Format: YYYY-MM-DD." }, "orderBy": { "title": "Order By", "enum": [ "", "-public_timestamp", "public_timestamp", "title", "-popularity" ], "type": "string", "description": "Sort order. Default is GOV.UK relevance.", "default": "" }, "maxItems": { "title": "Max Items", "minimum": 1, "maximum": 1000000, "type": "integer", "description": "Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000" } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```