---

Data.gov.uk Dataset Scraper 🌐📊🇬🇧

The Data.gov.uk Dataset Scraper is a powerful and scalable Apify Actor designed to extract structured dataset listings directly from Data.gov.uk search result pages.

It enables open data discovery, public sector research, dataset monitoring, government data analysis, academic research, machine learning data collection, and structured dataset generation from the United Kingdom's official open data portal.

---

🎯 What This Scraper Does

Simply provide one or more Data.gov.uk search URLs and the scraper handles everything automatically.

✅ Extracts structured dataset listings from Data.gov.uk

✅ Supports bulk URL scraping

✅ Automatically processes search result pages

✅ Handles pagination automatically

✅ Applies maxItemsPerUrl limits

✅ Extracts dataset metadata and publication details

✅ Captures publisher information

✅ Collects dataset URLs and descriptions

✅ Generates clean and structured datasets

✅ Ready for analytics and research workflows

✅ Export-ready output format

---

📊 Data Extracted

🌐 Dataset Information

Field	Description
🆔 datasetId	Unique dataset identifier
📄 title	Dataset title
🏢 publishedBy	Publishing organization
🕒 lastUpdated	Dataset last update date
📝 description	Dataset description
🔗 datasetUrl	Dataset page URL
🌐 sourceUrl	Search URL source

---

🛠 How to Use

1️⃣ Configure Input

Provide one or multiple Data.gov.uk search URLs:

{
  "urls": [
    "https://www.data.gov.uk/search?filters%5Btopic%5D=Business+and+economy",
    "https://www.data.gov.uk/search?filters%5Btopic%5D=Health"
  ],
  "maxItemsPerUrl": 20
}

2️⃣ Run the Actor

• Loads Data.gov.uk search result pages

• Processes dataset listings automatically

• Extracts structured dataset information

• Collects publisher and update metadata

• Applies maxItemsPerUrl limits

• Stops automatically when limits are reached

3️⃣ Export the Dataset

Download your results in multiple formats:

✅ JSON

✅ CSV

✅ Excel

✅ XML

✅ HTML

---

⚙️ Input Configuration

📥 Input Example

{
  "urls": [
    "https://www.data.gov.uk/search?filters%5Btopic%5D=Business+and+economy",
    "https://www.data.gov.uk/search?filters%5Btopic%5D=Health"
  ],
  "maxItemsPerUrl": 20
}

Input Fields

Field	Type	Description
urls	array	List of Data.gov.uk search URLs
maxItemsPerUrl	integer	Maximum number of datasets to collect per URL (0 = unlimited)

---

📤 Output Example

{
  "datasetId": "a24c061b-d57e-4889-abbc-eead787f38e6",
  "title": "Bioscience and Health Technology Database",
  "publishedBy": "Department for Business, Energy and Industrial Strategy",
  "lastUpdated": "21 August 2020",
  "description": "This dataset is used by Departmental officials to analyse and produce annual reports and statistics within the UK Life Sciences sector.",
  "datasetUrl": "https://www.data.gov.uk/dataset/a24c061b-d57e-4889-abbc-eead787f38e6/bioscience-and-health-technology-database",
  "sourceUrl": "https://www.data.gov.uk/search?filters%5Btopic%5D=Business+and+economy"
}

---

📊 Output Explanation

Use Case	Description
📊 Open Data Research	Collect structured public datasets
🏛 Government Monitoring	Track newly published government datasets
📈 Data Science Projects	Build datasets for analytics and machine learning
📚 Academic Research	Gather datasets for studies and publications
🤖 Automation Pipelines	Feed open data into workflows and dashboards

---

🌍 Why Use This Scraper?

📊 Discover public datasets at scale

🏛 Monitor government open data publications

📈 Build research-ready structured datasets

🌍 Scrape multiple search topics simultaneously

⚡ Fast and automated extraction

🤖 Automation-ready output

📦 Bulk URL scraping support

🧠 Ideal for analysts, researchers, journalists, and data scientists

🚀 Scalable for both small and enterprise workloads

---

❓ FAQ

How does this scraper work?

The scraper loads Data.gov.uk search result pages and extracts structured dataset information including titles, descriptions, publishers, update dates, and dataset URLs.

Can I scrape multiple searches in one run?

Yes. You can provide multiple search URLs inside the urls array.

Does the scraper collect publisher information?

Yes. Publishing organizations are extracted whenever available.

Can I monitor new datasets over time?

Yes. You can schedule recurring Apify runs to monitor newly published or updated datasets.

Is the data collected live?

Yes. Data is extracted directly from Data.gov.uk during every run.

What export formats are supported?

JSON, CSV, Excel, XML, and HTML.

Can I use the extracted data commercially?

Yes. The extracted data can be used for analytics, research, automation, monitoring, and commercial applications.

What happens if the scraper fails?

The Actor includes retry mechanisms and automated error handling to improve reliability.

How long does a run take?

Most extractions complete within minutes depending on the number of URLs and requested results.

---

🚀 How to Use

1️⃣ Sign up — Create a free Apify account

2️⃣ Find the tool — Search for "Data.gov.uk Dataset Scraper" in the Apify Store

3️⃣ Configure URLs — Add one or multiple Data.gov.uk search URLs

4️⃣ Run it — Start the Actor and wait for extraction

5️⃣ Export data — Download results in JSON, CSV, Excel, XML, or HTML

---

⚠️ Disclaimer

This tool is an independent solution and is not affiliated with, endorsed by, or sponsored by Data.gov.uk or the UK Government.

---

💸 Pricing

This scraper runs on a pay per events subscription model.

You only pay for successful runs.

💳 Price: $1.19 / 1000 results

---

Related Actors

If you're interested in other Open Data, Research, Government, Analytics, Marketplace, Jobs, Real Estate, or Lead Generation scraping solutions, explore more tools:

(Coming soon)

---

📬 Support

⭐⭐⭐⭐⭐ Leave a 5-star rating if you like this tool

---

🌍 PrimeScrape

Built for scalable web data extraction & automation.

Contact us for custom scraping solutions or enterprise requests via Apify or by email.