Django Weblog Scraper

Get Django Weblog posts as clean JSON, CSV or Excel, from the Apify API or on a schedule. No login, about $1.00 for 1,000 posts.

What does Django Weblog Scraper do?

Django Weblog Scraper extracts structured data from djangoproject.com. Collect posts from the official Django weblog on djangoproject.com (title, URL, author, publication date and summary), following older-posts pagination, to track Django releases, security announcements and community news. It works as an API for djangoproject.com data: run it from Apify Console, on a schedule, or from your own code, and get clean, typed JSON with numbers as numbers and dates in ISO 8601.

What you get

Data	6 fields per item: url, title, postUrl, author, publishedAt, summary
Formats	JSON, CSV, Excel, HTML, or the Apify API
Price	$1.00 per 1,000 posts, pay per result
Access	Public djangoproject.com data only: no login, no cookies, robots.txt respected

Why use Django Weblog Scraper?

• Developers and maintainers: follow releases and security announcements without checking the site.
• Newsletters and digests: pull the latest posts with dates, authors and summaries.
• Market and community research: analyse topics and publishing frequency over time.
• AI agents and RAG: give an LLM the latest official posts as structured data.

Main features:

• Follows pagination up to maxPages pages per start URL and stops at maxItems results.
• Polite by default: respects robots.txt, at most maxConcurrency parallel requests and a delay between requests.
• Checks every result against field validators, so layout changes show up as clear data-quality warnings.
• Runs on the Apify platform: scheduling, API access, integrations, monitoring and datasets you can export.

What data can Django Weblog Scraper extract?

Field	Type	Description
url	string	Index page the post was scraped from
title	string	Post title
postUrl	string	Absolute URL of the post
author	string	Post author name (null if not shown)
publishedAt	ISO 8601 date	Publication date (ISO 8601, YYYY-MM-DD)
summary	string	Summary or excerpt text shown on the index page

How to scrape djangoproject.com

1. Open Django Weblog Scraper in Apify Console and go to the Input tab.
2. Enter what to scrape (see the Input section below), for example the start URLs.
3. Set Max items to the number of results you need.
4. Click Start and wait for the run to finish.
5. Download the results from the Output tab, or fetch them with the API.

How much will it cost to scrape djangoproject.com?

This Actor is priced per result: $1.00 per 1,000 results, with no extra charge for platform usage. That is about $1.00 for 1,000 posts: 100 results cost $0.10 and 10,000 results cost $10.00. Set a maximum cost per run and the Actor stops when it is reached.

Input

See the Input tab for full configuration options.

Field	Type	Required	Description
maxItems	integer	no	Maximum number of items to return (0 = unlimited).
startUrls	array	no	Blog index pages to scrape, for example https://www.djangoproject.com/weblog/.
maxPages	integer	no	Maximum listing pages to follow per start URL (pagination).
maxConcurrency	integer	no	Maximum parallel requests (politeness; 1-10).
requestDelayMs	integer	no	Minimum delay between requests, in milliseconds (at least 250).

Example input:

{
  "startUrls": [
    {
      "url": "https://www.djangoproject.com/weblog/"
    }
  ],
  "maxItems": 60,
  "maxPages": 3,
  "maxConcurrency": 2,
  "requestDelayMs": 1000
}

Output

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel. Example results from a real run:

[
  {
    "url": "https://www.djangoproject.com/weblog/",
    "title": "DSF member of the month - Ken Whitesell",
    "postUrl": "https://www.djangoproject.com/weblog/2026/sep/24/dsf-member-of-the-month-ken-whitesell/",
    "author": "Sarah Abderemane",
    "publishedAt": "2026-09-24",
    "summary": "Ken Whitesell is the DSF member of the month for September 2026. Explore the story of a longtime Django community member since February 2017. He has been a helper in the Django forum for many years."
  },
  {
    "url": "https://www.djangoproject.com/weblog/",
    "title": "New Technical Governance Approved",
    "postUrl": "https://www.djangoproject.com/weblog/2026/sep/22/new-technical-governance-approved/",
    "author": "Tim Schilling and the Steering Council",
    "publishedAt": "2026-09-22",
    "summary": "Both the Steering Council and DSF Board have approved DEP 19, which implements our new technical governance. Thank you to everyone who read the document and participated in the process! This was the effort of two different DSF boards and dozens of community members."
  },
  {
    "url": "https://www.djangoproject.com/weblog/",
    "title": "Proposed change to DSF voting membership",
    "postUrl": "https://www.djangoproject.com/weblog/2026/sep/18/proposed-change-to-dsf-voting-membership/",
    "author": "Jeff Triplett and the DSF Board",
    "publishedAt": "2026-09-18",
    "summary": "A proposed bylaw change to how we count voting members for quorum. Comments are open until October 7."
  }
]

Integrations and API

• Apify API: start a run and get the results in one HTTP request:

curl -X POST "https://api.apify.com/v2/acts/compass_lab~django-weblog-scraper/run-sync-get-dataset-items?token=<YOUR_APIFY_TOKEN>" \
  -H "Content-Type: application/json" -d '{"startUrls": [{"url": "https://www.djangoproject.com/weblog/"}], "maxItems": 60}'

• Python (pip install apify-client):

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("compass_lab/django-weblog-scraper").call(run_input={"startUrls": [{"url": "https://www.djangoproject.com/weblog/"}], "maxItems": 60})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

• JavaScript (npm install apify-client):

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('compass_lab/django-weblog-scraper').call({"startUrls": [{"url": "https://www.djangoproject.com/weblog/"}], "maxItems": 60});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

• Make, Zapier, n8n, Google Sheets, webhooks: use the Apify integrations (Integrations tab) to send each run's results where you need them, or to start a run from your workflow.
• Schedules: run it hourly, daily or weekly from Apify Console (Schedules) and always have fresh posts.

Tips and advanced options

• Keep Max items and Max pages as low as you need: fewer pages means a faster, cheaper run.
• Raise Delay between requests if the site responds slowly; keep Max concurrency low to stay polite.
• Missing values are null. Fields that often come back empty are listed in the run log as data-quality warnings.

FAQ, disclaimers and support

Is it legal to scrape djangoproject.com?

Checked 2026-09-30. robots.txt (https://www.djangoproject.com/robots.txt) only disallows /admin and /checklists; /weblog/ and ?page=N are allowed for all user agents. The site links a code of conduct and a trademark policy but publishes no terms of use restricting automated access. Blog posts are copyright their authors / the DSF: this actor collects titles, authors, dates, summaries and links only; do not republish full post bodies. Keep request rates low (defaults: 2 concurrent, 1 s delay).

Our Actors are ethical and do not extract any private user data, such as email addresses, gender, or location. They only extract what the user has chosen to share publicly. We therefore believe that our Actors, when used for ethical purposes by Apify users, are safe. However, you should be aware that your results could contain personal data. Personal data is protected by the GDPR in the European Union and by other regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers.

How many results can I get?

Up to maxItems per run (0 means no limit), as many as the source lists. Each result is one dataset item, and you are only charged for items that are saved.

Can I run it on a schedule or from my own code?

Yes. Schedule it in Apify Console (Schedules), or call it with the run-sync-get-dataset-items endpoint or the Python/JavaScript clients shown in Integrations and API above.

What are the limitations?

• Only what the blog index shows is collected: title, URL, date, author and summary, not the full post text.
• The author isn't shown for every post; it is null then.
• If djangoproject.com changes its page layout, results may be missing until the Actor is updated; the run log warns when fields stop matching.

Where can I get help?

Report problems or ideas on the Issues tab. To call this Actor from your own code, see the API tab.

Related actors

the same clean, typed output across sources, so you can combine them in one dataset.

Actor	What it scrapes	Price
Breezy HR Jobs Scraper	Job listings from Breezy HR	$1.00 / 1,000
Open Canada Dataset Catalogue Scraper	Dataset records from Open Canada	$1.00 / 1,000
data.gouv.fr Dataset Catalogue Scraper	Dataset records from data.gouv.fr	$1.00 / 1,000
Greenhouse Jobs Scraper	Job listings from Greenhouse	$1.60 / 1,000
Lever Jobs Scraper	Job listings from Lever	$1.60 / 1,000
Python Jobs Scraper (python.org)	Job listings from Python Jobs Scraper (python.org)	$2.00 / 1,000
Recruitee Jobs Scraper	Job listings from Recruitee	$1.00 / 1,000
We Work Remotely Jobs Scraper	Job listings from We Work Remotely	$2.50 / 1,000
Working Nomads Jobs Scraper	Job listings from Working Nomads	$2.50 / 1,000