An Apify Actor that automatically scrapes Claude Prompt Library data and converts it into a clean structured dataset.

This scraper extracts AI prompt information including titles, descriptions, categories, tags, authors, prompt content, examples, and source URLs.

Features

✅ Scrape Claude prompt libraries automatically
✅ Extract complete prompt information
✅ Collect titles and descriptions
✅ Extract categories and tags
✅ Capture author information
✅ Extract update dates
✅ Collect prompt examples
✅ Keyword filtering
✅ Category filtering
✅ Maximum item limit
✅ Duplicate URL removal
✅ Export data to JSON, CSV, Excel, and other Apify formats

---

Input Configuration

The Actor accepts the following inputs:

startUrls

List of Claude Prompt Library pages to scrape.

Example:

[
  {
    "url": "https://example.com/prompts"
  }
]

---

maxItems

Maximum number of prompts to scrape.

Example:

100

---

category

Filter prompts by category.

Example:

Marketing

---

keyword

Search prompts containing specific keywords.

Example:

SEO

---

includeContent

Enable or disable full prompt extraction.

Example:

true

---

includeTags

Enable or disable tag extraction.

Example:

true

---

proxyConfiguration

Configure Apify Proxy for reliable scraping.

Example:

{
  "useApifyProxy": true
}

---

Output Dataset

Each scraped item contains:

{
  "title": "AI Marketing Assistant",
  "category": "Marketing",
  "description": "Create marketing strategies using AI",
  "prompt": "Act as a marketing expert...",
  "tags": [
    "marketing",
    "ai"
  ],
  "author": "Author Name",
  "updatedDate": "2026-01-01",
  "example": "Example output",
  "url": "https://example.com/prompt"
}

---

How It Works

Start Actor

↓

Read Input

↓

Load Start URL

↓

Download Web Page

↓

Find Prompt Links

↓

Open Each Prompt

↓

Extract Data

↓

Apply Filters

↓

Save Dataset

↓

Finish

---

Use Cases

AI Researchers

Collect and analyze thousands of AI prompts.

Prompt Engineers

Build custom prompt databases.

Developers

Create AI applications using structured prompt data.

Marketers

Research AI-generated content workflows.

Businesses

Build internal AI knowledge libraries.

---

Installation

Install dependencies:

$pip install -r requirements.txt

---

Run Locally

Login to Apify:

$apify login

Run the Actor:

$apify push

---

Dataset Export

Apify automatically supports:

• JSON
• CSV
• Excel
• XML
• RSS

---

Technology Stack

• Python
• Apify SDK
• HTTPX
• BeautifulSoup
• LXML

---

Project Structure

claude-prompt-library-scraper/

├── .actor/
│   └── actor.json

├── src/
│   ├── main.py
│   ├── scraper.py
│   └── __init__.py

├── INPUT_SCHEMA.json
├── dataset_schema.json
├── requirements.txt
├── Dockerfile
└── README.md

---

Support

For issues, improvements, or feature requests, update the Actor source code or contact the developer.

---

License

MIT License