🚀 Actor Factory - AI-Powered Web Scraper Generator

Transform any website into a professional Apify scraper using Cursor's AI chat! This factory automates the entire process of creating production-ready web scrapers with just a few simple commands.

🎯 What is Actor Factory?

Actor Factory is an intelligent system that leverages Cursor's AI capabilities to automatically generate complete web scrapers for any website. Instead of spending hours coding scrapers from scratch, you simply describe what you want to scrape, and the AI handles everything - from code generation to documentation.

Perfect for:

• 🏢 Business analysts who need data but don't code
• 🧑‍💻 Developers who want to speed up scraper development
• 📊 Data researchers who need quick, reliable data collection
• 🚀 Entrepreneurs building data-driven products

✨ Key Features

• 🤖 AI-Powered: Uses Cursor chat to understand requirements and generate code
• ⚡ Fast Setup: Create a complete scraper in minutes, not hours
• 📋 Production Ready: Generates proper Apify actors with documentation
• 🎨 Professional Output: Creates business-friendly READMEs and documentation
• 🔧 Multiple Templates: Supports different scraping patterns (API, browser, etc.)
• 📊 Built-in Testing: Includes test configurations and validation

🏗️ Project Structure

actor-factory/
├── 📁 prompts/           # AI prompts and templates
│   ├── STEPS.md         # steps scraper creation process
│   ├── GUIDE-WRITE-README.md  # README generation guide
│   └── REFERENCE-REPOSITORIES.md  # Template examples
├── 📁 reference-repositories/  # Working scraper templates
├── 📁 repositories/     # Your generated scrapers
└── 📄 index.ts         # Main factory script

🚀 Quick Start with Cursor Chat

Prerequisites

1. Install Bun (if not already installed):

   $curl -fsSL https://bun.sh/install | bash
   

2. Install dependencies:

   $bun install
   

Step 1: Open Cursor Chat

1. Open this project in Cursor
2. Press Ctrl+L (or Cmd+L on Mac) to open the chat panel
3. Make sure you're in the actor-factory directory

Step 2: Create Your First Scraper

Example conversation with Cursor chat:

You: I want to create a scraper for a real estate website called "PropertyFinder.com". 
    It should scrape property listings with details like price, location, bedrooms, 
    bathrooms, and property type. The website has search filters for location and 
    property type.

Cursor: I'll help you create a PropertyFinder scraper! Let me start by following 
        the actor factory process...

Step 3: Follow the AI-Guided Process

The AI will automatically:

1. 📋 Analyze your requirements and ask clarifying questions
2. 🏗️ Set up the project structure in the repositories/ folder
3. 📝 Create the SCRAPER.md file with your specifications
4. 🔧 Generate the complete scraper code (main.ts, api.ts, mapper.ts, types.ts)
5. ⚙️ Configure Apify settings (.actor/actor.json, input_schema.json)
6. 📊 Create test configurations and sample data
7. 📖 Generate professional documentation and README
8. ✅ Test and validate the scraper

💬 Example Cursor Chat Workflows

Workflow 1: E-commerce Product Scraper

You: Create a scraper for an e-commerce site that collects product information

Cursor: What's the website URL and what specific product data do you need?

You: https://example-store.com - I need product name, price, description, 
    images, availability, and ratings

Cursor: Perfect! I'll create an e-commerce scraper. Let me set up the project 
        structure and generate the code...

Workflow 2: Job Board Scraper

You: I need to scrape job listings from a job board

Cursor: Which job board and what job details are you looking for?

You: Indeed.com - job title, company, location, salary, job description, 
    and application link

Cursor: I'll create a job board scraper using the browser automation template. 
        This will handle the dynamic content loading...

Workflow 3: API-Based Scraper

You: Create a scraper that uses an API instead of browser automation

Cursor: What API endpoint and what data structure are you working with?

You: https://api.example.com/data - it returns JSON with user profiles

Cursor: Great! I'll use the API template which is faster and more reliable 
        than browser automation...

🎯 Supported Scraper Types

1. 🌐 Browser Automation Scrapers

• For dynamic websites with JavaScript
• Uses Playwright with stealth features
• Handles complex interactions and forms
• Example: E-commerce sites, social media, job boards

2. ⚡ API Scrapers

• For websites with public APIs
• Fast and reliable data collection
• No browser overhead
• Example: GitHub, Hugging Face, news APIs

3. 🔍 Search-Based Scrapers

• For sites with search functionality
• Handles pagination and filtering
• Example: Real estate, classified ads, directories

4. 🛡️ Anti-Bot Protected Scrapers

• For sites with bot protection
• Uses proxy rotation and stealth techniques
• Example: LinkedIn, some e-commerce sites

📋 The Steps Process (Automated by AI)

The AI follows this comprehensive process automatically:

1. Setup: Clone templates, analyze target website
2. Configuration: Update settings, create input schemas
3. Development: Generate and test scraper code
4. Documentation: Create README, pricing, final testing

You don't need to know these steps - the AI handles everything!

🎨 Generated Output Examples

Professional README

The AI creates business-friendly documentation with:

• Clear value propositions
• Step-by-step usage instructions
• Real output examples
• Pricing information
• Integration guides

Complete Scraper Code

• main.ts: Main scraper logic
• api.ts: API interactions
• mapper.ts: Data transformation
• types.ts: TypeScript definitions
• Dockerfile: Container configuration

Apify Configuration

• actor.json: Scraper metadata
• input_schema.json: User input form
• pricing: Cost structure

🔧 Advanced Usage

Custom Templates

You can reference different template types:

You: Use the "yatco-actor" template for this API scraper
You: Use the "hubspot-marketplace-scraper" template for anti-bot protection
You: Use the "greatschools-scraper" template for browser automation

Batch Processing

You: Create scrapers for these 5 websites: [list URLs]
Cursor: I'll create all 5 scrapers in parallel. Let me start with the first one...

Custom Requirements

You: Add proxy support and rate limiting to this scraper
You: Include data validation and error handling
You: Add support for multiple output formats

🚀 Testing Your Scrapers

Quick Test

cd repositories/your-scraper-name
apify run --purge

Check Results

# View scraped data
cat storage/datasets/default/000000001.json

Deploy to Apify

$apify push

💡 Pro Tips for Cursor Chat

1. Be Specific

❌ "Create a scraper for a website" ✅ "Create a scraper for Amazon product listings that gets title, price, rating, and availability"

2. Provide Context

❌ "Scrape this site" ✅ "I need to scrape job listings from Indeed for market research. I want job title, company, location, salary range, and job description."

3. Mention Requirements

❌ "Make it work" ✅ "The site uses JavaScript heavily, so I need browser automation. Also, add proxy support for rate limiting."

4. Ask for Modifications

❌ "It's not working" ✅ "The scraper is getting blocked. Can you add stealth features and proxy rotation?"

🛠️ Troubleshooting

Common Issues

Q: The AI isn't following the process correctly A: Be more specific about your requirements. Reference the STEPS.md file if needed.

Q: Generated code has errors A: Ask the AI to "fix the linter errors" or "debug the scraper code"

Q: Scraper is getting blocked A: Request "add anti-bot measures" or "use the stealth template"

Q: Data format is wrong A: Ask to "update the mapper.ts to fix the data structure"

Getting Help

1. Check the prompts/ folder for detailed guides
2. Look at reference-repositories/ for working examples
3. Ask Cursor chat for specific help with any step
4. Test incrementally - don't try to scrape everything at once

📊 Success Metrics

After using Actor Factory, you should have:

• ✅ Complete scraper code that runs without errors
• ✅ Professional documentation ready for business users
• ✅ Proper Apify configuration for deployment
• ✅ Test data showing expected output format
• ✅ Pricing structure configured correctly

🎉 What's Next?

Once your scraper is created:

1. Test it locally with sample data
2. Deploy to Apify for production use
3. Share with your team or clients
4. Schedule regular runs for ongoing data collection
5. Integrate with other tools (Zapier, Make, etc.)

🤝 Contributing

Want to improve the Actor Factory? The AI can help you:

• Add new template types
• Improve the generation process
• Create better documentation
• Add new features

Just ask Cursor chat: "Help me improve the actor factory by adding [feature]"

---

Ready to create your first scraper? Open Cursor chat and say: "I want to create a scraper for [website] that collects [data types]"

The AI will guide you through the entire process! 🚀