Reel.farm Scraper - Apify Actor

Scrapes profiles, slideshows, prompts, and images from reel.farm with immediate data output.

Features

• Immediate Output: Data becomes available as soon as scraping starts
• Gradual Data Flow: CSV files update every 10 records
• Apify Residential Proxy: Uses rotating residential IPs
• Parallel Processing: Downloads images in parallel
• Batch Processing: Processes profiles in manageable batches
• Progress Tracking: Real-time progress reports
• Fault Tolerant: Data is saved incrementally

Input Configuration

Required:

• session_token: Your reel.farm session token (from browser cookies)

Optional:

• test_mode: Process only first 5 profiles (default: false)
• max_workers: Parallel workers for image downloads (default: 10)
• batch_size: Profiles per batch (default: 5)
• max_slideshows_per_profile: Limit slideshows per profile (default: 10)

Output Files

Data becomes available immediately:

Within 5 seconds:

• scraper_started.txt - Start timestamp
• progress_initial.json - Initial configuration
• reel_templates_started.csv - Empty CSV with headers

Within 30 seconds:

• dashboard_summary.json - Dashboard data summary
• profile_list.json - List of profiles to process
• status_now.json - Current status

During processing (updated frequently):

• reel_templates_latest.csv - Latest complete CSV
• reel_templates_incremental_*.csv - Incremental CSVs
• progress_report.json/txt - Updated progress
• batch_*_*.json - Batch and profile results

After completion:

• reel_templates_final.csv - Complete final CSV
• final_summary.json - Final statistics
• All images in Key-Value Store

How to Use

1. Get your session token:

   • Log into reel.farm
   • Open browser Developer Tools
   • Go to Application → Cookies
   • Copy value of __Secure-next-auth.session-token

2. Configure the actor:

   • Set session_token with your token
   • Choose test_mode: true for quick test
   • Adjust other parameters as needed

3. Run the actor:

   • The actor will start immediately
   • Data will appear in Key-Value Store and Dataset
   • Monitor progress through progress_report.txt

4. Download results:

   • Use Apify's export options
   • Download CSV files directly from Key-Value Store
   • Export Dataset as JSON, CSV, Excel, etc.

Expected Output Fields

CSV files contain these exact fields:

1. nickname - Profile display name
2. unique_id - Profile username/ID
3. slideshow_name - Original slideshow name
4. views - Formatted views (e.g., 292.7k)
5. likes - Formatted likes (e.g., 48.2k)
6. business_type - Business category
7. image_url - Main image URL
8. prompt - Generated prompt text
9. image_path - Path to downloaded images

Performance Notes

• Full scraping: ~230 profiles, ~2000 slideshows, 4-8 hours
• Test mode: 5 profiles, 30-60 minutes
• Memory usage: 4-8 GB recommended
• Timeout: Set to 24 hours for full scraping

Error Handling

If scraping fails:

• All data up to failure point is preserved
• Progress reports show last known state
• Can restart from last batch

Support

For issues or questions:

1. Check progress reports for error messages
2. Verify session token is valid
3. Ensure sufficient memory allocation
4. Contact support if problems persist