Batch Metamorph Actor 🔄

This Actor demonstrates the Apify metamorph operation by processing multiple URLs with configurable delays, then transforming into another Actor to complete the work in a single continuous run.

What This Actor Does

1. Accepts multiple URLs in requestListSources format (compatible with Crawlee)
2. Processes URLs sequentially with a configurable delay between each
3. Stores processing metadata in the key-value store to demonstrate data persistence
4. Metamorphs into dz_omar/universal-downloader with all URLs as a batch job

Key Features

🔄 Metamorph Operation

The metamorph operation allows one Actor to transform into another Actor mid-execution while preserving all storage data (Dataset, Key-Value Store, Request Queue). This happens seamlessly in a single run with one billing cycle.

⏱️ Configurable Delays

Add delays between URL processing to avoid overwhelming target servers or respect rate limits.

📊 Data Persistence

All data stored before the metamorph (processing logs, timestamps, metadata) remains accessible after the transformation.

Input Configuration

The Actor accepts input in this format:

{
  "input": [
    { "url": "https://httpbin.org/bytes/1024" },
    { "url": "https://httpbin.org/bytes/2048" },
    { "url": "https://httpbin.org/bytes/4096" }
  ],
  "delaySeconds": 3
}

Input Fields

• input (Array, required): List of URLs in requestListSources format
  • Compatible with Crawlee's RequestList sources
  • Supports both { "url": "..." } objects and plain strings
• delaySeconds (Integer, optional): Fixed delay between processing each URL
  • Default: 2 seconds
  • Range: 0-60 seconds

How It Works

1. Phase 1 - Your Actor:

   • Processes each URL with delays
   • Stores metadata for each step
   • Logs progress and timing

2. Phase 2 - Metamorph:

   • Transforms into universal-downloader
   • Passes all URLs as batch input
   • Continues in the same run ID

3. Phase 3 - Target Actor:

   • Downloads all files
   • Stores results in dataset
   • Completes the workflow

Example Output

Console Logs:

Processing 3 URLs with 3 second delay between requests
Processing URL 1/3: https://httpbin.org/bytes/1024  
Waiting 3 seconds before next URL...
Processing URL 2/3: https://httpbin.org/bytes/2048
Waiting 3 seconds before next URL...  
Processing URL 3/3: https://httpbin.org/bytes/4096
This is the last URL - METAMORPHING into universal-downloader...
[Then universal-downloader logs continue...]

Key-Value Store:

• BATCH_INFO: Overall batch information
• PROCESSING_0, PROCESSING_1, etc.: Individual URL processing logs

Dataset:

• Download results from universal-downloader

Benefits of Metamorph

✅ Single Run Billing: Pay for one continuous run instead of multiple Actor calls
✅ Data Persistence: All storage survives the transformation
✅ Seamless Integration: No manual coordination between Actors
✅ Simplified Workflows: Chain complex operations easily
✅ Cost Effective: Reduce overhead of multiple Actor starts

Use Cases

• Batch File Processing: Download multiple files with rate limiting
• Workflow Orchestration: Chain specialized Actors together
• Complex Scraping: Analyze targets, then use appropriate scrapers
• Data Pipeline: Transform data through multiple processing stages

Learn More About Metamorph

Official Documentation

• Apify Metamorph Documentation
• Actor Programming Interface

Technical Details

• Storage Persistence: All default storages (Dataset, Key-Value Store, Request Queue) are preserved
• Input Handling: Metamorphed Actor receives input via INPUT-METAMORPH-1 key (handled automatically by Actor.getInput())
• Billing: Single run ID for the entire workflow
• Limitations: Runtime limits apply to the total metamorph chain

Related Concepts

• Crawlee RequestList: Compatible input format
• Apify SDK: JavaScript SDK for Actor development
• Actor Development: Complete development guide

Error Handling

The Actor includes error handling for:

• Invalid URL formats in input
• Missing or malformed input data
• Metamorph operation failures
• Delay parameter validation

Deployment

1. Push to Apify platform: apify push
2. Configure input in Console or via API
3. Run and monitor the metamorph workflow
4. Check both phases in the same run logs

Note: This is an educational demonstration of the metamorph operation. In production, consider additional error handling, input validation, and monitoring based on your specific needs.

🤝 Support & Resources

Getting Help

• 📧 Email: fridaytechnolog@gmail.com
• 🐙 GitHub: DZ-ABDLHAKIM
• 🐦 Twitter: @DZ_45Omar
• 🔧 Apify: dz_omar