All-In-One Data Scraper

One actor. Four scraping surfaces. One pricing surface.

All-In-One Data Scraper is designed as a Swiss-army-knife actor for users who need more than one scraping workflow without paying for a different actor every time. Instead of forcing separate purchases for local leads, Apollo-style sales research, Instagram pulls, and general web extraction, this actor puts those modes behind one input schema.

The current build focuses on returning complete, useful records instead of thin proof-of-life responses:

• web returns article blocks, fallback text previews, contact points, social links, metadata, media, and links
• google_maps can follow each discovered place into its detail pane for enriched lead fields such as address, website, phone, hours, plus code, and review counts
• apollo returns company cards, structured company profiles, FAQs, product links, and lead-style outputs from Apollo pages
• instagram uses Instagram's public profile surface for richer profile metadata and recent post records on public accounts

Modes

• web: general website and content extraction with articles, metadata, media, and links
• google_maps: local business and place extraction from Google Maps searches
• apollo: Apollo-focused discovery and Apollo page scraping for company and people data
• instagram: Instagram profiles, posts, hashtags, places, and search surfaces

What It Returns

Every dataset item uses the same top-level contract:

• mode
• contentType
• url
• title
• query
• summary
• exports
• plus mode-specific fields such as places, companies, people, profile, posts, images, videos, links, contactPoints, textPreview, companyProfile, featuredFaqs, and highlights

Input Examples

Web

{
  "scrapingMode": "web",
  "startUrls": [
    { "url": "https://example.com" }
  ],
  "maxRequestsPerCrawl": 5,
  "includeImages": true,
  "includeVideos": true
}

maxRequestsPerCrawl is a hard budget for unique page requests across initial URLs, discovered links, and detail pages. The Actor never increases it internally. Retry attempts are configured separately with maxRequestRetries.

The request queue is persistent within the Actor run. After an automatic restart, initial URLs are added idempotently, missing seeds are restored, and already queued URLs do not consume the budget again. A resumed queue that already exceeds the configured cap fails closed instead of silently crawling more than requested.

Google Maps

{
  "scrapingMode": "google_maps",
  "searchTerms": [
    "dentists seattle",
    "roofing contractors tacoma"
  ],
  "googleMapsResultLimit": 10,
  "followGoogleMapsPlaceDetails": true
}

Apollo

{
  "scrapingMode": "apollo",
  "apolloQueries": [
    "openai",
    "b2b saas sales intelligence"
  ],
  "apolloDiscoveryLimit": 5,
  "followDiscoveredApolloUrls": true
}

Or scrape direct Apollo URLs:

{
  "scrapingMode": "apollo",
  "apolloStartUrls": [
    { "url": "https://www.apollo.io/companies/example" }
  ]
}

Instagram

{
  "scrapingMode": "instagram",
  "instagramInputType": "profiles",
  "instagramHandles": [
    "instagram"
  ],
  "instagramResultLimit": 12
}

Hashtags, places, posts, and search are also supported through instagramInputType.

Session Cookies

Apollo and Instagram can expose more data when run against authenticated pages. If you have a valid session, pass cookies in the encrypted sessionCookiesJson input as a JSON array. Malformed explicit cookie input stops the run with a clear authentication error instead of silently continuing while signed out.

Only the encrypted field is accepted; legacy clear-text cookie aliases are rejected. Cookies are validated before crawling, injected directly for navigation, and are not copied into Crawlee's persisted SessionPool state. If Instagram redirects the requested target to a login or challenge shell, the run fails clearly instead of publishing that shell as a successful profile.

Public Apollo company routes such as /comp/oracle produce a public company profile from the route's company-specific title, heading, and description when people-level data is unavailable. Generic, removed, or empty Apollo pages are rejected rather than emitted as valid lead records.

Proxy Configuration

Use proxyConfiguration to select Apify Proxy or a custom proxy. The same configuration applies to every scraping mode.

Failure Records

Retry attempts are logged internally. A dataset error record is written only after Crawlee declares the request permanently failed, and each request can produce at most one permanent failure record in a running process. Crawlee may re-serve in-progress requests after an infrastructure restart; because Apify Dataset writes and Request Queue acknowledgements are separate services, no scraper can promise transactional exactly-once dataset writes across a hard process crash. Consumers that require that guarantee should deduplicate by the record's source URL and request identity.

Product Intent

This actor is built around one idea: if your workflow needs two or more scraping categories, you should not have to buy two or more separate actors with unrelated pricing logic just to finish one job.