Apollo Scraper - Pay per leads avatar
Apollo Scraper - Pay per leads
Try for free

Pay $5.00 for 1,000 results

View all Actors
Apollo Scraper - Pay per leads

Apollo Scraper - Pay per leads

curious_coder/apollo-scraper
Try for free

Pay $5.00 for 1,000 results

Scrape apollo to get b2b leads with emails, work history, company website, company phone numbers, etc

User avatar

How to scrape past 1000 leads with one agent

Closed

pookNast opened this issue
a month ago

Apollo.io imposes a daily limit of 1000 email credits per set of API credentials. This limit presents a challenge when attempting to scrape lists that exceed 1000 results in a single run. Additionally, the 'Net New' list contains newly discovered prospects and does not carry over to another set of Apollo credentials.

Given these constraints, I seek guidance on the best approach to scrape lists larger than 1000 results. Specifically, I would like to know if a workaround allows me to scrape the entire list in one run, or if it is not possible due to the daily credit limit.

If scraping the entire list in a single run is not feasible, would the alternative be to have the agent (scraper) on a 24-hour cooldown period? During this cooldown, the agent would pause the scraping process and resume on the target URL and 'Net New' list after the 24 hours have elapsed, continuing from where it left off until the entire list is completed.

I would appreciate any insights, recommendations, or best practices on efficiently scraping large lists using Apollo.io API while adhering to their daily credit limit. Please let me know if there are any alternative approaches or strategies that can help overcome this limitation.

Thank you for your assistance in finding a suitable solution to this issue.

User avatar

Thanks for creating issue, let me check and get back to you

User avatar

pookNast

17 days ago

Hello Tech Support,

I am encountering a recurring issue with our scraping tool when trying to extract lists of over 1000 contacts. Here’s a detailed breakdown

1Consistent Data Extraction Across Days: When using a single agent (let's call it Agent 1) to scrape data, it successfully pulls the first 1000 contacts (equivalent to 10 pages) on Day 1. However, on subsequent days, even if I alter the "starting page" parameter, Agent 1 continues to extract the same 1000 contacts. This suggests that changing the "starting page" does not refresh the results as intended.
2
3Ineffective Starting Page Feature: The tool does not seem to recognize changes in the starting page. This is problematic because I aim to use the same URL with different agents to scrape an entire list over multiple days, but each agent ends up pulling the same initial set of contacts.
4
5Filter Adjustment Limitation: Adjusting the filters to generate new results is not a feasible solution, as the tool appears to only scrape the first 1000 contacts and ignores the specified page number beyond this limit.

Could you provide guidance or a workaround on how to use the scraper to efficiently extract different sets of 1000+ contacts from the same URL over consecutive days? Is there a way to ensure that the "starting page" feature functions as expected, or are there alternative methods we should consider for extracting larger datasets?

Thank you for your assistance.

User avatar

Already addressed in another issue

Developer
Maintained by Community
Actor metrics
  • 136 monthly users
  • 79.6% runs succeeded
  • 0.9 days response time
  • Created in Aug 2023
  • Modified about 1 month ago