🔥Apollo Scraper - Scrape upto 50k Leads
Pricing
$1.20 / 1,000 leads
🔥Apollo Scraper - Scrape upto 50k Leads
Scrape up to 50,000 leads per search URL.
4.1 (238)
Pricing
$1.20 / 1,000 leads
2459
Total users
48K
Monthly users
14K
Runs succeeded
>99%
Issues response
6 hours
Last modified
3 days ago
how do i avoid duplicate runs and the actor scraping the same leads
Closed
for my project i need to scrape it batches of 500 max however every time it has the same results or very few net new
the scraper always starts scraping from page 1.
You can fetch upto 50k leads in one run.
oritsetsemaye9
automated reply? do better
why are you fetching in batches of 500.
oritsetsemaye9
Make operations cost and quality control
Mike.fanelli.lightsaber
From Page 1. Makes sense. That's why I need to do one run, to avoid duplication. PROBLEM IS: n8n stops due to memory errors when I use it so I have to break up the batches. Unless someone can recommend a good way to do it.
Mike.fanelli.lightsaber
I've seen added a wait state and a batch limit of 25 to process at a time. Getting to be expensive testing though
Your are saying n8n stops due to memory errors.
Are you running the actor on n8n and waiting for the results?
You caņ just run the actor and fetch the results from apify after the run is completed.
Mike.fanelli.lightsaber
I did twice. and a few $ later. I guess I don't know how. I am kinda of a rookie in n8n. I have an automation to enrich but it all goes into Airtable etc etc...
I think you are using "Run Actor synchronously and get dataset items" apify api right?
You can create a task on apify, then use just "Run Actor" apify api.
You can setup something on n8n, that will check regularly that if the run is completed or not.
After its completed, you can send data to airtable.
Also, if your goal is to just send data to airtable then instead of using n8n, you can use apify integrations and connect airtable.
Mike.fanelli.lightsaber
Yes, I will need to add a batch node, and loop and optimize it so it processes in smaller chunks. it had all be done in n8n, it's cleaner and in one place is ideal.
