Hello thank you for using our actor.

Facebook sometimes returns duplicates while paginating. the log was just indicating that those duplicates have been skipped.

The scraper will keep paginating until reaching max_items. You need either to increase your timeout or decease max_items. We are thinking of adding a new parameter max_page in order to limit the number of pages to check. what do you think about it?

A fix that hides that log will be released

We will be testing a new feature to stop the scrapper if no more new items are returned in 3 consecutive pages.

Fixed. Thank you for reporting this issue.

Looks like the same issue is happening. Only scrapped 24 items.

Fetched page 6 Reached max consecutive pages with no new items Exiting Actor ({"exit_code": 0})

Here is the Run: yG4Nx4NgStiHML6sh

We are investigating to see if there is a recent change in facebook that causes this issue

Any updates?

I checked other scrapers available on Apify, and they have the same issue. It seems Facebook has made a change that limits the number of listings viewable without logging in. We could consider adding support for cookies to bypass this restriction. Would you be interested in this functionality?

Also, unlike our actor, other scrapers don’t handle duplicates either across runs or within the same run.

I've attached a screenshot comparing it to another scraper on Apify. I used the same URL, and it returned 749 results. I removed duplicates via the CSV file; there were 74. So still 695 unique results.

I'm less concerned about duplicates if scraping the whole list costs the same amount. If it doesn't reduce costs, I'm fine manually removing them via the CSV file. I'm looking to scrape hundreds of thousands of listings, so whatever Actor can reduce the number of compute units I'd use.

Is it possible to refine what fields are scraped to decrease the number of compute units? There are currently tons of fields I have no use for, and it could be a valuable feature (if it reduces the cost per listing), if we could control what fields are scraped.

Or, even an option to scrape just the 'Description', none of the other stuff (images, videos, etc.)

Hello again :)

First of all, I come with good news . Thanks to your comparison and feedback, we were able to fix a major issue in our actor. It really helped us improve it, so thank you for that!

I had hoped to offer you free access to the actor for a month or two as a thank you, but unfortunately, the platform doesn't allow us to do that at the moment (You can check with them if it's possible ;) )

Regarding the scraping process:

It's not currently possible to skip specific pages ahead of time, since we need to access the detail page first to extract any meaningful fields.

As for choosing which fields to scrape by the time we reach that step, the detail page has already been loaded, meaning the compute cost has already been incurred.

That said, I do want to highlight that our deduplication process — both across multiple runs and within the same run — helps reduce costs significantly. When a duplicate listing is detected, we skip the detail scraping altogether, which avoids unnecessary compute usage. This should help lower your cost per listing, especially at large scale.

I encourage you to compare compute usage with other similar actors on Apify. And if you still find that our actor is on the higher side cost-wise, feel free to reach out — we’d be glad to explore further optimizations.

Thanks again for your detailed testing and support — it’s really appreciated!