The data was obtained repeatedly.

Hi!

I checked by adArchiveId and its different ads:

https://www.facebook.com/ads/library/?active_status=active&ad_type=all&country=ALL&id=1270134520709520&is_targeted_country=false&media_type=all&search_type=page&view_all_page_id=83353586320

https://www.facebook.com/ads/library/?active_status=active&ad_type=all&country=ALL&id=2795204787306704&is_targeted_country=false&media_type=all&search_type=page&view_all_page_id=83353586320

You can check the rest by replacing id=1270134520709520 with values of adArchiveId

For scraping the actor just follow internal Facebook logic - if there are ads with the same content (snapshot) but different id each ad will be saved as a separate dataset item

In the list https://www.facebook.com/ads/library/?active_status=active&ad_type=all&country=US&is_targeted_country=false&media_type=all&search_type=page&view_all_page_id=83353586320 its clearly expressed that "92 ads use this creative and text" so if you do not want all the ads please consider option to get only totals.

I´m going to close the issue now, but if there is anything else we could help with, please let us know.

Thank you!

I tried using the totalCount option, but it only returned 30 pieces of data. I want to get more data, and the effect should be consistent with the page, that is, no duplicate ads should appear. What should I do?

I tried using collationId for deduplication, and found that after deduplication, only 14 pieces of data remained out of 200. Is there any way to improve this situation?

Hi!

Please try deduplication with the transform function https://console.apify.com/view/runs/6WnNrkIphpoQnPQ8U

(items) => items.map(x => ({ ...x, collationId: x.collationId || x.adArchiveID })) to keep snapshots with the same content as unique dataset items and not to lose single ads without collation id.

Sorry, I couldn't find where to use this function. Could you please provide a screenshot?

Do you mean to first use this actor to obtain the data, and then use another actor to filter the data?

Could you please add an option to return the filtered data directly?

Hi!

Sorry for the confusion, I thought you already tried https://apify.com/lukaskrivka/dedup-datasets

Full JSON input for the shared run:

1{
2  "appendDatasetIds": false,
3  "batchSizeLoad": 50000,
4  "datasetIds": [
5    "A0jKtixo8Qqgddegr"
6  ],
7  "fields": [
8    "collationId"
9  ],
10  "nullAsUnique": false,
11  "output": "unique-items",
12  "parallelLoads": 10,
13  "parallelPushes": 5,
14  "preDedupTransformFunction": "(items) => items.map(x => ({ ...x, collationId: x.collationId || x.adArchiveID }))",
15  "uploadBatchSize": 500,
16  "verboseLog": false,
17  "mode": "dedup-after-load",
18  "outputTo": "dataset"
19}

Please paste it, then switch to visual input form and you will see when each input parameter appears.

Generally, we try not to add unique features if alternative actor-to-actor integration exists.