Error: Importing to Clay

Hi, I am not too familiar with the Clay integration, but I think this means that your run of the Zillow Scraper produced a lot of data, which then couldn't be imported into Clay due to size restrictions on their side.

While I would love to assist, I don't see what we could change in the Actor to help with your issue. Do you have any suggestion in mind?

Looking forward to hearing back from you.
Best regards,
Matěj

Hi!

I think what could help is the ability to select which fields to pull and which not to pull. I really only need a few pieces such as the address, detail URL, all carousel photos and maybe some pieces of info.

If I was able to pull just that, the file size would be much smaller.

Is that possible?

I see it is in the ability to export but not at the actor/task level.

Hm, I see. We generally try to avoid creating options to only scrape selected fields, for multiple reasons:

• the scraper downloads the whole data object at once, so it's easier to just push it to the dataset instead of adding extra logic for filtering fields
• it's useful in case you later realize you wanted more fields - you don't have to rescrape anything, you just download more fields of the existing dataset, thus saving money
• it was causing confusion - you have the option to choose fields in two different places - at Actor startup, and at export

However, I think we could add this "selected fields export" option to the Clay export integration. I asked our team about it, we will update you hear once we have some progress. Thanks for your patience.

Sounds good, thanks!

Hi, the Clay integration is developed by an external Clay team. I asked them about the state of this feature today and I'll let you know the details when I hear back from them.

Hi again, Clay team replied that they currently have a long backlog of more urgent tasks so they won't be able to resolve this any time soon 😔

I was thinking of possible workarounds and you could try out an integration with Merge, Dedup & Transform Datasets as an intermediate step. First, you would need to navigate to Zillow Detail Scraper Integrations > Add integration > Connect Actor or Task > Merge, Dedup & Transform Datasets.

There you could specify the fields you want to keep from the Zillow Detail Scraper's output (in the Limit fields to load for faster execution section), such as:

• addressOrUrlFromInput
• address
• originalPhotos

You would also need to add the following value to the Datset IDs input field to make the integration work properly: {{resource.defaultDatasetId}}. (See the attached screenshot.)

Once the integration is set up, it triggers a new run of Merge, Dedup & Transform Datasets automatically on each successful run of Zillow Detail Scraper. See the example of such a run: https://console.apify.com/view/runs/mEcAhjUunfyTZOGmN

As a last step, you could set up your Clay integration in Merge, Dedup & Transform Datasets.