1 day trial then $5.00/month - No credit card required now
API / JSON scraper
1 day trial then $5.00/month - No credit card required now
Scrape any API / JSON URLs directly to the dataset, and return them in CSV, XML, HTML, or Excel formats. Transform and filter the output. Enables you to follow pagination recursively from the payload without the need to visit the HTML page.
Filter out columns on dataset
Open
Is there a way to filter out columns returned from the JSON scraper?
yes, you can filter anything from the output. in the "Filter / map output" field, you can delete keys from the data variable.
but if you want to filter from the generated dataset, you can use Apify dataset columns parameter
clever_dictionary
Hi Paulo,
Thanks for your response. I am a bit of newbie so need a little more help to do the above.
The JSON link is: https://api-2.curalate.com/v1/media/dXdIobAfKjMmUENb?filter=productId:beadboard-wide-dresser&filter=ssort=Latest&limit=30
In Apify this produces: https://api.apify.com/v2/datasets/zibdwKAxTHBHfnEiC/items?clean=true&format=html&limit=1000
I just want the columns: data/data/items/0/products/0/price/display data/data/items/0/products/0/price/isOnSale data/data/items/0/products/0/price/saleDisplay
So how do I do this with either the "Filter / map output" field? Currently it is only showing this:
async ({ request, response, addRequest, flattenObjectKeys, context, Apify, _, moment, data, input, customData }) => { return { url: request.url, data }; }
Or should I use the Apify dataset columns parameter as you suggest? However I can't find that option.
Thanks!
Paulo Cesar- 13 monthly users
- 99.8% runs succeeded
- days response time
- Created in May 2022
- Modified 4 months ago
