API / JSON scraper avatar
API / JSON scraper
Try for free

1 day trial then $5.00/month - No credit card required now

View all Actors
API / JSON scraper

API / JSON scraper

pocesar/json-downloader
Try for free

1 day trial then $5.00/month - No credit card required now

Scrape any API / JSON URLs directly to the dataset, and return them in CSV, XML, HTML, or Excel formats. Transform and filter the output. Enables you to follow pagination recursively from the payload without the need to visit the HTML page.

User avatar

Filter out columns on dataset

Open

clever_dictionary opened this issue
3 months ago

Is there a way to filter out columns returned from the JSON scraper?

User avatar

yes, you can filter anything from the output. in the "Filter / map output" field, you can delete keys from the data variable. but if you want to filter from the generated dataset, you can use Apify dataset columns parameter

User avatar

clever_dictionary

2 months ago

Hi Paulo,

Thanks for your response. I am a bit of newbie so need a little more help to do the above.

The JSON link is: https://api-2.curalate.com/v1/media/dXdIobAfKjMmUENb?filter=productId:beadboard-wide-dresser&filter=ssort=Latest&limit=30

In Apify this produces: https://api.apify.com/v2/datasets/zibdwKAxTHBHfnEiC/items?clean=true&format=html&limit=1000

I just want the columns: data/data/items/0/products/0/price/display data/data/items/0/products/0/price/isOnSale data/data/items/0/products/0/price/saleDisplay

So how do I do this with either the "Filter / map output" field? Currently it is only showing this:

async ({ request, response, addRequest, flattenObjectKeys, context, Apify, _, moment, data, input, customData }) => { return { url: request.url, data }; }

Or should I use the Apify dataset columns parameter as you suggest? However I can't find that option.

Thanks!

Developer
Maintained by Community
Actor metrics
  • 17 monthly users
  • 99.4% runs succeeded
  • 55.7 days response time
  • Created in May 2022
  • Modified 4 months ago