Pay $9.00 for 1,000 results
Macy's Scraper
Pay $9.00 for 1,000 results
Macy's web scraper to crawl product information including price and sale price, color, and images. Extract all data in a dataset in structured formats.
I'll be subscribing today
Closed
Hi Gustavo, Thank you for being prompt and helping out. I'm talking to Apify rep what subscription plan may be a best fit for my volume. One thing I want to ask you is, if I were to run the actor on monthly basis, how do I run the actor efficiently where I don't extract the same data/product from one month to the following month? For example: In September, I extract item #123456 in Women's Clothing. In October, the same item #123456 still exist in one of Women's Clothing pages. How does this work? Will it extract the same data regardless of that we collected it already in previous month?
Yes, the actor keeps track of scrapped items but only during the same run. I believe that is possible to make it keep a permanent record of items it already scrapped but that will require some customization.
82society
How do I proceed with customization? Is it something you have to customize? or should I inquire Apify?
Let me check if Apify already has a solution to this
I can add this feature to the scrapper, but I will need a few days to test it
82society
Can I subscribe today and run the actor for now? Meanwhile, you can add the feature? So that on my next run, it'll avoid duplicates?
Or should I just wait and run it after you complete adding the feature? Also, how many days will it take?
Unfortunally you will have to wait for the new feature. I need to make a change on how the actor store the scrapped items so you can persist that cross multiple runs. I think I would be able to do it until the end of the week but is probably better to wait until next Monday.
82society
Okay, thanks Gustavo.
82society
Please kindly let me know when it's complete.
Sure
It is done Andrew,
On the actor input you now have an option to add the products request queue name. This will create a request queue of the scrapped products and it will persist across multiple runs.
You just have to use the same name.
Gustavo Rudiger- 3 monthly users
- 98.8% runs succeeded
- days response time
- Created in Dec 2019
- Modified 3 months ago
