Google Shopping avatar
Google Shopping
Try for free

3 days trial then $30.00/month - No credit card required now

View all Actors
Google Shopping

Google Shopping

dan.scraper/google-shopping
Try for free

3 days trial then $30.00/month - No credit card required now

High-quality Google Shopping Scraper. Rich data results.

User avatar

I can only download the first query result

Closed

Abdes opened this issue
a year ago

Hello, only first query result is actually downloaded, and all remaining ones are not. This is the run ID https://console.apify.com/actors/9CNJ84uoMLnX0Hhm2/runs/PtBpISngRwEKNn3yu#output

User avatar

Hello Abdessamad, As I see in the JSON results. There are 34 search queries. Most of them are successful and there are some searches are empty.

I enabled saveHTML option and find out that Google blocks some of our searches. This is related to the Apify proxy and I will notify Apify developer to fix this.

For now, I think it's better to retry these failed keywords again?

Thanks

User avatar

Here is my search with HTML option: https://api.apify.com/v2/datasets/njakYUtJmDX6EvPyp/items?clean=true&format=json.

I sent the support requests to Apify and Apify will help us to solve proxy issues.

User avatar

Abdes

a year ago

Hello,

Actually the problem is not the failed ones but the fact that you cannot download any of the results except the first one. Have you tried?

User avatar

Dear Abdessamad,

I downloaded your searches in different formats: JSON, CSV(https://api.apify.com/v2/datasets/gP75tbmswgwqFZ2gG/items?clean=true&format=json), ... and it contains all 34 keyword searches and results.

Could you share how you download it? or a video so I can support

Thanks

User avatar

Abdes

a year ago

Sure –

  1. I click on export 34 results below [cid:image001.png@01D95E37.CAABCFD0]

  2. I select all fields in excel or csv (it gives same result) [cid:image002.png@01D95E37.CAABCFD0]

Let me know what you think,

User avatar

Oh. I see. Right now the response is JSON. To get the CSV, I will support csvFriendly option. Please wait.

Right now I recommend using JSON data and other tools to convert to CSV.

User avatar

Abdes

a year ago

Sure – it will take time before you figure out? Just to know whether I look for alternatives.

User avatar

No worry. I will finish in this weekend.

User avatar

Abdes

a year ago

Hello – hope you are well,

Any update on the issue raised?

User avatar

Hello Abdessamad, Thanks for waiting. The csvFriendlyOutput is supported. Enable this flag and you will get list of all shopping results.

Example: https://api.apify.com/v2/datasets/qUFHbZuDsxQ4dU3qi/items?attachment=true&clean=true&format=csv Table: https://api.apify.com/v2/datasets/qUFHbZuDsxQ4dU3qi/items?clean=true&format=html

Please let me know if you have any request.

User avatar

Abdes

a year ago

Thank you – where should this flag be enabled?

[cid:image001.png@01D960B3.1A17F350]

User avatar

The csvFriendlyOutput is one of the input parameters. You need to enable this when making the request.

User avatar

Let me support this. Will be available this weekend.

User avatar

I udpated the Google Shopping actor. You can sort or filter results by filterValue param:

The default is relevance.

You also can filter by price, band, ...

User avatar

glass_fox

a year ago

Hi, Please the dataset does not contain a search query, can you please add it? Thank you!

User avatar

The result contains the search query: https://api.apify.com/v2/datasets/liOfrmo83iPxzqncy/items?clean=true&format=json

Do you need to search query with csvFriendlyOutput enabled?

User avatar

glass_fox

a year ago

Yes, and also for table view (excel, HTML table...)

User avatar

I included the query in each items: https://api.apify.com/v2/datasets/39df4ntuGrBxj7FQM/items?clean=true&format=json. Let me know if you need any feature.

User avatar

The HTML table works too: https://api.apify.com/v2/datasets/39df4ntuGrBxj7FQM/items?clean=true&format=html

Please enable csvFriendlyOutput: true in your search. Thanks

User avatar

The new run will include the query: https://api.apify.com/v2/datasets/1h5dudRRDeVnaMIIR/items?clean=true&format=html. Let me know if you request any feature.

User avatar

All requests will be made via proxy. Apify will provide proxy. Mean Google will not block your IPs. The proxy is all around the world. All scrape content product use proxy to prevent blocking from Google.

User avatar

glass_fox

a year ago

Hi, I scrape 100 queries in one run. But the scraper skips some queries. "WARNING: Logging is too fast, some lines were skipped." How can I filter out the skipped queries after each run to make it easier to get complete data? Thanks!

User avatar

glass_fox

a year ago

Please can you answer?

User avatar

Hi Mirka,

"WARNING: Logging is too fast, some lines were skipped." didn't affect to the results. The scraper didn't skip any queries. All queries and results will be in result file, can export to csv, html table, json....

Please let me know if you have any issue. I'm happy to answer

User avatar

glass_fox

a year ago

Hi, the problem still persists. When I have 100 queries (1 page) in 1 scraping run, I expect an export with 6000 rows. But the export never includes that. Currently last run 5400 rows. I need a list of queries that were not scraped after each run. Thanks, M.

User avatar

Hello Mirka,

Are you using csv_friendly enabled?

Thanks,

User avatar

glass_fox

a year ago

Yes

User avatar

Okay. Let me returns the list of empty results keywords

User avatar

glass_fox

a year ago

Thanks! How long do you think it will take?

User avatar

I'm pretty busy this week. Maybe Thursday or Friday. Is this okay?

User avatar

glass_fox

a year ago

Yes, thanks!

User avatar

glass_fox

a year ago

It will be done soon?

User avatar

Hi Mirka,

Yes, I am working on it. Will notify you soon.

Thanks for checking.

User avatar

glass_fox

a year ago

Thanks!

User avatar

Thanks for your patience. I added the error field. If the query is empty, it has error: No results found

https://api.apify.com/v2/datasets/Ve7BgUqdMhdBfj6j0/items?clean=true&format=html

Developer
Maintained by Community
Actor metrics
  • 11 monthly users
  • 0 stars
  • 100.0% runs succeeded
  • Created in Mar 2023
  • Modified about 1 year ago