7 days trial then $20.00/month - No credit card required now

Google News Scraper

lhotanova/google-news-scraper

Try for free

7 days trial then $20.00/month - No credit card required now

Gets featured articles from Google News with title, link, source, publication date and image.

Back to issues Create new issue

scraper returning wrong links

Open

dynamic_harvest opened this issue

The scraper is returning wrong links "https://news.google.com/rss/articles/CBMib2h0dHBzOi8vd3d3LnBybmV3c3dpcmUuY29tL25ld3MtcmVsZWFzZXMvYW1lcmljYW4tY2VudHVyeS1sYXVuY2hlcy1jYWxpZm9ybmlhLW11bmljaXBhbC1ib25kLWV0Zi0zMDIyMDAyMzcuaHRtbNIBAA?oc=5&hl=en-IN&gl=IN&ceid=IN:en"

starting with "https://news.google.com/"

on clicking this link you get redirected to the correct website. This is breaking all my flows, this has been happening since last week, I was getting correct links before that

dynamic_harvest

After

dynamic_harvest

Before

Kristýna Lhoťanová (lhotanova)

Hello, thank you for reporting this bug and I'm sorry for a late reply. I'm currently reworking the Actor to address this issue as well as other bugs. I'll keep you updated about the progress here.

Kristýna Lhoťanová (lhotanova)

Hi again, there has been a recent change of Google News API that caused this bug. The Actor has been fixed today, see the example run: https://console.apify.com/view/runs/oqDhZjFezcTVZHdAi

The Actor first extracts the links in the https://news.google.com/ format from the API, then it decodes them to the actual target links and opens the target pages to extract preview images. It needs to be done this way because the Actor doesn't use a web browser that could resolve the redirects automatically. It uses HTTP requests only to keep the expenses low. Google News has recently made the encoding of target URLs more difficult to deal with, so the Actor now uses rather a hacky way to decode the links. Hopefully the Google News API will be stable now and won't break the Actor's flow again.

If you encounter any other issues, please report them using new issue threads 🙏

boothdev

Hi, we seems to be getting a lot of links like that causing it to show a good amount of failures on every run.

boothdev

Any update on this as we are seeing quite a few links come up as https://news.google.com/_/DotsSplashUi/data/batchexecute?rpcids=Fbv4je

Add comment

Developer

Kristýna Lhoťanová

Actor Metrics

93 monthly users
16 stars
>99% runs succeeded
40 days response time
Created in Oct 2022
Modified 3 months ago

Categories

News

Google News Scraper - Cheap

bot_kevin/Google-News-Scraper

Easily scrape news from Google News page in .json format.

bot

Tweet Scraper 2024 | $0.25/1000 Tweets | Pay-Per-Result API

kaitoeasyapi/twitter-x-data-tweet-scraper-pay-per-result-cheapest

Only $0.25/1000 tweets for Twitter scraping, 100% reliability, swift data retrieval.This incredible low price is almost too good to be true.Thanks to our large-scale operations and efficient servers, we can offer you rock-bottom prices that no competitors can match. Don't miss this opportunity !

Kaito Easy API

246

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, AI overviews, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule runs, or integrate with other tools.

Apify

50.8k

253

🔥 Fast Indeed Jobs Scraper

memo23/apify-indeed-cheerio

Web scraper for Indeed.com job listings. Add Indeed URLs to customize searches. Set max jobs, concurrency, and other parameters to optimize performance while respecting site resources. Efficiently extracts job data, balancing speed with ethical practices. Ideal for gathering targeted job market info

Muhamed Didovic

Traffic Generator (Youtube, Web, Etsy, Behance and many more!)

epctex/traffic-generator

Maximize your website's performance and visibility with our Traffic Generator. Drive targeted traffic, simulate page views, and stress-test against potential threats. With the power to generate millions of visits, it's the ultimate solution for boosting your online presence.

epctex

25.8k

184

🔥 LinkedIn Jobs Scraper

bebity/linkedin-jobs-scraper

ℹ️ Designed for both personal and professional use, simply enter your desired job title and location to receive a tailored list of job opportunities. Try it today!

Bebity

4.3k

Download HTML from URLs

mtrunkat/url-list-download-html

This actor takes a list of URLs and downloads HTML of each page.

Marek Trunkát

8.2k

Google Trends Scraper

emastra/google-trends-scraper

Scrape data from Google Trends by search terms or URLs. Specify locations, define time ranges, select categories to get interest by subregion and over time, related queries and topics, and more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Emiliano Mastragostino

3.8k

Twitter Tweets and Profiles Scraper

web.harvester/twitter-scraper

Easily search and extract tweets from profiles or directly using a URL with our tool. Scrape tweets and replies effortlessly. Download data in formats like JSON, CSV, XML, RSS, or HTML Table, perfect for integration with various applications, databases, and social media analytics tools.