Analyze market trends with the Google Finance Scraper. Extract stock prices, company information, market caps, trading volumes, and financial metrics automatically. Great for financial research, data analysis, and monitoring investment opportunities.
Fixed - Actor was returning 0 results on every run
Root cause was a combination of three bugs in src/main.py:
HTTP redirects were never followed.www.google.com/finance/quote/<ticker>
now permanently 302-redirects to www.google.com/finance/beta/quote/<ticker>
(Google's page structure moved to a new "beta" layout). The impit.AsyncClient
instances in perform_request() did not pass follow_redirects=True (impit
defaults this to False), so every request returned an empty-bodied 302
response instead of the real page. Because the response status (302) wasn't in
the [403, 429] block-check and the empty body didn't match captcha/sorry,
this was silently treated as a "successful" fetch.
Crash on every single ticker.extract_google_finance_data() called
extract_price_from_js_vars(html_content), a function that was never defined
anywhere in the codebase. This raised a NameError on every run (guaranteed,
since the price-regex fallback above it could never match anything on the new
page anyway - see #3), which was swallowed by a broad except Exception,
causing the function to return None. Back in main(), if result: on None
silently skipped the ticker - no data pushed, no error record, no dataset row
at all. This is why the actor finished "successfully" while producing zero
output.
The HTML price scraper could never have matched real pages. Even with
redirects fixed, extract_current_price_from_html() looked for literal
data-price="...", price: ..., value: ... text/attributes in the page
HTML. Google Finance's quote page is a client-rendered SPA - none of that text
exists in the raw HTML. The real quote (price, change, %) is embedded as JSON
inside AF_initDataCallback({key: 'ds:N', ...}) script blocks used to hydrate
the page client-side.
Fixes applied:
perform_request(): added follow_redirects=True, max_redirects=5 to all three
AsyncClient(...) constructions (no-proxy, datacenter-proxy, residential-proxy
tiers) so the /finance/beta/quote/... redirect is followed and the actual page
HTML is retrieved.
Replaced the dead extract_current_price_from_html() regex scraper and the
undefined extract_price_from_js_vars() call with a working JSON-based
extractor (extract_quote_from_google_html() + helpers
_parse_af_init_data_blocks() / _find_quote_tuple()) that parses the page's
AF_initDataCallback payloads and locates the [price, change, changePercent, ...]
tuple for the requested ticker by content/shape rather than a fixed index path,
since the exact field layout differs slightly between equities, indices, mutual
funds, crypto pairs and forex pairs (verified against one live example of each).
main(): added an explicit else branch so a ticker that raises an exception
during extraction now still gets an error row pushed to the dataset instead of
vanishing without a trace.
Verified against live Google Finance + Yahoo Finance data for one URL of each
type in the actor's own default/prefill list:
.DJI (index), SWPPX:MUTF (mutual fund), GOOGL:NASDAQ (equity), ETH-BTC
(crypto pair), EUR-USD (forex pair) - all five now return a real current
price/change and 20-30 real historical data points for the default 1M period.
Notes
The legacy fetch_google_finance_api() fallback (google.com/finance/getprices)
now returns HTTP 404 - this endpoint appears to have been retired by Google.
It is not the cause of the 0-results bug (Yahoo Finance is tried first and
succeeds for all tested ticker types) and was left in place as a harmless,
already-guarded fallback rather than removed, to keep the fix minimal and
focused on the actual root cause.
