Google Jobs Scraper

Google's jobs results as a table. Give it a job title and a city and you get one row per advert: the company, the location, the board the advert came from, how long ago it was posted, the salary where Google has one, the full job description, and every site the job can be applied on.

The reason to read jobs through Google rather than one board at a time is that Google has already merged them. A single advert here typically carries six to eight apply links, one per board that listed it, so you see LinkedIn, Indeed, the company's own careers page and four aggregators in one row instead of deduplicating four scrapes by hand.

Ten jobs per search, and why that is not a setting

This is the one limit to plan around. A Google jobs search returns ten adverts and there is no page two: the surface does not page, and asking for more returns nothing rather than the next ten. Measured, twice, on the same query.

So the way to a bigger dataset is more searches, and the input is shaped for exactly that. Job searches takes a list, Locations takes a list, and every search runs once per location. Four titles across five cities is twenty queries and up to two hundred adverts.

That works in your favour more than it sounds: "python developer" in Austin and "python developer" in Denver return genuinely different companies, where page two of a single search would have returned more of the same.

What you get per advert

• title, company, location, via, posted, employment_type
• salary_text, salary_min, salary_max, salary_currency, salary_period
• description — the advert's own text
• qualifications, responsibilities, benefits — Google's own extraction from the original post, as three separate lists
• apply_options — every board the job can be applied on, with its link
• share_url, company_logo, job_id
• search, search_location, market, language, position

via is worth a second look. It names the board Google took the advert from, which is where the job actually lives, and it is often the company's own careers site rather than an aggregator. Filtering on it is the quickest way to separate first-party postings from reposts.

qualifications and responsibilities are the fields that make this cheaper than reading the boards yourself. Google has already pulled the requirements out of the prose, so a screen for "3+ years" or "Kubernetes" is a filter on a list rather than a search through paragraphs.

Salary is present on a minority of adverts

Two adverts in ten carried a salary on the page this was measured against, and that is Google's doing rather than a gap here. The rest genuinely do not publish one. salary_text keeps the line exactly as advertised because the same figure gets written a dozen ways across boards, and the parsed salary_min / salary_max / salary_period sit next to it as a convenience. When the two disagree, trust the text.

posted is similar: Google dates some adverts and leaves others undated. A null means undated, not missed.

Apply links

Every apply link comes back as Google's own redirect. Turn on Resolve apply links to follow each one through to the board it lands on and get the real URL instead, capped by Maximum apply links to resolve.

It is off by default because it costs an extra request per link and adverts carry up to eight of them, which on a forty-advert run is a couple of hundred requests to learn what the board's name already told you. Turn it on when you are going to open the links, leave it off when you are analysing the market.

Market and language

Country is the market the search is made for and Language is what the page comes back written in. They are separate on purpose: an English-language search of the German market is a normal thing to want, and pinning both to de would silently change the results as well as the wording.

Job results are regional, so the same title returns different companies per market. Berlin, London, Toronto, Chicago and Singapore all returned a full ten adverts when this was measured. Some city-and-title pairs return nothing at all, which the run reports as queriesWithNoJobs rather than as an error, so check that number before concluding a market is unavailable.

If you are comparing two countries, run them as two runs and keep market on every row, which it already is.

Reading the run summary

summary in the key-value store is worth a glance after every run:

• queriesSent against jobs tells you the yield per search
• queriesWithNoJobs is how many searches Google answered with nothing
• requestsRetried is how much work the run spent getting served; a high number against a healthy jobs count is normal, a high number against zero jobs is not
• withSalary says how much of the salary column is actually populated before you build anything on it

Errors

Code	Meaning
blocked	The search could not be read after every attempt
bad_input	No search term was given
no_sessions	The run has no Google session configured

A search Google answered with no jobs is not an error and does not appear here. It is counted in the run summary as queriesWithNoJobs, because a search nobody is hiring for and a search that was refused look identical if you only count rows, and only one of them is worth running again.

Raise Attempts per query if you see blocked. Google serves this surface to a fraction of attempts, so retries are normal and the default clears it comfortably.

Related actors

For the boards themselves, see the LinkedIn job actors and the XING job scraper for the German market. For the companies behind the adverts, the LinkedIn company actors.