Actor - Google Jobs Scraper

Google Jobs scraper

The Google Jobs data scraper supports the following features:

• Search any keyword and job title - Seach any keyword or job title on Google Jobs. Retrieve extensive set of results in seconds.

• Target your job results - Enable filtering, sorting or targeting your results by country, language. Even pin point location and draw a radius.

• Extremely detailed and enriched output, better visibility - Very detailed output with all the sections included. Salary, qualifications, social links, apply links and many more!

• Highly configurable, easy to use - Flexible configuration enables you to easily use the actor without hassle, and focus only on your needs.

• Optional CSV Friendly Output - Use the actor for data retrieval or API purposes. Optional CSV-Friendly field enables you to get all the results in one item or multiple items. Easy integration.

• Daily maintenance, fast support - We offer daily maintenance and a blazing fast customer support. Your satisfaction is the number one priority.

Bugs, fixes, updates, and changelog

This scraper is under active development. If you have any feature requests you can create an issue from here.

Input Parameters

The input of this scraper should be JSON containing the list of pages on Google Jobs that should be visited. Required fields are:

• startUrls: (Optional) (Array) List of Google Jobs URLs. URLs to start with. It should be Job Listing URL.

• maxItems: (Optional) (Number) You can limit scraped items. This should be useful when you search through the big lists or search results.

• endPage: (Optional) (Number) Final number of page that you want to scrape. The default is Infinite. This applies to all search requests and startUrls individually.

• queries: (Optional) (Array) The keywords/queries you want to search on Google Jobs. This field is required whenever startUrls is not present.

• countryCode: (Optional) (String) Country determines the Google Jobs search domain (e.g. google.es for Spain). This setting only applies to Search queries.

• languageCode: (Optional) (String) This code specifies the precise location for Google Jobs searches. It's used as the 'uule' query parameter in Google Jobs Search URLs. Generate the UULE code using this generator: https://padavvan.github.io/.

• radius: (Optional) (Number) Enables job searching within a specified radius, in kilometers.

• includeUnfilteredResults: (Optional) (Boolean) If checked, the lower-quality results that Google normally filters out will be included. This usually consists of a few hundred extra results.

• csvFriendlyOutput: (Optional) (Boolean) If checked, the crawler will return results in a structure suitable for CSV format. Only 'googleJobs' results are included. CSV headers would be: title, companyName, location, description, and more.

• proxy: (Required) (Proxy Object) Proxy configuration.

• extendOutputFunction: (Optional) (String) Function that takes a JQuery handle ($) as an argument and returns an object with data.

• customMapFunction: (Optional) (String) Function that takes each object's handle as an argument and returns the object with executing the function.

This solution requires the use of Proxy servers, either your own proxy servers or you can use Apify Proxy.

Tip

When you want to scrape over a specific list URL, just copy and paste the link as one of the startUrl.

If you would like to scrape only the first page of a list then put the link for the page and have the endPage as 1.

With the last approach that is explained above you can also fetch any interval of pages. If you provide the 5th page of a list and define the endPage parameter as 6 then you'll have the 5th and 6th pages only.

Compute Unit Consumption

The actor is optimized to run blazing fast and scrape as many items as possible. Therefore, it forefronts all the detailed requests. If the actor doesn't block very often it'll scrape 100 jobs in 2.5 minutes with ~0.1-0.15 compute units.

Google Jobs Scraper Input example

1{
2    "startUrls": [
3        "https://www.google.com/search?q=Software+Engineer+Jobs&uule=w+CAIQICIKY2FsaWZvcm5pYQ==&hl=en&gl=us&udm=8&jbr=sep:0"
4    ],
5    "queries": [
6        "teacher"
7    ],
8    "countryCode": "us",
9    "languageCode": "en",
10    "locationUule": "w+CAIQICIFdGV4YXM=",
11    "radius": 300,
12    "includeUnfilteredResults": false,
13    "csvFriendlyOutput": true,
14    "proxy": {
15        "useApifyProxy": true
16    },
17    "endPage": 5,
18    "maxItems": 100
19}

During the Run

During the run, the actor will output messages letting you know what is going on. Each message always contains a short label specifying which page from the provided list is currently specified. When items are loaded from the page, you should see a message about this event with a loaded item count and total item count for each page.

If you provide incorrect input to the actor, it will immediately stop with a failure state and output an explanation of what is wrong.

Google Jobs Export

During the run, the actor stores results into a dataset. Each item is a separate item in the dataset.

You can manage the results in any language (Python, PHP, Node JS/NPM). See the FAQ or our API reference to learn more about getting results from this Google Jobs actor.

Google Jobs Output

CSV Friendly

1{
2	"title": "Software Developer",
3	"companyName": "New York University",
4	"location": "New York, NY",
5	"via": "via ICIMS - NYU Jobs",
6	"description": "Position Summary\n\nDevelop applications using current technologies to support business processes in an agile (scrum) development environment. Troubleshoot and debug existing applications. Participate in the design, development, testing, and deployment of upgrades and enhancements for the capital project management application. Prepare technical specifications; design, develop and test technical... solutions; and provide technical expertise upon request. Write documentation and online help systems for newly developed applications. Develop and implement innovative ways to improve quality and functionality of applications and share suggestions and knowledge capital with application owners and team members. Participate in peer code review processes. Mentor and develop student employees.\n\nQualifications\n\nRequired Education:Bachelor's Degree or equivalent in Computer Science or Computer Systems Engineering or equivalent combination of education and experience.Required Experience:3+ years of full-time related experience. Demonstrated programming experience in a scrum environment. 3+ years of full-time related experience in developing and maintaining complex enterprise applications in a construction management application with Oracle Unifier, Oracle Primavera, Kahua, eBuilder or ProCore.Required Skills, Knowledge and Abilities:C#, Javascript, Typescript, CSS, HTML, React.JS,, SQL,, and full stack .NET experience. Strong technical knowledge, written and verbal communication and good interpersonal skills. Strong analytical ability.Preferred Skills, Knowledge and Abilities:Experience with Oracle uDesigner, Kahua kBuilder, or similar. Experience building api integrations using an integration platform such as MuleSoft or similar. Knowledge of Database architecture and data manipulation techniques. Experience with developing microservice applications. Experience with big data analytics. Experience with Jelly.\n\nAdditional Information\n\nIn compliance with NYC's Pay Transparency Act, the annual base salary range for this position is USD $85,500.00 to USD $104,500.00. New York University considers factors such as (but not limited to) scope and responsibilities of the position, candidate's work experience, education/training, key skills, internal peer equity, as well as, market and organizational considerations when extending an offer. This pay range represents base pay only and excludes any additional items such as incentives, bonuses, clinical compensation, or other items. NYU aims to be among the greenest urban campuses in the country and carbon neutral by 2040. Learn more at nyu.edu/nyugreen.EOE/AA/Minorities/Females/Vet/Disabled/Sexual Orientation/Gender Identity",
7	"jobHighlights": [
8		{
9			"title": "Qualifications",
10			"items": [
11				"Required Education:Bachelor's Degree or equivalent in Computer Science or Computer Systems Engineering or equivalent combination of education and experience",
12				"Required Experience:3+ years of full-time related experience",
13				"Demonstrated programming experience in a scrum environment",
14				"3+ years of full-time related experience in developing and maintaining complex enterprise applications in a construction management application with Oracle Unifier, Oracle Primavera, Kahua, eBuilder or ProCore",
15				"Required Skills, Knowledge and Abilities:C#, Javascript, Typescript, CSS, HTML, React.JS,, SQL,, and full stack .NET experience",
16				"Strong technical knowledge, written and verbal communication and good interpersonal skills"
17			]
18		},
19		{
20			"title": "Responsibilities",
21			"items": [
22				"Develop applications using current technologies to support business processes in an agile (scrum) development environment",
23				"Troubleshoot and debug existing applications",
24				"Participate in the design, development, testing, and deployment of upgrades and enhancements for the capital project management application",
25				"Prepare technical specifications; design, develop and test technical solutions; and provide technical expertise upon request",
26				"Write documentation and online help systems for newly developed applications",
27				"Develop and implement innovative ways to improve quality and functionality of applications and share suggestions and knowledge capital with application owners and team members"
28			]
29		},
30		{
31			"title": "Qualifications",
32			"items": [
33				"Required Education:Bachelor's Degree or equivalent in Computer Science or Computer Systems Engineering or equivalent combination of education and experience",
34				"Required Experience:3+ years of full-time related experience",
35				"Demonstrated programming experience in a scrum environment",
36				"3+ years of full-time related experience in developing and maintaining complex enterprise applications in a construction management application with Oracle Unifier, Oracle Primavera, Kahua, eBuilder or ProCore",
37				"Required Skills, Knowledge and Abilities:C#, Javascript, Typescript, CSS, HTML, React.JS,, SQL,, and full stack .NET experience",
38				"Strong technical knowledge, written and verbal communication and good interpersonal skills",
39				"Strong analytical ability",
40				"Experience building api integrations using an integration platform such as MuleSoft or similar",
41				"Knowledge of Database architecture and data manipulation techniques",
42				"Experience with developing microservice applications",
43				"Experience with big data analytics"
44			]
45		},
46		{
47			"title": "Responsibilities",
48			"items": [
49				"Develop applications using current technologies to support business processes in an agile (scrum) development environment",
50				"Troubleshoot and debug existing applications",
51				"Participate in the design, development, testing, and deployment of upgrades and enhancements for the capital project management application",
52				"Prepare technical specifications; design, develop and test technical solutions; and provide technical expertise upon request",
53				"Write documentation and online help systems for newly developed applications",
54				"Develop and implement innovative ways to improve quality and functionality of applications and share suggestions and knowledge capital with application owners and team members",
55				"Participate in peer code review processes",
56				"Mentor and develop student employees"
57			]
58		}
59	],
60	"applyLink": [
61		{
62			"title": "icims.com",
63			"link": "https://uscareers-nyu.icims.com/jobs/12183/software-developer/job"
64		},
65		{
66			"title": "higheredjobs.com",
67			"link": "https://www.higheredjobs.com/admin/details.cfm?JobCode=178439960"
68		},
69		{
70			"title": "hercjobs.org",
71			"link": "https://main.hercjobs.org/jobs/19449282/software-developer"
72		},
73		{
74			"title": "mendeley.com",
75			"link": "https://www.mendeley.com/careers/job/software-developer-25672393"
76		},
77		{
78			"title": "simplyhired.com",
79			"link": "https://www.simplyhired.com/job/wE0SSrM3o-ZJ4mqRmgyl5sYYOAcN8FIF79yQHjQcpaVHKHgDamVrYg"
80		},
81		{
82			"title": "magazine.org",
83			"link": "https://jobs.magazine.org/jobs/rss/19449282/software-developer"
84		},
85		{
86			"title": "getwork.com",
87			"link": "https://getwork.com/details/a175ebab149668617a0480a36cc7e03b"
88		},
89		{
90			"title": "adzuna.com",
91			"link": "https://www.adzuna.com/details/4170656790"
92		}
93	],
94	"extras": [
95		"health_insurance"
96	],
97	"metadata": {
98		"postedAt": "1 day ago",
99		"scheduleType": "Full-time",
100		"salary": "10 an hour"
101	},
102	"logo": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTAzHMaOaLeGJbAGcR8rkK4uTsEoBCdFgvPLDf0&s=0"
103}

Non-CSV Friendly

1{
2	"searchQuery": {
3		"term": "Software Engineer jobs",
4		"page": 1,
5		"type": "SEARCH",
6		"domain": "https://www.google.com",
7		"countryCode": "US",
8		"locationUule": null,
9		"resultsPerPage": 10
10	},
11	"url": "https://www.google.com/search?q=Software+Engineer+Jobs&uule=w+CAIQICIKY2FsaWZvcm5pYQ==&hl=en&gl=us&udm=8&jbr=sep:0",
12	"googleJobs": [...],
13	"filters": [
14		{
15			"name": "Remote",
16			"parameters": {
17				"uds": "ADvngMjcH0KdF7qGWtwTBrP0nt...",
18				"q": "software engineer remote"
19			},
20			"link": "https://www.google.com/search?sc..."
21		}
22	]
23}

Contact

Please visit us through epctex.com to see all the products that are available for you. If you are looking for any custom integration or so, please reach out to us through the chat box in epctex.com. In need of support? devops@epctex.com is at your service.