Actor - LinkedIn companies and jobs scraper

LinkedIn companies and jobs scraper

Main goal is to get companies per sector or industry, for example use keywords like retail or real estate and optional location, like country name or city, state, country. Can also get just jobs. No login required so public company details can be scraped without limits. In very rare cases when a company page is not reached without login, the actor will save the company URL along with the job lead and set notAvailableWithoutLogin in the dataset.

Input example

1{
2    "searchList": [ "web scraping" ],
3    "resultsLimit": 10,
4    "resultsType": "company",
5	"proxy": {
6            "useApifyProxy": true,
7            "groups": ["RESIDENTIAL"],
8	}
9}

Scraped data

The structure of each job item in output looks like below:

Job listing example (fastest output)

1{
2	"keyword": "web scraping",
3	"id": "3506286369",
4	"date": "2023-03-03",
5	"jobTitle": "Junior Data Analyst",
6	"jobUrl": "https://www.linkedin.com/jobs/view/junior-data-analyst-at-buffalo-groupe-3506286369",
7	"salaryRange": "$45,000.00 - $55,000.00",
8	"companyName": "Buffalo Groupe",
9	"companyUrl": "https://www.linkedin.com/company/buffalo-groupe",
10	"location": "Portland, Oregon Metropolitan Area"
11}

Company details from listing example

1{
2	"keyword": "web scraping",
3	"name": "Buffalo Groupe",
4	"url": "https://www.linkedin.com/company/buffalo-groupe",
5	"address": {
6		"type": "PostalAddress",
7		"streetAddress": "12700 Sunrise Valley Drive, Suite 300",
8		"addressLocality": "Reston",
9		"addressRegion": "VA",
10		"postalCode": "20191",
11		"addressCountry": "US"
12	},
13	"description": "Buffalo Groupe is a collective of agencies (Buffalo Agency, 54 Brands, Rawle Murdy, Longitudes Group), media properties (Sports Illustrated Golf, The First Call) and events (Friars Golf Club, NFL Alumni Association, MLB Alumni Association) that authentically serve the industries of golf, travel, fitness, outdoor, lifestyle, real estate and other sports. We focus on the “moments that matter” to elevate our partners through various digital marketing, creative, content, social, data and PR services.",
14	"numberOfEmployees": {
15		"value": 24,
16		"@type": "QuantitativeValue"
17	},
18	"logo": {
19		"contentUrl": "https://media.licdn.com/dms/image/C560BAQEa-ZkoUCwaPA/company-logo_200_200/0/1615822937424?e=2147483647&v=beta&t=NvEIg5rFhgObJcoBX-hz2p75SXlShCFqS5XxAw7CVIo",
20		"description": "Buffalo Groupe",
21		"@type": "ImageObject"
22	},
23	"slogan": "A collection of agencies and media properties serving audiences in active sport, outdoor rec and wellness industries.",
24	"sameAs": "http://www.buffalogroupe.com"
25}

Company details not reached without login example

Rarely happens, expected to be resolved by running again later

1{
2	"keyword": "web scraping",
3	"url": "https://www.linkedin.com/company/penn-state-university",
4	"notAvailableWithoutLogin": true,
5	"fromSearch": {
6		"id": "3512814688",
7		"date": "2023-02-11",
8		"jobTitle": "Part-Time Research Assistant",
9		"jobUrl": "https://www.linkedin.com/jobs/view/part-time-research-assistant-at-penn-state-university-3512814688",
10		"salaryRange": "",
11		"companyName": "Penn State University",
12		"companyUrl": "https://www.linkedin.com/company/penn-state-university",
13		"location": "University Park, PA"
14	}
15}

Job details example

1{
2	"keyword": "web scraping",
3	"datePosted": "2023-03-03T17:54:54.000Z",
4	"description": "<p>Longitudes Group, a Buffalo Groupe company, is an industry-leading sports marketing research and consulting company. As a pioneer in geographic-based demographic research in golf, fitness, outdoor sports and travel, Longitudes Group serves our clients by offering a unique view of their business and laying the foundation for high-impact planning and action. Our services include distribution planning and sales force productivity research, consumer demand modeling, customized market surveys and much more.</p><p><br></p><p>Buffalo Groupe, the parent company of Longitudes Group, is more than a holding company. It is a carefully assembled collection of agencies, events and media properties that authentically serve the escape industries of golf, outdoor recreation, travel, lifestyle, real estate and active sport. Together, our Collective provides clients the best match of resources based on their goals and needs.</p><p>We are headquartered near Washington, DC with offices in Charleston, Charlotte, Boulder, Portland and LA.</p><p><br></p><p><strong>New team member</strong> will begin as full-time with benefits</p><p>The ideal candidate is a highly technical, hard worker, who pays close attention to detail while demonstrating outstanding communication skills.</p><p><br></p><p>Our new team will demonstrate the following skills:</p><ul><li>High self -motivation and show outstanding team, collaborative skills to go the extra mile</li><li>Ability to work independently requiring minimal direction, execute large projects following established guidelines/procedures and take initiative to explore and solve problems.</li><li>Strong process orientation and time management skills to manage multiple project streams</li><li>Ability to meet multiple deadlines within short time frame</li><li>Ability to visualize data in easy-to-digest/meaningful formats</li><li>Keep up-to-date with sporting goods/outdoor industry, research, trends, and standard methodologies.</li></ul><p><br></p><p><strong>Required Skills:</strong></p><ul><li>Proficiency in MS Office.</li><li>Excel – data analysis and visualization.</li><li>Proficient in data pipeline procedures using either R or Python:</li><li>Explore</li><li>Clean</li><li>Analyze</li><li>Visualize</li><li>Report</li><li>SQL – beginner Create Read Update Delete.</li><li>Proficient with data presentation in either PowerPoint or google slides.</li></ul><p><br></p><p><strong>Desired Skills:</strong></p><ul><li>Excel – VBA macros, custom dashboard creation, Power Query.</li><li>Tableau.</li><li>Webscraping/Webcrawling with Python or internal software.</li><li>Experience in sports/recreation industry, passion for golf or outdoor sports.</li><li>Google Cloud Platform – BigQuery and APIs.</li></ul><p><br></p><p>Preferred Qualifications: BA or BS degree with 1-2 years of experience including potential experience in any of the following areas: Data management, market research, sports or recreation industries, marketing.</p><p><br></p><p>Responsibilities include:</p><ul><li><strong>Data Management</strong>: Ongoing online data collection, organization, and storage within internal databases. Online data sources can include and are not limited to: US Census, dealer locators, consumer datasets from credit bureaus or 3rd party aggregators.</li><li><strong>Data analysis</strong>: We have a wide range of clients who rely on many different types of data and insights that we provide. Successful candidate will have the ability to research potential data sources and enrichment opportunities for client projects while keeping the overall project budget in-mind. Plan and present research strategies both internally and to clients. Execute an analysis project from start to finish by importing and cleaning client data, to analyzing and presenting findings in easy-to-understand formats.</li></ul><p><br></p><p>Salary: $45,000 - $55,000</p><p><br></p><p><strong><span class=\"ql-cursor\"> </span>Work will begin hybrid office/remote if you are located in Portland, OR, or full-remote if located elsewhere.</strong></p>",
5	"employmentType": "FULL_TIME",
6	"hiringOrganization": {
7		"@type": "Organization",
8		"name": "Buffalo Groupe",
9		"sameAs": "https://www.linkedin.com/company/buffalo-groupe",
10		"logo": "https://media.licdn.com/dms/image/C560BAQEa-ZkoUCwaPA/company-logo_200_200/0/1615822937424?e=2147483647&v=beta&t=NvEIg5rFhgObJcoBX-hz2p75SXlShCFqS5XxAw7CVIo"
11	},
12	"identifier": {
13		"@type": "PropertyValue",
14		"name": "Buffalo Groupe"
15	},
16	"image": "https://media.licdn.com/dms/image/C560BAQEa-ZkoUCwaPA/company-logo_100_100/0/1615822937424?e=2147483647&v=beta&t=bTNAoHIcdMbkzRya5LWwwO11ruxgu3Y9HQLHtumjWGQ",
17	"industry": "Advertising Services",
18	"jobLocation": {
19		"@type": "Place",
20		"address": {
21			"@type": "PostalAddress",
22			"addressCountry": "US",
23			"addressLocality": "Portland",
24			"addressRegion": "Oregon Metropolitan Area",
25			"streetAddress": null
26		},
27		"latitude": 45.553497,
28		"longitude": -122.90853
29	},
30	"skills": "",
31	"title": "  Junior Data Analyst",
32	"validThrough": "2023-04-02T17:54:54.000Z",
33	"educationRequirements": {
34		"@type": "EducationalOccupationalCredential",
35		"credentialCategory": "bachelor degree"
36	},
37	"experienceRequirements": {
38		"@type": "OccupationalExperienceRequirements",
39		"monthsOfExperience": 12
40	},
41	"jobLocationType": "TELECOMMUTE",
42	"applicantLocationRequirements": {
43		"@type": "Country",
44		"name": "United States"
45	},
46	"baseSalary": {
47		"@type": "MonetaryAmount",
48		"currency": "USD",
49		"value": {
50			"@type": "QuantitativeValue",
51			"minValue": 45000,
52			"maxValue": 55000,
53			"unitText": "YEAR"
54		}
55	}
56}