Indeed Scraper

2024-11-09

Fixes

• Temporarily disabled company detail pages for start URLs due to overaggressive blocking by Indeed
  • We are monitoring the situation and will re-enable these URLs once it's layed off

2024-11-06

Fixes

• Temporarily disabled the parseCompanyDetails input option due to overaggressive blocking by Indeed for company pages
  • We are monitoring the situation and will re-enable this option once it is resolved

2024-10-19

Fixes

• Fixed blocking issues on specific Indeed domains (e.g. ph.indeed.com)

2024-09-14

Fixes

• Re-added back previously blocked countries for search, as Indeed has unblocked them

2024-08-29

Fixes

• Removed old non-working countries for search from input
• Due to recent blocking changes on Indeed, we are unable to scrape jobs from the following website domains: (your input will fail if you use these)
  • ['TH', 'KR', 'SG', 'PH', 'ID', 'HK', 'AU']
  • Learn more at https://console.apify.com/actors/hMvNSpz3JnHgl5jkh/issues/3GLnmwPIA53ZkoLjy

2024-07-22

Fixes

• Fixed new blocking method by Indeed
• Fixed job links extraction from company profile pages

2024-07-08

Fixes

• Temporarily increase the memory limit to work around a memory leak in the new unblocking mechanism

Changes

• Set followApplyRedirects to be false by default. It is causing significant slowdown. We are working on making this option fast again but you will need to explicitly turn it on.

2024-07-01

Fixes

• Fixed external apply links extraction by greatly reducing their blocking rate
• Further reduced blocking rate for job detail pages
• Added warning for quoted queries in startUrls (containing double quotes)
  • These are currently blocked very heavily by Indeed, so it's recommended to avoid them

2024-05-09

Fixes

• Search for other countries now works again
• Blocking rate slightly decreased

2024-05-09

Features

• Added maxItemsPerSearch input field that limits results for each URL in startUrls

2024-04-19

Fixes

• Fixed blocking when URLs contained double quotes

2024-02-19

Fixes

• Fixed scraping search results after Indeed.com changed page layout

2024-01-04

Features

• Added isExpired field

2023-12-08

Fixes

• Fix extracting postedAt, which caused the scraper to fail completely

2023-10-17

Fixes

• Fix extracting jobType

2023-09-21

Fixes

• Fix extracting rating and reviewsCount on job detail page

2023-08-30

Fixes

• The location field now strips css noise.

2023-08-22

Changes

• The scraper now uses RESIDENTIAL proxies, which should result in less 403 errors.

2023-08-21

Features

• The scraper can now scrape company jobs if you provide a URL in the format of https://www.indeed.com/cmp/Google/jobs. To get it go to the company page, "Jobs" tab.

2023-08-08

Input changes:

• Proxies have been removed from input due to the difficulties of keeping the scraper at reasonable price and functionality with a lot of them. The default Apify proxy group is always used instead.

Fixes:

• Jobs that are marked as ads are now extracted correctly

2023-07-30

Fixes:

• Now skips links with /pagead/ segment in them, because it used to result in a lot of errors

2023-06-22

BREAKING CHANGES:

• Top-level companyLogo property was removed (it used to be null anyways). Now, to get the company logo, enable input option "Scrape company details".

Features

• New field under company info, companyLogo

2023-06-15

Features

• Added input parameter to make the scraper follow externalApplyLink and push the final redirect url to the output.

2023-06-08

Fixes

• Fixed externalApplyLink being always null

Improvements

• New searchInput field in the output that duplicates some input properties

2023-31-05

Improvements

• Added an option to scrape detailed company info (with "parseCompanyDetails" input property)

2023-03-06

Improvements

• Added new "descriptionHTML" field to the output;

2023-01-17

Improvements

• Added new "companyLogo" field to the output;

2022-11-07

Improvements

• Added new "jobType" field to the output;

2022-09-13

Improvements

• Added "saveOnlyUniqueItems" option to the input;

2022-01-31

Improvements

• Added new "externalApplyLink" field to the output;
• SDK update to 2.3.0.

2022-01-31

Improvements

• SDK update to 2.2.2.

2022-01-31

Improvements

• Added new "rating" field to the output;
• "reviews" field changed to "reviewsCount".

2022-01-12

• SDK update to 2.2.1.

2021-12-08

• SDK update to 2.1.0;
• Removed deduplication of results.

2021-09-21

• Added postedAt and scrapedAt fields

2021-15-07 - New SDK version (1.1.2).

Bugs

• Changed selectors (reason why we were getting 0 results);
• Fixed issue with broken pagination logic + implemented logic about limited pagination;

Improvements

• Improved logging during the run.