Reed.co.uk Scraper avatar

Reed.co.uk Scraper

Try for free

1 day trial then $30.00/month - No credit card required now

Go to Store
Reed.co.uk Scraper

Reed.co.uk Scraper

lexis-solutions/reed-co-uk-scraper
Try for free

1 day trial then $30.00/month - No credit card required now

The Reed.co.uk scraper is a web scraping tool that retrieves job postings from Reed.co.uk, a job search website in the UK 🇬🇧

TE

companyProfileURL is empty for some results

Closed

testtheworkout opened this issue
6 months ago

Hello, I've noticed that the 'companyProfileURL' is blank for some records and not for others. Every record has a 'companyName' and when clicking on the record I can see no reason why they shouldn't all have a 'companyProfileURL' as there is a clickable name present for all.

An example would be:

This role has a companyProfileURL: https://www.reed.co.uk/jobs/senior-data-manager/52918589

This role does not: https://www.reed.co.uk/jobs/data-scientist/52915607

They both have a companyName and from looking at the record we should also have the 'companyProfileURL'.

I imagine there will also be other values missing for these records as well but I am not sure what other columns would be impacted.

Thanks for looking into this, and any other information you need, please let me know.

If no companyProfileURL is available, is there another identifier unique identifier that could be added to the file? I am trying to identify who is posting each record and I don't think companyName is unique so I am struggling on the other records

lexis-solutions avatar

Hi, if you open the links from the job page, you can see they are different kinds of pages.

  1. https://www.reed.co.uk/company-profile/willmott-dixon-1340 is a company page, with a banner, description, etc.
  2. https://www.reed.co.uk/jobs/adecco-uk-limited-45420/p45420 is not a company page, but a list of jobs posted by the employer.

To ensure there is always a value that can be used as a key, we've added a new field "companyNameURL", this is the href value from the job page on the employer's name.

This field can be either of the types, but will be present, if the company name leads somewhere.

Hope this is helpful, feel free to reopen the issue if there's anything else we can improve.

Thanks!

Developer
Maintained by Community

Actor Metrics

  • 7 monthly users

  • 9 stars

  • >99% runs succeeded

  • Created in Mar 2023

  • Modified 3 months ago

Categories