Craigslist Scraper avatar

Craigslist Scraper

Try for free

2 hours trial then $25.00/month - No credit card required now

Go to Store
Craigslist Scraper

Craigslist Scraper

ivanvs/craigslist-scraper
Try for free

2 hours trial then $25.00/month - No credit card required now

Extract data from classified advertisements on Craigslist. Scrape contact details from jobs, housing, items wanted, items for sale, services, community service, gigs, events and resumes listed on Craigslist. Download listings data in JSON, XML, Excel, and other versatile

IM

Sample URL

Closed

aimeb opened this issue
2 years ago

I tried running this actor using the sample URL provided in the documentation. I get the following error:

TypeError: Cannot read properties of undefined (reading 'split')

Is this actor broken?

ivanvs avatar

Hi, page HTML is changed a little bit. So there are some changes that needs to be done in order for scraper to continue working. I've done some testing and fix a problem with some of categories that I have noticed. It should be working now. I will continue testing just to be sure there is no additional changes that I haven't noticed yet.

Thank you for informing me about this issue, and yes it seems that for a few hours scraper was broken :(

IM

aimeb

2 years ago

Thank you, seems to work now. Do you have an option to collect contact information that is available after clicking on the Reply button? Also, any option to avoid duplicate post scraping by limiting to ads that are in the specific requested city?

ivanvs avatar

Currently I don't have option regarding Replay button. But thank you for suggestion, I like it so I will try to implement that, it seems like a nice addition that other users would also appreciate.

The second thing I don't think I understand what is the problem that you want to point out. If you can give me additional info, maybe I would understand that problem better and than I could implement that also.

IM

aimeb

2 years ago

I plan on searching multiple cities with the same search terms. When I search a city that only has a few results CL will also append additional results ("more from nearby areas") with ads from neighboring cities. When I then run another actor to do the same search on a nearby city I will be duplicating the scraping on many ads.

https://fortcollins.craigslist.org/search/gra?bundleDuplicates=1&hasPic=1&query=truck&srchType=T#search=1~list~0~0

https://rockies.craigslist.org/search/gra?bundleDuplicates=1&hasPic=1&query=truck&srchType=T#search=1~list~0~0

ivanvs avatar

Ok, I think I understand the problem that you have. Let me think if I can do something in scraper to fix that issue.

IM

aimeb

2 years ago

Great thanks. Perhaps if your actor accepts multiple search url's it could gather all the ad ID's and throw away duplicates before navigating/scraping those?

ivanvs avatar

Hi A B, I've added both collecting of emails and multi search url's. For collecting emails you should enable proxy in order to have better success rate.

IM

aimeb

2 years ago

That sounds promising. Unfortunately my trial is expired. Are you able to reset the trial period so that I can try these features out?

ivanvs avatar

Sadly but I don't see that option. I have only number of users that are using my actor, but I don't have any additional info regarding that.

Developer
Maintained by Community

Actor Metrics

  • 14 monthly users

  • 5 stars

  • >99% runs succeeded

  • 6.7 days response time

  • Created in Sep 2022

  • Modified 20 days ago