Propertyfinder Scraper

🏡 What is Propertyfinder Real Estate Properties Scraper?

This Propertyfinder properties Scraper will enable you scrape any sale/rent listing from collection from propertyfinder.ae.

You can simply take your listing url from browser and enter it into this actor. This actor will crawl through all pages of particular listing and generate dataset for you.

Listing url is something you get when you perform the search on propertyfinder site. Example listing urls :

• https://www.propertyfinder.ae/en/search?c=2&fu=0&l=11-18&ob=nd&page=1&rp=y
• https://www.propertyfinder.ae/en/search?c=1&l=11-18-21-262-348-563&ob=nd&page=1
• https://www.propertyfinder.ae/en/search?c=1&l=11-18-21-262-348-563&ob=nd&page=1&t=35

You can fine tune your listing by applying various filters and then simply copy that URL and start scraping it using this actor. Additionally, in case you have list of specific prooperties to scrape, you can also provide hadnpicked collection of each property URL to scrape.

🚪 All in one propertyfinder scraper

All the propertyfinder country specific portals are supported by this single scraper :

Propertyfinder.ae

Propertyfinder.qa

Propertyfinder.bh

Propertyfinder.eg

Propertyfinder.sa

🚪 What can this Propertyfinder Scraper do?

📈 Extract Propertyfinder market data listings on Propertyfinder

👀 This actor is not just scraper but also has monitoring capability. You can turn on monitoring mode and it will give you only newly added properties compared to your previous scrapes.

📩 This actor also helps yu to identify which properties are not listed anymore. Please refer to Identifying delisted properties

⬇️ Download Propertyfinder real estate data in Excel, CSV, JSON, and other formats

📚 How do I start scraping with this scraper?

1. Register for your free Apify account here
2. You don't need to provide your credit card details for free acount. Just click on "Get Started" button on above link and complete the registration only.
3. Free account comes with reasonable credits to try out this actor. This actor also comes with free trial of 3 days without any commitment/upfront charge.
4. Run this actor and verify the scraped data. Apify has huge integration possibilities. You can download the data or push the data into any 3rd party platform directly.

🌳 What Propertyfinder data can I extract using this tool?

📝	📝
Listing Title	Full Address
Listing URL	Permit Number
Agent BRN	Completion Date
Bathrooms	Bedrooms
Broker	Agent Phone
Listing Type	Property Type
Latitude	Longitude
Furnishing Type	Freehold
Text Description	Formatted HTML Description
Amenities	Images
Price	Size
Listing Date	Property Age
Similar Transactions	RERA
Agent Information	Broker Information

⬇️ Input

For simple usecase, you just need to provide browser url of propertyfinder search result page & that's all. You can leave other fields as they are to be sensible defaults.

Input example

1{
2    "listUrls": [
3        {
4            "url": "https://www.propertyfinder.ae/en/search?c=1&l=11-18-21-262-348-563&ob=nd&page=1"
5        }
6    ],
7    "propertyUrls": [
8        {
9            "url": "https://www.propertyfinder.ae/en/plp/buy/villa-for-sale-abu-dhabi-al-karamah-al-musalla-area-liwa-village-9524137.html"
10        }
11    ],
12    "threads" : 3,
13    "fullScrape": true,
14    "monitoringMode": false,
15    "enableDelistingTracker" : false,
16    "addEmptyTrackerRecord" : false
17}

You can either provide listUrls to search properties from or provide propertyUrls directly to crawl.

Understading monitoring mode :

• fullScrape : This option is by default turned on. When enabled it always force actor to scrape complete listing from all pagination pages regardless of monitoring is enabled or not.

• monitoringMode : This option when turned on will only scrape newly added property listings compared to previously scraped properties by this actor. It's important to turn off fullScrape setting if you are using this mode. If you keep fullScrape on, it will re-scrape complete listing again.

• enableDelistingTracker : This option when turned on will start tracking date against each property under Apify Key Value store. This KV store can be queried later to find out which properties are delisted.

• threads: Number of parallel threads to run for scraping

• addEmptyTrackerRecord : This option when turned on will add empty record having only id of property to Apify dataset. This helps you identify whether property is still listed compared to your own database in incremental mode.

⬆️ Output

The scraped data is stored in the dataset of each run. The data can be viewed or downloaded in many popular formats, such as JSON, CSV, Excel, XML, RSS, and HTML.

Output example

The result for scraping a single property like this:

1{
2  "id": "9523636",
3  "url": "https://www.propertyfinder.ae/en/plp/buy/villa-for-sale-dubai-al-mizhar-al-mizhar-1-9523636.html",
4  "title": "Huge 6 Bedrooms Villa for Sale On Prime Location",
5  "displayAddress": "Al Mizhar 1, Al Mizhar, Dubai",
6  "bathrooms": 6,
7  "bedrooms": 6,
8  "addedOn":"2023-09-25T07:13:30+00:00",
9  "broker": "WAED ALEASRIA PROPERTIES",
10  "agent": "Jihad Itani",
11  "agentPhone": "+971506312497",
12  "verified": false,
13  "reference": "goldenrose-4393236",
14  "permit_number": "6533910000",
15  "agentBrn": "CN-1761158",
16  "freehold": true,
17  "completionDate": null,
18  "priceDuration": "sale",
19  "propertyAge": null,
20  "propertyType": "villa",
21  "price": 4900000,
22  "coordinates": {
23    "latitude": 25.246215,
24    "longitude": 55.441623
25  },
26  "type": "sale",
27  "sizeMin": "15,000 sqft",
28  "furnishing": "NO",
29  "features": [
30    "Unfurnished",
31    "Built in Wardrobes",
32    "Covered Parking",
33    "Maids Room",
34    "Private Garden",
35    "Study"
36  ],
37  "agentInfo":{
38    "id": "215858",
39    "image": "https://www.propertyfinder.ae/images/pf_agent/picture/16f48a2b818cefb651d0dff1aa711434a852ebfd/desktop",
40    "is_super_agent": false,
41    "name": "Muhammad Numan Siddique",
42    "user_id": 76541,
43    "email": "numansiddique@taurusfloor.ae",
44    "social": "https://www.linkedin.com/in/numan-siddique-02117b243",
45    "languages": [
46        "English",
47        "Hindi",
48        "Urdu",
49        "Punjabi"
50    ],
51    "slug": "muhammad-numan-siddique",
52    "avg_whatsapp_response_time": 857,
53    "total_properties": 33,
54    "position": "Real Estate Agent ",
55    "years_of_experience": 5
56  },
57  "brokerInfo": {
58    "id": "5928",
59    "logo": "https://www.propertyfinder.ae/broker/1/178/98/MODE/f1c7c7/5928-logo.jpg?ctr=ae",
60    "name": "Taurus Real Estate",
61    "address": "Office 232, Building Al Wasl Tower, Al Nahda, Al Nahda St, Dubai, ",
62    "email": "yaseen@taurusfloor.aa",
63    "phone": "0554539438",
64    "slug": "taurus-real-estate",
65    "total_properties": 293,
66    "license_number": "794945",
67    "is_exclusive": false
68  },
69  "description": "Waed Aleasria Properties LLC Is delighted To Present You this Huge 6 Bedrooms Villa with Nice view, Majlis and Maid Room for Sale on Prime Location in Al Mizhir 1.\nProperty Highlights:\n* Affordable Price\n* 6 Bedrooms with 2 Master Bedrooms\n* Maid Room\n* Covered Parking\n* Private Garden\n* 3 Majlis\n* Store\n* 6 Bathrooms\n* Kitchen\n* Size: 15000 sqft\nAMENITIES: -\n* Children play Area\n* Public Parking\n* Restaurants\n* Shops\n* Mall\n* Public Transport\n* Public Park\nAl Mizhar 1 is a sub-community in Al Mizhar. The community is divided into two parts, Al Mizhar 1 and Al Mizhar 2. Al Mizhar 1 is a quiet neighborhood with a well-structured road system. It is one of the most budget-friendly areas in the city.\nLocated close to Sharjah, Al Mizhar 1 is a large sub-community with all major amenities a resident may desire. From schools, supermarkets, entertainment options to mosques, everything is available within the vicinity. Located close to Mushrif Park\nThe community boasts many educational institutions\nFOR FURTHER DETAILS PLEASE CALL: Display phone number| Company Name: Waed Aleasria Properties LLC RERA ORN: 31176. Address: Rega Al Buteen, Al Salmiya Tower, 3rd Floor, Office no: 301, Deira.\nDisplay phone number",
70  "descriptionHTML" : "<div><p>HTML version of above description would be here</p></div>",
71  "images": [
72    "https://www.propertyfinder.ae/property/99733391fda9de64572f2c454fa721f1/1312/894/MODE/a14561/9523636-9ad25o.jpg?ctr=ae",
73    "https://www.propertyfinder.ae/property/bed4d06eb382daf61819679fa38b2478/1312/894/MODE/c6229b/9523636-58dbdo.jpg?ctr=ae",
74    "https://www.propertyfinder.ae/property/f10c541f11513783db7ed26716e9bfd9/1312/894/MODE/d26b16/9523636-1a21eo.jpg?ctr=ae",
75    "https://www.propertyfinder.ae/property/78b5dba2e45ae8d8baf2547faa0457b4/1312/894/MODE/32a6ba/9523636-91450o.jpg?ctr=ae"
76  ],
77  "similarTransactions": {
78    "buy": [
79        {
80            "amount": 1510000,
81            "date": "2024-02-06T00:00:00+00:00",
82            "size": 649
83        },
84        {
85            "amount": 1000000,
86            "date": "2024-02-05T00:00:00+00:00",
87            "size": 664
88        }
89    ],
90    "rent": [
91        {
92            "price": 105000,
93            "date": "2024-02-15T00:00:00+00:00",
94            "size": 654
95        },
96        {
97            "price": 55650,
98            "date": "2024-02-12T00:00:00+00:00",
99            "size": 662
100        },
101    ]
102  }
103}

❓Deduplication handled

If multiple list urls contains overlapping results, they will get deduplicated within same run data.

❓Limitations

Since Propertyfinder allows only 100K properties per listing/search result, you might want to break down your listing urls into smaller area if it has more than 100K results. Good News is that even if multiple list urls contains overlapping results, they will get deduplicated within same run data.

🔎 Identifying delisted properties

This actor provides you monitoring mode configuration using which you can get only incremental updates about newly added properties. In case, you also want to identify which properties are delisted from platform, you can use any of the following techniques with the help of this actor.

1. Running Always in full scraper mode : Run this actor always in full scrape mode and cross check the new incoming batch of data with your existing database. If any property that exists in yoru database but not in newly scraped data batch, that means it's not listed anymore

2. Use Key Value Store generated by scraper : If your are monitoring very large batch of data and you don't want to scrape everything all the time, this method involves bit of technicality but achieves the goal efectively. Apify has storage feature called Key-value store. When you run this scrape, this scraper stores every single property in key value store along with timestamp in propertyfinder-properties store. Inside this store, key is property id itself and value is timestamp like this

   { lastSeen : '2023-11-02T05:59:25.763Z'}
   

   Whenever you run this scraper, it will update the timestamp against particular id if it finds property on the platform. e.g. if we have 2 proprties with id prop1 and prop2 and we scraped them both on November 1, key value storage would look like this :

   1prop1 -> { lastSeen : '2023-11-01T05:59:25.763Z'}
   2prop2 -> { lastSeen : '2023-11-01T05:59:25.763Z'}
   

   Now if you run this scraper again on December 1 and prop1 is not on the platform anymore but prop2 is still there, key value storage would change like this :

   1prop1 -> { lastSeen : '2023-11-01T05:59:25.763Z'}
   2prop2 -> { lastSeen : '2023-12-01T05:59:25.763Z'}
   

   That means if any property has lastSeen less than latest batch you loaded, that property is delisted now. You can directly iterate through whole Key value storage using Apify key value storage API to identify this. Please refer to this API documentation to do the same. Please remember store name generated by this scrape will be propertyfinder-properties.

   Alternatively, you can iterate through your existing database active properties and use this API to identify listing status.

   For this approach to work, it's important that you enable this feature via enableDelistingTracker (Enable Delisting tracker) input.

🙋‍♀️ For custom solutions

In case you need some custom solution, you can contact me : dhrumil@techvasu.com

Or learn more about me on github : https://github.com/dhrumil4u360