Quora Scraper with authentication(login)

Overview

Quora Scraper is a specialized tool designed for the Apify platform, enabling users to efficiently extract data from Quora. This scraper is capable of searching Quora using specific queries and can handle multiple search URLs simultaneously. While authentication is not mandatory, logging in can significantly increase the breadth of accessible data, particularly in terms of the number of questions retrieved.

Features

Multiple Search Queries: Supports scraping based on multiple search URLs, formatted as https://quora.com/search?q=your_query. Required Authentication over cookies: Users can log in to access a broader range of questions and answers. Customizable: Flexible configuration options to suit various scraping needs. Data Extraction: Efficiently scrapes questions, answers, user profiles, and other relevant data from Quora.

How to Use

1. Set Up: Ensure you have an Apify account and access to the Apify platform.
2. Input Configuration: Input your search URLs in the format https://quora.com/search?q=your_query. You can add multiple URLs for a broader search scope.
3. Authentication: Quora has made some changes so in order to access data from search you need to have account on it or to be logged in. You can input your cookies from quora.com site to access the data. Cookies are used to authorize actor with quora.com. Install EditThisCookie chrome extension. Login to your quora.com account. Click on the extension and export the quora.com cookies and insert into the input.
4. Run the Scraper: Execute the scraper on the Apify platform. You can customize settings like the number of pages to scrape, delay between requests, etc.
5. Data Collection: The scraper will output data in your chosen format, which can include JSON, HTML, CSV, Excel, or others as supported by Apify.

Output Structure

The output data typically includes:

• Questions: Titles, details, and metadata of the questions.
• Answers: number of answers, along with author information (next step would be to add answers for each question which would be configurable).
• User Profiles: Information about users who have asked or answered questions.
• Metadata: Additional information like question URLs, timestamps, etc.

Best Practices

• Rate Limiting: Respect Quora's terms of service by avoiding overly frequent requests.
• Error Handling: Implement robust error handling to manage request failures or unexpected data formats.
• Data Storage: Utilize Apify's data storage solutions for efficient data management.

Examples

Here's a simple example of how to set up a search for "Socrates":

1{
2  "searchUrls": ["https://quora.com/search?q=socrates"],
3  "cookies": "input your cookies from quora.com site"
4}

Request customization: webtester83@gmail.com

If you need anything else scraped, or this actor customized email me: webtester83@gmail.com

If you'd like an API of this service (no apify fee, just usage fee for the API), contact me: webtester83@gmail.com