No credit card required

Extended GPT Scraper

drobnikj/extended-gpt-scraper

No credit card required

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.

Back to issues Create new issue

4o mini model (for cost concern)

Closed

tak_lai opened this issue

Hi, since website scraping involved large input token and out token, any plans to add model 4o mini soon? Many thanks

Lukáš Průša (lukas.prusa)

Hi Tak, thanks opening this issue and your suggestion!

Yes, we will 100% add the 4o-mini model shortly :) We will also most likely add it to our Pay Per Result version of this Actor.

I will keep you updated here, thanks!

Lukáš Průša (lukas.prusa)

Hi again, thanks for your patience!

We've had some other issues with the scraper, so we kept in the beta for a bit too long now. The 4o-mini model is accessible on the latest (default) version :) We've also set it as the default one for the Pay Per Result Actor version.

Try it out and let me know how it works, thanks!

tak_lai

Thanks , I will try it. Really appreciate your effort

tak_lai

Hello , I just spotted another issue, which may relate to playwright. I tried to use playwright and selenium to scrape https://www.newbalance.com/pd/made-in-usa-990v6/U990V6-45189.html I am not sure why it is failed when using playwright. Since Extended GPT Scraper use playwright so it shared same result(retrieve blank page). It works using selenium in my local but failed to use playwright in my local. I just want to share this limitation as it is not the problem on this actor. I am truly thankful of your effort on this actor

Lukáš Průša (lukas.prusa)

Hi, thanks for sharing this inside. This looks to be some issue in playwright, or at least it's getting detected and blocked by the website.

I've tried a few settings like residential proxies and waiting longer for the dynamic content to load, but I guess it's still getting blocked... This looks a bit out of our scope as this is most likely an issue somewhere else, so unfortunately we won't be fixing this. But good luck with the website and I hope you will be able to scrape it well with selenium :)

Add comment

Developer

Jakub Drobník

Actor Metrics

89 monthly users
66 bookmarks
84% runs succeeded
1.1 days response time
Created in Jun 2023
Modified a month ago

Categories

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

36.5k

📩📍 Google Maps Email Extractor

lukaskrivka/google-maps-with-contact-details

Extract Google Maps contact details. Scrape websites of Google Maps places for contact details and get email addresses, website, location, address, zipcode, phone number, social media links. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Lukáš Křivka

15.6k

483

Instagram Scraper

apify/instagram-scraper

Scrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

74k

Facebook Posts Scraper

apify/facebook-posts-scraper

Extract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

18.2k

283

Google Maps Extractor

compass/google-maps-extractor

Extract data from hundreds of places fast. Scrape Google Maps by keyword, category, location, URLs & other filters. Get addresses, contact info, opening hours, popular times, prices, menus & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

29.3k

730

Contact Details Scraper

vdrmota/contact-info-scraper

Free email extractor and lead scraper to extract and download emails, phone numbers, Facebook, Twitter, LinkedIn, and Instagram profiles from any website. Extract contact information at scale from lists of URLs and download the data as Excel, CSV, JSON, HTML, and XML.

Vojta Drmota

28k

319

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

6.4k

120

Instagram Post Scraper

apify/instagram-post-scraper

Scrape Instagram posts. Just add one or more Instagram usernames and get your data in seconds including text, hashtags, mentions, comments, images, URLs, likes, locations, and metadata. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

25.8k

268

Web Scraper

apify/web-scraper

Crawls arbitrary websites using the Chrome browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

Apify

76.1k

455