No credit card required

Smart Article Extractor

lukaskrivka/article-extractor-smart

No credit card required

📰 Smart Article Extractor extracts articles from any scientific, academic, or news website with just one click. The extractor crawls the whole website and automatically distinguishes articles from other web pages. Download your data as HTML table, JSON, Excel, RSS feed, and more.

All issues Create new issue

Does not support Hebrew

Closed

abiding_flare opened this issue

I've tried to run this scrapper on Hebrew websites and it didn't work. Is there a way to add Hebrew?

Lukáš Křivka (lukaskrivka)

Hi Makan,

I'm sorry but this scraper very likely will perform poorly for Hebrew (it might work on some websites).

For less detailed parsing, you can try https://apify.com/apify/website-content-crawler which should work better for non-English

abiding_flare

Thank you for your quick response, I do need all of the parsing. It is an amazing tool and exactly what I need. (I tried it in English with great results).

abiding_flare

Could this help with adjusting the scrapper to work in Hebrew? https://stackoverflow.com/questions/1365510/where-can-i-find-a-list-of-hebrew-stop-words

Lukáš Křivka (lukaskrivka)

I don't think that will be enough. There is quite a complicated parsing library behind it. If you know some good for Hebrew, we could add it as a backend.

abiding_flare

Will this help? https://github.com/amir-zeldes/HebPipe Thank you

abiding_flare

Hi, Do you think this https://github.com/amir-zeldes/HebPipe could help? I tried to run the scraper with no minimum words per article and got some information but not the full text. https://console.apify.com/view/runs/A4SU2iteF1ZF8Mkg3

I would appreciate if this could work for Hebrew.

abiding_flare

Hi, Is there any update on this? Thank you!

Lukáš Křivka (lukaskrivka)

Hi Makam, I'm sorry but this is a big feature and we don't see enough demand to justify the development cost just yet.

abiding_flare

Hi, Is there a way for us to pay for such a development?

Lukáš Křivka (lukaskrivka)

Hi Makam, You have 2 options

Ask some of our partners or freelancers to build this from scratch.
Reach out to https://apify.com/enterprise

The best bet for Hebrew is probably using GPT so maybe using https://apify.com/drobnikj/gpt-scraper would help here

Lukáš Křivka (lukaskrivka)

I will close this issue now, we will keep your request in mind for this scraper in case an easier path forward is found

Add comment

Developer

Lukáš Křivka

Actor metrics

172 monthly users
74.9% runs succeeded
2.8 days response time
Created in Nov 2019
Modified about 2 months ago

Categories

News

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Apify

43.5k

Website Content Crawler

apify/website-content-crawler

Automatically crawl and extract text content from websites with documentation, knowledge bases, help centers, or blogs. This Actor is designed to provide data to feed, fine-tune, or train large language models such as ChatGPT or LLaMA.

Apify

13k

Twitter Scraper

quacker/twitter-scraper

Scrape tweets from any Twitter user profile. Top Twitter API alternative to scrape Twitter hashtags, threads, replies, followers, images, videos, statistics, and Twitter history. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Quacker

24.3k

GPT Scraper

drobnikj/gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.

Jakub Drobník

4.3k

AI Product Matcher

equidem/ai-product-matcher

Match products across multiple e-commerce websites. Use this AI product matching Actor whenever you need to find matching pairs of products from different online shops for dynamic pricing, competitor analysis or market research.

Matěj Sochor

308

Google Trends Scraper

emastra/google-trends-scraper

Scrape data from Google Trends by search terms or URLs. Specify locations, define time ranges, select categories to get interest by subregion and over time, related queries and topics, and more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Emiliano Mastragostino

3.2k

AI Web Agent

apify/ai-web-agent

Use natural language prompts to browse the web, click on elements, fill and submit forms, extract data, and take screenshots using the OpenAI API.

Apify

412

Twitter URL Scraper

quacker/twitter-url-scraper

Copy any Twitter URL and extract Twitter usernames, profile photos, follower count, tweets, hashtags, favorite count, and more. Export scraped datasets, run the scraper via API, schedule and monitor runs or integrate with other tools.

Quacker

4.3k

Extended GPT Scraper

drobnikj/extended-gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.

Jakub Drobník

855