Keyword Extractor: key phrases from text and pages (YAKE, RAKE)

Pull the key phrases out of many documents at once. Paste texts, send JSON records with your own IDs, list web pages, or point it at an Apify dataset, and get the top phrases of each document, ranked and scored.

Methods

• YAKE (default) scores each word by its casing, its position, its frequency, how many sentences it appears in, and how varied its neighbors are, then scores phrases of up to 3 words from those. It needs no training data and works on a single document.
• RAKE splits the text at stop words and punctuation into candidate phrases, and scores each phrase by how often its words appear and how many other words they appear with. It favors longer phrases.

Both are written for this actor from their published descriptions and run inside it. No text is sent to any outside service.

For each document

• keywords: the top phrases, best first, each with keyword (in its most common capitalization), score (the method's raw score: lower is better for YAKE, higher is better for RAKE), relevance (1 for the best phrase, smaller for the rest, comparable across both methods), count (times the phrase appears) and words
• topKeywords: the first 10 phrases as plain text
• language: the stop word list used, wordCount, textPreview
• For web pages, title. The page title is analyzed together with the main text.
• Every other field of a record you sent, such as your id

On the Python Software Foundation's About page, YAKE returns "Python", "Python Software Foundation", "Welcome to Python", "Python Software" and "Python Package Index" among the top phrases.

Input

• Texts: one document per entry
• Texts with IDs: JSON objects such as {"id": "review-17", "text": "...", "rating": 5}
• Web pages: the title and main text of each page. Menus, footers and scripts are left out.
• Apify datasets: each item is one document. The field is found for you (text, markdown, content, body, description, article, review, comment), or name it.
• Method, language (auto by default), keywords per text (default 20), longest phrase (default 3 words), shortest word (default 3 letters), similar phrase cut off (default 0.9), extra stop words

Stop word lists cover 69 languages, from the spaCy and stopwords-iso projects (both MIT licensed). Auto picks the list that matches the most words of the text. A language written without spaces between words, such as Japanese, is not split into words, so whole sentences come back as phrases. Use it on languages that put spaces between words.

Output

Dataset: one item per document, in input order. A document that cannot be analyzed gets ok: false and an error, such as HTTP 404, an empty text, or fewer than 3 words. The other documents still run.

Pricing

Pay per event: one event per document analyzed. A document that fails is not charged. The run stops when it reaches the maximum charge you set.

Scope and conduct

Each web page is downloaded once. Nothing is stored beyond the run's own storage.

Support

Questions and problems: the Issues tab on this page. Replies are written by an AI assistant on behalf of the developer and reviewed by the developer.