📚 Grokipedia.com Scraper | Fast Wikipedia-Like Data Extraction (2025)

Extract comprehensive knowledge from Grokipedia at lightning speed This powerful API scraper delivers rich, structured data including search metadata, citations, and full article content—all processed in parallel for maximum performance.

Whether you're conducting academic research, building knowledge bases, or analyzing information networks, this scraper provides reliable access to Grokipedia's extensive collection with search relevance scoring, view counts, and context-aware metadata that traditional scrapers miss.

Features

🔍 Powerful Search Capabilities

• Full-text search - Find articles across Grokipedia's entire knowledge base with relevance scoring
• Flexible query syntax - Use natural language search terms to discover relevant content
• Result limiting - Control the number of results (1-10000) to match your research needs
• Search metadata - Get context including total match count, search time, and result rankings

📄 Flexible Page Access Methods

• Direct URL support - Scrape individual pages by providing Grokipedia URLs
• Search-based discovery - Start with search queries and automatically fetch all matching pages
• Dual-mode operation - Seamlessly switch between search mode and direct page access
• URL parsing - Automatically detects search URLs vs. page URLs for smart execution

⚙️ Granular Data Control

• Toggle citations - Include or exclude references and citations to optimize data size
• Content control - Choose whether to include full article content (~100KB+ per article)
• Format selection - Convert markdown to HTML for direct web rendering, or keep markdown for processing flexibility
• Selective extraction - Fine-tune what data you need to reduce processing time and costs
• Structured output - Consistent JSON format with typed fields for easy integration

Use Cases

📊 Academic Researchers & Scientists

• Literature review automation - Search for research topics and extract comprehensive article collections with citations for academic papers
• Data validation - Cross-reference Grokipedia content with primary sources using included citation lists
• Longitudinal studies - Track article evolution over time by monitoring view counts and modification timestamps
• Knowledge graph construction - Build connected datasets using linkedPages to map information relationships
• Bibliometric analysis - Analyze citation patterns and reference networks across related topics
• Research dataset creation - Generate structured datasets for meta-analyses with controlled content inclusion

✍️ Content Creators & Knowledge Workers

• Content ideation - Search trending topics and analyze high-relevance articles to identify content opportunities
• Fact-checking workflows - Extract articles with citations for verification of claims and sources
• Topic research - Gather comprehensive background information including snippets and highlights for writing projects
• Knowledge base population - Build internal wikis and documentation by extracting structured article data
• SEO keyword research - Analyze relevance scores and search metadata to understand content performance

🤖 Data Scientists & ML Engineers

• Training dataset generation - Extract large volumes of structured text with metadata for machine learning models
• Information extraction - Parse article content to identify entities, relationships, and semantic patterns
• Trend analysis - Monitor viewCount and recentViews metrics to identify emerging topics
• Content classification - Use search relevance scores and categories to build topic taxonomies
• Text similarity analysis - Compare snippets and highlights across search results for clustering
• Quality assessment - Leverage qualityScore and fixedIssues data to filter high-quality content

🔬 Knowledge Management Teams

• Competitive intelligence - Monitor specific topics by scraping and analyzing related articles at scale
• Content audit - Extract comprehensive article metadata to assess coverage gaps in knowledge domains
• Information architecture - Map linkedPages relationships to understand information hierarchies
• Search optimization - Analyze titleHighlights and snippetHighlights to understand query-content matching
• Citation tracking - Build reference networks by extracting and analyzing citation data across articles

Quick Start

Basic Search Query

{
  "searchQuery": "artificial intelligence",
  "limit": 10
}

This simple configuration searches for "artificial intelligence" and returns the top 10 matching articles with full content and citations.

Advanced Direct URL Access

{
  "url": "https://grokipedia.com/Artificial_intelligence",
  "includeCitations": true,
  "includeContent": false
}

Fetch a specific page by URL with citations but without full content to reduce data size for metadata-only analysis.

HTML Format for Web Integration

{
  "searchQuery": "blockchain technology",
  "limit": 10,
  "convertMarkdownToHtml": true,
  "includeContent": true
}

Convert article content to HTML for direct rendering in web applications without client-side markdown processing.

Complete Configuration

{
  "searchQuery": "machine learning",
  "limit": 50,
  "includeCitations": true,
  "includeContent": true,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Full-featured setup with search, moderate result limit, all data included, and residential proxies for maximum reliability.

Input Parameters

Parameter	Type	Description	Default
searchQuery	string	Search query to find Grokipedia articles. Either this OR 'url' must be provided. Example: "artificial intelligence"	None (required unless url provided)
url	string	Direct Grokipedia URL (search or page). Either this OR 'searchQuery' must be provided. Examples: "https://grokipedia.com/search?q=ai" or "https://grokipedia.com/Artificial_intelligence"	None (required unless searchQuery provided)
limit	integer	Maximum number of search results to scrape. Only applies to search mode. Range: 1-10000	10
minRelevanceScore	integer	Filter out low-relevance results. 0 = all results, 100 = good matches, 500+ = very relevant. Only applies to search mode. Range: 0-10000	0
includeCitations	boolean	Include citations/references from articles in the output	true
includeContent	boolean	Include full article content. Warning: Can be large (~100KB+ per article). Disable for metadata-only extraction	true
convertMarkdownToHtml	boolean	Convert markdown content and description to HTML format. Useful for web rendering or when markdown processing is not available	false
proxyConfiguration	object	Apify proxy configuration for reliable API access. Supports useApifyProxy flag and apifyProxyGroups array	{"useApifyProxy": false}

Note: searchQuery and url are mutually exclusive—provide one or the other, not both.

Output

Each scraped article is returned as a structured JSON object containing:

• Basic Information - slug, title, description (markdown or HTML based on convertMarkdownToHtml)
• Content - Full article text in markdown or HTML format (if includeContent: true)
• Metadata - categories, lastModified, contentLength, version, language, quality flags
• Statistics - totalViews, recentViews, dailyAvgViews, qualityScore, lastViewed
• Media - images[] with captions, URLs, positions, and dimensions
• Relationships - linkedPages (indexed and unindexed slugs)
• Citations - citations[] with id, title, description, URL (if includeCitations: true)
• Quality Metrics - fixedIssues[] documenting content improvements
• Search Context (search mode only) - _search_metadata with query, totalCount, searchTimeMs, resultIndex, relevanceScore, viewCount, snippet, highlights
• Timestamps - scraped_at, scraped_at_timestamp

Example Output

{
  "type": "page",
  "slug": "Artificial_intelligence",
  "title": "Artificial Intelligence",
  "description": "Comprehensive overview of AI technology, history, and applications",
  "content": "# Ai\n\nArtificial intelligence (AI) is a machine-based system that, for a given set of human-defined objectives, can make predictions, recommendations, or decisions influencing real or virtual environments through processes such as learning from experience, adapting to new inputs, and executing tasks associated with human cognitive functions like reasoning and problem-solving.[](https://csrc.nist.gov/glossary/term/artificial_intelligence)[](https://www.nibib.nih.gov/science-education/science-topics/artificial-intelligence-ai) Originating as a formal field in the 1950s with foundational work on symbolic reasoning and early neural networks, AI has evolved through cycles of optimism and setbacks, driven by advances in computational power, data availability, and algorithmic innovations like backpropagation and transformer architectures... <snip>",
  "metadata": {
      "categories": [
        "AI",
        "A.I.",
        "Artificial Intelligence"
      ],
      "lastModified": "1761585482",
      "contentLength": "183951",
      "version": "1.0",
      "lastEditor": "system",
      "language": "en",
      "isRedirect": false,
      "redirectTarget": "",
      "isWithheld": false
  },
  "stats": {
    "totalViews": "119644",
    "recentViews": "119644",
    "dailyAvgViews": 3988.13330078125,
    "qualityScore": 1,
    "lastViewed": "1762188888"
  },
  "images": [
    {
      "caption": "AI neural network visualization",
      "url": "https://grokipedia.com/images/ai-network.jpg",
      "position": 1,
      "width": 1200,
      "height": 800
    }
  ],
  "linkedPages": {
    "indexed": ["Machine_learning", "Neural_networks", "Deep_learning"],
    "unindexed": ["Future_of_AI"]
  },
  "citations": [
    {
      "id": "cite_1",
      "title": "The Quest for Artificial Intelligence",
      "description": "Cambridge University Press",
      "url": "https://example.com/ai-history"
    }
  ],
  "fixedIssues": [],
  "_search_metadata": {
    "query": "artificial intelligence",
    "totalCount": 1247,
    "searchTimeMs": 142.5,
    "resultIndex": 0,
    "relevanceScore": 0.98,
    "viewCount": 4532891,
    "snippet": "Artificial intelligence (AI) is intelligence demonstrated by machines, as opposed to natural intelligence...",
    "titleHighlights": ["Artificial", "Intelligence"],
    "snippetHighlights": ["artificial intelligence", "machines", "intelligence"]
  },
  "scraped_at": "2025-11-03T10:30:45Z",
  "scraped_at_timestamp": 1730632245
}

Pricing

This actor uses a Pay Per Result (PPR) pricing model at $3.00 per 1,000 pages extracted.

Pages Scraped	Cost
100 pages	$0.30
500 pages	$1.50
1,000 pages	$3.00

API Integration

Python Example

from apify_client import ApifyClient

# Initialize the Apify client
client = ApifyClient("YOUR_APIFY_API_TOKEN")

# Prepare the actor input
run_input = {
    "searchQuery": "quantum computing",
    "limit": 25,
    "includeCitations": True,
    "includeContent": True,
    "proxyConfiguration": {
        "useApifyProxy": True
    }
}

# Run the actor and wait for completion
run = client.actor("YOUR_USERNAME/grokipedia-scraper").call(run_input=run_input)

# Fetch results from the dataset
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"Title: {item['title']}")
    print(f"Slug: {item['slug']}")
    print(f"Views: {item['stats']['totalViews']:,}")

    # Access search metadata if available
    if '_search_metadata' in item:
        print(f"Relevance: {item['_search_metadata']['relevanceScore']}")
        print(f"Position: {item['_search_metadata']['resultIndex']}")

    print(f"Citations: {len(item.get('citations', []))}")
    print("---")

JavaScript Example

import { ApifyClient } from 'apify-client';

// Initialize the Apify client
const client = new ApifyClient({
    token: 'YOUR_APIFY_API_TOKEN',
});

// Prepare the actor input
const input = {
    searchQuery: "quantum computing",
    limit: 25,
    includeCitations: true,
    includeContent: true,
    proxyConfiguration: {
        useApifyProxy: true
    }
};

// Run the actor and wait for completion
const run = await client.actor("YOUR_USERNAME/grokipedia-scraper").call(input);

// Fetch results from the dataset
const { items } = await client.dataset(run.defaultDatasetId).listItems();

items.forEach((item) => {
    console.log(`Title: ${item.title}`);
    console.log(`Slug: ${item.slug}`);
    console.log(`Views: ${item.stats.totalViews.toLocaleString()}`);

    // Access search metadata if available
    if (item._search_metadata) {
        console.log(`Relevance: ${item._search_metadata.relevanceScore}`);
        console.log(`Position: ${item._search_metadata.resultIndex}`);
    }

    console.log(`Citations: ${item.citations?.length || 0}`);
    console.log('---');
});

Advanced Usage

Bulk Research with Multiple Searches

Run multiple searches to build comprehensive datasets across different topics:

{
  "searchQuery": "renewable energy",
  "limit": 100,
  "includeCitations": true,
  "includeContent": true
}

Then create separate runs for related topics like "solar power", "wind energy", "hydroelectric" to build a complete energy research database.

Citation-Only Extraction for Literature Review

Extract metadata and references without full content to minimize data size and processing time:

{
  "searchQuery": "machine learning applications",
  "limit": 50,
  "includeCitations": true,
  "includeContent": false
}

Perfect for bibliometric analysis where you only need citation networks and article metadata.

Content-Heavy Scraping with Quality Filtering

For building knowledge bases, extract full content and use quality metrics to filter results:

{
  "searchQuery": "artificial intelligence ethics",
  "limit": 75,
  "includeCitations": true,
  "includeContent": true
}

Post-process results by filtering for stats.qualityScore > 0.85 to ensure high-quality content.

Targeted Page Collection via Direct URLs

When you have specific pages to scrape, use direct URL mode with batch processing:

{
  "url": "https://grokipedia.com/Deep_learning",
  "includeCitations": true,
  "includeContent": true
}

Set up multiple runs with different URLs for parallel extraction of known pages.

HTML Output for CMS or Web Apps

Extract content in HTML format for direct integration with content management systems or web applications:

{
  "searchQuery": "machine learning",
  "limit": 20,
  "convertMarkdownToHtml": true,
  "includeContent": true,
  "includeCitations": true
}

The convertMarkdownToHtml option transforms both content and description fields from markdown to HTML, making it ready for immediate web rendering without additional processing.

Proxy-Enhanced Reliability

For large-scale scraping or when facing rate limits, enable proxies:

{
  "searchQuery": "biotechnology",
  "limit": 100,
  "includeCitations": true,
  "includeContent": true,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Understanding Relevance Scores

FAQ

What data can I extract from Grokipedia articles?

You can extract comprehensive article data including titles, full content, descriptions, metadata (categories, modification dates, versions), statistics (views, quality scores), linked pages, citations/references, and quality improvement records. When using search mode, you also get search-specific metadata like relevance scores, view counts, snippets, and highlight positions.

Is a Grokipedia account or login required?

No login or authentication is required.

What's included in search metadata?

Search metadata (_search_metadata) includes the original query, total match count, search execution time, result position (resultIndex), relevance score, view count, snippet preview, and highlighted terms in both title and snippet. This context is crucial for understanding why a result was returned and its ranking.

Can I scrape individual pages without searching?

Yes, absolutely. Use the url parameter with a direct Grokipedia page URL (e.g., "https://grokipedia.com/Artificial_intelligence"). The actor will fetch just that single page with all its details, bypassing the search functionality entirely. This is perfect for targeted extraction of known pages.

How much does it cost to scrape 1,000 articles?

At $3.00 per 1,000 pages, scraping 1,000 articles costs exactly $3.00. Scraping 100 pages costs $0.30, 500 pages costs $1.50, and 10,000 pages costs $30.00. Pricing is per page regardless of content size or citation count.

Can I filter or limit the data returned?

Yes, you have fine-grained control. Use the limit parameter (1-10000) to cap search results. Set includeContent: false to exclude full article text and reduce data size by ~90%. Set includeCitations: false to omit reference lists. You can also filter results in post-processing using quality scores, view counts, or relevance scores.

Can I get the content in HTML format instead of markdown?

Yes, set convertMarkdownToHtml: true to convert both the content and description fields from markdown to HTML format. This is useful for direct web rendering, CMS integration, or when you don't have markdown processing capabilities. The conversion happens automatically and preserves all formatting including headers, links, images, and emphasis.

Can I use this for real-time monitoring?

While the actor itself is designed for batch extraction, you can set up scheduled runs via Apify's scheduling feature to monitor specific topics or pages at regular intervals (hourly, daily, weekly). Each run produces a timestamped dataset, allowing you to track changes over time. Use metadata.lastModified to detect content updates.

How do I export the scraped data?

All scraped data is stored in Apify's dataset storage and can be exported in multiple formats: JSON, CSV, Excel (XLSX), XML, RSS, or HTML. You can download directly from the Apify Console or use the Apify API to programmatically fetch results in your preferred format.

Getting Started

Step 1: Set Up Your Apify Account

Create a free account at apify.com if you don't have one already. No credit card required to start—the free tier includes generous usage limits for testing.

Step 2: Configure Your Scraping Task

Navigate to the Grokipedia API Scraper in the Apify Store and click Try for Free. Configure your input:

• Choose between searchQuery (for topic-based extraction) or url (for specific pages)
• Set your limit to control the number of results
• Toggle includeCitations and includeContent based on your data needs
• Optionally enable proxyConfiguration for enhanced reliability

Step 3: Run and Monitor

Click Start to launch your scraping run. Monitor progress in real-time through the Apify Console. You'll see logs showing search results found, pages being fetched in parallel, and completion status. Most runs complete in under 2 minutes for typical configurations.

Step 4: Export and Integrate

Once complete, export your data in JSON, CSV, Excel, or other formats directly from the Console. Alternatively, integrate the actor into your workflows using the Apify API with Python, JavaScript, or other languages. Set up scheduled runs for automated data collection or webhook notifications for real-time integration.

Support

• 📧 Email: max@mapa.slmail.me
• 📖 Found a bug?: Use the issues tab and describe your issue
• 🔧 Feature Requests: Contact via email or issues tab for additional features

Legal Compliance

This Grokipedia.com scraper extracts publicly available data from Grokipedia's website. Users must comply with Grokipedia.com terms of service and applicable data protection regulations for their intended use.

---

🚀 Start Extracting Grokipedia Data Now

Try Grokipedia API Scraper →

Extract comprehensive Wikipedia-like knowledge at scale with parallel processing, rich metadata, and zero authentication—your research deserves the best data infrastructure.

---

Built with ❤️ for researchers, data scientists, and knowledge workers worldwide. Happy scraping!