Contact

If you encounter any issues or need to exchange information, please feel free to contact us through the following link: My profile

Scryfall.com Scraper: Extract Card Data & Pricing Intelligence

Introduction

Scryfall stands as "a fast, powerful, comprehensive Magic: The Gathering card search" engine that has revolutionized how players and developers access Magic card data. With "a large set of keywords and expressions you can use to find Magic: The Gathering cards", Scryfall's advanced search capabilities enable precise filtering across the entire Magic database.

However, manually extracting data from complex search results becomes impractical when dealing with large datasets. Whether you're analyzing market trends across specific card sets, building comprehensive databases for tournament preparation, or conducting research on Magic's design evolution, the need for automated bulk data extraction from Scryfall's search results is critical. Our Scryfall Search Scraper addresses this challenge by automating the extraction of detailed card information from any Scryfall search query, enabling efficient processing of hundreds or thousands of cards at once.

Scraper Overview

The Scryfall Search Scraper is a specialized tool designed to process Scryfall's advanced search URLs and extract comprehensive card data from paginated search results. Unlike simple card scrapers that target individual card pages, this scraper handles the complexity of Scryfall's search system, processing multiple pages of results and extracting detailed information from each card found.

Key Capabilities:

• Advanced Query Processing: Handles complex Scryfall search syntax including set codes, rarity filters, collector number ranges, and multi-parameter queries
• Pagination Management: Automatically processes multiple pages of search results to capture complete datasets
• Flexible Result Limits: Configurable items per URL to control data volume and processing time
• Proxy Integration: Built-in residential proxy support ensures reliable access and prevents rate limiting
• Structured Data Output: Delivers consistent, structured card information ready for analysis or integration

Target Applications:

• Market analysis and price tracking across card categories
• Set completion tracking and collection management
• Tournament meta analysis and deck building research
• Academic research on Magic card design and power level
• Database building for MTG-related applications and tools

User Categories:

• Data analysts and market researchers
• MTG application developers and database administrators
• Competitive players and tournament organizers
• Content creators and MTG community builders
• Academic researchers studying game design

Input and Output Details

Example url 1: https://scryfall.com/search?order=set&q=set%3Apdft+is%3Astamped&unique=prints

Example url 2: https://scryfall.com/search?order=set&q=set%3Afca+border%3Aborderless+-type%3Aplaneswalker&unique=prints

Example url 3: https://scryfall.com/search?order=set&q=e%3Atdm+cn%E2%89%A51+cn%E2%89%A4286&unique=prints

Example Screenshot of card information page:

Input Format

The scraper accepts configuration through a JSON object with several key parameters:

Scrape with URLs:

{
  "max_retries_per_url": 2, // Maximum number of retry attempts for each URL
  "proxy": { // Proxy configuration to avoid bot detection
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL" 
    ],
    "apifyProxyCountry": "US" // Choose a country that matches your target data location
  },
  "max_items_per_url": 20, // Total number of items you want to scrape
  "urls": [ // Card list page URLs to scrape
    "https://scryfall.com/search?order=set&q=set%3Apdft+is%3Astamped&unique=prints",
    "https://scryfall.com/search?order=set&q=set%3Afca+border%3Aborderless+-type%3Aplaneswalker&unique=prints",
    "https://scryfall.com/search?order=set&q=e%3Atdm+cn%E2%89%A51+cn%E2%89%A4286&unique=prints"
  ],
  "ignore_url_failures": true // Continue scraping even if some URLs fail
}

The urls parameter: List of card list page URLs that you want to scrape. You can add URLs one by one, or use the Bulk edit section to add a prepared list.

The ignore_url_failures parameter: If set to true, the scraper will continue running even if some URLs fail to be scraped after reaching the maximum number of retries. This ensures that one problematic URL doesn't stop your entire scraping job.

When you provide a list of URLs for scraping, all options in the "Scrape with search filters" section will be disabled. The system will only collect data from the URLs you specified.

Scrape with Search Filters:

{
  "max_retries_per_url": 2, // Maximum number of retry attempts for each search filter
  "proxy": { // Proxy configuration to avoid bot detection
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL" 
    ],
    "apifyProxyCountry": "US" // Choose a country that matches your target data location
  },
  "max_items_per_url": 20, // Total number of items you want to scrape
  "keyword": "black lotus", // Search keyword to find cards
  "sort_by": "usd", // Sort cards by specific criteria
  "order": "desc", // Order results ascending or descending
  "unique": "prints", // Filter by uniqueness
  "page": 1 // Starting page number for search results
}

The keyword parameter: The search keyword to find cards (e.g., "black lotus", "lightning bolt", "counterspell").

The sort_by parameter: Sort cards by various criteria:

• "name" - Name
• "released" - Release Date
• "set" - Set/Number
• "rarity" - Rarity
• "color" - Color
• "usd" - Price: USD
• "tix" - Price: TIX
• "eur" - Price: EUR
• "cmc" - Mana Value
• "power" - Power
• "toughness" - Toughness
• "artist" - Artist Name
• "edhrec" - EDHREC Rank
• "review" - Set Review

The order parameter: Order results in ascending or descending order:

• "" - Auto (default sorting order)
• "asc" - Ascending
• "desc" - Descending

The unique parameter: Filter cards by uniqueness:

• "cards" - Cards (show unique card names)
• "prints" - Prints (show all printings of each card)
• "art" - Unique Art (show cards with unique artwork)

The page parameter: Starting page number for scraping, useful for continuing interrupted scrapes or targeting specific result ranges.

When using search filters for scraping, you need to leave the urls field empty (or set it to null) in the "Scrape with URLs" configuration.

General Options:

The max_items_per_url parameter: Limits the number of cards extracted from each card list page or search results page. The default value is 20, providing a manageable batch size while allowing for comprehensive data collection.

The max_retries_per_url parameter: Sets the maximum number of retry attempts for each URL or search filters if the scrape is detected as a bot or the page fails to load. The default value is 2, providing a good balance between thoroughness and efficiency.

The proxy parameter: Proxy configuration is essential for maintaining anonymity and avoiding detection. The residential proxy option ensures that your scraping activities appear as legitimate browsing, reducing the risk of being blocked or rate-limited. You should choose a country that matches the location of the website you're scraping (e.g., US for scryfall.com).

Output Format

You get the output from the Scryfall.com Card Search Scraper stored in a tab. The following is an example of the Information Fields collected after running the Actor.

[ // List of card information
  {
  "url": "https://scryfall.com/card/eoe/18/focus-fire",
  "name": "Focus Fire (Edge of Eternities)",
  "description": "{W} • Instant • Focus Fire deals X damage to target attacking or blocking creature, where X is 2 plus the number of creatures and/or Spacecraft you control. • Edge of Eternities (EOE) #18 • Illustrated by Borja Pindado • Magic: The Gathering, MTG",
  "sku": "a9ddfcbc-0f84-4315-aaa3-ca54ff64d7de",
  "brand": {
    "@type": "Brand",
    "name": "Magic: The Gathering",
    "url": "http://magic.wizards.com"
  },
  "image": [
    "https://cards.scryfall.io/png/front/a/9/a9ddfcbc-0f84-4315-aaa3-ca54ff64d7de.png?1752946622",
    "https://cards.scryfall.io/large/front/a/9/a9ddfcbc-0f84-4315-aaa3-ca54ff64d7de.jpg?1752946622",
    "https://cards.scryfall.io/normal/front/a/9/a9ddfcbc-0f84-4315-aaa3-ca54ff64d7de.jpg?1752946622",
    "https://cards.scryfall.io/small/front/a/9/a9ddfcbc-0f84-4315-aaa3-ca54ff64d7de.jpg?1752946622"
  ],
  "buy_urls": [
    {
      "title": "TCGplayer articles about this card",
      "url": "https://partner.tcgplayer.com/c/4931599/1830156/21018?subId1=card-detail-infinite-articles&trafcat=tcgplayer.com%2Fsearch%2Farticles&u=https%3A%2F%2Fwww.tcgplayer.com%2Fsearch%2Farticles%3FproductLineName%3Dmagic%26q%3DAnticausal%2BVestige"
    },
    {
      "title": "TCGplayer decks with this card",
      "url": "https://partner.tcgplayer.com/c/4931599/1830156/21018?subId1=card-detail-infinite-decks&trafcat=tcgplayer.com%2Fsearch%2Fdecks&u=https%3A%2F%2Fwww.tcgplayer.com%2Fsearch%2Fdecks%3FproductLineName%3Dmagic%26q%3DAnticausal%2BVestige"
    },
    {
      "title": "Search MTGTop8 for this card",
      "url": "https://mtgtop8.com/search?MD_check=1&SB_check=1&cards=Anticausal+Vestige"
    },
    {
      "title": "Moxfield decks with this card",
      "url": "https://www.moxfield.com/decks/public?filter=Anticausal+Vestige&utm_medium=card-detail-moxfield&utm_source=scryfall"
    },
    {
      "title": "Card analysis on EDHREC",
      "url": "https://edhrec.com/route/?cc=Anticausal+Vestige"
    },
    {
      "title": "Card stats on Cube Cobra",
      "url": "https://cubecobra.com/tool/card/35372b69-6086-44e0-9f7c-681e362e5142"
    },
    {
      "title": "Open on Scryfall Tagger",
      "url": "https://tagger.scryfall.com/card/eoe/1"
    }
  ],
  "toolbox": [
    {
      "title": "Buy on TCGplayer$1.67",
      "url": "https://partner.tcgplayer.com/c/4931599/1830156/21018?subId1=card-detail-buy-nonfoil&u=https%3A%2F%2Fwww.tcgplayer.com%2Fproduct%2F641717%3Fpage%3D1"
    },
    {
      "title": "Buy foil on TCGplayer✶ $2.11",
      "url": "https://partner.tcgplayer.com/c/4931599/1830156/21018?subId1=card-detail-buy-foil&u=https%3A%2F%2Fwww.tcgplayer.com%2Fproduct%2F641717%3FPrinting%3DFoil%26page%3D1"
    },
    {
      "title": "Buy on Cardmarket€1.52",
      "url": "https://www.cardmarket.com/en/Magic/Products?idProduct=833832&referrer=scryfall&utm_campaign=card_prices&utm_medium=text&utm_source=scryfall"
    },
    {
      "title": "Buy foil on Cardmarket✶ €1.98",
      "url": "https://www.cardmarket.com/en/Magic/Products?idProduct=833832&isFoil=Y&referrer=scryfall&utm_campaign=card_prices&utm_medium=text&utm_source=scryfall"
    },
    {
      "title": "Buy on Cardhoarder0.07",
      "url": "https://www.cardhoarder.com/cards/142615?affiliate_id=scryfall&ref=card-profile&utm_campaign=affiliate&utm_medium=card&utm_source=scryfall"
    }
  ],
  "image_and_data": [
    {
      "title": "Download PNG image (0.96 MB)",
      "url": "https://cards.scryfall.io/png/front/3/5/35372b69-6086-44e0-9f7c-681e362e5142.png?1752946556"
    },
    {
      "title": "Download art crop (46.29 KB)",
      "url": "https://cards.scryfall.io/art_crop/front/3/5/35372b69-6086-44e0-9f7c-681e362e5142.jpg?1752946556"
    },
    {
      "title": "Copy-pasteable text",
      "url": "https://api.scryfall.com/cards/35372b69-6086-44e0-9f7c-681e362e5142?format=text&pretty=true"
    },
    {
      "title": "Copy-pasteable JSON",
      "url": "https://api.scryfall.com/cards/35372b69-6086-44e0-9f7c-681e362e5142?format=json&pretty=true"
    },
    {
      "title": "Report card issue",
      "url": "https://scryfall.com/contact?problem_uri=https%3A%2F%2Fscryfall.com%2Fcard%2Feoe%2F1%2Fanticausal-vestige&title=Problem+with+Anticausal+Vestige+%28Edge+of+Eternities+%231%29&topic=card"
    }
  ],
  "from_url": "https://scryfall.com/search?order=set&q=e%3Aeoe+cn%E2%89%A51+cn%E2%89%A4276&unique=prints"
}, // ... Many other card details
]

Field Purposes and Applications:

• URL: Direct link to individual card page for reference and verification
• Name: Official card name for database indexing and search functionality
• Description: Complete Oracle text including abilities and flavor text for game rules and content analysis
• SKU: Unique identifier combining set code and collector number for inventory management
• Brand: Consistent "Magic: The Gathering" designation for product categorization
• Image: Primary card image URL for visual display and recognition systems
• Buy URLs: Marketplace links for price tracking and purchase flow integration
• Toolbox: Combined metadata including card type, mechanics, colors, and special properties for filtering and categorization
• Image and Data: Multi-resolution image collection supporting various display requirements from thumbnails to high-quality prints

Usage Guide

Implementation Workflow

Option A - URL-Based Workflow:

1. Construct Search Queries: Use Scryfall's advanced search syntax to create precise queries on the website. Scryfall's comprehensive keyword system supports filtering by set, rarity, color, type, and numerous other attributes. Copy the complete URLs from your browser.

2. Configure Processing Limits: Set max_items_per_url based on your data needs. For large datasets, consider multiple smaller queries rather than one massive extraction.

3. Optimize Proxy Settings: Choose proxy locations that align with your target market or analysis focus. US locations typically provide optimal performance for Scryfall.

4. Execute Batch Processing: Process search URLs sequentially to maintain server stability and data accuracy.

5. Validate and Clean Data: Review extracted data for completeness, particularly for special card types like double-faced cards or promotional releases.

Option B - Filter-Based Workflow:

1. Define Search Parameters: Configure your search using the keyword parameter to find specific cards or card types (e.g., "planeswalker", "artifact", card names).

2. Set Sorting and Filtering:

   • Use sort_by to organize results by price, rarity, release date, or other criteria
   • Set order to "asc" or "desc" based on your needs
   • Configure unique to control whether you want unique cards, all prints, or unique artwork

3. Configure Processing Limits: Set max_items_per_url and starting page number based on your data needs.

4. Optimize Proxy Settings: Choose US proxy locations for optimal performance with Scryfall.

5. Execute and Monitor: Run the scraper and monitor progress. The system will automatically build search URLs based on your filters.

6. Validate and Clean Data: Review extracted data for completeness, particularly for special card types.

Best Practices

Choose the Right Method:

• Use URL-based scraping for complex queries with advanced Scryfall syntax (e.g., set:khm cn≥1 cn≤286)
• Use filter-based scraping for simpler keyword searches with standard sorting and filtering

Query Optimization:

• For URL-based: Use specific search parameters to reduce unnecessary data processing. For example, add unique=prints to avoid duplicate printings if analyzing card mechanics rather than market data
• For filter-based: Set the unique parameter appropriately to control result granularity

Rate Limiting: Respect Scryfall's servers by spacing requests appropriately. The retry mechanism helps handle temporary failures without overwhelming the system.

Data Validation: Cross-reference critical data points, especially for newly released sets or promotional cards that may have limited marketplace integration.

Error Handling:

• For URL-based: Monitor for query syntax errors in URLs, as malformed search parameters can result in empty result sets or scraping failures
• For filter-based: Ensure keywords are spelled correctly and test with small batches first

Sorting Strategy:

• Sort by usd, eur, or tix when analyzing market prices
• Sort by released or set for chronological analysis
• Sort by edhrec for Commander format popularity insights

Enable Failure Handling: Always set ignore_url_failures to true to prevent one failed query from stopping your entire scraping job.

Common Challenges and Solutions

Complex Query URLs: Scryfall search URLs contain encoded parameters that must be preserved exactly. Always test queries in Scryfall's interface before scraping.

Pagination Boundaries: Some searches may return fewer results than expected due to Scryfall's result limits or query-specific filtering.

Image Availability: Different card printings may have varying image quality or availability, particularly for older or promotional releases.

Benefits and Applications

Operational Efficiency

Manual data collection from Scryfall search results is extremely time-consuming and prone to human error. This scraper transforms hours of manual work into minutes of automated processing, enabling rapid dataset creation for time-sensitive analysis or real-time market monitoring.

Real-World Applications

Market Intelligence: Track pricing trends across specific card categories, sets, or rarities by processing targeted search queries and monitoring marketplace links.

Collection Analytics: Build comprehensive databases for collection management, enabling advanced filtering, valuation tracking, and completion analysis.

Competitive Analysis: Extract tournament-legal cards matching specific criteria for meta analysis and deck building optimization.

Academic Research: Generate datasets for studying Magic's design evolution, power level analysis, or market dynamics across different card categories.

Business Value

For MTG retailers, application developers, and content creators, automated search data extraction enables dynamic inventory management, real-time price monitoring, and enhanced user experiences through comprehensive card databases.

Conclusion

The Scryfall Search Scraper bridges the gap between Scryfall's powerful search capabilities and the need for bulk data extraction. By automating the processing of complex search queries and delivering structured card data, this tool empowers users to focus on analysis and application rather than manual data collection.

Whether you're building the next generation of MTG tools, conducting market research, or managing extensive card collections, this scraper provides the foundation for data-driven success in the Magic: The Gathering ecosystem.

Related Actors

• Scryfall.com Card Details Scraper: A specialized data extraction solution engineered to harvest comprehensive card information from Scryfall.com's.

Your feedback

We are always working to improve Actors' performance. So, if you have any technical feedback about Scryfall.com Card Search Scraper or simply found a bug, please create an issue on the Actor's Issues tab in Apify Console.