Welcome to GraphQL Extractor

🕸️ About GRAPHQL

GraphQL is a data query and manipulation language for APIs that allows a client to specify what data it needs ("declarative data fetching"). A GraphQL server can fetch data from separate sources for a single client query and present the results in a unified graph.[2] It is not tied to any specific database or storage engine.

🕸️ About This Actor

Trying to be Universal GraphQL Scraper Actor

Web scraping is so much easier now, thanks to GraphQL. It’s like you ask websites for data, and they just serve it up on a silver platter, no fuss. Honestly, it's made scraping feel less like a chore and more like a friendly chat with the internet. Now, here's hoping Amazon.com jumps on the GraphQL bandwagon soon—fingers crossed!

🕸️ Examples (Use Cases)

Example #1: https://apify.hashnode.dev

Scraping blogs from https://apify.hashnode.dev

URL :

https://apify.hashnode.dev/api/graphql

Command :

query ($host: String!, $after: String, $first: Int!, $filter: PublicationPostConnectionFilter) {
  publication(host: $host) {
    id
    posts(after: $after, first: $first, filter: $filter) {
      edges {
        node {
          __typename id title subtitle slug publishedAt  url brief
        }
        __typename
      }
      pageInfo { hasNextPage endCursor __typename }
      totalDocuments
      __typename
    }
    __typename
  }
}

Variables :

{
    "host": "apify.hashnode.dev"
}

ACTOR INPUT :

{
    "limit": 200,

    "url": "https://apify.hashnode.dev/api/graphql",
    "variables": "{ \"host\": \"apify.hashnode.dev\", \"first\": 10 }",
    
    "query": "query ($host: String!, $after: String, $first: Int!, $filter: PublicationPostConnectionFilter) {\r\n publication(host: $host) {\r\n id\r\n posts(after: $after, first: $first, filter: $filter) {\r\n edges {\r\n node {\r\n __typename id title subtitle slug publishedAt url brief\r\n }\r\n __typename\r\n }\r\n pageInfo { hasNextPage endCursor __typename }\r\n totalDocuments\r\n __typename\r\n }\r\n __typename\r\n }\r\n}",

    "cursor.step"	: 25,
    "cursor.next"	: "after",
    "cursor.limit"	: "first",

    "parse.root"	: "publication.posts",
    "parse.list"	: "edges",
    "parse.item"	: "node",
    "parse.total"	: "totalDocuments",
    "parse.next"	: "pageInfo.endCursor"

}

🕸️ GraphQL Quick Guide

Lesson #1: BASIC

SYNTAX: 
	{ commandName(parameter: ARGUMENT) { FIELD_LIST } }

Example :

{ 
	search(name:"pants" count:10) { id name __typename } 
}

JSON Input :

{
	"query": "{ search(name:\"pants\" count:10) { id name __typename } }"
}

Lesson #2: VARIABLES

SYNTAX: 
	query ($VARIABLE: TYPE) { commandName(parameter: $VARIABLE) { FIELD_LIST } }

Example :

query ($text:String $limit:Int) { 
	search(name:$text count:$limit) { id name __typename } 
}

JSON Input :

{
	"query": "query ($text:String $limit:Int){ search(name:$text count:$limit){id name __typename} }",
	"variables": { "text": "pants", "limit": 10 }
}

Lesson #3: FRAGMENT

SYNTAX:
	query ($VARIABLE: TYPE) { commandName(parameter: $VARIABLE) { ... FRAGMENT_NAME } }
	fragment FRAGMENT_NAME on TYPE { FIELD_LIST }

Example: Without Fragment

query ($text:String $limit:Int) {
	search(name:$text count:$limit) { 
		id name sku description url specifications __typename 
		variations { id name sku description url __typename  }
		similarProducts { id name sku description url __typename }
	} 
}

Example: With Fragment

query ($text:String $limit:Int) { 
	search(name:$text count:$limit) { 
		... ProductInfo 
		specifications 
		variations { ... ProductInfo  }
		similarProducts { ... ProductInfo }
	} 
}

fragment ProductInfo on Product { id name sku description url __typename }