Pricing

from $0.50 / 1,000 processed job records

Try for free

Go to Apify Store

Job Feed Deduplicator

Try for free

Deduplicate job feeds from multiple sources without collapsing conflicting requisitions. Strips tracking links and returns stable canonical jobs with source provenance and confidence.

Pricing

from $0.50 / 1,000 processed job records

Rating

0.0

(0)

Developer

Mehdi Badawi

Actor stats

Bookmarked

Total users

Monthly active users

6 days ago

Last modified

Categories

Jobs

Developer tools

Automation

You can access the Job Feed Deduplicator programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

{
  "openapi": "3.0.1",
  "info": {
    "version": "0.1",
    "x-build-id": "8UUjU3Pk3xFfSWv3m"
  },
  "servers": [
    {
      "url": "https://api.apify.com/v2"
    }
  ],
  "paths": {
    "/acts/mehdi_badawi~job-posting-deduplication/run-sync-get-dataset-items": {
      "post": {
        "operationId": "run-sync-get-dataset-items-mehdi_badawi-job-posting-deduplication",
        "x-openai-isConsequential": false,
        "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
        "tags": [
          "Run Actor"
        ],
        "requestBody": {
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/inputSchema"
              }
            }
          }
        },
        "parameters": [
          {
            "name": "token",
            "in": "query",
            "required": true,
            "schema": {
              "type": "string"
            },
            "description": "Enter your Apify token here"
          }
        ],
        "responses": {
          "200": {
            "description": "OK"
          }
        }
      }
    },
    "/acts/mehdi_badawi~job-posting-deduplication/runs": {
      "post": {
        "operationId": "runs-sync-mehdi_badawi-job-posting-deduplication",
        "x-openai-isConsequential": false,
        "summary": "Executes an Actor and returns information about the initiated run in response.",
        "tags": [
          "Run Actor"
        ],
        "requestBody": {
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/inputSchema"
              }
            }
          }
        },
        "parameters": [
          {
            "name": "token",
            "in": "query",
            "required": true,
            "schema": {
              "type": "string"
            },
            "description": "Enter your Apify token here"
          }
        ],
        "responses": {
          "200": {
            "description": "OK",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/runsResponseSchema"
                }
              }
            }
          }
        }
      }
    },
    "/acts/mehdi_badawi~job-posting-deduplication/run-sync": {
      "post": {
        "operationId": "run-sync-mehdi_badawi-job-posting-deduplication",
        "x-openai-isConsequential": false,
        "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
        "tags": [
          "Run Actor"
        ],
        "requestBody": {
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/inputSchema"
              }
            }
          }
        },
        "parameters": [
          {
            "name": "token",
            "in": "query",
            "required": true,
            "schema": {
              "type": "string"
            },
            "description": "Enter your Apify token here"
          }
        ],
        "responses": {
          "200": {
            "description": "OK"
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "inputSchema": {
        "type": "object",
        "properties": {
          "jobs": {
            "title": "Job records",
            "maxItems": 5000,
            "type": "array",
            "description": "Array of raw job posting objects from job boards, scrapers, or ATS feeds.",
            "items": {
              "type": "object",
              "properties": {
                "id": {
                  "type": "string",
                  "title": "Id",
                  "description": "Id value."
                },
                "title": {
                  "type": "string",
                  "title": "Title",
                  "description": "Title value."
                },
                "company": {
                  "type": "string",
                  "title": "Company",
                  "description": "Company value."
                },
                "location": {
                  "type": "string",
                  "title": "Location",
                  "description": "Location value."
                },
                "isRemote": {
                  "type": "boolean",
                  "title": "Is Remote",
                  "description": "Is Remote value."
                },
                "applyUrl": {
                  "type": "string",
                  "title": "Apply Url",
                  "description": "Apply Url value."
                },
                "description": {
                  "type": "string",
                  "title": "Description",
                  "description": "Description value."
                },
                "requisitionId": {
                  "type": "string",
                  "title": "Requisition Id",
                  "description": "Requisition Id value."
                },
                "source": {
                  "type": "string",
                  "title": "Source",
                  "description": "Source value."
                },
                "postedAt": {
                  "type": "string",
                  "title": "Posted At",
                  "description": "Posted At value."
                },
                "salary": {
                  "type": "string",
                  "title": "Salary",
                  "description": "Salary value."
                }
              }
            },
            "default": []
          },
          "fixturePath": {
            "title": "Fixture file path",
            "type": "string",
            "description": "Optional local file path to load JSON job fixtures from (for testing and offline execution)."
          }
        }
      },
      "runsResponseSchema": {
        "type": "object",
        "properties": {
          "data": {
            "type": "object",
            "properties": {
              "id": {
                "type": "string"
              },
              "actId": {
                "type": "string"
              },
              "userId": {
                "type": "string"
              },
              "startedAt": {
                "type": "string",
                "format": "date-time",
                "example": "2025-01-08T00:00:00.000Z"
              },
              "finishedAt": {
                "type": "string",
                "format": "date-time",
                "example": "2025-01-08T00:00:00.000Z"
              },
              "status": {
                "type": "string",
                "example": "READY"
              },
              "meta": {
                "type": "object",
                "properties": {
                  "origin": {
                    "type": "string",
                    "example": "API"
                  },
                  "userAgent": {
                    "type": "string"
                  }
                }
              },
              "stats": {
                "type": "object",
                "properties": {
                  "inputBodyLen": {
                    "type": "integer",
                    "example": 2000
                  },
                  "rebootCount": {
                    "type": "integer",
                    "example": 0
                  },
                  "restartCount": {
                    "type": "integer",
                    "example": 0
                  },
                  "resurrectCount": {
                    "type": "integer",
                    "example": 0
                  },
                  "computeUnits": {
                    "type": "integer",
                    "example": 0
                  }
                }
              },
              "options": {
                "type": "object",
                "properties": {
                  "build": {
                    "type": "string",
                    "example": "latest"
                  },
                  "timeoutSecs": {
                    "type": "integer",
                    "example": 300
                  },
                  "memoryMbytes": {
                    "type": "integer",
                    "example": 1024
                  },
                  "diskMbytes": {
                    "type": "integer",
                    "example": 2048
                  }
                }
              },
              "buildId": {
                "type": "string"
              },
              "defaultKeyValueStoreId": {
                "type": "string"
              },
              "defaultDatasetId": {
                "type": "string"
              },
              "defaultRequestQueueId": {
                "type": "string"
              },
              "buildNumber": {
                "type": "string",
                "example": "1.0.0"
              },
              "containerUrl": {
                "type": "string"
              },
              "usage": {
                "type": "object",
                "properties": {
                  "ACTOR_COMPUTE_UNITS": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATASET_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATASET_WRITES": {
                    "type": "integer",
                    "example": 0
                  },
                  "KEY_VALUE_STORE_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "KEY_VALUE_STORE_WRITES": {
                    "type": "integer",
                    "example": 1
                  },
                  "KEY_VALUE_STORE_LISTS": {
                    "type": "integer",
                    "example": 0
                  },
                  "REQUEST_QUEUE_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "REQUEST_QUEUE_WRITES": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATA_TRANSFER_INTERNAL_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATA_TRANSFER_EXTERNAL_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_SERPS": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_UNBLOCKER_UNITS": {
                    "type": "integer",
                    "example": 0
                  }
                }
              },
              "usageTotalUsd": {
                "type": "number",
                "example": 0.00005
              },
              "usageUsd": {
                "type": "object",
                "properties": {
                  "ACTOR_COMPUTE_UNITS": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATASET_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATASET_WRITES": {
                    "type": "integer",
                    "example": 0
                  },
                  "KEY_VALUE_STORE_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "KEY_VALUE_STORE_WRITES": {
                    "type": "number",
                    "example": 0.00005
                  },
                  "KEY_VALUE_STORE_LISTS": {
                    "type": "integer",
                    "example": 0
                  },
                  "REQUEST_QUEUE_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "REQUEST_QUEUE_WRITES": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATA_TRANSFER_INTERNAL_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATA_TRANSFER_EXTERNAL_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_SERPS": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_UNBLOCKER_UNITS": {
                    "type": "integer",
                    "example": 0
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

Job Feed Deduplication and Canonicalization Tool OpenAPI definition

OpenAPI is a standard for designing and describing RESTful APIs, allowing developers to define API structure, endpoints, and data formats in a machine-readable way. It simplifies API development, integration, and documentation.

OpenAPI is effective when used with AI agents and GPTs by standardizing how these systems interact with various APIs, for reliable integrations and efficient communication.

By defining machine-readable API specifications, OpenAPI allows AI models like GPTs to understand and use varied data sources, improving accuracy. This accelerates development, reduces errors, and provides context-aware responses, making OpenAPI a core component for AI applications.

You can download the OpenAPI definitions for Job Feed Deduplicator from the options below:

OpenAPI.json

If you’d like to learn more about how OpenAPI powers GPTs, read our blog post.

You can also check out our other API clients:

Job Feed Deduplicator API in Python

Job Feed Deduplicator API in JavaScript

Job Feed Deduplicator API through CLI

Job Feed Deduplicator API

GEO/AEO Brand Visibility Analyzer

mehdi_badawi/geo-aeo-visibility

Measure brand mentions, citations, explicit recommendation order, and run-to-run variance across AI-search observations you supply. Deterministic analysis without extra inference cost.

Mehdi Badawi

Influencer Deliverable Reconciler

mehdi_badawi/influencer-deliverable-reconciler

Turn a campaign manifest and collected post data into one verdict per creator deliverable: posted, late, missing, edited, removed, requirements failed, or unknown. Does not scrape social platforms.

Mehdi Badawi

Jobgether Jobs | Normalized

nomad-agent/normalized-jobgether-jobs-scraper

Unofficial independent Actor for bounded public Jobgether job searches. Returns strict normalized records with source links, descriptions, dates, locations, compensation when published, and application details.

Nomad Dev

California Prop 65 Notice Watchlist Processor

mehdi_badawi/proposition-65-watchlist

Normalize and compare California Prop 65 notice snapshots, then flag new or changed records that match your watchlist. Preserves source links and uncertainty; informational only.

Mehdi Badawi

Marketplace Settlement Ledger Normalizer

mehdi_badawi/settlement-ledger-normalizer

Convert marketplace settlement CSV, TSV, or JSON exports into exact, currency-separated ledger rows with source provenance. Missing amounts stay unresolved instead of being guessed.

Mehdi Badawi

Qatar Living Jobs Scraper

debonaire_sedan/qatar-living-jobs-scraper

Export every job on Qatar Living with company, city, salary, required languages and applicant counts. Flags expired listings the site still displays.

Mohammad Salman Farsi Khan

CSV Cleaner & Deduplicator

rock-ai-tools/csv-cleaner-deduplicator

Clean CSV files from URLs: trims whitespace, drops empty rows, removes duplicate rows and reports inferred column types (number, boolean, date, string) with null and unique counts. No scraping, pure local parsing.

Rock AI Tools

Lever Jobs Scraper

tacps126/lever-jobs-scraper

Scrape all open jobs from any Lever career page — just type the company name. Fast, low-cost and clean: location, remote/hybrid, seniority, salary and skills in one consistent format. Monitor new jobs on a schedule.

Tapaswai Ashok Choudhary

Dataset Deduplicator & Cleaner

starshaped_bullsnake/dataset-deduplicator-cleaner

Remove duplicate rows from an Apify Dataset or inline JSON. Deduplicate by selected fields, trim strings, and write the cleaned rows to a new Dataset.

Starshape Tools

SmartRecruiters Jobs Scraper

tacps126/smartrecruiters-jobs-scraper

Scrape all open jobs from any SmartRecruiters career page — just type the company name. Fast, low-cost and clean: location, remote/hybrid, seniority, salary and skills in one consistent format. Monitor new jobs on a schedule.

Tapaswai Ashok Choudhary