There's An AI For That Scraper | TAAFT

Slug: fatihtahta/theres-an-ai-for-that-scraper

Overview

This actor is built upon the ValidatedMails.com architecture for contact enrichment workflows.

There's An AI For That Scraper collects structured AI tool records, including names, listing identifiers, summaries, categories, pricing signals, engagement metrics, media links, and optional contact details. It can collect from task pages, search result pages, individual tool pages, or keyword-based searches, then return the results in a clean JSON format. There's An AI For That is a widely used directory of AI products and use cases, which makes it valuable for research, discovery, enrichment, and monitoring. The actor automates repetitive collection work so teams can gather the same fields consistently across runs. That reduces manual effort and makes the output easier to use in spreadsheets, dashboards, and downstream systems.

Why Use This Actor

• Market research and analytics: Map AI categories, compare tool positioning, and track pricing, verification, and engagement signals across segments.
• Product and content teams: Build comparison pages, maintain internal research lists, and identify tools for specific use cases or editorial coverage.
• Developers and data engineers: Feed analytics warehouses, ETL pipelines, internal search tools, and APIs with structured AI tool records.
• Lead generation and enrichment: Identify relevant tools, websites, and optional contact details for outreach, ecosystem mapping, or vendor research.
• Monitoring and competitive tracking: Re-run categories or keyword searches on a schedule to watch for new tools, listing changes, and market movement over time.

Input Parameters

Provide any combination of URLs, queries, and filters to control what the actor collects.

Parameter	Type	Description	Default
startUrls	string[]	One or more There's An AI For That URLs to collect directly. Supported page types: task pages, search result pages, and individual tool pages.	["https://theresanaiforthat.com/task/creativity/", "https://theresanaiforthat.com/s/llm+observability/"]
queries	string[]	One or more keywords to search on There's An AI For That, such as task names, use cases, categories, product types, or specific tool names.	–
getContacts	boolean	Whether to enrich records with available contact details such as emails, phone numbers, and social profile links. Allowed values: true, false.	false
limit	integer	Maximum number of listings to save for each query and each start URL. Minimum value: 10.	50000
proxyConfiguration	object	Optional Apify connection settings for larger runs, recurring jobs, or broader collections.	{"useApifyProxy": true}

Example Input

{
  "startUrls": [
    "https://theresanaiforthat.com/task/creativity/",
    "https://theresanaiforthat.com/s/llm+observability/"
  ],
  "queries": [
    "meeting assistant",
    "seo automation"
  ],
  "getContacts": true,
  "limit": 500,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}

Output

6.1 Output destination

The actor writes results to an Apify dataset as JSON records. And the dataset is designed for direct consumption by analytics tools, ETL pipelines, and downstream APIs without post-processing.

6.2 Record envelope (all items)

Each dataset item is a single tool record. For downstream normalization and deduplication, use this stable envelope:

• type (string, required): Logical record type. Always tool.
• id (string, required): Stable tool identifier at the top level of the record.
• url (string, required): Canonical tool listing URL, available as source_metadata.listing_url.

Recommended idempotency key: type + ":" + id

If the same tool is discovered through multiple URLs, queries, or repeat runs, use this key for deduplication and upserts.

6.3 Examples

Example: tool (type = "tool")

{
  "id": "where-to-eat",
  "name": "Where to Eat?",
  "identity": {
    "slug": "where-to-eat"
  },
  "source_metadata": {
    "listing_url": "https://theresanaiforthat.com/ai/where-to-eat/",
    "source_page_url": "https://theresanaiforthat.com/task/personal/?ref=nav_bar",
    "scraped_time": "2026-03-21T07:05:01Z",
    "page_index": 1,
    "extraction_strategy": "dom",
    "seed": {
      "id": "1dcd70feabb0",
      "type": "url",
      "value": "https://theresanaiforthat.com/task/personal/?ref=nav_bar"
    }
  },
  "descriptions": {
    "summary": "AI-powered restaurant picks so that everyone in the group can agree !",
    "meta_description": "Where to Eat? by Helpful Extensions is an AI-powered tool designed as an aide in finding the perfect restaurant for groups with different dietary needs, budgets, and preferences. This extension aims to ease decision-making when dining out by offering suggestions everyone in the group could agree on...."
  },
  "links": {
    "visit_url": "https://chromewebstore.google.com/detail/where-to-eat-by-helpful-e/nakdpkdhdglelechpiijgjcoedepdmip?pli=1&ref=taaft",
    "task_page_url": "https://theresanaiforthat.com/task/restaurant-recommendations/"
  },
  "classification": {
    "primary_task": "Restaurant recommendations",
    "topics": [
      "Personal",
      "Food",
      "Restaurants",
      "Restaurant recommendations"
    ],
    "is_featured": false,
    "is_verified": true
  },
  "media": {
    "logo_url": "https://media.theresanaiforthat.com/icons/where-to-eat.png?height=207",
    "open_graph_image_url": "https://media.theresanaiforthat.com/where-to-eat.png"
  },
  "pricing": {
    "pricing_summary": "Released 1y ago 100% Free"
  },
  "engagement": {
    "stats_summary": "Restaurant recommendations 382 No ratings",
    "views": "380",
    "engagement": "5"
  },
  "extraction_status": {
    "detail_page_fetch_status": "success",
    "detail_page_status_code": 200,
    "detail_page_final_url": "https://theresanaiforthat.com/ai/where-to-eat/"
  },
  "contacts": {
    "emails": [
      "helpfulextensions@gmail.com"
    ],
    "phone_numbers": [
      "+61 400 688 989"
    ],
    "social_profiles": {
      "github": "https://github.com/helpfulextensions2025/where-to-eat-privacy/blob/main/privacy-policy.md"
    }
  }
}

Field reference

Tool fields (type = "tool")

• id (string, required): Stable tool identifier.
• name (string, required): Tool name as displayed.
• identity.slug (string, optional): Listing slug.
• identity.startup_id (string, optional): Source startup identifier when available.
• identity.page_body_id (string, optional): Page-level identifier when available.
• source_metadata.listing_url (string, required): Canonical tool listing URL.
• source_metadata.source_page_url (string, optional): URL where the tool was discovered.
• source_metadata.scraped_time (string, required): Extraction time in ISO 8601 format.
• source_metadata.page_index (integer, optional): Page index associated with discovery.
• source_metadata.extraction_strategy (string, optional): Source extraction label.
• source_metadata.seed.id (string, required): Seed identifier for the originating input.
• source_metadata.seed.type (string, required): Seed type, such as url or query.
• source_metadata.seed.value (string, required): Original input value used for discovery.
• descriptions.summary (string, optional): Primary short description.
• descriptions.tagline (string, optional): Secondary short tagline.
• descriptions.meta_description (string, optional): Meta description text.
• descriptions.long_description (string, optional): Extended description text.
• descriptions.description_paragraphs (array[string], optional): Description paragraphs when available.
• links.visit_url (string, optional): External destination or website URL.
• links.task_page_url (string, optional): Related task page URL on There's An AI For That.
• links.follow_action_url (string, optional): Additional public action URL when available.
• classification.primary_task (string, optional): Main task or use-case label.
• classification.task_label (string, optional): Secondary task label when provided separately.
• classification.topics (array[string], optional): Associated topic names.
• classification.topic_urls (array[string], optional): Topic page URLs.
• classification.is_featured (boolean, optional): Whether the listing is marked as featured.
• classification.is_verified (boolean, optional): Whether the listing is marked as verified.
• classification.is_boosted (boolean, optional): Whether the listing is marked as boosted.
• media.logo_url (string, optional): Logo or icon image URL.
• media.open_graph_image_url (string, optional): Open Graph image URL.
• media.hero_image_urls (array[string], optional): Additional hero or gallery image URLs.
• pricing.pricing_summary (string, optional): Pricing text as displayed.
• pricing.price (number, optional): Parsed numeric price when available.
• pricing.currency (string, optional): Currency code associated with pricing.price.
• pricing.featured_at (string, optional): Featured date text when available.
• engagement.stats_summary (string, optional): Combined engagement summary text.
• engagement.video_views (string, optional): Video views text when available.
• engagement.views (string, optional): Listing views text when available.
• engagement.engagement (string, optional): Engagement count or summary text.
• creators.profiles (array[object], optional): Creator profile entries when available.
• extraction_status.detail_page_fetch_status (string, optional): Detail page fetch outcome.
• extraction_status.detail_page_status_code (integer, optional): Detail page HTTP status code.
• extraction_status.detail_page_final_url (string, optional): Final resolved detail page URL.
• extraction_status.detail_page_error_class (string, optional): Error class when detail collection fails.
• extraction_status.detail_page_error_message (string, optional): Error message when detail collection fails.
• contacts.emails (array[string], optional): Email addresses found.
• contacts.phone_numbers (array[string], optional): Phone numbers found.
• contacts.social_profiles. (string, optional): Social profile URL keyed by network name.

Data guarantees & handling

• Best-effort extraction: fields may vary by region, session, availability, and UI experiments.
• Optional fields: null-check in downstream code.
• Deduplication: recommend type + ":" + id.

How to Run on Apify

1. Open the Actor in Apify Console.
2. Configure your URLs, search queries, and optional contact enrichment settings.
3. Set the maximum number of outputs to collect.
4. Click Start and wait for the run to finish.
5. Download results in JSON, CSV, Excel, or other supported formats.

Scheduling & Automation

Scheduling

Automated Data Collection

You can schedule recurring runs to keep your dataset fresh without manual work. This is useful for ongoing category tracking, repeated keyword monitoring, and regularly refreshed internal datasets.

• Navigate to Schedules in Apify Console
• Create a new schedule (daily, weekly, or custom cron)
• Configure input parameters
• Enable notifications for run completion
• (Optional) Add webhooks for automated processing

Integration Options

• Webhooks: Trigger downstream actions when a run completes
• Zapier: Connect to 5,000+ apps without coding
• Make (Integromat): Build multi-step automation workflows
• Google Sheets: Export results to a spreadsheet
• Slack/Discord: Receive notifications and summaries
• Email: Send automated reports via email

Performance

Estimated run times:

• Small runs (< 1,000 outputs): ~2-3 minutes
• Medium runs (1,000-5,000 outputs): ~5-15 minutes
• Large runs (5,000+ outputs): ~15-30 minutes

Execution time varies based on filters, result volume, and how much information is returned per record.

Compliance & Ethics

Responsible Data Collection

This actor collects publicly available AI Companies and Tools information from https://theresanaiforthat.com for legitimate business purposes. Common use cases include research, enrichment, monitoring, and internal discovery workflows. Users are responsible for ensuring that their collection, storage, and use of the data complies with applicable laws, regulations, and the target site's terms. This section is informational and not legal advice.

• AI software research and market analysis
• Vendor discovery and enrichment
• Competitive tracking and internal directories

Best Practices

• Use collected data in accordance with applicable laws, regulations, and the target site's terms
• Respect individual privacy and personal information
• Use data responsibly and avoid disruptive or excessive collection
• Do not use this actor for spamming, harassment, or other harmful purposes
• Follow relevant data protection requirements where applicable (e.g., GDPR, CCPA)

Support

For help, use the Issues tab or the actor page in Apify Console. Include the input used with sensitive values redacted, the run ID, the expected versus actual behavior, and, if helpful, a small output sample so the issue can be reproduced more quickly.