Public Competition & Hackathon Finder

An inexpensive Apify Actor for discovering public competitions, hackathons, and developer events. It reads public list pages, filters records by keyword, removes duplicates, and writes structured results to the default dataset.

It intentionally does not log in, bypass anti-bot systems, use residential proxies, or collect private/restricted data. Crawlee's robots.txt support is enabled, blocked URLs are skipped, and local/private-network source URLs are rejected.

What it extracts

Each dataset item contains:

• title, url, deadline
• startDate, endDate
• organizer, prize
• location, online
• summary
• sourceUrl, sourceName, scrapedAt

Missing information is returned as null; the Actor does not invent prizes, deadlines, or organizers.

Supported pages

The extractors run in this order:

1. The current public Major League Hacking season page (built-in adapter).
2. Standard Schema.org Event JSON-LD, which is the most maintainable format.
3. Optional CSS selectors supplied in the input.
4. A conservative generic event-card fallback.

When sourceUrls is empty, the Actor automatically chooses the current MLH season page. It makes one HTTP request per source page and does not follow event links.

Input example

{
  "sourceUrls": [],
  "keywords": ["AI", "open source", "data"],
  "maxItems": 20,
  "includePast": false,
  "requestTimeoutSecs": 30,
  "selectors": {}
}

For a custom public list page without JSON-LD:

{
  "sourceUrls": ["https://example.org/public-competitions"],
  "keywords": ["AI"],
  "maxItems": 50,
  "selectors": {
    "itemSelector": ".competition-card",
    "titleSelector": "h2",
    "urlSelector": "a.details",
    "deadlineSelector": ".deadline",
    "organizerSelector": ".organizer",
    "prizeSelector": ".prize",
    "locationSelector": ".location",
    "summarySelector": "p.summary"
  }
}

Selectors are shared across all URLs in one run. Use separate tasks/runs when different websites need different selector sets.

Run locally

Requirements: Node.js 22+ and npm.

npm install
npm test
npm start

Local input is read from storage/key_value_stores/default/INPUT.json. Results appear under storage/datasets/default/.

To test Pay Per Event logging locally:

$ACTOR_TEST_PAY_PER_EVENT=true ACTOR_USE_CHARGING_LOG_DATASET=true npm start

The local charging log appears under storage/datasets/charging-log/. Local test events use a placeholder price; real prices are configured in Apify Console.

Deploy to Apify

1. Install and authenticate the Apify CLI: npm install -g apify-cli, then apify login.
2. From this project folder, run apify push.
3. In Apify Console, open the Actor, build it, and run it first with a small maxItems value.
4. Inspect the Output table and logs before publishing it to Apify Store.

The .actor directory includes the current input, output, and dataset schemas required for a Store-ready Actor definition.

Pay Per Event setup

The code saves each visible dataset item with the custom event name competition-result:

await Actor.pushData(event, 'competition-result');

In the Actor's Monetization / Pricing setup in Apify Console:

1. Choose Pay per event.
2. Keep the recommended synthetic apify-actor-start event. Apify currently defaults it to $0.00005 per start and covers the first five seconds of compute.
3. Add a custom event named exactly competition-result with a title such as “Competition result”.
4. Start testing around $0.001–$0.003 per result, then adjust only after measuring real run costs and comparable Store Actors. This range is a starting hypothesis, not a revenue guarantee.
5. Set a small minimum maximum-charge amount that covers one start and at least one result.
6. For clearer buyer pricing and agentic-payment eligibility, leave Pay per event + usage off after your cost tests are complete.

Do not also enable the synthetic apify-default-dataset-item event while the custom competition-result event is active, or one dataset item may be charged twice.

The SDK returns a ChargeResult; the Actor stops saving results when the user's run spending limit prevents another charge.

Publishing checklist

• Replace example screenshots and add a clear Store description.
• Test with 5, 20, and 100 results and record compute cost per result.
• Confirm each default/custom source permits the intended automated access and attribution.
• Keep memory at 256 MB unless real runs show it is insufficient.
• State that source pages can change and that missing fields remain null.
• Complete Apify identity and payout requirements before expecting paid Store distribution.

中文快速开始

这个 Actor 默认抓取当前 MLH 赛季的公开黑客松列表；sourceUrls 留空即可。先修改 storage/key_value_stores/default/INPUT.json，然后运行 npm install && npm test && npm start。上传时执行 apify login 和 apify push。

计费请在 Apify Console 里选 Pay per event，并新建名称完全一致的 competition-result 事件。建议先小规模跑几次测真实成本，再定价；不要同时开启 apify-default-dataset-item，否则可能重复计费。

License

MIT