# Women in Tech Inspiration Feed (`katerinahronik/women-in-tech-inspiration-feed`) Actor Women in Tech Inspiration Feed: recent posts about women in tech, collected into one stream. Part of PyCon Singapore PyLadies track. - **URL**: https://apify.com/katerinahronik/women-in-tech-inspiration-feed.md - **Developed by:** [Kateřina Hroníková](https://apify.com/katerinahronik) (community) - **Categories:** Automation, News - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Women in Tech Inspiration Feed — Apify Actor A small [Apify Actor](https://apify.com/actors) (in Python) that collects recent posts about **women in tech** into one stream, then lets you export or schedule them. It's the Level-3 finale of the *"Apify Web Scraping & Parsing Workshop"* (PyLadies @ PyCon Singapore). The two scripts one level up (`../01_pyladies_html_scrape.py`, `../02_devto_api.py`) teach the moves; this Actor wraps them into something you can deploy to the cloud and run on a schedule. ← **start with the [demo hub README](../README.md)** if you haven't. --- ### What does it do? You give it a few **tags** and it pulls recent posts from three sources, normalizes them into one shape, drops stale/duplicate ones, and saves the rest to a dataset: - **Dev.to** → official JSON API → *Level 1* (`source: "dev.to"`) - **Mastodon** → public JSON API, no auth → *Level 1* (`source: "mastodon"`) - **Medium** → blocks scrapers, so we read its RSS feed → *Level 3* (`source: "medium"`) The result is a tidy, exportable feed of inspiration you could email yourself every Monday. --- ### Prerequisites - [Apify account](https://console.apify.com) (free tier is plenty) — only needed to deploy - Python 3.10+ (the Docker image uses 3.13) - [Apify CLI](https://docs.apify.com/cli/) for the cloud parts: ```bash npm install -g apify-cli apify login ```` *** ### Input Configure these in the Apify Console UI, or in a local `INPUT.json` (see below). | Field | Type | Default | What it does | |---|---|---|---| | `tags` | string\[] | `["womenintech", "pyladies"]` | Topics to follow. Each becomes a Dev.to tag, a Mastodon hashtag, **and** a Medium RSS feed. | | `sources` | string\[] | `["devto", "mastodon", "medium"]` | Where to collect from: any of `devto`, `mastodon`, `medium`. | | `maxAgeDays` | integer | `30` | Keep only posts newer than this. #womenintech/#pyladies are sparse — bump to 90 if the feed looks empty. | | `limit` | integer | `50` | Max posts to save. | > ⚠️ **Tags are a union (OR), not an intersection.** `["womenintech", "python"]` returns > women-in-tech posts **and** every recent Python post — not their overlap. A busy tag > like `#python` will flood and bury the on-theme posts, so keep this list focused. ### Output One item per post, every source normalized to the same shape: | Field | Example | |---|---| | `name` | author / publication name | | `excerpt` | short summary (HTML stripped) | | `link` | URL to the post | | `tags` | `["womenintech", "career"]` | | `source` | `"dev.to"`, `"mastodon"`, or `"medium"` | | `published_at` | ISO 8601 timestamp | | `cover_image` | URL or `null` | *** ### Run it locally No cloud account needed — the SDK uses a local `./storage` folder. ```bash cd 03_apify_actor pip install -r requirements.txt ## Option A: via the Apify CLI (uses input_schema.json defaults) apify run ## Option B: plain Python (set your own input first — see below) python -m src ``` To customize the input for `python -m src`, create `storage/key_value_stores/default/INPUT.json`: ```json { "tags": ["womenintech", "pyladies"], "sources": ["devto", "mastodon", "medium"], "maxAgeDays": 90 } ``` Results land in `storage/datasets/default/*.json` — one file per post. > **Tip:** the dataset *appends*. Delete `storage/datasets/default/` between runs if you > don't want old items mixed with new ones. *** ### How the code works The whole Actor is in [`src/main.py`](src/main.py), and `main()` reads as a 4-step pipeline: ``` COLLECT → for each tag, fetch each source (Dev.to API + Mastodon API + Medium RSS) into one list FILTER → drop posts older than maxAgeDays DEDUPE → same post can appear under two tags; keep one, sort newest-first OUTPUT → Actor.push_data() writes each item to the dataset ``` The building blocks mirror the teaching scripts: ```python ## Level 1 — the easy win: data already arrives as clean JSON resp = await client.get(DEVTO_API, params={"tag": tag, "per_page": 30}) posts = resp.json() ## Level 1 — Mastodon's public API (the one you can find live in DevTools) resp = await client.get(MASTODON_TAG.format(tag=tag), params={"limit": 40}) posts = resp.json() ## Level 3 — Medium blocks scrapers, so we read its RSS feed instead resp = await client.get(MEDIUM_FEED.format(tag=tag)) # custom User-Agent! feed = feedparser.parse(resp.content) ``` `async with Actor:` is the Apify lifecycle — it reads the run's input, wires up logging, and finalizes the run on exit. Everything else is plain Python you already understand. *** ### Deploy to Apify ```bash apify push ``` Your Actor is now live at [console.apify.com](https://console.apify.com/actors) under **My Actors**. Run it from the UI, tweak the input, and watch the dataset fill up. ### Schedule & export **Run on a schedule** — e.g. every Monday at 8:00 for a weekly digest: 1. Open your Actor in the Apify Console 2. **Schedules** → **+ New schedule** 3. Cron: `0 8 * * 1` (Mondays at 08:00) **Export the results:** - Dataset → **Export** → CSV / JSON / Excel - Or pipe it straight to **Gmail/Slack/Google Sheets** via [Apify integrations](https://docs.apify.com/platform/integrations) *** ### Troubleshooting | Symptom | Cause / fix | |---|---| | Import error `cannot specify both default and default_factory` | An old `apify<3` resolved. Pin **`apify~=3.4`** (already in `requirements.txt`). | | Medium returns nothing | Medium blocks `feedparser`'s default User-Agent. We fetch the RSS with `httpx` (custom UA) first, then parse the bytes — don't hand a URL straight to `feedparser`. | | Feed is empty / too small | `#womenintech` and `#pyladies` are sparse (often 0 posts in a 7-day window). Raise `maxAgeDays` (try 90) or add `devto`+`medium` both. | | Feed full of off-topic posts | A broad tag like `#python` is in `tags` — remember tags are OR'd. Trim to on-theme tags. | *** ### Where to next - **Try the exercises:** [`../EXERCISES.md`](../EXERCISES.md) — easy → hard, including "add a `keyword` input" and "add a third source." - [Apify Academy — Web scraping for Python devs](https://docs.apify.com/academy) - [Apify Python SDK docs](https://docs.apify.com/sdk/python/) - [Browse thousands of ready-made Actors](https://apify.com/store) # Actor input Schema ## `tags` (type: `array`): Topics to follow. Each becomes a Dev.to tag (and a Medium RSS feed). ## `sources` (type: `array`): Where to collect posts from. ## `maxAgeDays` (type: `integer`): Only keep posts published within this many days. Tip: #womenintech/#pyladies are sparse — 30 keeps the feed full; set 7 for a strict 'this week'. ## `limit` (type: `integer`): Maximum number of posts to save. ## Actor input object example ```json { "tags": [ "womenintech", "pyladies" ], "sources": [ "devto", "mastodon", "medium" ], "maxAgeDays": 30, "limit": 50 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "tags": [ "womenintech", "pyladies" ] }; // Run the Actor and wait for it to finish const run = await client.actor("katerinahronik/women-in-tech-inspiration-feed").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "tags": [ "womenintech", "pyladies", ] } # Run the Actor and wait for it to finish run = client.actor("katerinahronik/women-in-tech-inspiration-feed").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "tags": [ "womenintech", "pyladies" ] }' | apify call katerinahronik/women-in-tech-inspiration-feed --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=katerinahronik/women-in-tech-inspiration-feed", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Women in Tech Inspiration Feed", "description": "Women in Tech Inspiration Feed: recent posts about women in tech, collected into one stream. Part of PyCon Singapore PyLadies track.", "version": "0.0", "x-build-id": "IsM9Wjz49EurdtfUA" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/katerinahronik~women-in-tech-inspiration-feed/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-katerinahronik-women-in-tech-inspiration-feed", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/katerinahronik~women-in-tech-inspiration-feed/runs": { "post": { "operationId": "runs-sync-katerinahronik-women-in-tech-inspiration-feed", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/katerinahronik~women-in-tech-inspiration-feed/run-sync": { "post": { "operationId": "run-sync-katerinahronik-women-in-tech-inspiration-feed", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "tags": { "title": "Tags", "type": "array", "description": "Topics to follow. Each becomes a Dev.to tag (and a Medium RSS feed).", "default": [ "womenintech", "pyladies" ], "items": { "type": "string" } }, "sources": { "title": "Sources", "type": "array", "description": "Where to collect posts from.", "items": { "type": "string", "enum": [ "devto", "mastodon", "medium" ], "enumTitles": [ "Dev.to (JSON API)", "Mastodon (JSON API)", "Medium (RSS)" ] }, "default": [ "devto", "mastodon", "medium" ] }, "maxAgeDays": { "title": "Max age (days)", "minimum": 1, "maximum": 365, "type": "integer", "description": "Only keep posts published within this many days. Tip: #womenintech/#pyladies are sparse — 30 keeps the feed full; set 7 for a strict 'this week'.", "default": 30 }, "limit": { "title": "Max items", "minimum": 1, "maximum": 500, "type": "integer", "description": "Maximum number of posts to save.", "default": 50 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```