G2.com Category Listings (MongoDB)

Returns stored G2.com software listings for one or more category URLs.

Input and output match coder_luffy/g2-scraper field for field, so this is a drop-in replacement for it — except that records are read from MongoDB instead of scraped from G2.com.

Input

Field	Type	Default	Description
urls	array of strings	required	One or more G2.com category page URLs. A bare slug such as crm also works.
maxItems	integer	25	Maximum listings to return across all URLs. Max 50000.
requestTimeoutSecs	integer	30	Per-query timeout in seconds, 5 to 120.
enrichFromProductPage	boolean	false	Accepted for input compatibility, but ignored — see below.

{
  "urls": [
    "https://www.g2.com/categories/aca-compliance",
    "https://www.g2.com/categories/it-financial-management-itfm"
  ],
  "maxItems": 25
}

maxItems is a total across all URLs, not a per-URL cap. Categories are read in the order given until the total is reached; the rest are skipped. Duplicate URLs pointing at the same category are read once.

All of these resolve to the same category:

https://www.g2.com/categories/data-labeling/
https://www.g2.com/categories/data-labeling
http://g2.com/categories/data-labeling?page=3
www.g2.com/categories/data-labeling
data-labeling

An entry that resolves to neither a URL nor a slug is skipped with a warning; the run continues with the rest. If nothing resolves, the run fails.

enrichFromProductPage exists so that input copied from coder_luffy/g2-scraper is accepted verbatim. There are no product pages to fetch when reading from MongoDB, so setting it to true logs a warning and changes nothing.

Output

One dataset item per matching document in g2list.catalog_listings, carrying the same 25 fields in the same order as coder_luffy/g2-scraper:

{
  "productId": 103210,
  "productUuid": null,
  "productName": "NavigateHCR",
  "productSlug": "navigatehcr",
  "productUrl": "https://www.g2.com/products/navigatehcr/reviews",
  "logoUrl": "https://images.g2crowd.com/uploads/product/image/large_detail/...",
  "vendor": "NavigateHCR",
  "vendorUrl": "https://www.g2.com/sellers/navigatehcr",
  "ratingAvg": 5,
  "numReviews": 1,
  "entryLevelPrice": null,
  "productDescription": null,
  "prosHighlights": [],
  "consHighlights": [],
  "userSentiment": null,
  "typicalUsers": null,
  "industries": null,
  "marketSegment": null,
  "consultingServicesUrl": null,
  "category": "ACA Compliance",
  "categoryId": null,
  "pageNumber": 1,
  "sourceUrl": "https://www.g2.com/categories/aca-compliance",
  "scrapedAt": "2026-08-12T19:40:38.144Z",
  "error": null
}

Records come back in pageNumber then _id order, which is stable across runs.

An unknown category is not an error — it contributes no items and logs a warning. A run whose categories are all unknown succeeds with an empty dataset.

How fields are filled

A document's own value always wins. When a field is absent, it is derived if possible, and otherwise null (or [] for the two list fields). So as richer records land in MongoDB, they pass straight through and the nulls fill themselves in — no code change needed.

Output field	Source
productId productName productUrl logoUrl vendor vendorUrl ratingAvg numReviews prosHighlights consHighlights userSentiment pageNumber	Stored directly
productSlug	Stored, else parsed from productUrl
category	Stored, else categoryName
sourceUrl	Stored, else built from the category slug
scrapedAt	Stored, else ingestedAt, as an ISO string
consultingServicesUrl	Stored, else /products/<slug>/imp built from productUrl
productUuid entryLevelPrice productDescription typicalUsers industries marketSegment categoryId error	Stored, else null

Against catalog_listings as it stands, 12 of the 25 columns are filled on every record, consultingServicesUrl productSlug category sourceUrl scrapedAt are derived, and the remaining 8 are null because that data is not in the collection.

Fields present in catalog_listings but not in the reference format — _id, isAiVerified, solutionType, prosDetail, consDetail — are dropped, so the output stays field-for-field identical to the reference actor.

Environment variables

The connection string is read from the environment rather than from the actor input, so the password never lands in the run's input record or the Console UI.

Variable	Required	Default	Description
MONGODB_URL	yes	—	MongoDB connection string
MONGODB_DB	no	g2list	Database name
MONGODB_COLLECTION	no	catalog_listings	Collection holding the listings

On the Apify platform, add MONGODB_URL under Actor → Settings → Environment variables and tick Secret. Locally it is read from .env in the project root:

MONGODB_URL=mongodb+srv://user:password@host/?appName=g2list

Running locally

npm install
npm test
echo '{ "urls": ["aca-compliance"], "maxItems": 25 }' > storage/key_value_stores/default/INPUT.json
npm start

Results are written to storage/datasets/default/.

npm test covers the field mapping — that the output carries the reference actor's 25 fields in its order, that sparse documents are derived correctly, and that full scraper documents pass through untouched.

Deploying

npm install -g apify-cli
apify login
apify push

.env is excluded by .dockerignore, so set MONGODB_URL on the actor before the first run.

Scaling note

catalog_listings holds ~242,000 listings across ~1,737 categories. It is indexed on categorySlug, which is the field this actor queries.

Measured on the largest category, emerging-ai-software (6,000 docs): the query examines exactly the 6,000 matching documents via the index and returns in ~13 ms, with the sort done in memory well inside MongoDB's 32 MB limit. Results stream through a cursor and are pushed in batches of 500, so memory stays flat regardless of category size.

If categories grow much beyond this, a compound index turns the sort into an index scan:

db.catalog_listings.createIndex({ categorySlug: 1, pageNumber: 1, _id: 1 })