Angelcam Car Detector (Apify actor)

Pulls frames from your Angelcam cameras through the read-only Angelcam MCP server and tells you whether a car is parked there and what colour it is. Detection runs locally inside the actor (YOLOv8n via ONNX Runtime on CPU, ~50 ms per frame) – no AI API keys, no per-image cost.

Brand/model is deliberately not reported: open-source make classifiers are unreliable, especially on European fleets. If you need brand later, add a vision-LLM call only when a new car appears (the carPresent flip) – that keeps cost negligible.

One dataset row per camera:

{
  "cameraId": 123456, "cameraName": "Garage", "source": "live",
  "capturedAt": "2026-08-21T09:41:33Z", "frameSize": "2560x1440",
  "carPresent": true, "inRegion": true, "vehicleType": "car",
  "color": "blue", "colorShare": 0.74, "secondaryColor": "white", "nightMode": false,
  "confidence": 0.90, "box": { "x1": 0, "y1": 2, "x2": 121, "y2": 213 },
  "vehiclesInFrame": 3, "allVehicles": [ ... ],
  "inferenceMs": 63,
  "frameUrl": "https://api.apify.com/v2/key-value-stores/.../records/frame-123456-..."
}

frameUrl points to the analysed frame with boxes drawn: green = your region, red = the vehicle chosen for the row, yellow = other vehicles. Each box is labelled with its colour, type and confidence (e.g. red car 87%), and every entry in allVehicles carries its own color / colorShare, not just the chosen one.

How it works

1. Auth – the MCP server is OAuth-protected (PKCE + rotating refresh tokens). The actor exchanges a refresh token for an access token on every run and stores the rotated refresh token in the named key-value store angelcam-auth, so you only seed it once.
2. Cameras – from cameraIds, or from cameraQuery via the MCP find_camera tool, or all online cameras.
3. Frame – source: "live" grabs one JPEG from the MJPEG live stream (get_live_stream_url) and closes the connection; source: "snapshot" uses the periodic snapshot (get_camera_snapshot), which can be 30+ minutes old (snapshotAgeMinutes).
4. Detection – YOLOv8n (COCO) finds car / truck / bus boxes. With a region, the vehicle whose box overlaps the spot by ≥ minRegionOverlap is chosen (inRegion); otherwise the largest vehicle in frame.
5. Tiling (only when needed) – a 2560×1440 frame squeezed into the detector's 640×640 input shrinks distant cars to a few pixels, so the actor can slice the region into overlapping tileSize tiles and run the detector on each. It costs ~1 inference per tile (≈1.3 s for a 1100×500 region at 320 px), and a car larger than one tile can be fragmented, so the tiled pass always merges in a whole-frame pass and tiledDetection: "auto" (default) runs the cheap whole-frame pass first, escalating to tiles only when that finds nothing in the region. Typical cost: 50 ms when a car is there, ~1.3 s when the spot looks empty.
6. Colour – HSV vote over the middle band of the vehicle box (skips windows/roof and road/shadow). Names: white, black, silver, gray, red, orange, yellow, green, blue, purple, brown, other. Monochrome (IR/night) frames report unknown unless clearly white/black and set nightMode: true.

One-time setup

npm install
npm run login          # opens browser → Angelcam login → prints clientId + refresh_token

Put the printed clientId / refresh_token into the actor input (angelcamClientId, angelcamRefreshToken) or env vars ANGELCAM_CLIENT_ID / ANGELCAM_REFRESH_TOKEN (set them as secret env vars on the Apify actor).

Or do it in one go – seeds .env and the Apify CLI secrets angelcamClientId / angelcamRefreshToken (referenced from .actor/actor.json), then redeploy:

node src/auth-login.mjs | npm run -s reseed
apify push

The refresh token rotates on every use. If you run the actor from two places with the same seed token, one may get locked out – run npm run login again for a fresh one.

The chain can also die on its own (invalid_grant from the token endpoint): the MCP server expires or revokes refresh tokens after a while (a token rotated on 3 Sep was rejected on 15 Sep). Re-login + reseed + apify push fixes it; the actor notices the new seed and ignores the stale rotated token stored in angelcam-auth. If it keeps happening, schedule the actor often enough to keep the token fresh.

The authorization server lives on mcp.angelcam.com even when angelcamMcpUrl points at the old angelcam-mcp-server.vercel.app host – the actor discovers the token endpoint from /.well-known/oauth-protected-resource, so either URL works.

Run

# local
mkdir -p storage/key_value_stores/default
echo '{"cameraIds":["123456"],"source":"live","region":{"x":0.3,"y":0.1,"w":0.4,"h":0.35}}' \
  > storage/key_value_stores/default/INPUT.json
set -a; source .env; set +a
APIFY_LOCAL_STORAGE_DIR=./storage npm start

# Apify
npm i -g apify-cli
apify login && apify push

Schedule it on Apify (e.g. every 5 min) to get a time series of presence per camera; flips of carPresent / inRegion are your arrive/leave events.

Input reference

field	default	notes
cameraIds	[]	Angelcam camera IDs (strings or numbers)
cameraQuery	–	e.g. "garage"; used when cameraIds is empty
source	live	live or snapshot
region	–	{x,y,w,h} parking-spot box, fractions 0‑1 or pixels
minRegionOverlap	50	% of vehicle box inside region to count as inRegion
confThreshold	45	YOLO confidence % cut-off
tiledDetection	auto	auto (tile only if the whole-frame pass finds nothing) / always / never
tileSize	320	tile edge in source px; smaller = more zoom on distant cars, slower
saveFrames	true	store annotated JPEG, link in frameUrl
angelcamMcpUrl	https://mcp.angelcam.com/api/mcp
angelcamClientId	env ANGELCAM_CLIENT_ID	from npm run login
angelcamRefreshToken	env ANGELCAM_REFRESH_TOKEN	seed only; rotated token lives in KV store angelcam-auth

Verified on a real camera

Tested on a live outdoor camera overlooking a street-side car park:

#123456 Car Park: car=true inRegion=true red car conf=0.871 (3 vehicles, 49 ms)

Correct on every run — two cars in the foreground bays both boxed, the red one picked because it overlaps the region most, with decisive colour votes (e.g. 0.51 red vs 0.48 silver on the two foreground cars).

Camera aiming dominates everything else. The same camera previously pointed slightly higher, which put the lot behind a concrete parapet with only the top 10-25% of each car (roof slivers) visible. Full-frame detection found 0 of ~6 cars; tiling recovered only 1-3, and swapping in yolov8s / yolo11s changed nothing - the limit was occlusion, not model capacity. After tilting the camera down so whole cars are visible, the plain full-frame pass finds them at 0.87 confidence in 50 ms. If detection is unreliable, re-aim the camera before touching any setting here.

Example input: one camera, one parking spot

{
  "cameraIds": ["123456"],
  "source": "live",
  "region": { "x": 0.20, "y": 0.62, "w": 0.44, "h": 0.36 },
  "confThreshold": 35,
  "minRegionOverlap": 40
}

Swapping the model

Any YOLOv8/YOLO11 ONNX export with the standard images → output0 layout works, float32 or float16 (auto-detected). Point YOLO_MODEL_PATH at it, or replace models/yolov8n.onnx. Pre-exported weights: huggingface.co/unity/inference-engine-yolo (models/yolov8s.onnx, models/yolo11s.onnx).

Limits

• Colour is a heuristic: two-tone cars, heavy reflections, sodium street lights and wet bodywork reduce accuracy; colorShare tells you how decisive the vote was.
• Heavily occluded cars (only roof visible) are missed regardless of model or tiling – see "Verified on a real camera" above.
• vehiclesInFrame counts vehicles in the searched area: the whole frame normally, or the padded region when the run escalated to tiles (tiled: true).
• Live streams are capped at 10 concurrent consumers per camera (Angelcam limit).
• Dockerfile uses node:22-bookworm-slim (glibc) because onnxruntime-node has no Alpine/musl build.