Object Detection and Image Tagging in bulk (DETR)

List image URLs and get back the objects in each picture: what each one is, how confident the model is, and where it sits as a bounding box. Each image also gets a count per label and a short list of tags.

Detection runs inside the actor with DETR ResNet-50, so no image is sent to an outside API and there's no key to supply. The model was trained on the COCO 2017 object detection dataset and is released under the Apache 2.0 license [1]. Its model card reports an average precision (AP) of 42.0 on COCO 2017 validation [1].

What you get for each image

• objects: one entry per detection with label, score from 0 to 1, box in pixels (xmin, ymin, xmax, ymax) and boxNormalized as fractions of the width and height
• objectCount, labelCounts such as {"person": 4, "sports ball": 1}, and labels, the unique labels with the most frequent first
• width, height and inputFormat of the image as read, after it's turned upright from its EXIF orientation
• processingSeconds for the detection itself
• annotatedImageUrl when you turn on annotated images: a JPEG copy with the boxes and labels drawn on it, stored in the run's key value store

Labels come from the COCO label list the model ships with: person, bicycle, car, motorcycle, airplane, bus, train, truck, boat, traffic light, fire hydrant, street sign, stop sign, parking meter, bench, bird, cat, dog, horse, sheep, cow, elephant, bear, zebra, giraffe, hat, backpack, umbrella, shoe, eye glasses, handbag, tie, suitcase, frisbee, skis, snowboard, sports ball, kite, baseball bat, baseball glove, skateboard, surfboard, tennis racket, bottle, plate, wine glass, cup, fork, knife, spoon, bowl, banana, apple, sandwich, orange, broccoli, carrot, hot dog, pizza, donut, cake, chair, couch, potted plant, bed, mirror, dining table, window, desk, toilet, door, tv, laptop, mouse, remote, keyboard, cell phone, microwave, oven, toaster, sink, refrigerator, blender, book, clock, vase, scissors, teddy bear, hair drier and toothbrush. The model names only objects on that list.

Input

• Image URLs (required). Direct links to JPEG, PNG, WebP, GIF, BMP, TIFF and other formats Pillow reads. For an animated image, the first frame is analyzed. Transparent areas are filled with white.
• Minimum confidence, 0.9 by default. Try 0.5 to 0.7 when small or partly hidden objects matter, and expect more wrong labels at the lower end.
• Only these labels, for example person or car. Leave it empty to get every label.
• Maximum objects per image, keeping the highest scores (0 means no limit).
• Save annotated images, off by default.
• Maximum file size, 25 MB by default.

Output

The dataset has one row per image. A link that fails to download, answers with an error status, or isn't an image gets one row with ok false and the reason in error, and the run moves on to the next image. An OUTPUT record in the key value store gives images processed, images failed and whether the run stopped at your spending limit.

Speed and memory

The default run memory is 2,048 MB. On Apify at that setting, the 640 by 480 sample photo took 1.35 seconds to analyze and the run peaked at 566 MB of memory. Bigger photos cost little extra time, because every image is resized to 800 pixels on its short side (1,333 at most on the long side) before detection: on one CPU thread, a 12 megapixel photo took 2.1 seconds against 1.6 for the 640 by 480 one. Apify gives a run one full CPU core for every 4,096 MB of memory and a share of a core below that [2], so a run with more memory gets through images faster.

Pricing

Pay per event: one event per image processed. Images that fail aren't charged. When the run reaches the maximum charge you set, it stops before the next image, and every result row in the dataset is a paid one.

Support

Questions and problems: the Issues tab on this page. Replies are written by an AI assistant on behalf of the developer and reviewed by the developer.

References

[1] Hugging Face, facebook/detr-resnet-50 model card, sections "Training data" and "Evaluation results", and the license tag. https://huggingface.co/facebook/detr-resnet-50 [2] Apify documentation, Usage and resources, section "CPU". https://docs.apify.com/platform/actors/running/usage-and-resources