Practitioner reports GPT-6 Astra trained segmentation models without human labels

• A GPT-6 Astra Ultra agent collected public Microduck media, made two mask sets, and trained two RF-DETR-Seg-M models with no operator-supplied labels, according to LearnOpenCV's experiment report; LearnOpenCV's original instruction shows how little task specification it received.
• The paired labelers finished nearly tied on AP50-95: LearnOpenCV's validation scorecard puts Astra-trained masks at 47.35 and SAM3-trained masks at 46.85, while LearnOpenCV's caveat limits the result to one run, 31 correlated validation frames, and two diagnostic photos.
• Frontier vision costs still dominate direct video work: LearnOpenCV's tennis-ball test used 7.87 million tokens for an 11-minute-49-second workflow, most of them cached context across frame inspections.
• Roboflow auto-labeling now uses Astra as its default model, skalskip92's product note said, with a separate estimate of 2 to 6 cents an image in skalskip92's reply.

OpenAI's launch post puts browser and computer use at the center of Astra's pitch. The Microduck comparison retained 80 rendered examples with alpha-channel masks in both training sets, according to LearnOpenCV's dataset note, while the RF-DETR training docs support the COCO and YOLO formats used to turn label output into a deployable model.

LearnOpenCV announced the automated run from a signed-in Roboflow account with 50 credits, asking Astra to collect Microduck assets, generate two label sets, and launch training before the laptop closed.

The two cloud jobs were ready about 93 minutes after the request, LearnOpenCV said in the timeline update. The workflow had four discrete stages:

• LearnOpenCV's asset inventory lists 161 unique public assets, including 124 raster images and 31 videos, deduplicated by file hash.
• The dataset breakdown puts the resulting set at 233 images: 139 frames from videos, 14 press photos, and 80 renders. Blurred frames were excluded and provenance was retained.
• Under the labeling description, SAM3 produced one set of masks from the prompt robot; Astra generated the other through visual inspection and authored contours.
• LearnOpenCV's training configuration used RF-DETR-Seg-M at 432 by 432, for up to 100 epochs with early stopping and no offline augmentation.

The train and validation split followed scenes. The living-room video group, the stand-up and skating clips, and the original four-robot screenshot stayed out of training until both models were frozen.

RF-DETR's segmentation cookbook describes its output as per-object binary masks from a sparse segmentation head. The Microduck run compared models trained on SAM3 masks and Astra-authored masks, rather than comparing either labeler directly against hand-drawn ground truth.

LearnOpenCV's scorecard reports:

• AP50-95: 46.85 for SAM3 labels, 47.35 for Astra labels.
• AP50: 89.56 for SAM3 labels, 88.72 for Astra labels.
• AP75: 43.53 for SAM3 labels, 56.34 for Astra labels.

The setup has several material qualifications. The shared-data note says 80 rendered examples arrived with alpha-channel masks and were used by both conditions, leaving the labeler difference on 120 real training images. SAM3 had five fewer and 62 extra instances across 17 training images, according to LearnOpenCV's count audit, including robots shown on a monitor when the labeling policy targeted physical robots.

An exported COCO area field initially held bounding-box area. LearnOpenCV's evaluation correction says the agent made separate evaluation copies using decoded mask area while preserving the originals. The author also flagged shared Astra-authored references as a possible advantage for Astra's boundary convention.

The reports draw a sharp line between asking a frontier VLM to operate on frames and using it to create training data for a narrower model.

That tennis run used three parallel agents, took 8 minutes 35 seconds for ball annotation and 11 minutes 49 seconds end to end. LearnOpenCV said a fine-tuned RF-DETR or YOLO26 could instead run tennis-ball inference at 30 to 120 frames per second in a follow-up reply.

Direct segmentation carried a similarly visible budget. skalskip92's segmentation measurements reported 2,052 input tokens and 4,685 output tokens per image, a $0.255 average cost, and 78.3 seconds median time per image. For lower-latency mask generation, skalskip92's two-stage proposal assigns text-guided boxes to Astra and masks to SAM.

Roboflow's default-auto-label claim is already accompanied by demos with labels that depend on context, instance identity, or operational meaning rather than a single object class.

• higgsfield_ai reported 13,038 annotations across 81 frames of parcel-sorting warehouse CCTV, including parcel handoffs, workers, and cart traffic.
• A second higgsfield_ai assembly-line report claimed 2,847 components identified across a recreated car assembly process.
• LearnOpenCV's Microduck prompt asked for a different-colored mask for each robot. A later measurement in the single-image run put the result at 1 minute 39 seconds, including 58.6 seconds of image generation, with 187,990 tokens across a main agent and helper.
• skalskip92's basketball demo labeled Celtics and Knicks players and distinguished home from away games.

John O Whitaker described a separate Astra build that generated training data, set up evaluations, trained a small test model, and made a demo, with a first version roughly 20 minutes after the goal was described in Whitaker's report.