# Historical segmentation refiner replay

Experiment date: **3 September 2026**. Public export: **4 September 2026**.

This is a descriptive replay of segmentation refiners on **50 previously reused development images** from COCO: 30 object cases and 20 wall-surface cases. Every version receives the same saved parent mask and native RGB image. It is not an end-to-end rendering benchmark or a measurement of current production performance.

## Files

- `per-case.csv`: 250 rows (50 cases × 5 stages), with four measured scores, public COCO image IDs, category, dimensions and recorded photo license.
- `per-case.json`: the same per-case scores plus reference identifiers, recorded image hashes and verified frozen-response hashes. No images or mask bitmaps are included.
- `aggregate.csv` / `aggregate.json`: unweighted means across all 50 cases for each historical stage. JSON also includes paired comparisons.
- `paired-changes.csv`: direction-aware counts of improved, declined and unchanged cases, plus numeric deltas, for three stage comparisons.
- `specification.json`: metric definitions, boundary tolerance, experiment scope, source and model identities, source hashes, input construction, missing metadata and limitations.
- `checksums.json`: SHA256 digests of the exported files.

## Reading the results

Scores range from 0 to 1. Higher IoU and boundary F1 are better; lower leakage and missed rate are better. Multiply a score difference by 100 to express **percentage points**, not relative percentage improvement. The key `current` names the historical Test refiner at commit `5ceddcf` on the experiment date; it does not claim to describe today's product.

Saved raw parent → historical Test refiner:

| Metric | Raw mean | Test refiner mean | Delta, percentage points | Improved / declined / tied cases |
|---|---:|---:|---:|---:|
| IoU ↑ | 0.754154 | 0.758022 | +0.386833 | 35 / 15 / 0 |
| Boundary F1 ↑ | 0.491288 | 0.514673 | +2.338468 | 37 / 13 / 0 |
| Leakage ↓ | 0.108287 | 0.110304 | +0.201741 (worse) | 20 / 30 / 0 |
| Missed rate ↓ | 0.208901 | 0.204692 | −0.420895 | 42 / 8 / 0 |

The opt-in line-recovery candidate changed **0 of 50 cases** against that Test refiner. Neither passed the registered boundary-F1 and missed-rate promotion bars. Show the leakage worsening and zero-change result whenever presenting the improvement metrics.

## Verification and scope

The export was independently recomputed from the recorded per-case rows: all 20 stage means agree with the saved aggregate within 5.56e-17; all 12 paired comparisons were recomputed using a 1e-12 tie tolerance. All 50 original frozen-response file hashes match the replay's recorded hashes. This verifies the score aggregation and response integrity; it does not rerun inference or remeasure masks from pixels.

Original train/holdout labels are retained only for provenance. All 50 images had informed development, so these are not fresh holdout results. Localization uses reference-derived boxes with deterministic padding, excluding automatic localization from the measured task. The records lack complete checkpoint, hardware and upstream inference metadata. No speed, cost, competitor, full-rendering quality, statistical-significance or current-production claim is supported.

Only derived scores, identifiers, hashes and source metadata are distributed. COCO and COCO-Stuff remain the sources for original data; recorded photo licenses do not constitute complete attribution for a future image publication. No customer images, signed URLs or private account details are included.

Public references: [COCO](https://cocodataset.org/), [COCO-Stuff](https://github.com/nightrome/cocostuff), [COCO-Stuff labels](https://github.com/nightrome/cocostuff/blob/master/labels.txt), [SAM 2](https://github.com/facebookresearch/sam2).