Ox Alpha

What it nailed

• Complete artifact set with real DOCX, PPTX, XLSX, PDF, HTML, markdown, and manifest files.
• Excellent handling of the benchmark's central absurdities: dingoes as nonstandard animals, Alaska/Australia mismatch, tiny market, import-created demand, exotic-animal law, support-language liability, and ethics.
• Strong cross-document source-of-truth discipline around recognized revenue, units, price, launch dates, budget gates, TAM, and import-program posture.
• Research file prioritizes official regulatory sources, covers all required jurisdictions, and correctly labels fictional competitors as scenario-only.
• Dashboard passed rendered desktop and mobile visual review with no reported defects.
• Copy variants are meaningfully differentiated while maintaining legal and ethical guardrails.

Where it slipped

• Research claims are well cited in the artifacts, but the scorer prompt does not include an independent live URL verification pass, so perfect research credit is not warranted.
• Some market-size and adjacent-segment inputs remain low-confidence proxy assumptions, albeit disclosed.
• Minor cross-document date ambiguity appears around standard/general availability versus the Sept. 15 DTC and Oct. 6 retail external dates.
• At least one extracted pricing-workbook analysis formula appears to reference the wrong cells for TCO comparison, suggesting small spreadsheet-polish issues.
• Several external-facing assets appropriately require counsel review before publication, so they are not final public-ready copy without human signoff.

What it nailed

• Complete required artifact set with an openable SQLite database, generator script, reports, frontend data payloads, and screenshots.
• Substantial audit/review schema with source records, conflicts, flagged items, rejected items, OCR evidence, and price history.
• Good reviewer-facing static dashboard that renders cleanly on desktop and mobile with only a minor cosmetic clipping issue.
• Useful normalization scaffolding for statuses, payment methods, dates, price eras, customer name variants, and service aliases.
• Source inventory and sensitive-file treatment are thoughtfully represented, even though some claims are not independently verified.

Where it slipped

• Benchmark-specific cap triggered by three or more primary canary failures: ghost/test quarantine, SVC-007/The Works conflict, and department-code normalization.
• Canonical orders are not actually canonical: 2,876 jobs are loaded versus the obstacle-key order universe of about 1,003, with duplicates flagged but still promoted into the clean orders/revenue layer.
• The service model incorrectly preserves The Works as a separate service bucket instead of resolving it to Full Detail Package/SVC-006 while keeping SVC-007 as Express Wash.
• Customer merge audit is broken or unpopulated: customer_merges has zero rows despite duplicate-merge claims.
• Multimodal handling is weak: OCR evidence is low confidence and misses or misreads important handwritten receipt details.
• Documentation contains material internal contradictions around JSON recovery, merge counts, and test-data quarantine.

What it nailed

• [100-piece-lunar-rover] Complete primary artifact exists and renders as a polished browser-based assembly guide.
• [100-piece-lunar-rover] Structured 100-piece rover concept with IDs, step IDs, visible stud geometry, wheels, cockpit, equipment rack, lights, and sample container.
• [100-piece-lunar-rover] Clean completed-state visual presentation with readable desktop and mobile UI and no operator-major visual defects for this isolated case.
• [100-piece-lunar-rover] Instruction flow is sensibly chunked into 27 steps and 5 chapters.
• [250-piece-rescue-helicopter] Complete required isolated-case artifact is present as artifacts/index.html, with kitSpec.json also provided.
• [250-piece-rescue-helicopter] Exact 250-piece target and 53-step instruction structure are represented in machine-readable data.
• [250-piece-rescue-helicopter] The rendered completed model is recognizable as a rescue helicopter with pad, rotors, cockpit, skids, winch, and rescue detailing.
• [250-piece-rescue-helicopter] The browser guide is polished and largely reviewable, with the expected playback, scrubbing, step navigation, speed, completion, and camera controls.
• [500-piece-cyberpunk-food-stall] Required isolated-case index.html is present, with a separate kitSpec.json also provided.
• [500-piece-cyberpunk-food-stall] Substantial 501-piece structured kit spec with chapters, steps, part IDs, colors, positions, and dimensions.
• [500-piece-cyberpunk-food-stall] Viewer appears source-driven from KITSPEC rather than a purely separate static illustration.
• [500-piece-cyberpunk-food-stall] Completed model is recognizable as a cyberpunk food stall, with neon accents, signage, roof structure, cooking area, street base, and scooter.
• [500-piece-cyberpunk-food-stall] Rich browser UI includes timeline, speed, step navigation, complete-model view, camera/view controls, and orbit interaction.
• [1000-piece-airship-research-station] Delivers the required isolated case artifact, with a rich 974-piece airship research station and separate kitSpec.json.
• [1000-piece-airship-research-station] Strong prompt adherence: recognizable mountaintop research station, docking gantry, airship envelope, gondola, cargo pod, propellers, and final docking scene.
• [1000-piece-airship-research-station] Substantial structured build data with part IDs, dimensions, positions, rotations, chapters, and instruction steps.
• [1000-piece-airship-research-station] Visually impressive completed model that operator review found legible and coherent on desktop.

Where it slipped

• [100-piece-lunar-rover] Authoritative mechanical connectivity validation failed to measure any parts, so physical buildability is not proven.
• [100-piece-lunar-rover] Self-reported 0-collision and 0-floating validation claims are uncorroborated and were not credited as independent evidence.
• [100-piece-lunar-rover] The runtime kit spec was not extractable/serializable by the mechanical validator, reducing confidence in machine-readable source integrity.
• [100-piece-lunar-rover] Step-0 appears visually blank until interaction in one operator review.
• [100-piece-lunar-rover] Full interactive control behavior was not independently verified beyond rendered/deep-linked states.
• [250-piece-rescue-helicopter] Authoritative connectivity measurement shows catastrophic physical buildability failure: 208 of 250 parts floating and 147 separate components.
• [250-piece-rescue-helicopter] The artifact overclaims physical coherence/no floating geometry; those claims are contradicted by measured validation and receive no independent credit.
• [250-piece-rescue-helicopter] High near_miss_count indicates the spatial data is visually approximate rather than accurately engaged to the stud grid.
• [250-piece-rescue-helicopter] Minor rendered UI issues remain, including clipped/faded sidebar labels and cramped/cropped mobile framing.
• [500-piece-cyberpunk-food-stall] Independent mechanical connectivity validation failed to extract any parts, leaving buildability unverified.
• [500-piece-cyberpunk-food-stall] Large debug/status overlay remains visible over completed desktop and mobile renders.
• [500-piece-cyberpunk-food-stall] Step-0/ready state is visually misleading because it shows a dense ghosted/full-scene mass.
• [500-piece-cyberpunk-food-stall] Mobile completed view is crowded by overlay and stacked controls.
• [500-piece-cyberpunk-food-stall] Self-reported generator validation cannot be credited as independent proof of no floating parts, collisions, or correct stud coupling.
• [1000-piece-airship-research-station] Authoritative strict connectivity validation failed completely, leaving physical buildability unverified.
• [1000-piece-airship-research-station] Several physical claims depend on self-authored validation notes/tooling rather than independent measurement.
• [1000-piece-airship-research-station] The airship uses many rotated fabric/connector/decorative elements whose real stud-coupling and support are not proven.
• [1000-piece-airship-research-station] Minor but real UI defects: overlapping chapter/play prompts and cramped/cropped mobile presentation.
• [1000-piece-airship-research-station] The main HTML was not machine-extractable by the mechanical validator despite the separate kitSpec being present.

What it nailed

• Substantial artifact set with fact sheet, source register, visualization, documentation, beat pages, and screenshots.
• Primary visualization renders successfully on desktop and mobile according to independent visual review.
• Interactive design includes MET clock, telemetry HUD, mission event log, timeline scrubber, camera modes, speed controls, labels, and deep links.
• The visual design is generally polished and useful for video/screenshot workflows, with only minor low-contrast issues reported.

Where it slipped

• Material factual/internal-consistency errors in public-facing mission content prevent publication trust.
• Source support is not independently verified by the supplied validation evidence, and some important claims rely on generic or model-authored source assertions.
• The visualization package contradicts itself on core technical beats such as TLI propulsion and re-entry type.
• Minor low-contrast timeline/control text reduces reviewability, especially on desktop.
• README references a factsheet-top screenshot that is not present in the artifact inventory.