Kimi K3

What it nailed

• Exceptionally strong handling of the benchmark's core absurdities: dingo welfare, Alaska/Australia mismatch, import-created demand, legality, support-language liability, and ethics.
• Comprehensive artifact completion in real requested formats, including substantial DOCX/PPTX/XLSX/PDF/HTML outputs.
• Consistent canon-number system prevents the usual drift across deck, summary, GTM plan, dashboard, spreadsheets, and FAQ.
• Strong quantitative reconciliation of revenue, units, budget, runway, CAC/LTV, NPS, import attach-rate artifacts, and import-margin uncertainty.
• Fictional scenario competitors are labeled instead of being hallucinated as real market evidence.

Where it slipped

• Minor mobile dashboard navigation/header clipping and very small header/logo text were observed by one visual reviewer.
• Some research claims still need independent verification before external use; Australian export legality is explicitly left as an open official-source gap.
• A few citation/source-count details are imperfect, including source-count wording and at least some source-link/citation polish concerns.
• Some public-facing copy is very good but still requires counsel review because the product category and import program are legally sensitive.
• Quote-permission and some validation/manifest claims are model-authored assertions unless checked against the original source files.

What it nailed

• Substantial artifact set: SQLite database, deterministic migration script, static audit UI, frontend JSON/JS export, design notes, README, and migration report.
• Strong reviewer UX with clean desktop/mobile rendering and no visual defects reported by independent visual judges.
• Good coverage of many structured-source problems: source inventory, SVC-007 conflict, corrupted JSON partial recovery, duplicate images, status/payment normalization, price eras, and many typo/nickname merges.
• Useful provenance/review architecture with source_files, source_records, conflicts, rejected items, review flags, and provenance explorer.

Where it slipped

• Major canary failure: at least one obvious ghost/test record, Mickey Mouse with magic payment, appears to be retained as canonical payment/revenue rather than quarantined.
• Rejected-items evidence only covers Test Customer rows; there is no clear evidence that Mickey Mouse or Asdf Asdf were rejected.
• Handwritten receipt/image extraction is weak: several important images had no usable OCR and image-derived price/customer conflicts are only partially represented.
• Entity resolution is imperfect, with visible duplicate identities such as Amber and Amber Bilbow sharing contact evidence.
• Department-code normalization and full referential-integrity validation are not independently demonstrated.

What it nailed

• [100-piece-lunar-rover] Exactly 100 measured parts for the isolated lunar-rover case.
• [100-piece-lunar-rover] Strong single-spec structure: kitSpec drives the rendered model, UI manifest, and step-based assembly guide.
• [100-piece-lunar-rover] Completed rover is visually recognizable and recordable on desktop, with clear lunar-rover features.
• [100-piece-lunar-rover] Connectivity is mostly successful for a 100-piece build: 91 of 100 parts are in the largest strict component and floating_fraction is only 0.09.
• [250-piece-rescue-helicopter] Produced the required isolated case artifact with a structured kitSpec and a complete browser-based assembly guide.
• [250-piece-rescue-helicopter] Exact 250-piece count verified by independent extraction from the HTML/spec.
• [250-piece-rescue-helicopter] Connectivity is strong for this benchmark scale: only 5.6% floating under the strict mechanical validator.
• [250-piece-rescue-helicopter] Clear step/chapter organization and a recognizable rescue-helicopter silhouette with landing pad, rotors, skids, door, and winch.
• [250-piece-rescue-helicopter] Desktop visual presentation is polished and recording-friendly.
• [500-piece-cyberpunk-food-stall] Delivers the required isolated case with artifacts/index.html and a separate kitSpec.json.
• [500-piece-cyberpunk-food-stall] Exact 500-piece declared kit with structured part records, step IDs, dimensions, positions, and rotations.
• [500-piece-cyberpunk-food-stall] Strong cyberpunk food-stall concept with recognizable stall silhouette, neon signage, kitchen/counter details, accessories, and delivery scooter.
• [500-piece-cyberpunk-food-stall] Animated Three.js guide includes the expected reviewer controls and a polished, recording-friendly visual style.
• [500-piece-cyberpunk-food-stall] Desktop visual quality is strong; operator findings mark only minor issues for this case.
• [1000-piece-airship-research-station] Delivered the required artifacts/index.html for the isolated 1000-piece case, plus kitSpec.json and a build guide.
• [1000-piece-airship-research-station] Declared and measured part counts match at 1048 pieces, within the requested range.
• [1000-piece-airship-research-station] The kit is structured around a machine-readable kitSpec with chapters, steps, part IDs, and per-part geometry.
• [1000-piece-airship-research-station] Concept coverage is rich: mountain base, research station, gantry, airship envelope, gondola, cargo pod, instruments, and final docking sequence.

Where it slipped

• [100-piece-lunar-rover] Independent connectivity contradicts the model's self-validation claim: 9 parts are floating under strict stud/connector coupling.
• [100-piece-lunar-rover] High near_miss_count indicates spatial placements are often approximate rather than cleanly engaged on a strict stud grid.
• [100-piece-lunar-rover] Several small accessories such as control, windshield, lamp clips, antenna/sensor, and dish are not mechanically validated as attached.
• [100-piece-lunar-rover] Mobile completed view is crowded, with overlays and controls obscuring part of the model and parts-added list.
• [100-piece-lunar-rover] The viewer depends on CDN-loaded Three.js rather than being fully offline self-contained.
• [250-piece-rescue-helicopter] Build data integrity is limited by the validator contradicting the artifact's own support/connectivity claims.
• [250-piece-rescue-helicopter] 801 near misses is high relative to 250 parts, suggesting approximate placement rather than consistently strict stud-grid engagement.
• [250-piece-rescue-helicopter] Fourteen parts remain strictly floating, including several small accessories and rotor/chain-related pieces.
• [250-piece-rescue-helicopter] Mobile completed-state presentation has minor overlay/cropping defects.
• [500-piece-cyberpunk-food-stall] Mechanical connectivity validator could not extract or measure the model, leaving strict stud-coupling buildability unverified.
• [500-piece-cyberpunk-food-stall] Self-reported validation claims cannot substitute for independent physical measurement.
• [500-piece-cyberpunk-food-stall] Runtime kitSpec was not discoverable by the provided validator from index.html, reducing confidence in automated build verification.
• [500-piece-cyberpunk-food-stall] Mobile presentation has minor crowding/overlap/cropping issues in completed-state views.
• [500-piece-cyberpunk-food-stall] Very slow generation run: 26 API calls and roughly 7,256 seconds wall-clock time.
• [1000-piece-airship-research-station] Strict mechanical validation shows 35.78% of parts floating and 252 disconnected components.
• [1000-piece-airship-research-station] The near_miss_count of 6512 indicates widespread approximate placement rather than reliable stud-grid engagement.
• [1000-piece-airship-research-station] Rendered completed-state evidence is blocked by the intro modal, so the actual 1000-piece completed model cannot be visually reviewed.
• [1000-piece-airship-research-station] Controls are present in the HTML but not independently verified as usable because the operator screenshots never reach the visualizer.
• [1000-piece-airship-research-station] Self-reported validation claims are contradicted by operator visual findings and are not treated as evidence.

What it nailed

• Complete artifact package with fact sheet, interactive visualization, source list, documentation, shot links, and screenshots.
• Visualization renders successfully and presents a mission-specific 3D scene rather than a generic orbit shell.
• Strong mission beat coverage from launch through recovery, including crew and vehicle context.
• Good citation discipline in the artifacts, with bracketed source tags and explicit planned/as-flown labeling.
• Useful production features such as deep links, camera presets, event stepping, HUD telemetry, and hide-UI recording mode.

Where it slipped

• No independent source URL/content validation is provided for highly specific as-flown and post-mission claims.
• Some key research assertions depend on the artifact's own verification claims and secondary sources, so publication would require manual fact checking.
• Desktop render has a minor persistent loading-overlay concern.
• Mobile view is crowded, with small or truncated timeline labels and overlays covering much of the scene.
• The visualization depends on an external Three.js CDN for first load, so it is not fully offline/self-contained.
• Run speed was very poor, with roughly 9 hours wall-clock time, 50 API calls, and 12 retries.