Expert-reviewed scenarios, a public dataset, a failure history, and downloadable evidence behind every fit verdict.
A public, continuously-validated evidence base for the ItemFits spatial solvers: a frozen case corpus, a public failure ledger, and a downloadable dataset.
What this is
Every fit verdict ItemFits returns comes from deterministic geometry, not a model. This page is the public evidence base: a frozen corpus of scenarios whose expected verdict was fixed by human review before the solver ran, re-checked against the deployed build, with a public history of every disagreement. It measures agreement with an expert verdict on a curated corpus, not real-world fit success. Current corpus, growing.
Agreement by tier
| Validation tier | Cases | Agreement |
|---|---|---|
| Expert-reviewed | 6 | 100% |
| Direct geometry | 70 | 100% |
| Edge cases | 22 | 100% |
| Regression cases | 21 | 100% |
| Historical bugs | 4 | 100% |
Each tier reports the share of its scenarios on which the deployed solver matched the frozen expected verdict, with the scenario count it is drawn from. Agreement is comparison against an expert verdict, not a real-world success rate.
Coverage
| Solver | Cases |
|---|---|
| opening | 46 |
| hallway | 14 |
| container | 14 |
| room | 8 |
| elevator | 9 |
| turn | 9 |
| hybrid | 6 |
| vehicle | 8 |
| capacity | 3 |
| stairs | 6 |
123 scenarios across 10 solver families. Nondeterministic motion and maneuver solvers (RRT) are excluded by design. The vehicle, stair, and capacity solvers ship in production but are not yet represented in this corpus — they are outside the figures on this page. Current corpus, growing.
Failure history
| Date | Scenario | Expected | Actual | Resolution |
|---|---|---|---|---|
| - | Prod fit-check f4b5b669-f4b0-48b9-ac0f-8c6efb550e69 (user_feedback='inaccurate'). Every leaf solver returned FITS; the hybrid composition returned DOESNT_FIT carrying a POSITIVE +635mm margin. Root cause: faceToFloorFootprint switched on `orientation.face`, which only the opening solver emits — hallway/turn/stairs/elevator legs fell to a default branch that produced a {0,0} floor footprint, which then eliminated every room orientation (no_valid_orientation_after_path_filter) and fabricated a confident DOESNT_FIT. Fixed by deriving the footprint from the carried (across × along) axes for every path-solver orientation shape, and by failing OPEN (no orientation filter) rather than closed when a leg's shape cannot be read. Investigation: docs/investigation-2026-07-06-turn-multistep-composition-doesnt-fit.md. Regression guards: hybrid-0002 + hybrid-0003 (a genuine block must still block) and tests/hybrid-path-footprint-contract.test.js. | n/a | n/a | solver bug · fixed |
| - | Found while fixing hybrid-0001: the fabricated proof card it produced ('a 3659mm item in a 3658mm room' — an item dimension 1mm larger than the room, invented so a failing axis card could be rendered) was the fingerprint of a SECOND wrong verdict, not merely bad presentation. A 7'2" bookshelf that can only clear a 6'8" door lying flat has a CARRY footprint of (width × HEIGHT); the room and capacity solvers rest a rigid item on its (width × depth) base, always. Feeding the carry footprint in as a resting-orientation whitelist matched no orientation, eliminated every placement, and returned DOESNT_FIT at HIGH confidence for a bookshelf with metres of floor to spare. Fixed by only constraining the resting orientation when the item is actually carried in its resting pose (otherwise it is turned upright inside — surfaced as an explicit assumption), by failing OPEN in room-solver on a pose-basis mismatch, and by never fabricating a dimension or reporting a -1 sentinel as a clearance. The proof contract requiring a limiting card with a real margin was the signal the verdict was unjustifiable — satisfying it required lying. Regression guards: hybrid-0004, tests/hybrid-path-footprint-contract.test.js, src/lib/ai/solvers/__tests__/room-proof-steps.test.js. | n/a | n/a | solver bug · fixed |
| 2026-09-09 | The original axis-only rejection is not an all-orientation obstruction: a 2400mm axis exceeding each cab axis rules out axis-aligned placement only. The item diagonal is about 2546mm, below the cab diagonal of about 3177mm, and the 600mm minimum dimension clears the door and cab. These necessary bounds do not prove a fit either. Combined insertion, tilt and turning have no established entry-path witness or impossibility proof, so this access check must remain UNKNOWN. Corrected after the access-path proof guards in 686376bd9 exposed the stale expectation during deployment prebuild. Original inputs and row remain immutable; no solver bypass or open-bug exemption added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-09 | The original claim that a 6000mm pole cannot turn even tilted is unsupported without a measured ceiling or a restriction forbidding tilt. With sufficient headroom it could stand on its 400x400mm end and pass an 800mm corner. Failure of the checked planar and default-height tilt strategies therefore proves neither impossibility nor a feasible measured path; UNKNOWN is required. Corrected after the access-path proof guards in 686376bd9 exposed the stale expectation during deployment prebuild. Original inputs and row remain immutable; no solver bypass or open-bug exemption added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-09 | Width and headroom alone do not establish a straight stair carry: measured riser/tread, floor and ceiling profiles, carry pitch, and entry/exit staging are absent. No obstruction is proved either; the verdict must remain UNKNOWN. Corrected stale positive expectations after the access-path proof guards. Original inputs and rows remain immutable; no solver bypass or open-bug exemption added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-09 | A measured 90-degree landing rectangle does not establish a continuous carry through the stair flights and landing. Flight profiles, pitch and entry/exit staging are absent; UNKNOWN is required. Corrected stale positive expectations after the access-path proof guards. Original inputs and rows remain immutable; no solver bypass or open-bug exemption added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-09 | A measured 180-degree landing rectangle does not establish a continuous carry through both flights and the reversal. Flight profiles, pitch and entry/exit staging are absent; UNKNOWN is required. Corrected stale positive expectations after the access-path proof guards. Original inputs and rows remain immutable; no solver bypass or open-bug exemption added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-09 | An 800mm headroom measurement alone cannot establish a tight swept stair clearance. Without measured flight profiles, pitch and entry/exit staging, neither a complete carry nor an obstruction is proved; UNKNOWN is required. Corrected stale positive expectations after the access-path proof guards. Original inputs and rows remain immutable; no solver bypass or open-bug exemption added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-09 | Measured handrail intrusions establish usable width only. They do not establish the missing flight profiles, pitch and entry/exit staging needed for a continuous carry; UNKNOWN is required. Corrected stale positive expectations after the access-path proof guards. Original inputs and rows remain immutable; no solver bypass or open-bug exemption added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-09 | Feet/inches conversion preserves the supplied width and headroom but cannot supply missing flight profiles, pitch or entry/exit staging. A complete stair carry remains unproved; UNKNOWN is required. Corrected stale positive expectations after the access-path proof guards. Original inputs and rows remain immutable; no solver bypass or open-bug exemption added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-09 | A doorway and destination room fit cannot establish passage through the intermediate stairs. That leg supplies only width and height, without measured flight profiles or entry/exit staging; the composed path must remain UNKNOWN. Corrected stale positive expectations after the access-path proof guards. Original inputs and rows remain immutable; no solver bypass or open-bug exemption added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-18 | The owner reversed the stair half of decision D3 on 2026-09-18 (docs/2026-09-18-stair-grid-answers-p0-handoff.md; merged in b8c42d9d7, bfe354f29 and 5c8ac48f5): stair width and headroom, measured or from a picked stair type, are enough to answer, and answers built on typical data are hedged downstream by hedgeStairTypicalVerdict instead of withheld. The 2026-09-09 UNKNOWN/clarification expectation is therefore stale. It surfaced when the deployment prebuild ran after those merges, because the stair-P0 verification gates did not include this corpus. The superseding row restores the pre-2026-09-09 expectation with identical inputs. Original rows are unchanged; no solver bypass or open-defect exemption was added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-18 | The owner reversed the stair half of decision D3 on 2026-09-18 (docs/2026-09-18-stair-grid-answers-p0-handoff.md; merged in b8c42d9d7, bfe354f29 and 5c8ac48f5): stair width and headroom, measured or from a picked stair type, are enough to answer, and answers built on typical data are hedged downstream by hedgeStairTypicalVerdict instead of withheld. The 2026-09-09 UNKNOWN/clarification expectation is therefore stale. It surfaced when the deployment prebuild ran after those merges, because the stair-P0 verification gates did not include this corpus. The superseding row restores the pre-2026-09-09 expectation with identical inputs. Original rows are unchanged; no solver bypass or open-defect exemption was added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-18 | The owner reversed the stair half of decision D3 on 2026-09-18 (docs/2026-09-18-stair-grid-answers-p0-handoff.md; merged in b8c42d9d7, bfe354f29 and 5c8ac48f5): stair width and headroom, measured or from a picked stair type, are enough to answer, and answers built on typical data are hedged downstream by hedgeStairTypicalVerdict instead of withheld. The 2026-09-09 UNKNOWN/clarification expectation is therefore stale. It surfaced when the deployment prebuild ran after those merges, because the stair-P0 verification gates did not include this corpus. The superseding row restores the pre-2026-09-09 expectation with identical inputs. Original rows are unchanged; no solver bypass or open-defect exemption was added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-18 | The owner reversed the stair half of decision D3 on 2026-09-18 (docs/2026-09-18-stair-grid-answers-p0-handoff.md; merged in b8c42d9d7, bfe354f29 and 5c8ac48f5): stair width and headroom, measured or from a picked stair type, are enough to answer, and answers built on typical data are hedged downstream by hedgeStairTypicalVerdict instead of withheld. The 2026-09-09 UNKNOWN/clarification expectation is therefore stale. It surfaced when the deployment prebuild ran after those merges, because the stair-P0 verification gates did not include this corpus. The superseding row restores the pre-2026-09-09 expectation with identical inputs. Original rows are unchanged; no solver bypass or open-defect exemption was added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-18 | The owner reversed the stair half of decision D3 on 2026-09-18 (docs/2026-09-18-stair-grid-answers-p0-handoff.md; merged in b8c42d9d7, bfe354f29 and 5c8ac48f5): stair width and headroom, measured or from a picked stair type, are enough to answer, and answers built on typical data are hedged downstream by hedgeStairTypicalVerdict instead of withheld. The 2026-09-09 UNKNOWN/clarification expectation is therefore stale. It surfaced when the deployment prebuild ran after those merges, because the stair-P0 verification gates did not include this corpus. The superseding row restores the pre-2026-09-09 expectation with identical inputs. Original rows are unchanged; no solver bypass or open-defect exemption was added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-18 | The owner reversed the stair half of decision D3 on 2026-09-18 (docs/2026-09-18-stair-grid-answers-p0-handoff.md; merged in b8c42d9d7, bfe354f29 and 5c8ac48f5): stair width and headroom, measured or from a picked stair type, are enough to answer, and answers built on typical data are hedged downstream by hedgeStairTypicalVerdict instead of withheld. The 2026-09-09 UNKNOWN/clarification expectation is therefore stale. It surfaced when the deployment prebuild ran after those merges, because the stair-P0 verification gates did not include this corpus. The superseding row restores the pre-2026-09-09 expectation with identical inputs. Original rows are unchanged; no solver bypass or open-defect exemption was added. | n/a | n/a | corpus correction · corpus_corrected |
| 2026-09-18 | The owner reversed the stair half of decision D3 on 2026-09-18 (docs/2026-09-18-stair-grid-answers-p0-handoff.md; merged in b8c42d9d7, bfe354f29 and 5c8ac48f5): stair width and headroom, measured or from a picked stair type, are enough to answer, and answers built on typical data are hedged downstream by hedgeStairTypicalVerdict instead of withheld. The 2026-09-09 UNKNOWN/clarification expectation is therefore stale. It surfaced when the deployment prebuild ran after those merges, because the stair-P0 verification gates did not include this corpus. The superseding row restores the pre-2026-09-09 expectation with identical inputs. Original rows are unchanged; no solver bypass or open-defect exemption was added. | n/a | n/a | corpus correction · corpus_corrected |
Every solver disagreement with a correct scenario, and every later correction to a scenario, recorded as a public audit trail.
Methodology
Each scenario pairs structured item and space dimensions with an expected verdict frozen by human review before the solver runs.
The honesty contract
Every published figure carries its sample size. Anything below the threshold is suppressed, never shown as zero.
Validated against the deployed solver build on: October 2026 · 123 scenarios checked
Cite this study
ItemFits Public Solver Validation Program. Expert-reviewed scenarios validated against frozen expected verdicts; public dataset and failure history.
Take the data
The figures on this page, as a machine-readable file. Free to reuse with attribution to ItemFits. Permalink: https://itemfits.com/data/solver-validation
Enter your item and space above — get a verdict in seconds, then test the what-ifs.
Open the fit calculator