← Chimmani / RPT-004
Synthetic demo data
Report 004 · Architecture · Trustworthy AI over enterprise data

Getting an answer is easy.
Verifiable is the fun part.

Enterprise AI gets genuinely interesting the moment an answer has to hold up — in a board pack, in a filing, in a room with an auditor. That bar is a gift to an architect, because it rules out guesswork and points straight at a design. Three ideas I've been exploring: let the model choose and phrase while a deterministic core owns every number; give each answer a ledger it can be replayed from; and derive the vocabulary of entities from the ontology itself, so generation happens inside a space where only real things exist. Then size the model to the task on purpose. Everything below runs live — have a play.

S1

Three properties worth designing for

"Trustworthy" is vague until you decompose it. These three properties are specific enough to build toward and to test for — and each one turns out to be an engineering problem rather than a modelling one, which is what makes them tractable.

P1 · REPRODUCIBLE

Same question, same answer

Ask on Monday and on Thursday and get an identical figure, or a precise account of what moved. Achieved by routing every number through a versioned plan rather than a sampled generation.

Design target
Q: "TRIR for NL-03?"
Mon → 1.65  ·  Thu → 1.65
or a named delta.
P2 · ATTRIBUTABLE

Chain of custody by default

Every figure carries the records that produced it, the period definition applied, and what was excluded — so a reviewer can follow it down to source without asking anyone.

Design target
"€4.12M at risk"
→ 38 orders, listed
→ plan + hash attached.
P3 · GROUNDED

Only real things sayable

Entity names come from the ontology's own registry, so a plausible-but-invented identifier isn't something the system has to catch — it's something it can't produce.

Design target
Vocabulary = registry
ACME-0042 ✓
NORDIC-118 not emittable.
The encouraging part

None of the three depends on a better model — they're properties of the system built around the model. That means they're reachable today with ordinary engineering, and they get more valuable as models improve, not less.

S2

The split

One rule carries the whole design, and it's a liberating one: the model chooses the plan and phrases the result; a deterministic core computes every number. Green is deterministic code over the ontology, amber is generative. Drawing the line here means the model gets to do what it's genuinely excellent at — understanding intent and writing well — while arithmetic stays somewhere it can be checked.

The payoff

Swap the model and the prose changes while the number holds. That one property is what makes an output auditable, and it means you can adopt a better model tomorrow without re-validating a single figure. Try it live in the next section.

S3

Claim ledger & replay

This one is a working engine, not a mockup — the values below are computed in your browser from a synthetic object store, and every answer emits a ledger with a content hash over the plan and its resolved inputs. Replay re-executes and compares. Mutate nudges a source record so you can watch the drift get caught and named.

Question → plan → value → prose
Deterministic plan
Answer

Claim ledger
Not yet replayed
Try this

Switch the phrasing model at the top right. The wording changes; the value hash stays identical. Then mutate a record and replay — the ledger names exactly which input moved and by how much.

S4

Constrained generation

Prompting a model not to invent things is a request it will mostly honour. Deriving the permitted vocabulary from the ontology and rejecting anything outside it is a guarantee — and the nice part is how little machinery it takes. Same question, same intended output, two decoders.

Unconstrained
Ontology-constrained
Permitted entity tokens — derived from the ontology registry, not from a prompt
Scripted token stream illustrating the mechanism — no model runs in this page
Why the structural version wins

Notice that the unconstrained output doesn't look wrong — the invented IDs are well-formed and entirely plausible, which is exactly what makes them hard to spot by eye. Deriving the vocabulary from the registry turns a review burden into a property of the system, and it costs almost nothing to implement.

S5

Trust budget

Model right-sizing as a deliberate decision rather than a default. The rule this encodes surprises people: as consequence rises, you move work out of the model rather than into a bigger one. It's a cheerful finding — the highest-stakes work is usually the most deterministic, so the safest option is also the least expensive.

Task
Monthly volume 5,000 items
1001K5K25K100K500K
Consequence of being wrong Operational
InformationalOperationalFinancialRegulatory
Recommendation

Unit cost
Volume / mo
Review overhead
Monthly cost