Grounded Reasoning for Safety-Critical AI: An Ontology-Based Architecture and Evaluation Benchmark for Human Error Risk Assessment in Process Industries

Abstract#

Language models are entering environment, health and safety (EHS) workflows on the assumption that they behave like engineering infrastructure. We argue that their characteristic failure - a fluent, well-formatted substitution of a semantically adjacent but categorically inapplicable answer - is structural rather than incidental. A practitioner incident motivates the analysis: a persona-configured assistant, asked about overpressure protection for chemical reactors, returned pressure-relief-valve content when rupture-disk content was requested. Four independent mechanisms predict this outcome. One we verify directly from the governing standard: API 520 Part I documents both device classes in a single text with a shared definitions section, and "disc" is polysemous within it, naming both a valve's closure element and an entire non-reclosing device ^[api520p1]. The authoritative corpus is lexically adversarial precisely where the distinction carries consequence.

We contribute four artifacts. First, that structural account, assembled from retrieval, training-incentive, distributional-semantics and corpus-design evidence. Second, an auditable human-factors ontology that adopts - rather than proposes - the four-context performance-influencing-factor structure of IDHEAS-G (NUREG-2198) ^[nureg2198], with a generated crosswalk of 90 alignments to five prior frameworks and six asserted absences. Third, a preregistered benchmark of 68 clause-anchored EHS questions, every anchor verified against the live Electronic Code of Federal Regulations, with a three-arm (ungrounded / pseudo-grounded / authoritative) minimal-pair design. Fourth, a reanalysis of 2,801,064 OSHA establishment filings showing that the exposure denominator underlying every rate-based safety metric is corrupted at scale.

We report no evaluation of any production system.

(240 words)

1. Introduction#

1.1 An incident that was not bad luck#

An EHS executive built a question-answering assistant over his own written material and asked it about overpressure protection for chemical reactors. He wanted rupture disks. It gave him pressure relief valves - fluently, in the right format, with the right register, and completely inapplicable to the question he had asked. He caught it because he had spent decades in the field and knew what a rupture disk answer looks like.¹

The reflexive reading of that episode is that the model got unlucky, that a better prompt or a larger model would have fixed it. We think the opposite. The substitution is what one should expect from the system that was built, and it is over-determined by at least four mechanisms that operate independently of each other.

There was no grounding to fail. A system prompt plus pasted source material is parametric conditioning. It shifts a prior; it does not supply evidence. Under the definition that established retrieval-augmented generation, a RAG system pairs a parametric generator with a non-parametric retrieved memory and marginalizes over retrieved documents ^[lewis2020]. A persona-configured assistant has no retriever, no index, and no such marginalization. Every token in that answer came from pretraining. The label frequently attached to such systems - "digital twin" - asserts a synchronization with a physical or documentary referent that the architecture never had, and the misnomer is part of how these systems come to be trusted.

The requested content is long-tail. Parametric factual accuracy scales with how often a fact appears in pretraining ^{[kandpal2023]}, and parametric memory degrades specifically on low-popularity entities, which is also where retrieval delivers its largest gain ^[mallen2023]. Pressure-relief-valve content saturates the open web through vendor catalogs, sizing tutorials and forum threads. Rupture-disk specifics do not. The query landed in a sparse region and the model returned the dense neighbor.

Abstention is penalized. Training objectives and benchmark scoring reward a plausible guess over an admission of ignorance: declining scores zero, a confident wrong answer sometimes scores above zero ^[kalai2025]. Under uncertainty the model emits its highest-probability plausible completion. This is why the failure is not stochastic noise - the same gradient produces the same substitution repeatedly.

Distributional representations encode relatedness, not relation type. Rupture disks and pressure relief valves are exact taxonomic siblings under the parent category "pressure relief devices" in ASME BPVC Section VIII, Division 1, where rupture disks are classed as nonreclosing devices ^{[asme_viii_ug127]}. Siblings share contexts more heavily than almost any other lexical pair - the same sentences, tables, P&IDs and regulatory verbs. Supervised distributional methods learn prototypical hypernyms rather than the relation holding between a specific pair ^[levy2015]; hypernymy is not cleanly separable from other tight relations by distributional means ^{[shwartz2017]}; antonyms, which are maximally distinct semantically, sit close in embedding space and require explicit external constraints to separate ^[mrksic2016]. On the retrieval side, neural retrievers largely fail minimal pairs differing only by negation ^{[weller2024nevir]}. A retriever that cannot represent not cannot enforce rupture disk, not valve as a hard constraint rather than a soft preference.

The fourth mechanism is the one we consider decisive, and we verified it against the standard itself rather than inferring it. API Standard 520 Part I (9th ed., July 2014) documents pressure-relief valves and rupture disk devices in the same text, under one definitions section, with a Rupture Disk Device Specification Sheet annex sitting alongside the valve specification sheets. Inside that single document the word "disc" is genuinely polysemous: in a pressure-relief valve it names the closure element - lift is the travel of the disc, reseating pressure is where the valve disc reestablishes contact with the seat - whereas a rupture disk is an entire device ^[api520p1]. Any chunk-and-embed index built over API 520 will therefore return passages whose nearest neighbors are the wrong device class, because lexical and topical overlap peak exactly at the consequential distinction. ISO, by contrast, separates the two device families across separate parts of ISO 4126 ^[iso4126]. Which standards family an organization indexes is thus a safety-relevant corpus-design decision, and to our knowledge no prior work has framed it that way.

What makes this error class dangerous rather than merely wrong is that the two parameter spaces do not intersect. A valve has set pressure, blowdown, accumulation, backpressure, effective discharge area, coefficient of discharge, lift, chatter, reseating pressure - and it recloses. A rupture disk has marked burst pressure, burst pressure tolerance, manufacturing design range, specified disk temperature, lot - and it is one-shot ^{[nationalboard_burst]}. No arithmetic converts one answer into the other. The output is not approximately correct; it is categorically inapplicable while remaining fluent and locally plausible, which is extrinsic hallucination in its most hazardous form ^{[maynez2020, ji2023]}. The executive caught it because he held the domain model. Language models uncritically accept incorrect user premises ^[dahl2024] and conform to stated user belief ^[sharma2024], so the junior engineer who asks a leading question gets agreement rather than correction.

1.2 What is missing, and what is not#

It would be convenient if the gap here were a taxonomic one - if the field lacked the right categories for contextual risk factors and this paper could supply them. It does not, and we say so at the outset because the alternative framing would not survive review.

Human reliability analysis has been organizing contextual factors since THERP introduced the internal/external performance-shaping-factor distinction in 1983 ^[swain1983]. SPAR-H uses eight PSFs ^{[gertman2005]}; the NRC's good-practice guidance enumerates fifteen ^[nureg1792]; CREAM uses nine common performance conditions ^{[hollnagel1998]}; HSE organizes performance-influencing factors under job, person and organization headings ^[hse_pifs]; HFACS subdivides preconditions for unsafe acts into environmental, operator-condition and personnel factors ^{[shappell2000, wiegmann2003]}. Most directly, IDHEAS-G classifies twenty PIFs into exactly four context types - environment and situation, system, personnel, task ^[nureg2198]. Any four-way contextual split proposed today for process industries is that split. Proposing it again would be relabeling, and Boring has already documented that PSF proliferation - from single-factor models to fifty-plus - has not demonstrably improved HRA quality ^[boring2010]. Groth and Mosleh state the standing problem plainly: there is no standard PIF set, PIFs are not defined specifically enough for consistent interpretation across methods, and few rules govern PIF set creation ^[groth2012]. A fifteenth loosely-specified grouping makes that worse. Operational definitions with measurement procedures make it better.

So we adopt IDHEAS-G's structure and ask a different question. Every framework named above is analyst-rated: a trained analyst reads a scenario, performs a task analysis, and assigns a level per factor. None specifies how to derive factor levels from data an industrial operator already collects - OSHA 300/301 case detail, shift rosters, permit-to-work records, CMMS work orders, MES and ERP logs. That is a measurement problem, not a taxonomy problem, and it is where we locate the contribution.

The measurement problem has a specific and severe form in this domain: the denominator is not trustworthy. All rate-based safety analysis presupposes a well-defined exposure count. Hanecke and colleagues hit this with more than 1.2 million German accident records for 1994 and no working-time data, and had to construct estimated exposure models to say anything about risk as a function of hour at work ^{[hanecke1998]}. The same failure is present in US data. In our reanalysis of the pooled 2016–2024 OSHA Injury Tracking Application Form 300A panel (2,801,064 deduplicated establishment filings), filings failing a physical-plausibility screen constitute between 1.8% and 4.5% of records across a sensitivity grid, and at the primary specification carry 96.7% of all reported hours. A single filing - seven employees, zero recordables, 1.68 × 10¹³ declared hours - accounts for 88.8% of the total by itself. Screening changes the pooled aggregate total recordable incident rate from 0.134 to 3.983, a factor of 29.7; year by year the factor ranges from 1.39 to 249. We report this as a negative result as much as a positive one: the correction multiplier is not a constant, it is not portable between years, and an earlier version of this analysis over a narrower panel produced materially different figures ^{[chimmani2026]}. Anyone computing an industry benchmark from these filings without a plausibility screen is computing an artifact of a handful of data-entry errors.

Nor is there prior art that closes the gap on the engineering side. The ontology-for-process-hazard-analysis literature is real, coherent and thirty years old - signed-digraph automated HAZOP dates to the mid-1990s ^{[vaidhyanathan1995]}, and the modern ontology work is careful ^{[single2020_jlp, aziz2019]} - but its own state-of-research review reports that after more than three decades of effort very few approaches have been used in the chemical process industry, and that knowledge quality, not reasoning technology, is the binding constraint ^[single2019]. Published validations are single-vessel: a hexane storage tank, a gasoline storage tank. Aziz and colleagues are the outlier at 45 US Chemical Safety Board events ^[aziz2019]. Nothing has been run against a population of real regulatory filings. On the evaluation side, the RAG literature has a benchmark for negation ^{[weller2024nevir]} and one for a safety-critical regulated domain in medicine ^{[xiong2024mirage]}, but none for co-hyponym device-type discrimination in engineering standards, and none for process safety at all. The recent process-safety LLM work evaluates HAZOP, STPA and FRAM generation against expert-prepared worksheets ^{[lee2026hazop, charalampidou2024, kaya2025, elhosary2026]}; none performs a controlled grounded-versus-ungrounded comparison on standards-lookup questions, and none uses minimal pairs. Dokas reaches the same conclusion from a scoping review: existing LLM benchmarks do not assess what hazard analysis requires ^[dokas2026].

1.3 Contributions#

We state these precisely, and we separate what we adopt from what we add.

C1. A structural account of semantically adjacent substitution in safety-critical question answering (§2). We assemble four independently attested mechanisms - absent retrieval, long-tail frequency, penalized abstention, and relation-blind distributional geometry - into a prediction, and add a fifth that is specific to this domain and, to our knowledge, new: the authoritative corpus is itself lexically adversarial at the distinction that matters, verified from API 520 Part I's own definitions and its polysemous use of "disc" ^[api520p1]. This converts a folk explanation ("the model hallucinated") into a falsifiable claim about corpus design and retriever architecture, with a stated prediction - dense bi-encoder retrieval should show the largest co-hyponym confusion, lexical retrieval should be competitive where exact terminology carries the distinction, and the decisive intervention should be a device-type metadata filter rather than a better embedding.

C2. An auditable contextual-factor ontology that adopts IDHEAS-G's structure rather than proposing a rival (§3). The artifact encodes 20 factors under IDHEAS-G's four context types, 21 deterministic rules, and a generated crosswalk of 90 alignments to 78 named factors across five external frameworks, together with 6 asserted absences - cases where we claim no counterpart exists in a prior framework - and 7 documented coverage gaps. The crosswalk is generated from the ontology file and regression-tested against drift, so the paper's tables cannot silently disagree with the artifact. Seven of twelve citations backing the transcriptions are verified against primary documents; the remaining five are bibliographic, and CREAM's nine common performance conditions are transcribed at one remove from a secondary source rather than from Hollnagel's book. The novelty claim is not the categories. It is the machine-checked provenance discipline applied to them, and the explicit statement of where the four-way cut fails to cover something.

C3. An open, preregistered benchmark for grounding in safety-critical technical question answering (§4). 68 items, each keyed to a specific regulatory clause identifier following the provenance-annotation design of KILT ^{[petroni2021kilt]}, with every anchor text verified to sit inside the paragraph it cites - a check we added after discovering that our own initial verifier passed an item whose anchor lived in a different paragraph of the cited section. The design is three-arm (ungrounded / pseudo-grounded over a plausible but unauthoritative corpus / authoritative over version-pinned standards), scored on paired accuracy over minimal pairs following contrast-set methodology ^{[gardner2020, ribeiro2020checklist, weller2024nevir]}, with a distinct category-error arm measuring negative rejection ^{[chen2024rgb]} and abstention as a first-class outcome. The pseudo-grounded arm is the condition most enterprise deployments actually occupy and the one nobody measures; irrelevant retrieved context is known to degrade accuracy ^{[shi2023distracted]}, so it may perform worse than no retrieval at all. Six additional draft items are quarantined because we could not verify their clause identifiers, and the repository enforces by test that no quarantined item asserts one.

C4. A population-scale measurement result on the exposure denominator (§5). The reanalysis described above, with a fully pinned provenance chain - nine source files, SHA-256 digests, live re-download verification - a byte-reproducible pipeline, and 136 regression tests, including tests that fail when a number in the manuscript stops matching the generated table. Alongside it we report a simulation study establishing the arithmetic that governs any shift-level safety risk score: at the 2024 BLS private-industry total recordable rate of 2.3 per 100 full-time-equivalent workers ^[bls2026], a recordable case occurs roughly once per 10,870 eight-hour worker-shifts, so a classifier at 0.80 sensitivity and 0.95 specificity yields about one true event per 680 alerts. At that base rate, discrimination metrics are close to uninformative as decision criteria, and the standard imbalance corrections destroy calibration without improving discrimination ^{[goorbergh2022, carriero2025]}.

1.4 What we do not claim#

Grounding reduces hallucination; it does not remove it. Commercial legal research tools marketed on the strength of eliminating hallucination were measured, in a preregistered evaluation, at 17–33% hallucination ^[magesh2025]. We report deltas with confidence intervals under stated conditions and use no verb stronger than "reduces."

We do not call our rule layer automated reasoning. It performs deterministic forward evaluation over a fact base; the algorithmic ancestor of such systems is a pattern matcher by its author's own title ^[forgy1982], and the honest formal home for declarative terminating rule evaluation is Datalog ^[green2013]. No description-logic reasoner is run over our ontology - the file is parsed and structurally validated, never classified or checked for consistency - so we assert no soundness, completeness, decidability or complexity result. Where an unsound translation step feeds a downstream check, we follow the practice of marking the epistemic boundary explicitly, as AWS does when it states that its natural-language-to-logic step uses foundation models and may contain errors while only the validation step is mathematically sound ^{[aws_arc_docs]}. We also do not claim that protocol-level source binding solves grounding; the Model Context Protocol standardizes how a model reaches a source and makes the binding auditable, and its own specification locates trust outside itself ^{[mcp2025spec]}.

We do not claim causal effects. Deductive entailment is not causation, and the formalism for cause and effect is structural rather than deductive ^{[pearl2019, halpern2005]}. Our ontology's screening bands are ordinal labels produced by rules marked as convention; nothing in the artifact has been validated against injury or incident outcomes, and no effect size is reported for any factor-to-error-mode link.

We have not evaluated any production system. The benchmark harness has been exercised only against a mock adapter with a synthetic response profile, held under a synthetic/ directory with a seeded generator and declared headers, and the repository states that any headline number traceable to that directory is wrong. The answer keys were audited mechanically for provenance but not re-derived editorially by a second analyst; five key defects were found and fixed during audit, and we do not assume the remaining population is zero. The ontology's 90 crosswalk match strengths are one coder's unadjudicated judgement with no inter-rater reliability figure. The simulation study has never been fitted to real data and demonstrates properties of an estimator, not facts about workplace safety.

Finally, we do not present the hour-of-shift distribution as a discovery. Elevated early-shift accident counts are a named, twenty-five-year-old phenomenon ^[tucker2000], and once exposure is modeled, risk rises approximately exponentially with time on shift, more than doubling by the twelfth hour relative to the first eight ^{[folkard2003]}. Any unadjusted count distribution over incident timing describes when injuries are reported, not when risk is highest, and no public dataset supplies the hours-at-risk denominator needed to convert one into the other. We state that as a limitation rather than resolving it.

1.5 Theoretical position#

A paper that models risk as a weighted combination of contextual factors owes the reader an account of why it is not adopting the systems-theoretic alternative. Rasmussen argued that task analysis focused on human-error deviations should be replaced by modeling behavior-shaping mechanisms as work-system constraints ^{[rasmussen1997]}; Leveson's STAMP treats accidents as inadequate enforcement of safety constraints across a hierarchical control structure rather than as a sum of contributing factors ^{[leveson2004]}. We take these seriously and do not cite them as friendly authority. Our position is narrow: the ontology is a measurement instrument for factors that established frameworks already require an analyst to rate, not an accident model, and it makes no claim that summing factor levels explains an accident. Where the paper does describe causation, it does so in the systems-theoretic register, not the additive one. Readers who reject additive PSF models entirely will still find C1, C3 and C4 usable, since none of them depends on the ontology.

The broader empirical record also cuts against optimism, and we would rather state it than have a reviewer state it for us. The international HRA empirical study found human error probabilities varying substantially method to method for the same human failure events, though it also concluded that predictions mostly correlate with observed empirical difficulty ^[nureg2127]. French and colleagues concluded that the barriers to quantification in summative human reliability analysis "are currently too substantial" given available models ^[french2011]. Forty years of category refinement has not fixed quantification. Our claim is not that better categories will. It is that the categories cannot even be tested until they can be measured from data that exists, and that the data that exists is, at present, arithmetically unusable without a screen.

1.6 Roadmap#

Section 2 develops the structural account of adjacent substitution and its corpus-design mechanism. Section 3 presents the ontology, the crosswalk to IDHEAS-G, SPAR-H, CREAM, HSE PIFs and HFACS, and the rule layer, with its verification discipline and its limits. Section 4 specifies the benchmark: item construction, clause anchoring and verification, the three-arm protocol, the scoring rules, and the preregistered analysis plan. Section 5 reports the OSHA denominator reanalysis and the rare-event arithmetic that constrains any shift-level score. Section 6 discusses what a practitioner should do differently on Monday. Section 7 states threats to validity and what would falsify each contribution.

2. Related Work#

Three literatures bear on the argument of this paper, and they have largely not been read against each other. The first is human reliability analysis (HRA), which has spent forty years building and rebuilding lists of the contextual factors that shape human performance. The second is automated reasoning and neurosymbolic AI, which owns the vocabulary that safety practitioners have begun borrowing to describe rule-based tools. The third is the empirical literature on hallucination, retrieval augmentation, and attribution in language models, which has already established most of what can and cannot be claimed for a grounded assistant.

We treat each strand on its own terms before stating what is left open. Two of the conclusions in this section cut against positions we would prefer to hold. We state them anyway, because both are checkable in a few minutes by any reviewer drawn from the relevant community, and because being wrong about prior art is more expensive than having less to claim.

2.1 Human reliability analysis and performance shaping factors#

2.1.1 The canon#

The modern treatment of context in human error begins with THERP, the Technique for Human Error Rate Prediction ^[swain1983]. THERP introduced the apparatus that most later methods inherit: human failure events, task analysis, human error probabilities (HEPs), dependence, event trees, recovery, and, of direct relevance here, the split between internal and external performance shaping factors (PSFs). Nearly every factor list published since is a redrawing of that boundary.

Second-generation methods reorganized rather than abandoned it. CREAM replaced the PSF vocabulary with nine Common Performance Conditions covering organization, working conditions, human-machine interface and operational support, availability of procedures, number of simultaneous goals, available time, time of day, training and experience, and crew collaboration quality ^{[hollnagel1998]}. We note, because it matters for how much weight the reader should place on our transcription of that list, that we worked from a secondary source rather than from Hollnagel's book text, and the list should be checked against the original before it is quoted.

ATHEANA moved in the opposite direction, rejecting a fixed factor set in favor of a search for the error-forcing context that makes a given failure plausible ^[nureg1624]. SPAR-H settled on eight PSFs (available time, stress, experience and training, complexity, ergonomics including the human-machine interface, procedures, fitness for duty, and work processes), of which three are evaluated in the basic model and five default to nominal ^{[gertman2005]}. The NRC's good-practices guidance enumerates fifteen ^[nureg1792]. The UK Health and Safety Executive publishes a PIF list under three headings, job, person, and organisation, and states on the page itself that the list is not exhaustive ^[hse_pifs]. HFACS, built on Reason's Swiss cheese model, arranges its categories under four levels, with Preconditions for Unsafe Acts subdividing into Environmental Factors, Condition of Operators, and Personnel Factors ^{[shappell2000; wiegmann2003; reason1990; reason2000]}. Rasmussen's skill-rule-knowledge distinction underlies the error typologies these frameworks classify against ^{[rasmussen1983]}.

The most recent entry, and the one that governs the novelty question below, is IDHEAS-G, the NRC's integrated human event analysis methodology ^[nureg2198]. It organizes twenty PIFs into four categories of context. The report states that PIFs are classified according to the four types of context: environment and situation, system, personnel, and task. The twenty PIFs distribute across those categories as five, three, five, and seven respectively. We verified this against the published report rather than a summary of it.

2.1.2 The four contextual dimensions are not novel#

Practitioner writing in this space, including framing we ourselves have used, groups human error context into four dimensions: task, operational, human, and system. It is worth saying plainly what these are.

Two of the four labels are identical to IDHEAS-G's. The other two are direct synonyms. The cut-points are the same and the correspondence is one to one with a federal standard published in 2021. This is a relabeling, and we say so in the abstract as well as here. The four-way split is not our contribution and we make no originality claim on it. What we adopt from IDHEAS-G we adopt on the record, and where our boundaries differ from the NRC's we say where. One such difference is worth naming: IDHEAS-G files procedures and guidance under Personnel, whereas an industrial practitioner would more naturally file them under System or Task. That is a boundary disagreement, not a discovery, and it is an instance of exactly the ambiguity that Groth and Mosleh flagged ^[groth2012].

2.1.3 The field's own assessment of its factor lists#

The case against adding another factor taxonomy is made most forcefully from inside the HRA community. Dougherty's critique set the terms of the first- versus second-generation debate and remains the piece any paper claiming to advance human error analysis has to engage ^{[dougherty1990]}. Groth and Mosleh report that more than a dozen HRA methods use PIFs, that there is no standard PIF set and no framework for comparing them, that PIFs in circulation are not defined precisely enough for consistent interpretation across methods, and that few rules govern how PIF sets are created; their response is a data-informed collapsible hierarchy built from event data rather than a fifteenth list ^[groth2012]. Boring documents the proliferation directly, from single-factor models to methods with more than fifty PSFs, and asks whether expansion has improved HRA at all ^[boring2010].

The regulator has conceded the point about its own methods. NUREG-2198 attributes HEP variability to weak guidance for qualitative analysis and poor understanding of PIFs, and describes that variability as running method to method, analyst to analyst, and crew to crew ^[nureg2198]. The same report notes that existing methods were built for procedure-based response to internal at-power events in nuclear power plants and are not necessarily adequate for human actions in other domains, which is a direct caution against transplanting the apparatus into general chemical manufacturing without argument.

The empirical studies say less than they are sometimes reported to say, and we want to be accurate about them because we have previously been inaccurate about them ourselves. The International HRA Empirical Study compared method predictions against HAMMLAB simulator data and found substantial quantitative spread, on the order of one order of magnitude in loss-of-feedwater scenarios and up to two in steam generator tube rupture scenarios after outliers were censored, along with a disconnection between qualitative and quantitative analysis in several method applications ^[nureg2127]. It did not conclude that method rankings were uninformative. Read directly, the report finds that HRA predictions mostly correlate with the empirical difficulty of the human failure events, with named exceptions. An earlier draft of our own supporting repository asserted the stronger, contrary claim; we retrieved the report and corrected it. The US companion study reaches compatible conclusions ^[nureg2156]. French and colleagues, reviewing HRA for a management audience, judge the barriers to the quantification required for summative analysis to be too substantial at present given the state of validated behavioral and organizational models, and note that data are sparse in most contexts involving serious system failures ^[french2011].

Finally, the theoretical objection. Rasmussen argues that task analysis organized around action sequences and human-error deviations should give way to modelling the behaviour-shaping mechanisms of work-system constraints and boundaries ^{[rasmussen1997]}. Leveson's STAMP treats accidents as inadequate enforcement of safety constraints across a hierarchical control structure rather than as a sum of contributing factors, and rejects decomposition into contributing factors as an accident model ^{[leveson2004; leveson2012]}. Both positions are in tension with any additive contextual-factor scheme, including the one we adopt. We do not cite either as support for our structure. We take the tension as real and address it in Section 5: our factor structure is used to organize evidence and make derivations auditable, not as a claim that risk is the sum of four terms. Related critical work on the evolution of human error thinking, including the limits of the new view and of the Swiss cheese model, sets the same expectation that a factor-based scheme will be argued for rather than assumed ^{[read2021; lecoze2022; larouzee2020; hollnagel2009]}.

2.1.4 The gap is measurement, not taxonomy#

Every framework surveyed above is analyst-rated. SPAR-H, CREAM, NUREG-1792 and IDHEAS-G all require a trained analyst to read a scenario, perform a task analysis, and assign a level to each factor. None specifies how to derive factor levels from data an industrial operator already collects: shift rosters, permit-to-work records, CMMS work orders, MES and ERP logs, or the OSHA 300 and 301 case records themselves. That is a measurement problem, and it is where a contribution is available.

Three obstacles stand in the way, all documented in prior work.

The first is the exposure denominator. All HRA presupposes a well-defined count of opportunities. Hanecke and colleagues met this directly, analysing more than 1.2 million German accident records for 1994 with no working-time data available at national level, and were obliged to construct and compare estimated exposure models; they report accident risk rising exponentially beyond the ninth hour at work, together with an interaction between time of day and hours worked ^{[hanecke1998]}. This is the methodological template for the problem, and it is nearly thirty years old. Our own prior analysis of OSHA Injury Tracking Application establishment filings documents a US instance of the same failure in the reported denominator rather than in its absence ^{[chimmani2026]}. We restate the provenance of that work precisely in Section 4, because the numbers come from Form 300A summary filings and not from the case-detail files, and because the figures are sensitive to the panel and screen used.

The second is temporal resolution. PSF frameworks assign static per-scenario ratings and none represents time on task as a continuous covariate. The gap here is between HRA and chronobiology rather than in the literature as a whole. Elevated accident risk in the early hours of a shift is a named and long-published phenomenon, the two-to-four hour shift phenomenon ^[tucker2000], and Folkard and Tucker report that once exposure is accounted for, risk rises approximately exponentially with time on shift, more than doubling by the twelfth hour relative to the first eight, with a slight elevation from the second to the fifth hour ^{[folkard2003]}. Any observation of an early-shift peak in raw incident counts is a restatement of a known effect, and, more importantly, a count distribution is not a rate. We are explicit about this distinction in Section 4 and we do not present unadjusted counts as risk. Public data now permits the question to be asked on three independent sources, since OSHA case-detail filings carry the time work started and the time of incident ^{[osha_ita_case_detail_dict_2026]}, MSHA accident records carry shift begin time and accident time ^{[msha_accidents_definition]}, and the BLS case-characteristics series publishes an hours-worked-before-event dimension ^{[bls_ca_documentation]}. None of the three supplies hours at risk by shift hour, which is the denominator the question actually needs.

The third is the outcome variable and what may be claimed from it. Zero inflation and censoring are properties of establishment-level injury counts that no PSF framework accommodates. Beyond that, the safety analytics literature has been candid about the state of predictive validity. Poh and colleagues state in their own abstract that validated leading indicators capable of reliably classifying sites by risk level are lacking ^[poh2018]. Hallowell and colleagues find the four predictive families, risk assessment, precursor analysis, leading indicators, and safety climate, modelled independently of one another with no cross-validation between them ^{[hallowell2020]}. Xiong and colleagues, reviewing eighty-four construction SEM applications, report recurring problems with construct validity and sample size ^[xiong2015]. Baker, Hallowell and Tixier rebuilt an earlier prediction study using independent human annotation specifically to remove artificial correlation between predictors and predictands, which is a documented instance of target leakage inside safety machine learning, corrected by the original authors ^{[baker2020ai]}; the general phenomenon is catalogued across seventeen fields by Kapoor and Narayanan ^[kapoor2023]. Direction of effect is also contested: Beus and colleagues examined both directions between safety climate and injuries and report the injuries-to-later-climate relationship as the stronger one, which unsettles the arrow assumed in most safety SEM ^[beus2010]. We rely on secondary summaries for that directional finding and flag it as such rather than quoting effect sizes we have not read in the original.

Two further constraints govern what a data-derived factor score could legitimately claim. Calibration, not discrimination, is the property that matters when a score triggers action, and it is routinely omitted ^{[vancalster2019; vancalster2016; steyerberg2010; collins2015tripod]}. And the common remedies for class imbalance degrade calibration without improving discrimination, in logistic regression ^{[goorbergh2022]} and in machine learning models alike ^{[carriero2025]}. At a private industry total recordable case rate of 2.3 per 100 full-time equivalent workers ^[bls2026], a shift-level event is rare enough that these are not fine points. Hopkins reached a compatible conclusion twenty years ago without any of this machinery, arguing that the leading versus lagging distinction is largely beside the point and that what matters is whether, at the level of aggregation in question, enough countable events occur to speak of a rate at all ^{[hopkins2009]}. The process safety indicator standards are consistent with him: HSG254 defines leading indicators as active monitoring, a systematic check that intended actions are being carried out, paired with a lagging indicator per critical risk control system ^{[hse2006_hsg254]}, and API RP 754 places operating discipline and management system performance at Tier 4 within a tier structure whose precursor-to-major-accident premise API's own material describes as a belief ^{[api754_4th; ccps2009]}. Empirical tests of that premise at scale are few ^{[yorio_moore2018; moore2020heinrich]}.

2.2 Automated reasoning, neurosymbolic architectures, and terminology#

2.2.1 What the term means to the field that owns it#

Automated reasoning has a narrow technical meaning. Formal methods is the broad term, the use of logic to reason about models of systems; automated reasoning is the automation of formal methods. That hierarchy is stated by the head of AWS's automated reasoning organization ^{[cook2021blog]} and matches the encyclopedia treatment, which describes the field as giving an algorithmic description to a formal calculus ^[sep_ar]. Three commitments are load-bearing: a formal language with specified semantics, an inference procedure sound with respect to those semantics, and a characterized decidability and complexity result. Systems that lack all three are not doing automated reasoning under any definition the field recognizes.

The branches decide different questions and are not interchangeable. Propositional satisfiability descends from DPLL ^[davis1962] and became industrially usable with Chaff-era engineering ^{[moskewicz2001; biere2021handbook]}. Satisfiability modulo theories adds background theories such as arithmetic, arrays and strings ^{[demoura2008; barbosa2022; barrett2021smt]}. First-order theorem proving by resolution and superposition traces to Robinson ^{[robinson1965]} and is represented by systems such as Vampire ^[kovacs2013]; interactive proof assistants occupy a distinct position where automation is partial ^{[demoura2015lean; nipkow2002]}. Model checking decides whether a finite-state model satisfies a temporal-logic property ^[clarke1986]. Description logics are decidable fragments of first-order logic whose standard tasks are subsumption, consistency, classification and instance checking ^[baader2007]; OWL 2 DL corresponds to SROIQ under Direct Semantics, with EL, QL and RL profiles trading expressivity for tractability ^{[w3c_owl2; grau2008]}, and is served by reasoners such as HermiT ^{[motik2009; glimm2014]} and the polynomial-time EL classifier ELK ^{[kazakov2011]}. Answer set programming provides non-monotonic reasoning under stable-model semantics ^{[brewka2011; gebser2011]}. Deployed instances of the guarantee this machinery buys include Astrée on Airbus flight control code ^{[blanchet2003]}, the seL4 kernel proof ^[klein2009], and CompCert ^[leroy2009], with AWS's Zelkova encoding IAM policy semantics into SMT ^{[backes2018; cook2018cav]}, continuous proof obligations in the s2n TLS library ^{[chudnov2018]}, and TLA+ specification of storage services ^{[newcombe2015; brooker2025]}.

2.2.2 Two claims we do not make#

The first is that rule evaluation determines cause and effect. Deductive systems compute entailment. Material implication constrains models and licenses contraposition, which causation does not; the formalism for cause and effect is the structural causal model, and the ladder of association, intervention and counterfactual places rule evaluation on the first rung ^{[pearl2019; pearl2009]}. Actual causation has a structural-equation definition, not a deductive one ^{[halpern2005; halpern2016]}. The strongest paper in the process-safety ontology literature makes the same move implicitly: Aziz, Ahmed and Khan capture knowledge in OWL and then transform it into a multi-entity Bayesian network for the causal step, validated against forty-five US Chemical Safety Board events ^[aziz2019].

The second is that automated reasoning differs from machine learning by being logical rather than pattern-matching. Rete, the algorithm inside essentially every production rule engine, is titled as a pattern match algorithm by its author ^[forgy1982]. Modern SAT solvers use learned branching heuristics ^[liang2016], premise selection in theorem proving has used deep networks since DeepMath ^[alemi2016], and networks have been trained to solve SAT directly ^[selsam2019]. The defensible distinction is different and narrower: automated reasoning systems carry a proof obligation and can emit a checkable certificate, whether a model, a counterexample, an unsatisfiable core or a proof object, while statistical systems produce a score with no certificate. That is the distinction this paper uses.

Accordingly we name our components by what they are. Where reasoning is declarative, terminating, bottom-up rule evaluation with least-fixpoint semantics, the accurate term is Datalog or deductive database ^[green2013]. Where it is forward-chaining production rules, the accurate term is a production rule system. Where it is SPARQL over a graph, the accurate term is structured retrieval, not reasoning. We note that even published safety ontologies blur this: competency questions answered by ontology query are retrieval rather than entailment ^{[single2020_ijsse]}. A related architectural point constrains design. OWL's semantics are open-world and monotonic, while regulatory safety rules are frequently closed-world and defeasible, and the usual repair of attaching SWRL rules forfeits decidability, as the SWRL submission itself states ^[swrl2004]. Answer set programming is the better-founded choice where defaults and exceptions are required ^[brewka2011].

2.2.3 Formal knowledge models in process safety#

The relevant domain literature is real, coherent and small. Automated HAZOP by signed digraph modelling of causal propagation dates to Vaidhyanathan and Venkatasubramanian in the mid-1990s ^{[vaidhyanathan1995]}, and the three-part fault detection and diagnosis review supplies a discipline-native taxonomy of quantitative model-based, qualitative model-based and process-history-based methods that removes any need to borrow the term automated reasoning at all ^{[venkatasubramanian2003]}. OntoCAPE is the substantial domain ontology for chemical process engineering ^{[marquardt2010]}. The modern ontology-for-HAZOP work is the Single, Schmidt and Denecke cluster: a review of thirty years of HAZOP automation which reports that very few approaches have been used in the chemical process industry and that the quality of the required specific and generic knowledge is often insufficient ^[single2019]; an ontology-based computer aid that models causal relations between HAZOP concepts in order to suppress meaningless generated scenarios, validated on a hexane storage tank ^{[single2020_jlp]}; a competency-question-driven ontology applied to a gasoline storage tank ^{[single2020_ijsse]}; and an ontology-plus-NLP pipeline for extracting structured knowledge from chemical accident databases ^{[single2020_ssci]}. Comparable architectures appear in construction safety ^[ding2016]. Validation in this literature is at the scale of a single vessel or unit, with Aziz and colleagues at forty-five events as the outlier ^[aziz2019]. No work we located formalizes OSHA recordkeeping; the nearest intellectual ancestor is the formalization of the British Nationality Act as a logic program, including its isomorphism principle of keeping the formalization traceable to the statutory text clause by clause ^[sergot1986].

2.2.4 The formalization bottleneck#

Deciding entailment is largely solved and industrial. Getting the axioms right is not. Autoformalization, translating informal statements into formal ones, was framed as an LLM task by Wu and colleagues ^[wu2022]. Logic-LM is the canonical pattern of LLM translation followed by deterministic solving with solver errors driving refinement, and reports substantial gains over chain-of-thought prompting across logical reasoning datasets ^{[pan2023logiclm]}. The cautionary result is LINC, which found LLM-only and LLM-plus-prover pipelines succeeding at roughly equal rates on average while failing in distinct and complementary ways, meaning that attaching a prover relocates errors into the translation step rather than removing them ^{[olausson2023linc]}. Ganguly and colleagues measure SMT autoformalization accuracy swinging from a large gain on logical tasks to a large loss on factual ones, with token-entropy uncertainty measures failing to flag the errors; that work is a preprint and we treat it as such ^{[ganguly2025]}. AWS's Bedrock Automated Reasoning checks make the epistemic boundary explicit in vendor documentation, splitting translation, which uses foundation models and may contain errors, from validation, which uses SMT solvers and is described as mathematically sound, and scoping the guarantee to the parts of an input captured by policy variables ^{[aws_arc_docs; aws_what_is_ar]}. Those are primary sources for what AWS claims and how the system is arranged, not evidence that the claims hold.

The neurosymbolic literature supplies vocabulary for saying precisely which architecture a system implements ^{[kautz2022; hitzler2022; garcez2023]}, and the sceptical literature supplies the reason to use that vocabulary carefully: transformer performance on compositional tasks degrades with problem depth in a manner consistent with pattern matching over subgraphs ^[dziri2023], chain-of-thought text can misrepresent the actual cause of an output ^[turpin2023], and the ascription of reasoning to language models has been challenged directly in the peer-reviewed literature ^{[kambhampati2024]}. The formal methods community's own survey of its practice and terminology, and recent systematic work on AI applied inside formal methods, complete the picture ^{[garavel2020; stock2025]}.

Two literatures, the neurosymbolic one and the process-safety ontology one, have converged on the same bottleneck, knowledge acquisition and faithful formalization, without citing each other. We are not aware of prior work bridging them.

2.3 Grounding, hallucination, and retrieval evaluation#

2.3.1 Grounding has a technical definition#

Retrieval-augmented generation is defined by a parametric generator combined with a non-parametric retrieved memory, with the model marginalizing over retrieved documents ^[lewis2020]. A system prompt plus pasted document text is not this. It has no retriever, no index and no marginalization; it is parametric conditioning, which shifts the prior without supplying evidence. This distinction determines what may be concluded from any anecdote about such a system.

Hallucination taxonomies distinguish intrinsic from extrinsic, and factuality from faithfulness ^{[maynez2020; ji2023; huang2025]}. Retrieval reduces hallucination; the earliest direct evidence framed the result as reduction and not elimination ^{[shuster2021]}.

2.3.2 Why semantically adjacent substitution is expected#

Four mechanisms converge, and each is separately supported. Parametric factual accuracy scales with pretraining frequency, so long-tail technical content is where parametric memory fails ^{[kandpal2023]}, and it is also where retrieval delivers its largest gain ^[mallen2023]. Training and benchmark scoring reward a plausible guess over an abstention, so under uncertainty a model emits its most probable plausible completion rather than declining ^[kalai2025]; that work is a preprint. Distributional representations encode relatedness rather than relation type: antonyms sit close in embedding space and require explicit external constraints to separate ^[mrksic2016], supervised distributional methods learn prototypical hypernyms rather than the relation holding between a specific pair ^[levy2015], and hypernymy does not separate cleanly from other tight lexical relations by distributional means ^{[shwartz2017]}. On the retrieval side, neural retrievers largely fail minimal pairs differing only by negation ^{[weller2024nevir]}, so a retriever cannot be relied on to enforce a device-type constraint as a hard requirement rather than a soft preference. Term-frequency and attestation effects have been traced as sources of hallucination on inference tasks ^{[mckenna2023]}, and sycophancy compounds the problem when a user states a premise ^{[sharma2024; dahl2024]}.

The domain adds a fourth mechanism that is specific to engineering standards. Pressure relief valves and rupture disks are taxonomic siblings under a single parent category in the ASME code, with rupture disks classed as nonreclosing pressure relief devices ^{[asme_viii_ug127]}. API 520 Part I covers both device types in one document with a shared definitions section, and the word disc is polysemous inside it, naming the closure element of a valve in one place and an entire device in another ^[api520p1]. ISO, by contrast, separates the two across parts of ISO 4126 ^[iso4126]. Which standards family is indexed is therefore a corpus design decision with safety consequences. The parameter spaces do not intersect: burst pressure tolerance, manufacturing design range and specified disk temperature have no valve analogue ^{[nationalboard_burst]}, so a substituted answer is not approximately correct but categorically inapplicable while remaining fluent.

2.3.3 Grounding reduces error, it does not remove it#

The governing counterexample to any elimination claim is the preregistered evaluation of commercial legal research tools marketed on the absence of hallucination, which measured hallucination between 17 and 33 percent of the time ^[magesh2025]. Related profiling work in the same domain reports substantially higher rates for ungrounded models and finds that models poorly predict their own hallucinations and accept incorrect user premises ^[dahl2024]. Engineering taxonomies enumerate where retrieval pipelines break ^{[barnett2024]}; benchmark work shows weakness in noise robustness and negative rejection ^{[chen2024rgb]}; irrelevant retrieved context degrades accuracy, so naive top-k retrieval that returns sibling-device passages can make a grounded system worse ^{[shi2023distracted]}; position within a long context affects whether retrieved evidence is used ^{[liu2024lost]}; and behavior is unstable when retrieved evidence conflicts with parametric belief ^{[xie2024; longpre2021]}. Architectural responses include adaptive retrieval with self-critique ^{[asai2024selfrag]}, context-aware decoding ^[shi2024cad], and context-faithful prompting ^{[zhou2023contextfaithful]}. Retriever choice is itself an open variable, since dense retrieval generalizes poorly zero-shot and lexical baselines often win out of domain ^{[thakur2021beir; karpukhin2020dpr; robertson2009bm25; khattab2020colbert]}.

2.3.4 How grounding is measured#

Attribution has a peer-reviewed framework in AIS ^{[rashkin2023]}, with citation precision and recall as an automatic complement ^{[gao2023alce]} and atomic-fact decomposition for long-form answers ^{[min2023factscore]}. Automatic attribution evaluation is itself unreliable ^[yue2023], and the precedent for validating a metric against human judgment before trusting it is established ^{[honovich2022true]}, which places reference-free LLM-judged metrics in a screening role ^{[es2024ragas; saadfalcon2024ares]}. Benchmark design precedents include provenance-annotated task items ^{[petroni2021kilt]}, three-way supported/refuted/insufficient labelling with required evidence ^{[thorne2018fever]}, adversarial items where the statistically likely answer is wrong ^{[lin2022truthfulqa]}, contrast sets of minimally perturbed items ^{[gardner2020]}, capability-organized behavioral testing ^{[ribeiro2020checklist]}, and paired scoring of minimal pairs ^{[weller2024nevir]}. Calibration and selective prediction supply the safety-relevant outcome, since the argument is not that errors are fewer but that they announce themselves ^{[guo2017; desai2020; jiang2021; kadavath2022]}, with semantic entropy as an uncertainty signal invariant to paraphrase ^{[kuhn2023; farquhar2024]}, sampling-consistency detection as a black-box baseline ^{[manakul2023]}, and conformal methods as a route to a distribution-free bound ^[mohri2024]. The closest existing analogue to what we build is MIRAGE, a purpose-built RAG benchmark for a safety-critical regulated domain, which is medical ^{[xiong2024mirage]}.

2.3.5 Language models in process safety#

The process safety community is publishing actively in this space, and the existing work evaluates generation quality rather than grounding. Lee and colleagues had four multimodal models generate HAZOP worksheets from a P&ID against an expert reference and conclude that expert oversight remains necessary ^{[lee2026hazop]}. Charalampidou and colleagues reconstructed a verified STPA on a drone system with GPT-4 and report that a substantial share of generated unsafe control actions required correction ^{[charalampidou2024]}. Kaya and colleagues extend this to STPA and FRAM ^[kaya2025]. Dokas reviews LLM benchmarks against what hazard analysis actually requires, finds them inadequate, and proposes performance consistency as a metric, which is the precedent for treating run-to-run stability as an outcome rather than a nuisance ^[dokas2026]. Elhosary and Moselhi present knowledge-augmented HAZOP report generation, the closest published work to grounded LLM use in process safety specifically ^{[elhosary2026]}. Adjacent engineering domains supply comparison points in regulatory code compliance by fine-tuning ^[shi2025aei] and staged retrieval for construction safety compliance ^{[guo2025autcon]}. None of these performs a controlled grounded-versus-ungrounded comparison on standards-lookup questions, and none uses minimal pairs.

2.3.6 Protocols standardize access, not authority#

The Model Context Protocol is a JSON-RPC protocol with hosts, clients and servers exposing resources, prompts and tools. It standardizes how a model reaches a source and makes that binding auditable and swappable, which has real value in a regulated deployment. It does not establish that a source is authoritative, does not guarantee the retrieved passage is the right one, and does not prevent a model from ignoring what it retrieved. The specification locates trust outside itself, stating that tool descriptions should be considered untrusted unless obtained from a trusted server ^{[mcp2025spec]}, and adopting it introduces its own threat surface ^{[narajala2025mcp]}. The tool-use lineage it standardizes at the transport layer runs through ReAct and Toolformer ^{[yao2023react; schick2023toolformer]}. Governance expectations for a regulated-industry audience are set by the NIST AI Risk Management Framework, whose Measure function is where an evaluation of this kind sits ^[nist_airmf].

2.4 What remains open#

Reading the three strands together produces a short list of things nobody appears to have done, and it is a narrower list than the practitioner literature suggests.

The four contextual dimensions are not among them. That structure is IDHEAS-G's, and the honest framing is adoption with attribution.

What is open is measurement and evaluation. No published mapping derives PSF levels from routinely collected industrial administrative data at population scale; every framework surveyed is analyst-rated. No formalization of OSHA recordkeeping exists, and no safety ontology has been validated against a large population of real regulatory filings rather than a single vessel. Data plausibility has not been framed as a constraint-satisfaction problem, which is the framing that would yield the artifact a formal-methods reviewer asks for, an unsatisfiable core naming the constraints a filing violates. There is no public benchmark for overpressure protection or pressure-relief device selection, no benchmark anywhere for co-hyponym device-type discrimination in engineering standards, and essentially no study of the pseudo-grounded condition, that is, retrieval over a plausible but unauthoritative corpus, which is where most enterprise deployments actually sit and where retrieval may perform worse than none by lending false support. Existing hallucination taxonomies are organized by source relation rather than by consequence severity, which averages a wrong device type together with a wrong citation year. And there is no calibration or abstention study for clause-level engineering standards question answering.

Sections 3 through 6 address a subset of these. We are explicit in each case about which claims the artifacts establish and which they do not.

3. Problem Formalization#

This section states the two problems the paper addresses in a form precise enough to be argued with. Section 3.1 fixes notation. Section 3.2 defines semantically-adjacent substitution and separates it from the general notion of hallucination. Section 3.3 shows why the natural evaluation statistic for it is paired rather than per-item accuracy. Section 3.4 defines the grounding property we want and proves that it does not, by itself, exclude the error class of Section 3.2. Section 3.5 defines an auditable derivation trace and says exactly which guarantee it carries and which it does not. Section 3.6 states the second problem, instrumentation of context factors from routinely collected data. Section 3.7 lists the assumptions the rest of the paper depends on. Section 3.8 states what is deliberately left unformalized.

Throughout, we are careful to claim only properties that a reader can check against the artifacts. Where a property is a design commitment rather than a theorem, it is labelled as such.

3.1 Notation and setting#

Questions and answers. Let Q be a set of technical questions posed in natural language and A a set of natural-language answers. A system is a function f : Q → A ∪ {⊥}, where ⊥ denotes an explicit refusal or statement of non-knowledge. We treat abstention as a distinguished output rather than as a degenerate answer, because in a safety setting its utility differs in sign from a confident wrong answer, and because scoring regimes that give abstention zero credit are part of the reason systems guess ^[kalai2025].

Authoritative corpus. Let

D = { d₁, …, d_N },   dᵢ = ( idᵢ , textᵢ , metaᵢ )

be a finite set of clause-addressed passages drawn from published standards and regulations. idᵢ is a resolvable clause identifier - 29 CFR 1910.147(c)(6), 46 CFR 54.15-13(b)(3), ISO 4126-2, ASME BPVC VIII-1 UG-127 - not a chunk offset. metaᵢ carries at minimum the issuing body, the edition or eCFR snapshot date, and (where applicable) the entity type the passage governs. The requirement that idᵢ be a clause identifier rather than a byte range is load-bearing: it is what makes provenance checkable by a third party, and it follows the provenance-annotation design of KILT ^{[petroni2021kilt]}.

Membership in D is a curation decision, not a property discovered at query time. A retriever operating over vendor literature, a corporate wiki, or an executive's own website is retrieving; it is not retrieving from D.

Entity taxonomy. Let T = (E, ⊑) where E is a set of domain entities - device types, procedures, hazard classes, regulatory categories - and ⊑ is the subsumption relation asserted by the governing standards. Write

e ~ e'   iff   ∃ p ∈ E .  e ⊏ p  ∧  e' ⊏ p  ∧  e ≠ e'  ∧  e ⋢ e'  ∧  e' ⋢ e

and call e, e' co-hyponyms: taxonomic siblings under a common parent, with neither subsuming the other. T is adopted from the standards, not invented here. Rupture disks and spring-loaded pressure relief valves are co-hyponyms under "pressure relief device" because ASME BPVC Section VIII Division 1 places them there, with rupture disk devices classed as non-reclosing ^{[asme_viii_ug127]}; API 520 Part I documents both device types within one standard under a shared definitions section ^[api520p1]; ISO instead splits them across separate parts, safety valves in 4126-1 and bursting disc safety devices in 4126-2 ^[iso4126].

Attribute schemas. Each entity e ∈ E carries an attribute schema Attr(e) ⊆ Π, the set of parameters predicable of it, where Π is the union of all domain parameters. Define the signature of e relative to a sibling e' as

Sig(e | e') = Attr(e) \ Attr(e')

the parameters that belong to e and are not predicable of e' at all.

For the motivating pair, the schemas are disjoint on every parameter that governs a sizing or installation decision. A pressure relief valve is characterized by set pressure, blowdown, accumulation, superimposed and built-up backpressure, effective discharge area, coefficient of discharge, lift, and reseating pressure - the last of which presupposes that the device recloses ^[api520p1]. A rupture disk is characterized by marked burst pressure, burst pressure tolerance, manufacturing design range, specified disk temperature, lot, and operating ratio, and it does not reclose ^{[nationalboard_burst]}. Formally, for these two entities

Attr(rupture disk) ∩ Attr(relief valve) = ∅   on the sizing-relevant parameters,

so Sig(e | e') = Attr(e) for both members of the pair. No arithmetic maps a correct answer about one onto a correct answer about the other. This is what distinguishes the error class from an ordinary approximation error, and it is a property of the domain rather than of any model.

Subject and asserted entity. For q ∈ Q let subj(q) ∈ E be the entity the question is about, fixed by the question's construction. For an answer a ∈ A let Assert(a) ⊆ Π be the set of parameters the answer predicates of subj(q). The distinction between predicating and mentioning matters: an answer that says "a rupture disk has no blowdown, because it does not reclose" mentions blowdown without asserting it. In the implemented scorer this distinction is made syntactically, by detecting contrast constructions around a matched term, and it is reported alongside the terms that fired so that a human adjudicator can overrule it.

3.2 Semantically-adjacent substitution#

Definition 1 (semantically-adjacent substitution). Let q ∈ Q with subj(q) = e, and let a = f(q) ∈ A. The response a is a semantically-adjacent substitution with respect to q iff there exists e' ∈ E such that

1. (adjacency) e ~ e' under T;
2. (signature assertion) Assert(a) ∩ Sig(e' | e) ≠ ∅;
3. (non-contrast) the parameters in (2) are predicated of e, not denied of it;
4. (surface plausibility) a is fluent, internally consistent, and correctly formatted for the question type - it passes the checks a reader applies before checking the content.

Write SAS(q, a) when these hold.

Conditions (1)–(3) say the system answered about the wrong entity. Condition (4) is what makes the error a safety problem rather than a nuisance. An answer that is obviously malformed is caught by the reader; an answer that is well-formed and wrong is caught only by a reader who already holds the domain model.

Relation to existing hallucination taxonomies. In the intrinsic/extrinsic split ^[maynez2020] and the factuality/faithfulness split ^{[ji2023; huang2025]}, SAS is extrinsic: the content is not contradicted by any single retrieved passage, it is simply unverifiable as an answer to the question actually asked. SAS is a proper subclass, and the subclass matters because the existing taxonomies are organized by the relation between output and source, not by the consequence of the error. Under a flat hallucination rate, substituting the wrong device type and misciting an edition year contribute equally. In a process-safety setting they do not. We therefore report SAS as a separate outcome category rather than folding it into an aggregate error rate.

Why the class is over-determined rather than incidental. Four mechanisms push toward SAS independently, and they compose.

(i) Absence of retrieval. A system consisting of a persona prompt plus pasted reference text has no index, no retriever, and no marginalization over retrieved documents. Under the definition of retrieval-augmented generation as a parametric generator combined with a non-parametric memory ^[lewis2020], such a system has non-parametric memory of size zero. Every token comes from pretraining. Calling it grounded, or a digital twin, misdescribes it.

(ii) Long-tail sparsity. Parametric factual accuracy tracks how often a fact appears in pretraining ^{[kandpal2023]}, and parametric memory degrades on low-popularity entities precisely where retrieval would help most ^[mallen2023]. Relief-valve content is abundant on the open web; rupture-disk content is comparatively sparse. A query landing in the sparse region returns the dense neighbour.

(iii) Penalized abstention. Training objectives and benchmark scoring reward a plausible guess over ⊥ ^[kalai2025]. Under uncertainty the system emits its highest-probability plausible completion, which for a question about overpressure protection is relief-valve content.

(iv) A lexically adversarial corpus. Distributional representations encode that two terms are related, not how they are related. Antonyms sit close in embedding space and require external constraints to separate ^[mrksic2016]; supervised distributional methods learn prototypical hypernyms rather than the relation holding between a specific pair ^[levy2015]; hypernymy is not cleanly separable from other tight lexical relations by distributional means ^{[shwartz2017]}. Co-hyponyms are the maximal-confusion case under this account, because siblings share the sentences, tables, and regulatory verbs that drive similarity. The retrieval side shows the same limit: neural retrievers largely fail minimal pairs differing only by negation ^{[weller2024nevir]}, and a retriever that cannot represent not cannot enforce this device, not that one as a hard constraint. The domain then makes it worse. Within API 520 Part I the token disc/disk is polysemous: in a relief valve the disc is the closure element whose travel defines lift and whose contact with the seat defines reseating, while a rupture disk is an entire device ^[api520p1]. Chunk-and-embed an index over that document and the nearest neighbours of a rupture-disk passage include relief-valve passages.

Mechanism (iv) yields a design consequence that we state as a claim about engineering practice rather than a theorem: which standards family is indexed is a safety-relevant corpus design decision, because ISO separates the two device types across documents while ASME and API co-locate them ^{[iso4126; api520p1; asme_viii_ug127]}.

Minimal pairs. A minimal pair is (q, q') with subj(q) = e, subj(q') = e', e ~ e', and question stems identical except for the entity named and any parameter term entailed by that substitution. This is the contrast set construction ^{[gardner2020]} specialized to co-hyponyms, scored under the paired discipline of NevIR ^{[weller2024nevir]}. Lexical overlap between members is near-total by construction, so a system that succeeds by topical similarity fails on pairs systematically and visibly.

3.3 Why the reported statistic must be paired#

Let P = { (q₁, q₁'), …, (q_n, q_n') } be a set of minimal pairs and let corr(a) be the indicator that a satisfies the item's answer key. Define

per-item accuracy      Acc  = (1 / 2n) · Σᵢ [ corr(aᵢ) + corr(aᵢ') ]
paired accuracy        PA   = (1 /  n) · Σᵢ [ corr(aᵢ) ∧ corr(aᵢ') ]

Proposition 1. PA ≤ Acc, with equality iff every pair is answered either wholly correctly or wholly incorrectly. Moreover, let σ be any entity-blind policy: one whose output distribution is independent of subj(q) given the stem. On a pair set in which the two members have disjoint answer keys (Section 3.1), σ achieves PA(σ) = 0 while Acc(σ) may be as high as 1/2.

Proof. The first claim is immediate from 1[x ∧ y] ≤ (x + y)/2 for indicators. For the second, disjointness of the keys means no single response satisfies both members' keys; an entity-blind policy produces responses whose entity content is identical across the pair, so at most one member can be scored correct, giving corr(aᵢ) ∧ corr(aᵢ') = 0 for every i. A policy that always answers about the denser sibling scores correct on the half of the items whose subject is that sibling. ∎

Proposition 1 is trivial mathematics with a non-trivial consequence: per-item accuracy assigns partial credit to exactly the behaviour the benchmark exists to detect, and paired accuracy assigns none. We therefore take PA as the primary statistic and report Acc only alongside it. The same reasoning applies to the analysis: minimal pairs and cross-arm items are paired observations and must be analyzed with paired tests, not with independent-samples comparisons of arm-level accuracy.

Category-error items. A separate item class carries a false premise - the blowdown of a rupture disk, the set pressure of a bursting disc. For these the correct output is ⊥ together with a rejection of the premise. They measure negative rejection in the sense of RGB ^{[chen2024rgb]} and are scored and reported separately, never folded into Acc or PA. Folding them in would let a system that answers everything outscore a properly calibrated one, inverting the safety objective.

3.4 The grounding property#

We want a property strong enough to be worth asserting and weak enough to be true. We build it from attribution rather than from correctness, because attribution is checkable against a document while correctness is not.

Claim decomposition. An answer to a standards question bundles several independently right-or-wrong assertions: the entity, the governing clause, the numeric value, the applicable edition. Let C(a) = {c₁, …, c_m} be the atomic claims of a under a stated decomposition procedure ^{[min2023factscore]}, and let cite(a) ⊆ D be the passages the answer cites.

Attribution. Write d ⊨ c when a generic reader would agree that according to d, c - the AIS relation ^{[rashkin2023]}. Citation precision and recall in the ALCE sense ^{[gao2023alce]} are secondary summaries of the same relation.

Definition 2 (γ-grounded response). For γ ∈ (0, 1], a response a is γ-grounded in D iff

• G1 (support) |{ c ∈ C(a) : ∃ d ∈ cite(a) . d ⊨ c }| / |C(a)| ≥ γ;
• G2 (provenance) every d ∈ cite(a) lies in D and resolves to a clause identifier with a recorded edition or snapshot date;
• G3 (anchor locality) for each supported c, the span of d that supports c lies inside the paragraph d actually cites, not merely somewhere in the same section;
• G4 (declared scope) a carries Covered(a) ⊆ C(a), the set of claims for which a governing passage was retrieved, and asserts nothing about C(a) \ Covered(a);
• G5 (abstention under non-coverage) if Covered(a) = ∅ then a = ⊥.

G3 is not decoration. In building the benchmark corpus, an item cited 29 CFR 1910.157(d) while anchoring on text that lives in (e)(2); it passed a verifier that checked only whether the cited paragraph markers appeared somewhere in the section. The defect was invisible until a locality check was added. A grounding property without G3 admits citations that are real, plausible, and pointing at the wrong paragraph.

G4 follows the scoping discipline stated in AWS's Automated Reasoning checks documentation, where a VALID result "covers only the parts of the input captured through policy variables" and untranslated content is simply unvalidated ^{[aws_arc_docs]}. A guarantee whose scope is not declared is not a guarantee.

What Definition 2 does not give us. Grounding as attribution is orthogonal to answering the right question.

Proposition 2 (attribution does not exclude substitution). There exist q, a with SAS(q, a) and a 1-grounded in D under Definition 2 with Covered(a) = C(a).

Proof by construction. Let subj(q) be a rupture disk and let a state the set pressure, blowdown, and reseating behaviour of a spring-loaded pressure relief valve, citing the passages of API 520 Part I that define those terms ^[api520p1]. Every atomic claim of a is attributable to a cited passage; every citation is in D, resolves to a clause identifier, and anchors inside the cited paragraph. So G1–G5 hold with γ = 1. Yet Assert(a) ∩ Sig(relief valve | rupture disk) ≠ ∅ and the parameters are predicated of the rupture disk, so SAS(q, a). ∎

Proposition 2 is the formal core of the paper's architectural argument. Every claim can be true of the document cited, and the answer can still be categorically inapplicable to the question, because the failure is one of entity scope rather than of support. Retrieval quality does not fix it: the retrieved passages are genuinely relevant to the topic, and irrelevant-in-scope passages retrieved alongside relevant ones measurably degrade generation ^{[shi2023distracted]}, so naive top-k over a co-locating standard can make the grounded condition worse than the ungrounded one. We therefore add a separate requirement.

Definition 3 (entity-faithful response). a is entity-faithful to q iff ent(a) = subj(q), where ent(a) is the unique e ∈ E such that Assert(a) ⊆ Attr(e) when such an e exists, and undefined otherwise.

The property we actually want is the conjunction: γ-grounded ∧ entity-faithful. Definition 3 is what motivates a hard metadata filter on entity type at retrieval time rather than a better embedding, since the distributional evidence above says the representation does not encode the distinction we need it to enforce.

What we do not claim. Grounding reduces this error class; it does not eliminate it. Commercial retrieval-augmented systems in a regulated domain, marketed as hallucination-free, were measured under preregistration at 17–33 percent hallucination ^[magesh2025]. Models behave unstably when retrieved evidence conflicts with parametric belief ^[xie2024], and will override retrieved evidence with memorized entities ^{[longpre2021]}. Any claim in this paper is a measured delta with an interval under stated conditions. No sentence asserts that grounding prevents or guarantees against substitution.

Selective prediction. Because the safety objective is not "fewer errors" but "errors that announce themselves," the summary statistic is a risk-coverage curve: accuracy as a function of the fraction of items answered, with ⊥ as the abstention. Report accuracy at a stated precision together with the coverage it costs. Semantic entropy is the natural uncertainty signal here ^{[kuhn2023; farquhar2024]} because the wrong answer in this setting is fluent and paraphrase-stable, so token-level entropy would miss it.

3.5 Auditable derivation traces#

The second architectural property concerns conclusions the system derives rather than passages it retrieves. We state it over a deterministic rule evaluator and are deliberate about the term used.

The formalism, named precisely. Let P be a finite set of rules

h(x̄)  ←  ℓ₁, …, ℓ_k

over a function-free first-order signature, where each ℓⱼ is a positive or negated atom or a built-in comparison, and P is stratified: for every positive body dependency the head's stratum is at least the body's, and for every negative dependency it is strictly greater. Let F₀ be a finite set of ground extensional facts. Because the signature has no function symbols, the Herbrand base is finite, so bottom-up evaluation to a fixpoint per stratum terminates; and for a stratified program the perfect model M(P, F₀) is unique and independent of rule order. Data complexity for a fixed program is polynomial. These are the standard properties of Datalog with stratified negation ^[green2013].

This is deterministic rule evaluation with a definite model-theoretic semantics. It is not automated reasoning in the sense the term carries in computer science, where the commitments are a formal language with specified semantics, an inference procedure sound with respect to that semantics, and a characterized decidability and complexity result ^{[cook2021blog; sep_ar]}. We claim the first and third, and we claim soundness only relative to the program's own semantics - never relative to a semantics of the world. We also decline the weaker but commoner move of calling a production-rule system a reasoner: Rete, the algorithm underneath most such systems, is by its author's own title a pattern-matching algorithm and carries no completeness theorem relative to a logical semantics ^[forgy1982].

Why Datalog and not OWL. Description logics give decidable entailment over an open-world, monotonic semantics ^{[baader2007; w3c_owl2]}, with polynomial-time classification available in the EL profile ^{[kazakov2011]} and production reasoners for OWL 2 DL ^{[motik2009; glimm2014]}. Two properties of the safety domain argue against that choice here. First, the rules are defeasible: exceptions, and exceptions to exceptions, are the normal shape of regulatory text. Second, they are closed-world in places: no recorded exposure must sometimes be read as none occurred, which open-world semantics will not license. The standard repair is to bolt SWRL rules onto OWL, but the SWRL submission itself states that OWL DL "becomes undecidable when extended in this way as rules can be used to simulate role value maps" ^[swrl2004], forfeiting the decidability that motivated choosing a description logic. Answer set programming under stable-model semantics is the better-founded home for genuinely defeasible rules ^[brewka2011]. We use stratified Datalog because the rule base we need is stratifiable, which buys a unique model and a finite trace without the additional machinery.

The trace.

Definition 4 (derivation trace). For f ∈ M(P, F₀), a derivation trace τ(f) is a finite tree in which

• the root is labelled f;
• each internal node labelled g carries a triple (r, θ, S) where r ∈ P, θ is a ground substitution with head(r)θ = g, and S is the ordered sequence of facts satisfying body(r)θ, one child per positive literal;
• each negative literal of body(r)θ is recorded as a verified absence from the model of a strictly lower stratum;
• each leaf is a fact of F₀.

Definition 5 (auditable trace). τ(f) is auditable iff it satisfies:

• T1 (local checkability) for every internal node, head(r)θ = g and the children satisfy body(r)θ - verifiable from the node alone, without re-running the evaluator;
• T2 (grounded leaves) every leaf is in F₀ and carries its origin: a scenario input, an ontology assertion, or a declared constant;
• T3 (rule warrant) every r carries a warrant class w(r) ∈ {definition, literature, convention}, and when w(r) = literature, a citation to the source that supports it;
• T4 (finiteness) τ(f) is finite and renderable in full;
• T5 (no hidden quantities) no node introduces a numeric value that does not appear in a leaf or in a declared constant.

T3 is where most of the honesty lives. A rule that assigns an ordinal screening band because the author chose a threshold is marked convention and says so in the rendered trace; a rule encoding a published relationship carries the citation. A reader can then discount conclusions that rest on conventions without having to guess which ones do. The implemented engine renders each derived fact with its rule identifier, its warrant class, its citation key where one exists, and the facts that satisfied the body, bottoming out in leaves marked given.

Proposition 3 (replay). If τ(f) satisfies T1 and T2 then a verifier that re-applies each recorded (r, θ) bottom-up reproduces f using only F₀ and P, without trusting the system that produced τ.

Proof. By induction on the height of τ. Leaves are in F₀ by T2. For an internal node, T1 gives that the children satisfy body(r)θ and that head(r)θ = g, so g is derivable in one step from facts derivable by the induction hypothesis. ∎

Proposition 3 is deliberately modest. It says the trace is a checkable object, which is the property a rendered chain-of-thought does not have: chain-of-thought text can systematically misrepresent the actual cause of a model's output ^[turpin2023], and a fluent narration of a causal chain is not evidence that any inference occurred ^{[kambhampati2024]}.

The epistemic boundary. The system as a whole is a composition

Φ : NL → F₀        (formalization)          Eval : (P, F₀) → M(P, F₀)   (evaluation)

Eval has the properties above. Φ does not. Whether Φ is performed by a person, a form, or a language model, it is the step where the system's picture of the world is fixed, and it carries no guarantee. This is the split AWS states explicitly for Bedrock's Automated Reasoning checks: translation from natural language to logic "use[s] foundation models (LLMs)" and "may contain errors," while validation "use[s] mathematical techniques (through SMT solvers)" and "is mathematically sound" ^{[aws_arc_docs]}. The empirical literature says the same thing from the research side. Autoformalization accuracy swings by construct type, and token-entropy uncertainty measures fail to flag the errors ^{[ganguly2025]}; attaching a prover to a language model relocates errors into the translation step rather than removing them, with the combined pipeline failing in distinct rather than strictly fewer ways ^{[olausson2023linc]}. The formalization bottleneck is also the standing finding in the process-safety ontology literature, where three decades of automated-HAZOP work report that the quality of the encoded knowledge, not the reasoning machinery, is what limits detection ^[single2019].

We therefore state the guarantee in scoped form: for facts in Φ(x), the conclusions in M(P, Φ(x)) follow from P by Definition 4, and the trace can be replayed by Proposition 3. Nothing is asserted about whether Φ(x) is a faithful description of the plant. In the taxonomy of neurosymbolic architectures this is a pipeline in which a learned component produces symbols consumed by a symbolic component ^[kautz2022], the same shape as solver-in-the-loop systems ^{[pan2023logiclm]}, and it inherits that shape's known weak point.

Entailment is not causation. A rule h ← ℓ₁, …, ℓ_k is a constraint on models. It licenses contraposition, which causal claims do not, and it sits at the associational rung of the ladder of causation ^[pearl2019]. Nothing in Definitions 4 and 5 permits reading a derived fact as a cause of an outcome; the formalism for actual causation is a structural-equation account, not a deductive one ^{[halpern2005]}. Accordingly the rule base derives screening and attention signals, not causal attributions, and the paper avoids causal verbs for its outputs.

Regulatory rules as programs. Encoding a body of regulation as executable logic, with the formalization kept traceable to the statutory text clause by clause, has a precedent in the formalization of the British Nationality Act ^[sergot1986]. That isomorphism principle - one clause, one identifiable rule, inspectable side by side - is the design target for encoding 29 CFR 1904 recordability, and T3 is its operational form.

3.6 The instrumentation problem#

The second problem is measurement, and it is where the human-factors half of the paper sits.

The taxonomy is not the gap. We adopt the four-context performance-influencing factor structure published by the U.S. Nuclear Regulatory Commission in IDHEAS-G, which classifies its twenty PIFs "according to the four types of context: environment and situation, system, personnel, and task" ^[nureg2198]. We adopt it rather than propose an alternative. The field already has PIF sets ranging from a single factor to more than fifty, no standard set, no framework for comparing them, and factor definitions too loose for consistent interpretation across methods ^[groth2012]. Adding another grouping worsens a documented problem.

What is missing. Every established framework defines factor assignment as an expert function. Write it as

ℓ_expert : Scenario → L^K ,    L = {level₁ ≺ … ≺ level₅},  K = 20 factors

evaluated by a trained analyst after a task analysis. No framework specifies a measurable function

ℓ_data : X → L^K

from X, the space of records an industrial operator already collects - OSHA 300 and 301 case detail, shift rosters, permit-to-work records, CMMS work orders, MES and ERP logs - to factor levels at population scale. Whether ℓ_data can be constructed, and whether it agrees with ℓ_expert, is the open question. The NRC's own account is that existing methods "were developed for a procedure-based response to internal events occurring at-power in NPPs" and "are not necessarily adequate to model human actions ... in other domains" ^[nureg2198], so their application to general chemical manufacturing is itself unvalidated ground.

The denominator obstacle. Any attempt to validate ℓ_data against outcomes requires a rate, and a rate requires an exposure denominator. For establishment u over period t, write observed recordable count Y(u,t) and declared exposure N(u,t) in worker-hours. Two facts about N block the naive path.

First, N is self-reported and its distribution is heavy-tailed in a way that is not physically admissible: a small fraction of filings in the OSHA Injury Tracking Application declare a large majority of all reported hours, and screening them out moves the aggregate rate by more than an order of magnitude. The size of that effect depends on the panel, the file version, and the screening rule, so it is reported in the empirical section with the years, the file digests, and the rule stated, rather than carried here as a constant ^{[chimmani2026; osha_ita]}.

Second, at the resolution the human-factors question actually needs - hour of shift - no public source supplies a denominator at all. Incident timing is recoverable: OSHA ITA case detail records the time work started and the time of incident, and MSHA's accident file records shift begin time and accident time. Hours at risk by shift-hour are recorded nowhere. Consequently an observed peak in incident counts at a given hour of shift is a statement about when incidents occur, not about when risk is highest. The occupational-health literature settled this distinction long ago: an exposure-corrected analysis of German accident data had to build estimated exposure models precisely because working-time information was unavailable ^{[hanecke1998]}, and once exposure is accounted for, risk rises approximately exponentially with time on shift, with only a slight elevation from the second to the fifth hour ^{[folkard2003]}. The elevated-risk-early-in-shift effect is a named, published phenomenon, and any count-based observation of it is corroboration in a new sector, not a discovery. We therefore state count distributions as count distributions and never as rates.

Hopkins reaches the same conclusion from outside statistics: what determines whether an indicator is usable is whether, at the level of aggregation and over the period in question, there are enough countable events to speak of a rate at all ^{[hopkins2009]}. That is the constraint governing any per-shift or per-crew score, and it is a stronger objection to real-time risk scoring than any argument about which factors to include.

3.7 Assumptions#

The results in the rest of the paper hold under the following assumptions. Each is stated so that a reviewer can identify what fails if it does not hold.

• A1 (adopted taxonomy). T and the four-context PIF structure are taken from published standards and from IDHEAS-G ^{[asme_viii_ug127; api520p1; iso4126; nureg2198]}. No novelty is claimed for either. If a standard reclassifies an entity, T changes and the affected items must be rebuilt.
• A2 (attribute disjointness). For every minimal pair used, the two members' answer keys are disjoint in the sense of Section 3.1. This is verified per pair, not assumed for co-hyponyms in general; sibling entities with overlapping parameter spaces exist and are excluded from the pair set.
• A3 (corpus authority and legality). D consists of public, clause-addressed federal regulatory text. Copyrighted consensus standards (ASME BPVC, API 520 and 521, NFPA 70E, ISO 4126) are cited by clause identifier and never quoted or redistributed; no answer key depends on their text. The consequence is a real limitation: the corpus is grounded in regulation rather than in the consensus standards a practising engineer reaches for first, and the marine equipment subpart used for the pressure-relief items is a defensible proxy for the general chemical plant context, not the same thing.
• A4 (answer-key correctness). Provenance is verified mechanically for every item - that the cited section resolves, that the verbatim anchor appears in it, and that the anchor lies inside the cited paragraph. Editorial correctness of the keys was reviewed but not independently re-derived by a second domain expert. The number of remaining subtle key defects is not assumed to be zero.
• A5 (automatic scoring is a screen). The adjacency, concept-coverage, and citation scorers are heuristics that emit their evidence. Automatic attribution evaluation is itself unreliable ^[yue2023], so a human-adjudicated subsample with reported inter-rater agreement is the primary measure, and no model-as-judge score is reported as ground truth.
• A6 (version pinning). Every source is pinned by edition or snapshot date, and the corpus is re-verified against the pinned edition. Standards editions are not interchangeable, and edition-sensitive items are among the most diagnostic.
• A7 (evaluation determinism). Eval is deterministic given (P, F₀). All nondeterminism is confined to Φ and to the system under test. Results are reported with temperature, seeds, number of repeats, and run-to-run variance; a single-run number is not reported.
• A8 (no causal claim). Derived facts are entailments under P, not causes ^{[pearl2019; halpern2005]}.
• A9 (no probability claim). Ordinal screening bands are ordinal labels produced in part by rules marked convention. They have no link function, no exposure denominator, and no calibration, and are not probabilities.
• A10 (denominator). Where a rate is reported, the denominator and screening rule are stated. Where no admissible denominator exists - hour-of-shift being the principal case - counts are reported as counts.
• A11 (contamination). Portions of the source corpus circulate on the open web, so an ungrounded system may have memorized some answers, which would compress any measured grounding delta. This is probed with edition-specific numeric values and reported, not assumed away.
• A12 (synthetic data). Any simulated or generated data is confined to a synthetic/ directory, produced by a seeded generator, and carries a header declaring it synthetic. No synthetic figure appears as a headline result. The demonstration run shipped with the evaluation harness measures nothing about any system.

3.8 What is deliberately not formalized#

Severity weighting. SAS is reported as a separate outcome class, and an optional risk-weighted summary exists, but the risk tiers are an author judgement and are never used for headline counts. A validated severity-weighted hallucination taxonomy for process safety does not exist, and we do not claim to supply one.

Causal structure of accidents. The framework above is a contextual-factor model, and there is a live theoretical objection to that whole family. STAMP holds that accidents in complex sociotechnical systems arise from inadequate enforcement of safety constraints across a control structure rather than from a sum of contributing factors or a chain of failure events ^{[leveson2004]}. We do not resolve that tension and we do not cite STAMP as support. The scope claim is narrower than the objection: the ontology is a structured encoding of what established PSF frameworks already assert, built so that its conclusions carry traces, and it is not offered as an accident model.

Calibration to incident probability. Nothing here maps a derived band or score onto P(incident | unit, window). Doing so would require a link function, an exposure denominator, a stated time window, and out-of-sample calibration evidence, and at occupational base rates it would additionally require an event count far beyond what the available data supply. Section 3.6 states why that path is blocked; the paper does not walk down it and then hedge.

Retrieval architecture. Definitions 2 and 3 constrain what a grounded, entity-faithful response must satisfy. They do not prescribe an implementation. Whether the entity constraint is best enforced by a hard metadata filter, by lexical retrieval ^{[robertson2009bm25]}, by late interaction preserving token-level matching ^{[khattab2020colbert]}, or by a hybrid over dense retrieval ^{[karpukhin2020dpr]} is an empirical question, and the distributional evidence in Section 3.2 makes a falsifiable prediction about it that the evaluation is designed to test.

4. An auditable architecture for contextual human-error screening#

This section describes the reference implementation, ehs-human-factors-ontology: what is represented, how the representation is bound to its sources, what the inference layer computes, and what a conclusion carries with it when it comes out. The artifact is Python 3.9, standard library only, MIT licensed, 158 passing tests.

Two framing statements belong at the front, because both are load-bearing and both cut against the way this kind of system is usually sold.

The taxonomy is adopted, not invented. The four context dimensions encoded here are the four performance-influencing-factor context categories published by the U.S. Nuclear Regulatory Commission in IDHEAS-G, which states that "PIFs are classified according to the four types of context: environment and situation, system, personnel, and task" ^[nureg2198]. Two of the four labels used here are identical to the NRC's and two are synonyms. The twenty factors are IDHEAS-G's twenty PIFs, adopted without addition, deletion or subdivision. Section 4.3 sets out the correspondence in full and explains why claiming novelty here would be an error rather than a marketing choice.

The inference layer is a Datalog program, and is named as one. It is not automated reasoning in the sense that term carries in computer science, where the commitments are a formal language with specified semantics, an inference procedure sound with respect to that semantics, and a characterised decidability and complexity result ^{[cook2021blog; sep_ar]}. There is no SMT or first-order decision procedure here, no proof certificate in any standard format, and no soundness theorem relative to a formal semantics of the safety domain. What there is: a definite semantics for the program evaluated, guaranteed termination, and a derivation trace for every conclusion. Section 4.5 states the boundary precisely, following the practice AWS adopts for its Bedrock Automated Reasoning checks, where the natural-language-to-logic step is documented as fallible and only the solver step is claimed to be mathematically sound ^{[aws_arc_docs]}.

4.1 Design commitments#

Five commitments determined the shape of the implementation.

Provenance is a first-class object, not a bibliography. Every factor carries the source's own wording. Every correspondence between this vocabulary and an external framework is an individually cited assertion rather than an entry in a table. Every inference rule declares whether it restates a proposition from a cited source or encodes a threshold the author chose.

Absence is asserted, not left blank. A framework that has no counterpart for a factor gets an explicit negative assertion. A concept in an external framework that this vocabulary does not cover gets an explicit coverage-gap record. A crosswalk that records only its successes is a sales document.

Unassessed is not nominal. A factor omitted from a scenario is carried at an explicit unknown level with its own ordinal rank, never folded into the nominal case. Treating missing data as satisfactory is the standard way a screening instrument understates a hazard.

Nothing is quantified. The output is an ordinal screening label. No human error probability is computed and no multiplier is applied. SPAR-H's multipliers were fitted to nuclear power plant operations ^{[gertman2005]}, and IDHEAS-G states plainly that existing methods "are not necessarily adequate to model human actions ... in other domains" ^[nureg2198]. Porting the multipliers into general chemical manufacturing without revalidation would not be defensible, so they are absent rather than borrowed.

The output is a derivation, not an answer. The conclusion set is the less interesting half of what the engine returns. The trace is the deliverable.

4.2 Knowledge representation#

The ontology is a hand-written OWL/Turtle file of 867 lines (ontology/ehs-hfo.ttl), parsed by a hand-written parser for the Turtle 1.1 subset the file uses (src/ehs_hfo/turtle.py). The parser exists because the implementation is restricted to the standard library; it raises with a line and column on any construct outside the supported subset, and its module docstring enumerates what it does not support. It performs no entailment.

The file declares:

Factors. Each factor carries rdfs:label (the wording used here), ehs:verbatimLabel (the source's own wording, transcribed without alteration), ehs:inDimension, and one or more ehs:sourceRef citation keys. Where the literature supports it, a factor also carries ehs:predisposesTo (an error mode) or ehs:degradationRaisesDemandOn (a cognitive control level). Both properties are annotated in the ontology as theoretical claims taken from the cited source, with no effect size and no probability asserted or implied.

Levels. Five ordinal states: enhanced (rank 2), nominal (1), degraded (0), severely degraded (-1), unknown (-99). The ranks order the states and nothing else; the ontology says so in the property comment. The enhanced level is retained because SPAR-H is one of the few methods that admits beneficial PSF influence ^{[gertman2005]}. The unknown level is kept structurally distinct from nominal for the reason given in 4.1.

Control levels and error modes. Skill-based, rule-based and knowledge-based control follow Rasmussen's SRK framework ^{[rasmussen1983]}; the three error modes (skill-based slip or lapse, rule-based mistake, knowledge-based mistake) follow Reason's GEMS typology ^[reason1990], each bound to the control level at which it arises.

Boundary notes. Where frameworks disagree about which context a factor belongs to, the disagreement is recorded rather than resolved. IDHEAS-G files procedures under Personnel; a practitioner would commonly file them under System or Task. IDHEAS-G files mental fatigue under Task; an occupational-safety practitioner would file it under Human or Operational. Both boundary disagreements are annotated in the file. Boundary disagreement is not novelty; it is the ambiguity Groth and Mosleh identified when they reported that PIFs in use across methods are not defined specifically enough for consistent interpretation ^[groth2012].

Observable proxies. All twenty factors carry an ehs:observableProxy annotation naming a routinely collected industrial data element that has been proposed as an indicator of that factor: LMS completion and recency for training, CMMS overdue preventive-maintenance backlog for equipment and tools, hours elapsed since shift start for mental fatigue, permit records for workplace accessibility. The annotation property's own comment states that these proposals are unvalidated and that no study cited in the repository establishes that any of them measures the factor it is attached to. They exist to make the measurement question explicit, not to answer it. This is the point at which the architecture touches the genuine open problem identified in Section 2: every established PSF framework is analyst-rated, and none specifies how to derive factor levels from data an operator already collects. The ontology names candidate instruments and marks them as candidates.

4.3 The four context dimensions and the PSF crosswalk#

The correspondence to IDHEAS-G is one to one:

The binding is machine-checkable rather than merely stated in prose: every context dimension carries ehs:correspondsToIdheasCategory naming the IDHEAS-G category it renames, and a test fails if any dimension does not.

The same four-way cut is visible across the PSF canon. THERP's internal/external PSF split is its ancestor ^[swain1983]. CREAM's nine common performance conditions ^{[hollnagel1998]}, SPAR-H's eight PSFs ^{[gertman2005]}, the NRC's 15-PSF good-practice set ^[nureg1792], HFACS's Preconditions sub-tree ^{[shappell2000]} and HSE's Job/Person/Organisation headings ^[hse_pifs] all partition the same territory. Proposing a fifteenth loosely-defined grouping would make a documented problem worse: Boring records that PSF sets already range from single-factor models to more than fifty ^[boring2010], and Groth and Mosleh report that there is no standard set and few rules governing how sets are created or defined ^[groth2012].

The crosswalk (crosswalk/crosswalk.md, crosswalk/crosswalk.csv) maps each of the twenty factors to its counterparts in SPAR-H, CREAM, HFACS and the HSE PIF list. It contains 96 rows: 90 positive correspondences and 6 asserted absences. Three design decisions govern it.

Alignments are reified. Each is an individual with a local factor, an external factor, a match strength and one or more citation keys, rather than a direct skos:closeMatch triple. That costs verbosity and buys the ability to attach a source and a caveat to a single correspondence. It is what allows the crosswalk to record, for example, that one IDHEAS-G factor (time pressure and stress) spans two SPAR-H PSFs (available time; stress and stressors), with the note that anyone porting SPAR-H multipliers across that boundary is double counting.

Match strength is declared and constrained. Five strengths are defined: exact, close, broader, narrower, partial. The distribution across the 90 alignments is 34 close, 33 partial, 23 broader; narrower is unused in the current set and exact is deliberately unused, with a test that fails if any alignment ever claims it. Asserting exact identity between factors from frameworks written decades apart for different industries would claim more than the sources support. The partial count is not a defect to be minimised: it is where Groth and Mosleh's interpretability problem bites, made visible rather than smoothed over.

Absence is asserted twice over. Six ehs:noCounterpartIn assertions record that a named framework has no factor corresponding to a local one - all five operational-context factors have no SPAR-H counterpart, and scenario familiarity has no HFACS counterpart. Seven coverage-gap individuals record external concepts this vocabulary does not represent: CREAM's time-of-day CPC, HFACS personal readiness, HFACS failed-to-correct-problem and supervisory violations, HSE motivation, peer pressure and organisational learning. Each gap carries a comment explaining why. The time-of-day gap is the one that matters most for shift-based occupational settings, and it is inherited rather than introduced: IDHEAS-G has no circadian PIF either. The gap record states that closing it honestly would require an exposure denominator by hour of shift that no public dataset supplies, which is the problem Hanecke and colleagues had to build estimated exposure models to work around ^{[hanecke1998]}, and the reason an unadjusted count distribution over hour of shift is not a risk curve ^{[folkard2003]}. Twenty-one further entries record external items deliberately not crosswalked, with a reason: HFACS unsafe acts and their children are outcomes rather than context, HFACS mid-level nodes and the HSE headings are structural containers, and the four IDHEAS-G categories bind at the dimension level rather than factor by factor.

The crosswalk files are generated from the Turtle by tools/build_crosswalk.py, and a test fails if the committed files disagree with what the ontology generates, so the published table cannot drift from the source of truth.

4.4 The inference layer#

src/ehs_hfo/engine.py is a bottom-up Datalog evaluator with stratified negation as failure. Datalog is the accurate name for declarative, terminating, bottom-up rule evaluation with least-fixpoint semantics ^[green2013], and it is a stronger label than "rule engine": Rete, the algorithm underneath essentially every production rule system, is by its author's own title a pattern-matching optimisation rather than an inference calculus, and carries no soundness or completeness theorem relative to a logical semantics ^[forgy1982].

The evaluator's properties, each enforced in code:

• No function symbols. The Herbrand base of any program is finite, so bottom-up evaluation terminates.
• Stratified negation, checked at construction. Head and body predicates are assigned strata by iterating the standard constraint to a fixpoint; a negative dependency inside a recursive cycle raises StratificationError rather than silently picking an answer. For a stratified program the perfect model is unique and independent of rule order. The current rule base stratifies into two levels, the single negative dependency being the mitigation test in R06.
• Range restriction, enforced per rule. A variable appearing in the head, or under negation, or in a builtin guard, must be bound by a positive body literal. Violations raise at rule construction, not at evaluation.
• Deterministic output. The fact index is seeded in sorted order and alternative derivations are canonically sorted, so two evaluations of the same scenario produce byte-identical reports. A test enforces this. An audit record that changes between runs is not an audit record.
• PTIME data complexity. The implementation is naive iteration to a fixpoint per stratum, which is ample at the scale of one scenario: twenty factors, twenty-one rules, a few hundred derived facts.

The rule base (src/ehs_hfo/rules.py) contains 21 rules over a fixed predicate vocabulary. Extensional predicates come from two places and nowhere else: the ontology (factorInDimension, factorPredisposesTo, factorRaisesDemandOn, errorModeAtControl, and the three level predicates) and the scenario (factorLevel, mitigation). The translation from ontology to facts invents nothing; each fact restates an assertion already present in the Turtle. The scenario translation validates every identifier against the ontology and refuses unknown factors, levels or mitigation targets, and it emits an explicit unknown level for every factor the scenario does not mention.

Derived predicates run from degradedFactor and unassessedFactor through dimension roll-up (dimensionDegraded, multiDimensionDegradation), cognitive control demand (controlDemand, unsupportedControlDemand), error-mode flagging (elevatedErrorMode, aggravatedErrorMode, noRuleBasedFallback) and finally screeningBand.

Every rule declares a basis, and the distinction is the point. Sixteen rules are marked literature, meaning the proposition appears in the cited source; the dataclass constructor refuses a literature rule with no citation. Five rules are marked convention, meaning the author chose a threshold and no source supports it. Every screening-band rule (R30 through R34) is convention, and a test fails if one ever claims otherwise. The basis is printed in every derivation trace, because the difference between "Rasmussen says so" and "I picked two" is exactly what a reviewer is looking for and exactly what a scalar risk score destroys.

Two examples of the literature rules. R10 states that degrading a factor the ontology marks as raising demand on a control level creates that demand; scenario familiarity is the principal case, since a situation for which no stored rule applies must be handled by reasoning from a model of the system ^{[rasmussen1983]}. R22 states that an error mode flagged by one degraded factor and flagged again by a second, distinct degraded factor is recorded as aggravated. The rule's own description says what it does not claim: that two independent routes to the same failure class are open, not that risk has doubled. Nothing in the repository supports a multiplicative claim, which is the assumption SPAR-H makes when it multiplies its eight PSFs as though they were independent, and which the PIF interdependence literature treats as unresolved ^[groth2012].

Two further constraints on interpretation. The engine records that a control was claimed against a factor; it has no way to judge whether the control exists or works, and R05's description says so. And screeningBand is an ordinal label drawn from a four-value ordered set (no-flag, review, elevated, stop-and-review). It is not a probability, not a rate, and not calibrated against any outcome data. A test enforces monotonicity - degrading a factor never improves the band - which is a coherence property, not evidence of validity.

4.5 Source binding, and where the guarantee stops#

Every assertion in the ontology and every literature rule carries one or more short citation keys. The keys resolve to entries in CITATIONS.md, and each entry declares how it was verified: primary (the document was retrieved and the specific wording read in it), bibliographic (identifiers confirmed, text not read), or secondary (transcribed at one remove). Seven of the twelve entries are primary, including IDHEAS-G, where the four-context sentence, all twenty PIF names and each PIF's context assignment were confirmed against the published report; SPAR-H, where all eight PSF names were read including their parenthesised forms; the HSE PIF sheet, read in full; and both FAA HFACS reports ^{[shappell2000; shappell2006]}.

Three test-enforced invariants make the binding structural rather than clerical: every citation key used in the ontology or the rule base must resolve to a CITATIONS.md entry that declares a verification state; no bibliography entry may go unused; and every alignment must carry at least one source reference, or the ontology fails structural validation at load.

What this machinery does not establish is that a cited document says what the repository claims it says. That obligation is human, is discharged in docs/PROVENANCE.md, and is discharged unevenly. The weakest link is named there rather than left for a reviewer to find: CREAM's nine common performance conditions are transcribed from an open-access secondary source because Hollnagel's book was not consulted directly ^{[hollnagel1998]}. The transcription was checked against that secondary source; the source itself was not checked against the book. Anyone relying on the CREAM rows should verify them against the original.

The boundary is worth stating in the same terms AWS uses for its own hybrid system, where translation from natural language to logic is documented as fallible and only the SMT validation step is claimed to be sound, and where a valid result is scoped to cover "only the parts of the input captured through policy variables" ^{[aws_arc_docs]}. Here the analogous split is: the Datalog evaluation is deterministic, terminating and order-independent, and the derivation trace is complete with respect to the rules that fired. Everything upstream of that - whether the factor is the right factor, whether the analyst's level is the right level, whether the cited source supports the rule - is unverified by the machine. A sound evaluator downstream of an unverified formalisation gives a guarantee about the program, not about the plant. The autoformalisation literature reports the same asymmetry from the other direction: attaching a solver to a language model relocates errors into the translation step rather than eliminating them ^{[olausson2023linc]}, and translation accuracy on factual content can be substantially worse than on logical content ^{[ganguly2025]}.

4.6 Derivation traces#

Every derived fact records the rule that produced it and the facts that satisfied that rule's body. Alternative derivations of the same fact are all retained and canonically ordered. The trace therefore unwinds any conclusion to the scenario inputs and the cited literature, terminating in facts marked [given].

The worked example (examples/reactor_startup_nonroutine.json) is a hand-written illustration, not a record of a real event, and contains no site data. It describes a batch reactor restart after an unplanned trip: an operator who qualified three weeks ago and has not run this recovery, a recovery sequence not covered by an approved procedure, and a fixed outage window. Training, scenario familiarity, procedures, and time pressure are supplied as degraded; scenario familiarity as severely degraded; the remaining sixteen factors as nominal.

The engine derives that the task has been displaced to knowledge-based control while the training that control level depends on is itself degraded, that the rule-based fallback is gone because procedures are degraded too, and that two distinct degraded factors independently bear on knowledge-based mistakes. Abbreviated, the band trace reads:

Screening band: stop-and-review
  Ordinal label only. Produced by rules marked 'convention'. Not a probability, not a rate.

  screeningBand(stop-and-review)  <- R33-band-stop-and-review [convention; no source]
    rule: screeningBand(stop-and-review) :- severelyDegradedFactor(?F),
          not mitigatedFactor(?F), aggravatedErrorMode(?M).
    because: A severely degraded factor occurring alongside an aggravated error
             mode puts the scenario in the stop-and-review band. Threshold chosen
             by the author; no source supports it.
      severelyDegradedFactor(ehs:ScenarioFamiliarity)  <- R02-severely-degraded [literature; nureg2198]
          factorLevel(ehs:ScenarioFamiliarity, ehs:LevelSeverelyDegraded)  [given]
      aggravatedErrorMode(ehs:KnowledgeBasedMistake)  <- R22-aggravated-by-second-factor [literature; groth2012]
          degradedFactor(ehs:ScenarioFamiliarity)  <- R01-degraded [literature; nureg2198]
          factorPredisposesTo(ehs:ScenarioFamiliarity, ehs:KnowledgeBasedMistake)  [given]
          degradedFactor(ehs:TimePressureAndStress)  <- R01-degraded [literature; nureg2198]
          factorPredisposesTo(ehs:TimePressureAndStress, ehs:KnowledgeBasedMistake)  [given]

Two features are worth drawing out. The band rule announces itself as a convention with no source, because it is one; a reader who disagrees with the threshold can see exactly which rule to argue with and what changing it would affect. And the chain bottoms out in [given] facts supplied by the scenario or restated from the ontology, so there is nothing between the input and the conclusion that a reader cannot inspect. The report additionally prints assessment coverage (how many of the twenty factors were assessed), the full list of rules that fired with their bases and citations, and three standing caveats naming the NRC as the source of the taxonomy, disclaiming any probability interpretation, and stating that factor levels are analyst judgements the tool neither derives nor checks. A machine-readable form of the same result is available via --json.

This is the property the ungrounded retrieval failure in Section 1 lacks. A language model asked about overpressure protection returned semantically adjacent content with no attribution and no route back to a governing clause; the answer was fluent, well-formed, and inspectable only by someone who already knew the answer. A derivation trace is inspectable by someone who does not.

4.7 Synthetic corpus#

synthetic/ holds 48 fabricated scenarios produced by a seeded generator, used only to exercise the engine across a wider range of inputs than the single worked example provides. Every file carries a synthetic label, the generator is reproducible and a test verifies byte-reproducibility, the scenario dataclass carries a synthetic flag that reports print prominently, and synthetic/README.md states that any headline number traceable to that directory is wrong and should be removed. No result in this paper derives from it.

4.8 What the architecture does not provide#

Stated directly, because "ontology" and "reasoning" are both words that do more marketing than work.

No OWL reasoning runs anywhere. The Turtle file is parsed and structurally validated; it is never classified, never checked for consistency, and has not been verified consistent under OWL 2 Direct Semantics. A claim of ontology reasoning would require a DL reasoner such as HermiT or ELK computing entailments ^{[motik2009; glimm2014; kazakov2011]}, and none is in the loop. This is worth being precise about because published safety ontologies blur the same line: competency questions answered by ontology query are structured retrieval, not entailment ^{[single2020_ijsse]}. A knowledge graph is a data model; deductive capability comes from an attached ontology or rule layer, and most deployed graphs perform very little entailment ^[hogan2021].

No causal claim is made. The rules compute entailment over asserted facts. Deductive implication is not causation; the formalism for cause and effect is the structural causal model, and rule evaluation never leaves the associational rung ^{[pearl2019; halpern2005]}. The strongest work in the safety-ontology literature reaches the same conclusion architecturally: Aziz and colleagues capture knowledge in OWL and then transform it into a multi-entity Bayesian network for the causation step, precisely because deduction alone cannot carry causal content ^[aziz2019]. Nothing here attempts that step.

No probability, no calibration, no validation against outcomes. The screening band is ordinal and produced by rules marked convention. Nothing in the repository has been checked against injury or incident data. The factor-to-error-mode links are theoretical propositions from the cited literature with no effect size attached to any of them.

Factor levels are inputs. An analyst supplies them. The engine cannot derive a level from data and cannot tell whether a supplied level is right. The traceability is real; the inputs remain judgements. This is the same constraint that binds every framework crosswalked here, and it is why the observableProxy annotations are marked as a research agenda rather than a capability.

Match strengths are one person's judgement. No second coder, no adjudication, no inter-rater reliability figure. A reviewer who disagrees with specific rows would be on solid ground, and the reified representation is designed to make disagreeing with a single row possible.

Domain transfer is an extension, not an application. Every framework crosswalked here except HSE's was built for nuclear power or aviation, and IDHEAS-G says existing methods are not necessarily adequate outside their origin domain ^[nureg2198].

Finally, a theoretical position rather than a limitation. This is an additive contextual-factor model: it treats context as a set of factors that can each be degraded and reasons about what follows. Rasmussen argues against exactly this style of analysis, in favour of modelling behaviour-shaping mechanisms as work-system constraints ^{[rasmussen1997]}, and Leveson's STAMP treats safety as constraint enforcement across a control structure rather than as a sum of contributing factors ^{[leveson2004]}. Neither is cited here as support, because neither supports this. The position taken is narrower: a factor vocabulary is useful for structuring what an analyst already records and for making the recording auditable, and that usefulness is not an argument that accidents decompose into factors. If the systems-theoretic critique is right, the value of this architecture is as a disciplined bookkeeping and traceability layer, not as an accident model.

The intellectual ancestor for that framing is not the HRA literature but Sergot and colleagues' formalisation of the British Nationality Act as a logic program, and specifically their isomorphism principle: keep the formalisation traceable to the source text clause by clause, so that a disagreement about the encoding can be located and argued ^[sergot1986]. That is what ehs:verbatimLabel, per-alignment citations, per-rule bases and the derivation trace are for. Forty years of automated HAZOP research has repeatedly found that the binding constraint is not the reasoning technology but the quality and provenance of the encoded knowledge ^[single2019], and the ontology-for-HAZOP work that followed validates on single vessels ^{[single2020_jlp]}. Building the provenance apparatus first, and refusing to quantify until there is something to calibrate against, is the response to that record.

5. Evaluation Methodology#

5.1 What is contributed here, and what is not#

This section describes an evaluation apparatus: a question corpus, a scoring harness, an adapter interface, a preregistered analysis plan, and a set of threats to validity. The apparatus is the contribution. No system has been evaluated on it. The full three-arm experimental run described in Section 5.7 is future work.

We state this plainly at the outset because the alternative is worse. A benchmark paper that reports numbers from a fixture is a benchmark paper that has taught its readers nothing and has made a claim it cannot support. The repository (ehs-ai-grounding-eval) ships a mock adapter that produces a labelled demonstration run so that the harness can be exercised without an API key and so that a reader can see the report format. That mock composes its answers out of the corpus item it is answering, and its outcome probabilities are constants typed into a profile definition. It scores exactly as well as those constants say it will. Every response it emits carries the provenance string MOCK_DEMONSTRATION_FIXTURE_NOT_RESULTS, every run file it writes records "is_mock_demonstration": true, and suppressing the banner in a report raises an exception rather than printing quietly. The demonstration output measures nothing about any system and is not reported here as though it did.

What the apparatus can establish, once real systems are run through it: whether a system distinguishes co-hyponymous safety devices under minimal-pair contrast; whether it cites the governing clause, a merely adjacent clause, or nothing; whether it declines when a question rests on a false premise; how stable its answers are across repeated runs; and how those quantities differ across grounding configurations, on this item set.

5.2 The measured construct#

The object of measurement is whether an answer to a safety-critical technical question is attributable to an authoritative source, and specifically whether the system distinguishes entities that are taxonomic siblings under the governing standard.

The error class is co-hyponym substitution: answering about the semantically adjacent wrong entity. A rupture disk and a spring-loaded pressure relief valve are both pressure relief devices under ASME BPVC Section VIII Division 1, which classes rupture disks specifically as nonreclosing ^{[asme_viii_ug127]}. Their parameter spaces do not intersect. A relief valve has set pressure, blowdown, accumulation, backpressure, lift, and a reseating pressure, because it recloses. A rupture disk has a marked burst pressure, a burst pressure tolerance, a manufacturing design range, a specified disk temperature, and an operating ratio, and it is one-shot ^{[nationalboard_burst]}. No arithmetic converts one answer into the other. The substituted answer is fluent, correctly formatted, internally consistent, and categorically inapplicable, which is what makes it more dangerous than an obviously wrong answer: it survives casual review.

This is not a benchmark of hallucination in general, and it should not be described as one. It measures one error class chosen because it is structurally over-determined rather than incidental. Four mechanisms converge on it. A system prompt with pasted reference text supplies no non-parametric memory in the sense of retrieval-augmented generation ^[lewis2020], so the answer comes entirely from pretraining. Parametric factual accuracy tracks pretraining frequency ^{[kandpal2023]} and fails on low-popularity entities where retrieval would help most ^[mallen2023], and rupture-disk content is long-tail relative to relief-valve content. Training and evaluation reward a plausible guess over an abstention, so under uncertainty the model emits its highest-probability plausible completion ^[kalai2025]. And distributional representations encode that two terms are related rather than how ^{[mrksic2016; levy2015; shwartz2017]}, with retrievers demonstrably failing minimal pairs that differ only by a logical operator ^{[weller2024nevir]}.

The fourth mechanism is the one specific to this domain. API 520 Part I documents both device types in a single standard with a shared definitions section, and the word disc/disk is polysemous inside it: in a relief valve the disc is the closure element whose travel defines lift and whose contact with the seat defines reseating, while a rupture disk is an entire device ^[api520p1]. ISO splits the two across separate parts, 4126-1 for safety valves and 4126-2 for bursting discs ^[iso4126]; ASME and API co-locate them. Chunk-and-embed an index over the co-locating corpus and the nearest neighbours of a rupture-disk passage are relief-valve passages. Which standards family is indexed is therefore a safety-relevant corpus design decision, and it argues for device-type metadata filtering over undifferentiated top-k retrieval.

5.3 Corpus construction#

The corpus contains 68 items across eight regulatory domains: pressure relief devices (12), injury recordkeeping (10), lockout/tagout (10), confined space (8), machine guarding and electrical and flammable liquids (8), process safety management (8), respiratory protection and noise (7), and hazard communication (5). Of these, 63 are factual items and 5 are category-error items. Twenty-three form complete minimal pairs. Items are grouped into 44 families, which are the cluster unit for the bootstrap.

Each item was built by the same procedure: identify a pair of entities or conditions that a governing regulation treats separately and that share vocabulary; locate a public federal clause stating a substantive requirement for each member; write two question stems differing only in which member they ask about; write the reference answer from the clause text; write the adjacent wrong answer together with the signature vocabulary that would appear if a system answered about the wrong member; state in operational terms why the confusion is dangerous; and record the clause identifier, the URL, and a verbatim anchor span.

An item therefore carries an answer key with required_concepts (alternation groups, so several surface forms of the same claim match), forbidden_concepts, and declared numeric_facts; an adjacent_wrong block naming the sibling entity, its signature terms, and why the confusion is dangerous; and a source block with clause, anchor text, and access status.

Minimal pairs#

Pair members differ only in the entity or condition asked about: the rupture-disk member and the relief-valve member; the non-fire and the fire-case accumulation limit; the lockout and the tagout periodic inspection requirement. Lexical overlap is near-total by construction, so a system succeeding through topical similarity alone fails systematically and visibly. This follows the contrast-set methodology of Gardner et al. ^{[gardner2020]} and the capability-organised behavioural testing of Ribeiro et al. ^{[ribeiro2020checklist]}, and it borrows its scoring discipline from NevIR ^{[weller2024nevir]}: score paired accuracy, requiring both members correct, rather than averaging per-item accuracy. A system that answers every pressure-relief question with generic pressure-relief content can score respectably per item while getting the distinction wrong every time. Requiring both members strips that strategy of its reward.

Category-error items#

Five items rest on a false premise: the blowdown of a rupture disk, the set pressure of a bursting disc. The correct behaviour is to reject the premise. These measure negative rejection in the sense of Chen et al. ^{[chen2024rgb]} and abstention in the sense of Kalai et al. ^[kalai2025]. They are scored and reported separately and are never folded into overall accuracy, because a system that confidently answers everything would otherwise outscore a properly calibrated one, inverting the safety objective.

Structural invariants#

Four invariants are enforced by the test suite. Every item's own reference answer must satisfy its own answer key, or the key is wrong and every system measured against it is measured against a broken target. No adjacency signature term may appear in the item's own reference answer, or a correct response would be scored as an adjacent substitution, biasing the headline statistic in the direction that flatters the benchmark. Every minimal pair has exactly one a and one b with differing questions. Every item is marked verified and appears in the source verification report with a matching clause string.

The first two invariants were violated by four items during construction and each was caught and corrected: one item restated its paired item's figure in its own reference answer, one required a number its own question stem supplied, one had answer-key phrasings too narrow to match natural wording, and one spelled a number as a word where the scorer recognised only digits (fixed in the scorer). These are recorded because the invariants are load-bearing rather than decorative.

5.4 Source verification and the copyright constraint#

Every item asserts that a specific paragraph of a specific federal regulation says a specific thing. That assertion is the only reason the answer key can be called correct, so all of them are re-checked against the primary source and the result is committed. tools/verify_sources.py fetches each cited section from the eCFR versioner API and checks that the section resolves at a stated edition date; that the item's verbatim anchor text appears after whitespace, quote and dash normalisation; that the cited paragraph markers appear with case preserved, since CFR hierarchy distinguishes (a) from (A); and that the anchor sits inside the paragraph the item cites rather than merely somewhere in the same section. The committed report records 68 of 68 items verified with 0 failures across 19 CFR sections at eCFR edition 2026-08-01. The test suite re-checks the corpus against that report offline on every run and fails if a clause was edited after verification.

The locality check exists because it caught a real defect. An independent audit re-ran the verifier against live eCFR, reproduced 68/68, and then found that one item cited 1910.157(d) while its anchor text belonged to (e)(2). The three checks then in place could not see it, because a bare (d) marker occurs somewhere in almost any section that has a paragraph (d). The item was repointed to (d)(6) with an anchor genuinely located there, and the locality check was added. The general lesson is worth stating in a paper about grounding: a verifier refutes only the errors it was built to look for, and the count of items it passes says nothing about the errors it cannot see.

Two items take assigned protection factors from Table 1 of 29 CFR 1910.134, which the eCFR XML renders as a flattened run of cells whose row and column alignment is not machine-recoverable. For these the automated check confirms the mandatory-use sentence and the paragraph path, and the values were read from the table by a human. Their verification method is recorded as ecfr_api+human_table_read rather than presented as equivalent to the fully automated checks.

Copyright forces a substantive design constraint. ASME BPVC, API 520/521, NFPA 70E and ISO 4126 are copyrighted and are not fetched, quoted or redistributed. No item's answer key depends on their text. Where an ASME paragraph is named it appears as a cross-reference whose identifier is corroborated from a public federal source: 46 CFR 54.15-13 is titled "Rupture disks (modifies UG-127)" and its paragraph (a) states that UG-127 provides for rupture disks in series with spring-loaded valves, which confirms the identifier without reading ASME's text. A test enforces that no item uses a paywalled standard as its primary authority.

The cost is stated rather than buried: the corpus is grounded in federal regulation rather than in the consensus standards a practising engineer reaches for first, and 46 CFR subpart 54.15 is Coast Guard marine equipment regulation rather than the general chemical plant context the motivating incident came from. It was chosen because it contains substantive, device-specific requirements for both device types and is public. It is a defensible proxy and not the same thing.

Six drafted items could not be verified and were excluded rather than admitted: two NFPA 70E, two API 520, one API 754, one ISO 4126. In each case the clause identifiers or the numeric values are edition-dependent and no public primary source was available. They are retained in a quarantine file so the exclusions are visible rather than silent; the loader never reads that file, each entry stores its text under draft_question so it cannot be picked up as an item, and none asserts a clause identifier.

5.5 Scoring#

Scoring is lexical and deterministic. Given the same corpus and the same response text, the same score comes out. There are no embeddings and no learned components anywhere in the scoring path.

That choice has a real cost: a correct answer paraphrasing around every listed alternative is scored as a miss, so measured accuracy is a lower bound on substantive correctness, and the size of the gap varies by system. A verbose system that restates regulatory phrasing is scored more generously than a terse one that paraphrases, independent of correctness. The reason for accepting the cost is that the benchmark's subject is whether a system distinguishes near-synonymous technical entities, and using an embedding model as the scorer would make the measuring instrument the same mechanism whose failure is under study. A scoring disagreement would then be uninterpretable.

Decision procedure#

Five outcomes are assigned in fixed order.

1. UNSCORABLE. Adapter error or empty response. Never counted as wrong. An infrastructure failure and a confabulation are different events, and mixing them corrupts both rates.
2. ABSTAINED. An explicit declining cue with no substantive commitment. Hedging and then answering is answering. Detection is phrase-based and deliberately conservative, with commitment cues that cancel an abstention classification.
3. CORRECT. Required concept coverage at or above threshold, no forbidden concept asserted, and every declared numeric fact present. Both digits and spelled-out integers are recognised. A wrong number in a safety answer is not a partial-credit situation.
4. ADJACENT_SUBSTITUTION. Not correct, and asserts at least adjacent_min_terms of the adjacent entity's signature vocabulary, or at least one such term when coverage falls below adjacent_low_coverage. This is the failure class the benchmark exists to measure, reported separately because its consequences differ from those of a plainly wrong answer.
5. OTHER_INCORRECT. Everything else.

Assertion is distinguished from mention. "A rupture disk has no blowdown and does not reseat" is a correct answer that names what it rules out, and is not scored as a substitution. Contrast detection operates on a 70-character left window and is the single weakest link in automatic scoring (Section 5.8).

Citation grading#

Citations parsed from the response text and from any structured citation list are graded on a four-level scale: none, wrong, correct_section, correct_clause. A fifth quantity is tracked separately, unsupported_paywalled_citations: references to copyrighted consensus standards that the item does not list as its source or as a corroborating cross-reference. The asymmetry is deliberate. A fabricated CFR citation can be checked for free in thirty seconds; a fabricated ASME paragraph number cannot, which makes it a more expensive error for a reader to catch and therefore worth counting on its own.

Citation grading is structural. It confirms that a system named the right clause. It does not confirm that the system retrieved that clause, read it, or used it, so a system emitting a plausible identifier from parametric memory grades the same as one that retrieved it. Separating those requires instrumenting the retrieval step, which is the adapter author's responsibility. This is a narrower measure than attribution in the AIS sense ^{[rashkin2023]} or citation precision and recall in the ALCE sense ^{[gao2023alce]}, and it should not be reported as if it were either.

Abstention credit#

Under the safety policy, declining beats confabulating: correct 1.0, abstained on an answerable item 0.25, abstained on a category-error item 0.9, other incorrect −0.5, adjacent substitution −1.0. These weights are a stated policy, not an estimated quantity. No expert panel elicited them. They encode a judgement that a confident wrong answer about the adjacent device is worse than a plainly wrong one, and that both are worse than declining. Any number derived from them must name the policy. An accuracy policy giving abstention no credit is included for comparison, and reporting both makes the policy's effect visible rather than assumed.

Per-item risk tiers (37 high, 30 medium, 1 low) are likewise an author judgement. They feed an optional weighted summary and never the headline outcome counts.

Scoring thresholds are configuration, fixed before any system was run: coverage_threshold 0.75, adjacent_min_terms 2, adjacent_low_coverage 0.5, require_full_numeric true, contrast window 70 characters. Every reported number should name the configuration that produced it.

5.6 Statistical plan#

The design is paired, because every arm answers every item and most items belong to a minimal pair, and clustered, because pair members share a stem, a source and an author. The analysis matches both facts. The plan below is frozen in docs/preregistration.md against the corpus digest a9fd4f4c2222dfb9f6135ee39a94ca9b92168867b14a56a5a319d16cb3980528, so that a later result cannot be the product of choices made after seeing the data. Preregistration follows Magesh et al. ^[magesh2025], which is the current bar for evaluation work in a regulated domain.

• Primary outcome: paired accuracy over the 23 minimal pairs. Confirmatory.
• Secondary outcomes, all exploratory and labelled as such: adjacent-substitution rate; per-item accuracy; abstention rate on answerable items; decline rate on category-error items; citation grade distribution.
• Arm comparison: McNemar's exact test on discordant items. Exact rather than chi-square because discordant counts are often small at this corpus size. The code raises rather than silently comparing an intersection if two arms' item sets differ, so an independent-samples test cannot be run by accident.
• Intervals: percentile bootstrap resampling whole families with replacement, 2000 resamples, seed 20260904, so a minimal pair enters a replicate whole or not at all. A Wilson interval that ignores clustering is reported alongside so the difference is visible rather than assumed away.
• Selective prediction: risk-coverage curves rather than raw accuracy. For a safety application the useful summary is the coverage a system sustains at acceptable precision. Items reporting no confidence are excluded and the point count says so; a system that reports no confidence at all yields an empty curve, which is itself a finding about that system.
• Repeats: at least three per arm, at fixed and recorded temperature, reported separately with across-repeat spread. Runs are never averaged silently, because in a safety setting an answer that is correct only sometimes is not a correct answer. This treats reproducibility as an outcome rather than a nuisance, following Dokas ^[dokas2026].
• Alpha: 0.05, two-sided.

Runs scored under different configurations are never pooled, and a run file whose corpus digest differs from the current corpus raises rather than being silently included.

The preregistration also records, in advance, three outcomes that would count against the benchmark's own premise: that grounded and ungrounded do not differ on paired accuracy; that the pseudo-grounded arm outperforms the grounded arm; and that adjacent-substitution rates are near zero in every arm, which would mean the corpus failed to construct genuinely confusable items and would be reported as a limitation of the corpus rather than as evidence that systems handle the distinction well.

5.7 The grounded-versus-ungrounded protocol#

Systems enter through an adapter that turns an item into a response. Grounded and ungrounded systems are scored through the same interface on the same code path, so a measured difference cannot come from a difference in how they were asked or parsed. An adapter representing a real system must use only the question text and at most the item id; it must not read the answer key, the reference answer, or the adjacent-wrong description. The mock adapter does read them, because it is a fixture rather than a system under test. This rule is enforced socially, not mechanically, and a reviewer evaluating a reported result should inspect the adapter source.

Three arms are specified.

1. Ungrounded. Instruction or persona prompt over parametric memory, no retrieval. This is the observed failure case.
2. Pseudo-grounded. Retrieval over a plausible but unauthoritative corpus: vendor pages, blog posts, an internal wiki.
3. Grounded. Retrieval over a version-pinned authoritative corpus with device-type metadata filtering.

The primary comparison is grounded versus ungrounded on paired accuracy. The secondary comparison of interest is pseudo-grounded versus ungrounded, and its direction is not predicted. Arm 2 is where most enterprise deployments actually sit, it is the least studied condition in the literature, and it is where retrieval may plausibly perform worse than none: irrelevant retrieved context measurably degrades accuracy ^{[shi2023distracted]}, relevant evidence placed mid-context is underused ^{[liu2024lost]}, and unauthoritative sources lend confident-looking support to a wrong answer. Behaviour under conflict between retrieved evidence and parametric belief is unstable and framing-sensitive ^{[xie2024; longpre2021]}, which is another reason not to predict the direction.

Two further conditions are worth adding if resources permit, to separate "retrieval helped" from "instruction to obey retrieval helped": context-aware decoding ^[shi2024cad] and context-faithful prompting ^{[zhou2023contextfaithful]}.

Reporting rules are fixed: outcome composition rather than a single accuracy number, with adjacent substitutions and plainly wrong answers never summed; category-error items reported separately; every point estimate accompanied by an interval and by the across-repeat spread; and every mock run displaying its provenance banner.

Human adjudication is primary, automatic scoring is a screen#

Before any adjacent-substitution rate is published, a stratified sample of at least 20 percent of responses in each scored outcome class is adjudicated by a human blind to the arm, the adapter name and the automatic outcome. Correct responses are sampled alongside errors, because adjudicating only the errors measures the scorer's false-positive rate while leaving its false-negative rate unknown. The export module writes the blinded, shuffled sheet with a recorded seed and keeps the mapping back to arm and outcome in a separate key file. Inter-rater agreement is reported if more than one rater is used. The human-adjudicated figures are the primary report; the automatic scores are a screen. This position follows the finding that automatic attribution evaluation is itself unreliable ^[yue2023] and the precedent of validating a factual-consistency metric against human judgement before trusting it ^{[honovich2022true]}. It applies equally to LLM-judge metrics such as RAGAS ^{[es2024ragas]} and ARES ^{[saadfalcon2024ares]}, which are screens here and not results. No human adjudication has been performed on this corpus to date.

Contamination probe#

Portions of 29 CFR circulate widely on the open web, so an ungrounded system may have memorised some answers, which would compress the measured difference between arms and understate the value of grounding. This will be probed with edition-specific values and reported. It will not be assumed away. It has not yet been probed.

5.8 Threats to validity#

Ordered by how likely each is to change a conclusion.

The scorer is lexical. Measured accuracy is a lower bound on substantive correctness, and the gap is unknown and system-dependent. Mitigation is partial: answer keys use alternation groups with several surface forms, and the invariant that every reference answer must satisfy its own key catches the worst cases. It does not catch keys that are merely narrow.

The contrast heuristic misfires in both directions. A response of the form "this is not a relief valve, so blowdown does not apply, and the blowdown would be 7 percent" is under-penalised; a correct contrast whose cue and term are separated by more than 70 characters is over-penalised. The adjacent-substitution rate is the benchmark's headline error class and is therefore the number most exposed to scorer error. This is why human adjudication is required rather than optional.

Federal regulation is a proxy for consensus standards. Results do not transfer to ASME or API practice without further work. A system tuned to ASME and API convention may answer correctly in substance while missing the 46 CFR framing, and would be scored wrong.

The corpus is a construct, not a sample. Sixty-eight items, hand-built by one author, with the failure mode in mind. There is no sampling frame and no basis for generalising to "the questions a safety engineer asks." Item selection is adversarial by design, since items were chosen because a confusable neighbour exists, so error rates here should be expected to exceed those on a naturally occurring question distribution by an unknown amount. Author bias is unmitigated: the same person chose the entities, wrote the questions, wrote the reference answers, and wrote the adjacency signatures. An independent item set would be a substantial improvement, as would external review of the answer keys, which has not occurred.

Statistical power is limited. The cluster bootstrap resamples 44 units, so intervals are wide and the study is powered to detect large differences rather than modest ones. A null result should be read as "this corpus cannot resolve a difference of this size," not as "the arms are equivalent." McNemar's exact test is valid at small discordant counts, but validity is not power.

Verification checks provenance, not editorial quality. It cannot confirm that the anchor is the most relevant span, that the reference answer is complete, that the key captures what matters, or that a clause has not been superseded by guidance or a letter of interpretation.

Regulations change. The verification report is pinned to a single eCFR edition. The offline recheck confirms the report still covers the corpus; it cannot detect that a regulation changed after the report was written. Only a fresh online run does that.

Utility weights and risk tiers are policy, not measurement, and any number derived from them must name the policy.

Citation checking is structural, and does not distinguish a retrieved identifier from a recalled one.

The motivating incident is n = 1. The rupture-disk substitution described in Section 1 motivates this work and demonstrates nothing on its own. Every empirical claim must rest on the constructed item set.

5.9 What will not be claimed from this apparatus#

Recorded here so that it cannot be claimed later.

• That grounding eliminates hallucination. It does not. Commercial legal research tools marketed on their freedom from hallucination were measured, under preregistration, to hallucinate between 17 and 33 percent of the time ^[magesh2025], and the earliest direct evidence framed retrieval correctly as reduction rather than removal ^{[shuster2021]}. Any claim from this benchmark must be a measured difference with a confidence interval under stated conditions.
• Anything about a population of questions beyond this corpus.
• Any calibration claim about incident probability. Nothing here estimates one.
• Any generalisation from 46 CFR to ASME or API engineering practice.
• That a protocol-level source binding solves grounding. A standardised binding between a model and a source (MCP, for instance) makes the binding auditable and swappable, which has real value in a regulated deployment, and the specification itself locates trust outside the protocol by stating that tool descriptions should be considered untrusted unless obtained from a trusted server ^{[mcp2025spec]}. It does not establish that a source is authoritative, that the retrieved passage is the right one, or that the model used what it retrieved. Corpus curation, metadata filtering and evaluation do that work.

Finally, the author's prior work on public OSHA establishment filings ^{[chimmani2026]} concerns data quality in mandatory regulatory reporting. It establishes domain background and is cited for that reason. It is not evidence about retrieval or hallucination, and the two lines of work are kept separate throughout.

6. Data Foundations#

6.1 The dependency this paper cannot argue around#

Sections 3 through 5 make a claim about architecture: that a system whose conclusions carry derivation traces back to identified source clauses is auditable in a way that a system conditioned on a persona and a pasted document is not. That claim is about the relationship between a conclusion and a record. It says nothing about the relationship between the record and the world.

The distinction is the one AWS draws explicitly for its own deployed reasoning system. Their documentation states that a VALID result "covers only the parts of the input captured through policy variables," and their own worked example is a claim resting on a forged doctor's note that will be scored valid because no variable captures forgery ^{[aws_arc_docs]}. A sound procedure over a false premise returns a sound conclusion about the premise. Nothing about the soundness reaches the world.

For safety-critical question answering in process industries this is not a philosophical footnote, because the records in question are self-reported administrative filings that nothing validates at intake. This section quantifies how bad that is, using the public OSHA Injury Tracking Application (ITA) Form 300A corpus, and then states precisely what follows for a grounded architecture. The short version is that grounding and data validation are separate obligations, and satisfying one does not discharge the other.

All figures in this section were computed by the pipeline in repos/ehs-osha-analysis from nine ITA Form 300A files (CY2016 through CY2024) downloaded from osha.gov, with URLs, byte counts and SHA-256 digests pinned in the repository and verified on 2026-09-03 ^[osha_ita]. They are transcribed from the generated summary.json and output tables, and regression tests parse the headline values back out of the manuscript's source repository and fail if the prose and the tables disagree. Where a figure comes from the author's earlier work rather than from this pipeline, it is labelled as such.

6.2 What the corpus is, and what it is not#

OSHA requires annual Form 300A summary submission from establishments with 250 or more employees, and from establishments with 20 to 249 employees in the higher-hazard industry groups listed in 29 CFR 1904 Subpart E ^{[osha_ita_users_guide]}. The result is a mandated administrative collection over a selected slice of US industry. It is not a probability sample of the US workforce, and any national rate computed from it is a rate for that slice.

The screened aggregate reported below (about 3.98 recordable cases per 200,000 hours) sits well above the BLS Survey of Occupational Injuries and Illnesses private-industry total recordable case rate of 2.3 per 100 full-time equivalent workers for 2024 ^[bls2026]. That direction is what a universe skewed toward larger establishments in higher-hazard industries would produce. The two figures are not directly comparable and are not compared here; OSHA publishes its own ITA-versus-SOII comparison and it should be read before anyone treats the gap as a finding ^{[osha_ita_bls_comparison]}.

Three schema facts matter for anyone building a retrieval or reasoning layer over these files, because each is a silent-wrong-answer trap rather than a loud failure. Column order differs between the 2016–2022 and 2023+ files, and naics_year appears only from

1. ITA Data CY 2018.csv contains bytes that are not valid UTF-8 (0x92, a Windows-1252 right single quote), so a strict decoder raises and a permissive one silently substitutes. And the size field is not comparable across years: OSHA's summary data dictionary documents codes 1 (<20), 2 (20–249), 21 (20–99), 22 (100–249) and 3 (250+), and states that "code 2 was split to 21 and 22 with the collection of 2023 data" ^{[osha_ita_summary_dict]}. The files agree - among plausible filings, code 2 falls from 244,231 in 2022 to 41,343 in 2024 while codes 21 and 22 rise to 224,266 combined - so a pooled panel mixes one 20–249 band with two narrower bands covering the same establishments. The codes are documented; the pooled semantics are not.

None of these is exotic. They are the ordinary condition of regulatory data, and they are invisible to a chunk-and-embed index, which will happily retrieve a 2019 size value and a 2024 size value as if they denoted the same population.

6.3 The denominator failure#

An incident rate is a ratio:

TRIR = 200,000 × recordable cases / hours worked

The numerator is bounded by how many people work at a site. The denominator is a free-text number on a form. When rates are aggregated across establishments the standard estimator is a ratio of sums, so one filing with an impossible hours value can dominate the denominator of an entire industry, state or national figure while contributing nothing to the numerator. The aggregate is then biased toward zero, and it reads as good news.

Pooled across 2,801,064 deduplicated filings, 2016–2024, under a screen that flags filings outside 120–4,500 hours per average employee per year:

The concentration is extreme even within the flagged set. A single filing holds 88.81% of all hours ever reported to the ITA and contributes zero recordable cases: establishment 90427, reporting year 2019, declaring 7 employees and 16,831,620,723,179 hours worked. That is roughly 2.4 trillion hours per employee, against a physical ceiling of 8,760. The ten largest filings by declared hours hold 94.5% of all hours and 0.0008% of all cases. One flag does the entire job: hours per employee > 4,500 fires on 24,617 filings (0.879% of the corpus) carrying 96.65% of all hours. The missing-value and low-hours flags matter for establishment-level rates and are irrelevant to the aggregate.

The screened aggregate is not an artefact of where the bounds are drawn. Across a 25-cell grid of lower bounds (100–400 h) and upper bounds (3,500–6,000 h), the screened aggregate stays within 3.963–3.993. The flagged share is not similarly stable, running from 1.79% to 4.49% across the same grid, and that instability is the point rather than a weakness: the filings that move in and out of the flagged set as bounds shift carry almost none of the hours, so the correction they make is nearly identical. The conclusion survives the threshold choice; the count of flagged filings does not.

6.4 What does not replicate, and why the non-replication is the finding#

An earlier public repository by the same author ran the same class of plausibility screen over a narrower panel of OSHA establishment filings and reported a headline correction multiplier ^{[chimmani2026]}. That multiplier is not reproduced by the pipeline described here, it is superseded by the figures in this section, and it is not restated here as a quantity, because the reason it cannot be reproduced is also the reason it should never have been quoted as one.

The flagged share is stable and does replicate: 2.07% pooled here, and 1.56% to 2.60% across individual years. The multiplier is not a property of the corpus at all. It is a property of whichever years happened to contain an extreme filing:

The screened rate is stable across nine years (3.15–4.60). The unscreened rate is not (0.016–2.92). The multiplier between them ranges from 1.39× to 249×, and no year subset tried here reproduces the earlier figure. The correct statement of the finding is therefore structural and holds in every single year: a small minority of filings is internally implausible, those filings hold a disproportionate share of the denominator, and removing them raises the aggregate substantially. The specific multiplier is a property of the tail of the hours distribution in whatever slice was taken, and should not be quoted as a constant. We state this here rather than in a limitations paragraph because the earlier framing circulated, and correcting one's own published number in the body of a paper is cheaper than having a reviewer do it.

The methodological point generalizes beyond this corpus. Hanecke and colleagues faced the same problem in 1998 with more than 1.2 million German accident records and no exposure data, and had to construct and compare estimated exposure models before any rate could be computed at all ^{[hanecke1998]}. Hopkins reached the same conclusion from the other direction without any statistics, arguing that what determines whether a safety indicator is meaningful is not whether it is labelled leading or lagging but whether, at the level of aggregation in use, there are enough countable events to form a rate - the "zoom effect" ^{[hopkins2009]}. Both are arguments about denominators. Neither has an analogue in the performance-shaping-factor literature, which presupposes a well-defined opportunity count throughout ^[nureg2198], ^[groth2012].

6.5 Zero-recordable filings: overdispersion, not a hidden non-reporting population#

Among plausible filings, 37.1% of all establishment-years report zero recordable cases. For NAICS 325 (chemical manufacturing) the pooled figure is 37.4%, rising from 34.1% in 2016 to 41.3% in 2024. The prior repository's often-quoted figure of roughly 38% for chemical establishments is consistent with this panel's 37.4% ^{[chimmani2026]}.

The obvious reading is that a zero-inflated count model is required - that there exists a distinct population of establishments producing structural zeros through non-reporting. Fitting the models does not support that reading as the main story.

Four intercept-only models with an exposure offset (Poisson, NB2, ZIP, ZINB) were fitted by maximum likelihood and compared by AIC, BIC and observed-versus-expected zeros over the 30 largest NAICS 3-digit groups in 2024, each with at least 500 establishments, covering 306,711 establishment-filings. Industry selection reads only group size, never outcome values or fit statistics.

Two results point in opposite directions. Overdispersion is universal: Poisson never wins in any of the 30 industries, and the variance-to-mean ratio of recordable counts runs from 3.0 to 756 against the value of 1 that Poisson assumes. ZIP is never selected either, so adding structural zeros to a Poisson does not rescue it - the problem is the spread of the whole distribution rather than the zeros alone. Meanwhile zero-inflation is optional and, where present, small: in 14 of the 30 industries the ZINB inflation parameter collapses to the boundary (on the order of 1e-14) and NB2 wins outright, and where ZINB is selected the inflation probability ranges from 0.001 to 0.069. That is a few percent of establishments, not the 37–41% zero share that motivated fitting it. NB2 frequently predicts slightly more zeros than are observed.

Refitting the four models with a log-hours offset, establishment size-band dummies and NAICS 4-digit fixed effects within each 3-digit group (count_model_covariates_summary.csv in the repository, method in docs/COUNT_MODELS.md) leaves overdispersion intact in all 30 industries (NB2 beats Poisson at boundary-corrected p < 0.001 in every one; NB2 alpha median 0.79, minimum 0.16) but reduces the ZINB wins from 15 of 30 to 10 of 30, with the ZINB-vs-NB2 boundary test below 0.01 in 9. The intercept-only table above therefore overstates zero inflation: part of the excess zeros was mean heterogeneity across size and sub-industry. The conclusion that holds under both specifications is that overdispersion is universal and zero inflation beyond NB2 is present in a minority of industries.

Establishment size accounts for most of the rest. The zero share falls from 0.751 in the 1–19 employee band to about 0.08 above 500 employees, and the median establishment TRIR rises from 0.00 to 3.61 across the same range; a ten-person site has so little exposure that zero is the modal outcome. (The fall is not strictly monotonic - 0.073 at 250–499 rises to 0.082 at 500–999 - and we note it rather than smoothing it.) Any benchmark comparing a site against an industry median without conditioning on size is comparing it against a number driven by how large the other sites are.

This is a statement about distributional shape, not about reporting behaviour. An overdispersed process and a mixture of compliant and non-compliant reporters can generate similar count distributions, and these data cannot separate them. Nothing here is evidence that under-reporting is absent. What the result does establish is that the bare zero share is not evidence that it is present, which is how that statistic is ordinarily used.

The distributional facts also bound what any downstream predictive layer can do. A variance-to-mean ratio of 241 in the largest size band, combined with a base rate that puts one recordable at roughly one per 10,870 eight-hour worker-shifts at the 2024 BLS private-industry rate ^[bls2026], is a rare-event regime in which maximum-likelihood logistic regression underestimates event probabilities ^{[king_zeng2001]} and in which the standard imbalance corrections - random over- and undersampling, SMOTE ^[chawla2002] - degrade calibration by strongly overestimating minority-class probability without improving discrimination ^{[goorbergh2022]}, ^{[carriero2025]}. Discrimination metrics are close to uninformative here; calibration is the property that determines whether a score can be acted on ^{[vancalster2019]}, ^{[vancalster2016]}.

6.6 Benchmark bands move on their own#

Peer groups defined as NAICS 3-digit × size band, compared across eight adjacent year pairs over roughly 440 matched cells per pair, are highly reproducible in ordering (median Spearman rho 0.862–0.933 across the p25, p50, p75 and p90 bands) and 95.7–98.5% of publishable cells persist between adjacent years. But the level of each band moves by a median of 9.5–14.3% per year. An establishment sitting exactly on last year's p75 would be roughly a tenth of the way off this year's p75 without any change in its own performance.

This bears directly on the question-answering task. A retrieval system asked "is this site's TRIR above the industry 75th percentile?" will return an answer that is approximately a coin flip near the band edge, and it will return it with the same fluency whether the site is clearly above, clearly below, or inside the year-to-year noise. The band is a real quantity with a real sampling distribution; the answer is presented as a fact.

6.7 Hour-of-shift: a claim withdrawn, and the conditions for ever making it again#

The author's earlier work circulated a shift-timing claim about when during a shift injuries peak ^{[chimmani2026]}. It is withdrawn here rather than restated, and its numbers are not reproduced anywhere in this manuscript, for a reason that is structural rather than a matter of degree: the pipeline described in this section could not produce such a figure at all. Form 300A is an annual summary. It carries case counts and total hours worked and no time-of-day, time-employee-began-work, or narrative field at all ^{[osha_ita_summary_dict]}. Hour-of-shift requires Form 300/301 case-detail data, which OSHA began publishing on a different and narrower establishment universe with the 2023/2024 collection cycle ^{[osha_ita_case_detail_dict_2026]}. Any future hour-of-shift figure must name its own dataset, year range and N, must not be read as a subset of the 300A panel analysed here, and must not inherit this panel's filing count.

Two further constraints apply, and both are decisive for how the finding may be phrased.

First, the phenomenon is named and roughly twenty-five years old. Tucker, Sytnik, Macdonald and Folkard called it the "2–4 h shift phenomenon" ^[tucker2000], and Folkard and Tucker report "a slightly heightened risk from the second to the fifth hour" in their review of shift work, safety and productivity ^{[folkard2003]}. A large-scale corroboration in a new sector would be a real contribution. A discovery claim is not available, and no corroboration is offered in this manuscript.

Second, and more seriously, a peak in the count distribution is not a peak in risk. Any statistic of the form "such-and-such a share of injuries occurs in the first four hours" is by construction a share of counts, and counts are governed by how many people are at work in each hour of shift. Folkard and Tucker found that once exposure is accounted for, risk rises approximately exponentially with time on shift, more than doubling by the twelfth hour relative to the first eight ^{[folkard2003]} - a pattern that an unadjusted count distribution, dominated by the hours in which most people are working, will not show. Hanecke and colleagues had to build estimated exposure models precisely because the hours-at-work denominator was unavailable in the German data ^{[hanecke1998]}, and no public US dataset supplies an hours-at-risk-by-shift-hour denominator either. MSHA's Accidents file carries SHIFT_BEGIN_TIME and an accident time on essentially all records ^{[msha_accidents_definition]}, and BLS publishes an "hours worked before event" dimension in its case-characteristics series ^{[bls_ca_documentation]}, so cross-source comparison of the distribution is feasible. The exposure denominator is not.

The defensible statement is therefore about the observed distribution of reported incidents, explicitly not about risk, with the denominator gap stated as an open problem rather than assumed away. We adopt that phrasing throughout and recommend it to anyone reusing the figure.

6.8 How the two failures compose#

The preceding subsections describe defects in a data corpus. Sections 4 and 5 describe defects in ungrounded language models: parametric recall that degrades on long-tail content ^{[kandpal2023]}, ^[mallen2023], training and evaluation regimes that reward a plausible guess over an abstention ^[kalai2025], and distributional representations that encode relatedness without encoding relation type, so that taxonomic siblings are the maximal-confusion class. These are independent failure modes with independent causes. They compose in three specific ways, and the composition is worse than either alone.

A retrieved number is not a validated number. Retrieval-augmented generation supplies non-parametric memory and marginalizes over retrieved documents ^[lewis2020]. That machinery grounds an assertion in a document. It carries no commitment about whether the document is internally consistent, and none of the standard RAG evaluation metrics test for it: attribution frameworks score whether a claim is attributable to its cited source ^{[rashkin2023]}, which a filing declaring 2.4 trillion hours per employee satisfies perfectly. A system that retrieves establishment 90427's filing and reports a TRIR of 0.00 has produced a fully attributable, fully traceable, entirely wrong answer. The derivation trace this paper advocates certifies that the conclusion follows from the record. Whether the record is admissible is a separate predicate, and it has to be computed.

Aggregation hides the defect from the reader and from the model. The single filing above is visible at establishment level - 7 employees, 16.8 trillion hours - and invisible in an industry aggregate, where it appears only as a suspiciously low rate. The failure mode is therefore worst at exactly the level of aggregation at which executives and benchmarking tools operate. A model asked for an industry rate has no signal in the retrieved aggregate telling it the aggregate is broken, and the resulting answer is low-variance across paraphrase and across sampling, so consistency-based hallucination detectors ^{[manakul2023]} and semantic-entropy methods ^[kuhn2023], ^{[farquhar2024]} will score it confident. Uncertainty estimation detects the model's uncertainty. It does not detect the corpus's.

Grounding narrows the error class without eliminating it, and the residual is measurable. The governing empirical result is Magesh and colleagues' preregistered evaluation of commercial legal research tools marketed as hallucination-free, which measured hallucination rates between 17% and 33% ^[magesh2025]. There is no basis for expecting a process-safety deployment to do better, and every reason - hierarchical, cross-referenced, edition-versioned, table-heavy source documents - to expect it to face harder retrieval conditions. Any claim in this paper is therefore a measured delta under stated conditions, never an elimination.

The constructive consequence is that plausibility screening belongs inside the architecture rather than upstream of it, as a checkable predicate over each retrieved record. The screen described in §6.3 is six deterministic predicates over declared hours, declared employees and case components; we call it that, and not automated reasoning, because it has no decision procedure over a formal semantics, no complexity characterization, and emits no proof object. Stating a filing's admissibility as a constraint problem - given declared hours, employee count and case counts, is this record consistent with the physical and regulatory constraints? - would produce an artifact that a formal-methods reviewer would recognize, namely an unsat core naming which constraints a specific filing violates ^{[barrett2021smt]}, ^{[demoura2008]}. We have not built that, and we flag it as future work rather than as a contribution. The precedent for formalizing a body of regulation as executable logic, with the formalization kept traceable to the statutory text clause by clause, is forty years old ^[sergot1986]; we found no OSHA successor to it.

A closing note on verification that checks the wrong invariant. The benchmark corpus described in Section 7 originally reported 68 of 68 items verified against live eCFR text with zero failures. An independent audit found that one item cited 29 CFR 1910.157(d) while its anchor text actually appears in paragraph (e)(2): the verifier confirmed only that the cited paragraph markers appeared somewhere in the retrieved section, so a clause-to-anchor mismatch was structurally invisible to it. The headline "68/68, 0 failures" was true and weaker evidence than it looked. The defect was repaired, an anchor-locality check was added, and the corpus re-verified. We report this because it is the same failure as the OSHA denominator, one level up: a validation procedure that returns a clean result on a broken record, because the property it tests is not the property that matters. Leakage between predictor construction and outcome labels has the same shape and is documented inside safety machine learning specifically - Baker, Hallowell and Tixier rebuilt an earlier construction-injury prediction study with independent human annotation to eliminate artificial correlation between predictors and predictands ^{[baker2020ai]} - and across 294 papers in 17 fields more broadly ^[kapoor2023]. In each case the pipeline ran, the tests passed, and the number was wrong.

The practical requirement that follows is unglamorous and is the precondition for everything else in this paper: state the dataset, the year range, the N, the screen, and the denominator, separately, for every number. Report calibration rather than discrimination where a score will trigger an action ^{[steyerberg2010]}, ^{[collins2015tripod]}. And treat the admissibility of a retrieved record as something the system computes and exhibits, rather than something the retriever's confidence score is assumed to have covered.

7. Discussion and Limitations#

7.1 What the artifacts do and do not license#

This paper contributes four things: a structural account of why ungrounded retrieval substitutes semantically adjacent answers in safety-critical technical question answering; an auditable encoding of a performance-influencing-factor vocabulary with an explicit crosswalk to the established frameworks it is drawn from; a rule engine whose screening output carries a derivation trace; and an open, source-verified benchmark corpus for measuring grounding.

Three of the four are instruments rather than results. The benchmark has not been run against any real system. The ontology has never been fitted to outcome data. The structural-equation work is a set of simulation studies of an estimator, not an analysis of injuries. Only the OSHA denominator analysis reports empirical findings, and its findings are about the integrity of a reported quantity rather than about safety performance.

We state this at the head of the discussion because the most common failure in this literature is the reverse. A benchmark that has been built but not run, an ontology that has been encoded but not validated, and a model that recovers its own generating parameters are all legitimate contributions, and none of them is evidence that anything works in a plant.

7.2 The benchmark measures a construct, not a population#

The corpus holds 68 items across eight regulatory domains, of which 63 are factual and 5 rest on a false premise, organized into 23 complete minimal pairs and 44 question families. Every item's citation was re-checked against the eCFR versioner API, and the committed verification report records 68 of 68 verified with zero failures across 19 CFR sections. That number establishes provenance and nothing else. It says the cited paragraph exists and contains the quoted anchor. It does not say the answer key is a good answer.

Four limits follow, none of them incidental.

There is no sampling frame. The items were written by one author to be hard in a specific way. They are not a sample from a population of questions a practicing engineer asks, and no accuracy figure computed on them generalizes to such a population. This is the same limit that applies to contrast sets generally ^{[gardner2020]} and to behavioral test suites ^{[ribeiro2020checklist]}: the instrument is diagnostic, not representative.

Answer keys were audited mechanically, not re-derived editorially. An independent verification pass over the corpus found a provenance defect that the original 68-of-68 check had missed: an item cited 29 CFR 1910.157(d) while its verbatim anchor lived in paragraph (e)(2). The section-level check could not see it. A paragraph-locality check was added and the item repointed. The lesson is that the population of remaining subtle key defects is probably not zero, and a second editorial reviewer is a prerequisite for using this corpus in a published comparison.

Scoring is lexical, and that has a price. A paraphrase that avoids every listed surface form of a required concept is scored as a miss. The choice was made because the benchmark's subject is whether a system distinguishes near-synonymous technical entities, and scoring that with an embedding model would use as the measuring instrument the very mechanism under study. But the contrast heuristic that separates "a rupture disk has no blowdown" from an assertion of blowdown operates on a fixed character window and misfires in both directions. Any adjacent-substitution rate reported from this harness is a screen. Automatic attribution evaluation is itself unreliable ^[yue2023], which is why the established practice is a human-adjudicated subsample with inter-rater agreement reported ^{[rashkin2023, honovich2022true]}. No human adjudication has been performed here.

The corpus is grounded in federal regulation, not in the consensus standards practitioners actually open. ASME BPVC, API 520 and 521, NFPA 70E and ISO 4126 are copyrighted and cannot be redistributed, so no item's key depends on their text; where an ASME paragraph identifier appears it is corroborated through a public federal source. The substantive consequence is that 46 CFR subpart 54.15 is marine equipment regulation, and the motivating incident came from general chemical plant practice. Six drafted items whose clause identifiers or numeric values were edition-dependent and could not be confirmed from a public primary source were quarantined rather than guessed: two NFPA 70E, two API 520, one API 754, one ISO 4126. They are excluded by construction and kept visible. This is a defensible proxy. It is not the same corpus a working engineer queries, and a grounding result on federal regulation does not transfer to API practice without being demonstrated there.

Two further caveats bound any future comparison run through this harness. Portions of these regulations circulate on the open web, so an ungrounded arm may have memorized some answers, which would compress the measured grounding difference; this should be probed with edition-specific values rather than assumed away. And the abstention utility weights (correct 1.0, abstained on an answerable item 0.25, abstained on a false-premise item 0.9, other incorrect -0.5, adjacent substitution -1.0) are a stated policy encoding a judgement about the relative cost of outcomes. They are not estimated from anything, and no number derived from them should be reported without naming the policy.

7.3 The costs of the ontology approach#

The four contextual categories encoded here are adopted, not invented. IDHEAS-G organizes 20 performance-influencing factors into exactly four context categories, and states directly that "PIFs are classified according to the four types of context: environment and situation, system, personnel, and task" ^[nureg2198]. Task and System are identical labels; Operational Context and Human Context are synonyms for Environment and Situation, and Personnel. The correspondence is one to one, and it maps further onto CREAM's nine common performance conditions ^{[hollnagel1998]}, SPAR-H's eight PSFs ^{[gertman2005]}, NUREG-1792's fifteen, and HSE's job, person and organisation headings ^[hse_pifs]. Presenting this vocabulary as novel would be indefensible, and we do not. What is contributed is the encoding, the crosswalk with per-row citations and asserted absences, and the traceability. That is a smaller claim, and it is the one the artifact supports.

The ontology form itself has costs worth naming rather than assuming away.

No reasoner runs. The Turtle file is parsed and structurally validated. It is never classified, and it has not been verified consistent under OWL 2 Direct Semantics. The engine is a Datalog evaluator over 21 rules and 20 factors, which gives a definite least-fixpoint semantics, guaranteed termination, and a derivation trace ^[green2013]. It does not give a proof certificate in any standard format and there is no soundness theorem relative to a formal semantics of the domain. Calling this automated reasoning in the computer-science sense would be an overclaim. Automated reasoning carries a proof obligation and emits a checkable artifact ^{[cook2021blog, backes2018]}; a production rule system does neither, and the algorithm at the heart of that tradition describes itself as a pattern matcher ^[forgy1982]. The honest distinction between symbolic and statistical components is not logic versus pattern matching, since modern solvers use learned heuristics throughout ^{[liang2016, alemi2016]}. It is that one class of system can hand you an object you can independently check.

Deduction is not causation. Rules over a factor vocabulary compute entailment. Cause and effect requires an interventional formalism ^{[pearl2019, halpern2005]}. This is not pedantry about vocabulary: the strongest prior ontology work in process safety transforms its OWL model into a multi-entity Bayesian network precisely because deduction alone cannot carry causal content ^[aziz2019]. Nothing in this engine's output should be read as a causal claim about why an incident occurred.

The expressivity choice is a real fork with real consequences. OWL's semantics are open-world and monotonic. Safety rules are frequently closed-world and defeasible: "no recorded exposure" is not "exposure did not occur," and regulatory text is full of exceptions and exceptions to exceptions. The standard move of bolting SWRL rules onto OWL forfeits the decidability guarantee that motivated choosing a description logic in the first place, as the submission itself states ^[swrl2004]. Answer set programming under stable-model semantics is better founded when defaults are needed ^[brewka2011], and the OWL 2 EL profile buys polynomial-time classification when the task is really large-scale taxonomic classification ^{[w3c_owl2, kazakov2011]}. We chose Datalog and said so. A different application should choose deliberately rather than by default.

Factor levels are inputs, and this is the gap the work does not close. Every established framework requires a trained analyst to rate each factor per scenario after a task analysis. So does this one. The engine cannot derive a level from data and cannot tell whether a supplied level is right. The observableProxy annotations, which suggest what might be measured to assess a factor from routinely collected records, are speculation and are labelled as such. None has been validated. The measurement problem that motivates the whole line of work is therefore named here, not solved.

The crosswalk is one coder's judgement. Ninety alignments carry match strengths assigned by one person, with no second coder, no inter-rater reliability figure, and no adjudication. Disclosure in three places is not validation. A reviewer disagreeing with specific rows would be on solid ground, and the boundary disagreements are real: IDHEAS-G files procedures under Personnel where a practitioner might file them under System or Task. That is exactly the interpretive looseness Groth and Mosleh flagged as the field's standing problem, noting that PIFs in use are not defined specifically enough for consistent interpretation and that few rules govern the creation of PIF sets ^[groth2012].

Domain transfer is not licensed by the sources. Every framework crosswalked here except HSE's was built for nuclear power or aviation, and NUREG-2198 states that existing methods were developed for procedure-based response to internal events at power in nuclear plants and "are not necessarily adequate to model human actions ... in other domains" ^[nureg2198]. Applying this vocabulary to chemical manufacturing is an extension. It is also why SPAR-H's multipliers are absent: they were fitted to nuclear operations and porting them without revalidation would not be defensible.

Field uptake of this class of artifact has been close to nil. Thirty years of automated HAZOP research has produced very few approaches used in the chemical process industry, and the reported bottleneck is that the quality of the required specific and generic knowledge is often insufficient ^{[single2019, single2020_jlp]}. The neurosymbolic literature has arrived at the same bottleneck from the other side: attaching a prover to a language model relocates errors into the formalization step rather than eliminating them ^{[olausson2023linc]}, and autoformalization accuracy swings widely by task type with token-entropy uncertainty failing to flag the errors ^{[ganguly2025]}. Both literatures point at getting the axioms right, and neither has solved it.

There is a theoretical tension we do not resolve. This is an additive contextual-factor model. Rasmussen argues that task analysis focused on deviations should be replaced by modelling work-system constraints and boundaries ^{[rasmussen1997]}, and STAMP treats safety as constraint enforcement across a control structure rather than as a sum of contributing factors ^{[leveson2004, leveson2012]}. We do not cite either as support, because they are not support. A reader persuaded by the systems-theoretic position should read this artifact as recording assertions in a checkable form, which has value independent of whether the underlying accident model is the right one.

7.4 Statistical and causal-inference limits#

The structural-equation component of this work exists to bound what a latent variable model can support, and most of the answer is negative. Four simulation results, all reproducible from the repository, make the case.

Fit cannot support the direction of an arrow. Data generated from X -> Y and fitted under both X -> Y and Y -> X returns identical coefficients (0.4927), identical standard errors, and chi-square agreeing to better than 1e-6, because the two models imply the same covariance matrix. This is the equivalent-models result ^{[maccallum1993]}, and it means any procedure that selects a direction by comparing fit is selecting on nothing.

Reverse causation fits well. Data generated in a world where injuries degrade safety climate, then fitted as climate predicting injuries, returns beta = -0.570 with SE 0.015, CFI 0.9999, RMSEA 0.004, SRMR 0.007. Every conventional cutoff passes and the direction is backwards. This is not a contrived arrangement: injuries have been reported to predict later safety climate more strongly than climate predicts later injuries ^[beus2010].

An omitted common cause leaves no trace. With a confounder in the model the coefficient is 0.173 against a generating direct effect of 0.200; omit it and the coefficient becomes 0.495, an error of 0.295, while CFI is 1.000 and RMSEA is 0.000. Fit indices are computed from the covariance matrix of the indicators in the model, so a variable that was never measured cannot degrade them.

Coefficients conditional on a mediator are not what readers take them to be. With X -> M -> Y alongside X -> Y, the direct effect of X with M in the model is 0.182 and the total effect is 0.442. Both are correct estimates of different quantities, and a table listing several coefficients side by side as comparable contributions to risk without saying which is which cannot be interpreted ^{[westreich_greenland2013, cole_hernan2002]}.

None of this means structural models are useless. Structural coefficients are causal parameters relative to an assumed graph ^{[bollen_pearl2013]}. The point is that the causal content comes entirely from assumptions the analyst supplies, and those assumptions should be written down and defended rather than left implicit in an arrow diagram. Hedging with the word "predicts" while reasoning causally is less transparent than stating the causal claim and its conditions ^{[hernan2018, grosz2020]}.

Three further limits apply to any risk score of this shape.

A latent score is not a probability. It has no link function, no exposure denominator and no time window, so no threshold on it can be justified. Separately, factor scores are not uniquely determined by a fitted model: on the worked example, two equally valid sets of scores for the same factor correlate as low as 0.54 to 0.59 ^{[steiger1979, grice2001]}. A per-crew or per-shift score inherits that indeterminacy however well the model fits.

Discrimination is close to uninformative at safety base rates. At the 2024 private-industry total recordable case rate of 2.3 per 100 full-time equivalent workers ^[bls2026], the repository's arithmetic gives roughly one recordable per 10,870 eight-hour worker-shifts. A classifier at 80% sensitivity and 95% specificity yields a positive predictive value of 0.0015, or about 680 alerts per true event; at 80% and 99%, 137 alerts; at 90% and 99.9%, still 13. Across base rates spanning a factor of 600 the AUC in our simulation moves only between 0.677 and 0.692 with no trend, while the Brier score falls by more than two orders of magnitude. Reporting AUC without calibration is therefore close to reporting nothing that bears on deployment ^{[vancalster2019, steyerberg2010]}. The class-imbalance corrections commonly reached for at this point make it worse: random over- and undersampling and SMOTE degrade calibration by strongly overestimating minority-class probability with no gain in AUC, and the apparent sensitivity gains are obtainable by moving the threshold ^{[goorbergh2022, carriero2025]}.

The estimator itself carries known limits. It is limited-information and two-step rather than full-information maximum likelihood, so chi-square is an upper bound and a reviewer comparing against lavaan or Mplus will get different values. Analytic standard errors are too narrow, with coverage that does not improve with sample size; the bootstrap is the supported path, and the bootstrap coverage comparison itself rests on only 40 replications, giving a Monte Carlo standard error of about 3.4 percentage points. Ordinal indicators are treated as continuous, where polychoric correlations would be correct. Only recursive models with congeneric measurement blocks are supported. Nothing here has been fitted to real injury data; recovering a parameter from data generated by the same model checks the estimator, not the model, and certainly not any claim about workplace safety.

7.5 Limits of the empirical denominator analysis#

The one component reporting empirical findings analyses 2,801,064 deduplicated OSHA Injury Tracking Application Form 300A filings for reporting years 2016 through 2024 ^[osha_ita]. Under the default plausibility screen, 57,857 filings (2.07%) are flagged; they hold 96.68% of all reported hours and 1.37% of all reported cases. The pooled aggregate total recordable incident rate is 0.134 unscreened and 3.983 screened, a ratio of 29.7.

The most important limitation is what that ratio is not. It is not a constant. Across individual years the ratio ranges from 1.39 to 249.3, because it depends entirely on whether a given year happens to contain an extreme filing: one filing in reporting year 2019 declared 7 employees and 16,831,620,723,179 hours worked with zero recordable cases, and holds 88.8% of all hours ever reported to the ITA. The screened aggregate is stable across years (3.15 to 4.60); the unscreened one is not (0.016 to 2.92). The stable finding is structural: a small minority of filings is implausible, those filings hold a disproportionate share of the denominator, and removing them raises the aggregate substantially. The specific multiplier should not be quoted as a number.

This matters for how the author's prior work is cited. That repository ran the same class of screen over a narrower panel and reported a headline correction multiplier ^{[chimmani2026]}. The flagged share replicates closely here (2.07% pooled, 1.56% to 2.60% by year). The multiplier does not, and could not, because the panel, the screen and the year coverage differ; it is superseded by the figures above and is not restated here as a quantity. The correction it implies is that the multiplier is an artifact of panel composition.

Further limits:

• The screen identifies filings whose numerator and denominator cannot both be right. It does not identify which field is wrong, and it is not a fraud detector.
• The ITA universe is a mandated administrative collection from a selected slice (250 or more employees, plus 20 to 249 in listed higher-hazard industries under 29 CFR 1904 Subpart E), not a probability sample ^{[osha_ita_users_guide]}. The screened aggregate sits well above the BLS private-industry rate for 2024 ^[bls2026], which is what a universe skewed towards larger, higher-hazard establishments would produce. The two are not directly comparable and are not compared here. OSHA publishes its own comparison of the two series ^{[osha_ita_bls_comparison]}, and it should be read before anyone treats the gap as a finding.
• A zero recordable count can mean an incident-free year or an unrecorded one. The count models can measure how many zeros there are and whether they exceed what overdispersion predicts; they cannot separate those causes.
• The count models are intercept-only, carry no covariates, report no standard errors, and support no causal claim.
• Establishment identity across years is not verified, so no panel analysis is attempted.
• Exchangeability of these filings has not been established for causal work. The reporting-behaviour heterogeneity documented above is direct evidence against assuming it.

One provenance question must be settled before any hour-of-shift result appears alongside this analysis. Form 300A summary data contain hours worked and case counts but no time-of-day or time-employee-began-work field; hour-of-shift requires Form 300/301 case detail, which OSHA began publishing for a narrower establishment universe with the 2023/24 collection cycle ^[osha_ita]. The repository described here uses only 300A files and contains no hour-of-shift analysis. The author's earlier shift-timing figure is accordingly withdrawn from this manuscript rather than restated: it is not derivable from this panel, no committed artifact here reproduces it, and any future statement of it must carry its own dataset, year range and N.

If such a result is reported, two things follow. First, an unadjusted count distribution is not a rate. A peak in the second hour of shift describes when reported injuries occur, not when risk is highest, unless hours at risk by shift hour are controlled, and no public dataset supplies that denominator; the methodological precedent had to construct estimated exposure models for exactly this reason ^{[hanecke1998]}. Second, elevated early-shift accident risk is named, published prior art: the "2-4 h shift phenomenon" ^[tucker2000], with the exposure-adjusted picture showing risk rising approximately exponentially with time on shift so that the twelfth hour carries more than double the risk of the first eight ^{[folkard2003]}. Presented with those citations, a large-scale result in a new sector is independent corroboration, which is a real contribution. Presented without them, it is a rediscovery.

7.6 The gap between benchmark performance and field safety outcomes#

This is the limitation we most want a reader to carry away, because it is the one a good benchmark score most easily obscures.

Grounding reduces hallucination; it does not eliminate it. Commercial retrieval-augmented legal research tools marketed on freedom from hallucination were measured, in a preregistered evaluation, to hallucinate between 17% and 33% of the time ^[magesh2025], and hallucination rates in the same regulated domain have been profiled far higher for ungrounded models ^[dahl2024]. Retrieval can also make things worse: irrelevant retrieved context degrades accuracy ^{[shi2023distracted]}, relevant evidence positioned mid-context is underused ^{[liu2024lost]}, and behaviour is unstable when retrieved evidence conflicts with parametric belief ^[xie2024]. No sentence in this paper should be read as claiming that a grounded architecture prevents the substitution failure. The claim available is a measured difference with a confidence interval under stated conditions, and we have not yet measured it.

Beyond that, a benchmark score sits several unmeasured links away from a safety outcome. The chain runs: the retriever returns the governing clause; the generator answers from it rather than from memory; the engineer reads the answer; the engineer accepts or corrects it; the accepted answer changes a design or procedural decision; the changed decision alters barrier integrity; the altered barrier prevents an incident. This work instruments the first two links. The third and fourth are human-factors questions the benchmark does not touch, and they are not benign: models conform to stated user beliefs ^[sharma2024] and uncritically accept incorrect user premises ^[dahl2024], so a junior engineer asking a leading question is at greater risk than a senior one asking an open question, which is the inverse of what a screening tool should deliver. The motivating incident was caught by decades of experience. A tool whose safety depends on the reviewer already knowing the answer has not added a barrier.

The last two links are, at the granularity where such a tool would be deployed, close to unmeasurable. This is Hopkins's zoom effect: a rate is meaningful for an industry and meaningless for a site, and the question of whether an indicator measures how well safety is being managed turns on whether there are enough countable events in the relevant period at the relevant level of aggregation to speak of a rate at all ^{[hopkins2009]}. Our own arithmetic gives the scale: pinning a shift-level base rate to plus or minus 10% relative requires about 384 events, which at the 2024 national rate is 4.18 million worker-shifts of observation. A single site cannot run that experiment. This is also why we do not position any output of this work as a leading indicator in the sense the standards use. HSE defines a leading indicator as a routine systematic check that key actions or activities are undertaken as intended, paired with a lagging indicator for each critical risk control system ^{[hse2006_hsg254]}, and API RP 754's Tier 3 and Tier 4 indicators are challenges to safety systems and management-system performance ^[api754_4th]. In all three of the major frameworks a leading indicator is an assurance check on a named barrier, not a model output, and the precursor-to-major-accident relationship that underpins the tier logic is described by API itself as a belief. Claiming alignment without arguing for it would be unearned.

The field-level record should temper expectations independently of anything we did. There is a documented lack of validated leading indicators that reliably classify sites by safety risk ^[poh2018]; the four predictive families are modelled independently of one another with no cross-validation between them ^{[hallowell2020]}; construct validity is a recurring problem across the domain's structural-equation applications ^[xiong2015]; and target leakage has been identified and corrected inside this literature by its own leading group, which rebuilt an earlier prediction study with independent human annotation specifically to remove artificial correlation between predictors and predictands ^{[baker2020ai, kapoor2023]}. Forty years of category refinement in human reliability analysis has not resolved quantification: human error probabilities vary method to method and analyst to analyst ^{[nureg2198, nureg2127]}, the number of performance-shaping factors in use ranges from one to more than fifty without evidence that expansion improved anything ^[boring2010], and a considered review concluded that the barriers to the quantification needed for summative analysis are currently too substantial ^[french2011]. A paper proposing better categories without engaging that record would be answering a question the field has moved past. Our position is that the useful move is measurement and traceability rather than another taxonomy, and we have delivered the traceability while leaving the measurement open.

7.7 Generalizability#

The empirical work covers one country, one regulatory regime, one reporting instrument and one nine-year window. The benchmark covers eight US federal regulatory domains, indexed against federal regulation rather than the consensus standards used in practice. The ontology encodes frameworks developed for nuclear power and aviation and applies their vocabulary to chemical manufacturing, an extension the sources do not license. The simulation studies describe properties of an estimator under models we specified, which is by construction the most favourable setting an estimator ever sees.

The artifacts are also the work of one author. They have no external users, no independent replication, and no adoption record, and we describe them on that basis. Where a repository is cited in this paper it is cited as the provenance of a stated number and as a work sample, not as evidence that anyone else has found it useful.

7.8 What would change our minds#

We name the falsifiers, since a paper that cannot say what would refute it is not making a claim.

The structural account of adjacent substitution predicts that dense bi-encoder retrieval shows the largest co-hyponym confusion, that lexical retrieval may beat it where exact terminology carries the distinction, and that a hard device-type metadata filter helps more than a better embedding. If a retriever ablation on this corpus showed no such ordering, the mechanism as stated would be wrong. The minimal-pair design predicts that paired accuracy falls substantially below per-item accuracy for systems that pass on topical similarity; if paired and per-item accuracy tracked each other closely across arms, the item construction would have failed. The pseudo-grounded arm predicts that retrieval over a plausible but unauthoritative corpus can perform worse than no retrieval at all; if it reliably landed between the ungrounded and grounded arms, the argument for running three arms rather than two would collapse.

The ontology's central claim, that a derivation trace makes a screening conclusion auditable, would be undermined if analysts given the trace disagreed about factor levels as much as analysts given a spreadsheet. That is a testable human-factors study and it has not been run. And the whole line of work rests on a proposition it has not tested: that grounding failures of this class occur in the field at a rate that justifies the engineering. The motivating incident is n=1 and anecdotal. It motivates the work; it demonstrates nothing.

8. Conclusion and Future Work#

8.1 What this paper contributes#

Four things, none of them a new taxonomy.

A structural account of semantically adjacent substitution. The motivating incident, in which a configured assistant answered a rupture-disk question with pressure relief valve content, is over-determined rather than unlucky. Four mechanisms converge on it. No retrieval was present, so nothing in the system met the definition of retrieval-augmented generation and every token came from pretraining ^[lewis2020]. Rupture-disk content is long-tail relative to relief valve content, and parametric factual accuracy tracks pretraining frequency ^{[kandpal2023, mallen2023]}. Abstention is scored at zero while a confident guess sometimes scores above zero, so under uncertainty the model emits its most probable plausible completion ^[kalai2025]. Distributional representations encode that two terms are related, not how, which makes taxonomic siblings the maximal confusion class ^{[mrksic2016, levy2015, shwartz2017]}, and retrievers built on them fail minimal pairs that turn on a single semantic distinction ^{[weller2024nevir]}.

The fourth mechanism is the one we would defend hardest. API 520 Part I documents both device types in a single standard with a shared definitions section, and the word disc is polysemous inside it: in a relief valve it names the closure element whose travel defines lift and whose seat contact defines reseating, while a rupture disk is an entire device ^[api520p1]. ASME co-locates the two under one parent category ^{[asme_viii_ug127]}; ISO splits them across separate parts ^[iso4126]. Chunk-and-embed an index over the co-locating family and the nearest neighbours of a rupture-disk passage are relief-valve passages. Which standards family is indexed is therefore a safety-relevant corpus design decision, and it argues for device-type metadata filtering rather than undifferentiated top-k retrieval. We are not aware of prior work stating the failure in these terms.

An adopted ontology, published machine-readable with its crosswalk. The four context dimensions are the four PIF context categories of IDHEAS-G, adopted unchanged: environment and situation, system, personnel, task ^[nureg2198]. Two of our labels are identical to the NRC's and two are synonyms, and the twenty factors are the NRC's twenty. We say so first because it is what a reviewer would otherwise find and hold against the work, and because forty years of PSF proliferation is a documented problem rather than a gap to fill ^{[boring2010, groth2012]}. What the artifact adds is a machine-readable encoding, ninety reified alignments to SPAR-H ^{[gertman2005]}, CREAM ^{[hollnagel1998]}, HFACS ^{[shappell2000]} and the HSE PIF list ^[hse_pifs], six asserted absences, and a rule engine that emits a derivation trace for each conclusion. No OWL reasoner is run over the file; it is parsed and structurally validated, never classified. The ninety match strengths are one coder's unadjudicated judgement with no second coder and no inter-rater figure. Nothing in the ontology is validated against injury outcomes.

Terminological discipline about what the architecture does. We do not call deterministic rule evaluation automated reasoning. Rete is a pattern matching algorithm by its author's own title and carries no soundness theorem relative to a logical semantics ^[forgy1982]; automated reasoning in its field-internal sense requires a specified semantics, a sound procedure and a characterised complexity result ^{[cook2021blog]}. Where the artifact evaluates rules, we call it rule evaluation or Datalog-style evaluation ^[green2013]. We also do not claim the system determines cause and effect, because deductive entailment sits at the association rung and causation requires an interventional formalism ^{[pearl2019, halpern2005]}. The architectural pattern we follow is the one AWS documents for Bedrock Automated Reasoning checks: an unsound natural-language-to-logic translation step whose output feeds a sound validation step, with the boundary and the scope of the guarantee stated explicitly ^{[aws_arc_docs, backes2018]}. Autoformalization accuracy is the live risk in that split, and it is measurably unstable across task types ^{[ganguly2025]}.

An open benchmark, and no results from it. The corpus holds 68 items across eight regulatory domains, 63 factual and 5 category-error, organised into 23 complete minimal pairs and 44 question families, every item keyed to a clause in public federal text (29 CFR, 40 CFR, 46 CFR) with a verbatim anchor span. All 68 verify live against eCFR across 19 sections, with an anchor-locality check that catches an anchor sitting outside the paragraph it cites; six drafts whose sources could not be verified are quarantined rather than published. The primary statistic is paired accuracy, following the contrast-set and negation-retrieval methodology ^{[gardner2020, weller2024nevir]}, because a system answering every pressure-relief question with generic pressure-relief content can score respectably per item while getting the distinction wrong every time.

No system has been evaluated. Every number the harness prints out of the box comes from a mock adapter with outcome probabilities fixed in advance, and every artifact it writes is stamped as a demonstration fixture. We report no grounding delta, and we note that commercial retrieval systems in another regulated domain hallucinate 17 to 33 percent of the time under preregistered evaluation ^[magesh2025], so the delta, when measured, will be a reduction under stated conditions rather than an elimination.

8.2 Two empirical results that constrain the rest#

Reanalysis of 2,801,064 OSHA ITA Form 300A establishment filings covering 2016 through 2024 finds 2.07 percent failing an hours-per-employee plausibility screen while carrying 96.68 percent of all declared hours. Aggregate total recordable incident rate computed on the raw panel is 0.134; computed after screening it is 3.983, a factor of 29.7. This does not reproduce the 0.45 and 3.41 figures from the author's earlier repository ^{[chimmani2026]}, which used a different panel and screen; by year the multiplier here ranges from 1.39 to 249, so it is not a constant and should not be quoted as one. The screened figure is also not an official statistic and is not comparable to BLS SOII estimates without engaging OSHA's own published comparison of the two ^{[osha_ita_bls_comparison]}.

Simulation on data generated from known models bounds what a latent-variable risk score can support. For a five-latent model with three indicators each, 80 percent power for the test of close fit arrives at N = 153, reproducing MacCallum, Browne and Sugawara's published table to within one observation across 257 cells ^{[maccallum1996]}; separating a path of 0.45 from one of 0.30 at the same power needs about 770 observations, and separating -0.25 from -0.20 needs about 6,884. At the 2024 US private-industry recordable rate of 2.3 per 100 full-time equivalents ^[bls2026], a shift-level classifier at 80 percent sensitivity and 95 percent specificity yields roughly 680 alerts per true event, and across a 600-fold span of base rates the AUC moves only between 0.677 and 0.692. None of this is a finding about workplaces. It is a finding about estimators, and it rules out a class of claims that circulate without sample-size or calibration evidence.

8.3 What comes next#

Run the benchmark. Three arms, preregistered before any system is touched: ungrounded, pseudo-grounded over a plausible but unauthoritative corpus, and grounded over a version-pinned authoritative index with device-type metadata filtering. The middle arm is the one nobody measures and the one most enterprise deployments actually occupy. Analysis must be paired to match the design, retrieval and generation metrics must be reported separately, human adjudication must be the primary attribution result with automatic scoring as a screen ^{[rashkin2023, yue2023]}, and run-to-run variance must be an outcome rather than a nuisance ^[dokas2026]. Contamination in the ungrounded arm should be probed with edition-specific numeric values, not assumed away. Copyright forecloses redistributing ASME, API or ISO text; the releasable artifacts are the question set, the clause-keyed answer keys, the metadata schema and the harness.

Formalise recordability. We found no logical encoding of 29 CFR 1904, and the nearest precedent is forty years old ^[sergot1986]. Plausibility screening is structurally a constraint-satisfaction question: given declared hours, employee count and case counts, is a filing consistent with an admissible set of constraints? An SMT encoding would return the artifact formal-methods reviewers ask for, an unsat core naming which constraints a filing violates, in place of a Boolean flag. The formalism should be chosen deliberately, since bolting SWRL rules onto OWL forfeits the decidability that motivated the description logic ^[swrl2004] and defeasible regulatory rules with exceptions are better served by stable-model semantics ^[brewka2011] or by the polynomial-time EL profile where the work is really large-scale classification ^{[w3c_owl2, kazakov2011]}. There is currently no accepted way to report how much of a regulation a formalisation captured, which is itself worth proposing.

Instrument the contexts instead of rating them. Every framework from THERP through IDHEAS-G requires a trained analyst to assign a level per factor per scenario after a task analysis, and none specifies how to derive levels from data an operator already collects ^{[nureg2198, groth2012]}. That measurement problem, not the taxonomy, is where a contribution lives. Three public sources now make part of it approachable: OSHA ITA Form 300/301 case detail, which for the 2024 cycle holds 688,649 incident records with time started work and time of incident both present on 84.0 percent of them, fields defined in OSHA's case-detail dictionary ^{[osha_ita_case_detail_dict_2026]}; MSHA Accidents, with 274,567 records from 2000 onward and shift begin time on 99.6 percent ^{[msha_accidents_definition]}; and the BLS SOII case-characteristics series, which publishes an official hours-worked-before-event dimension ^{[bls_ca_documentation]}. Agreement across three independently collected sources would be a defensible claim; agreement is not guaranteed, and the schemas drift between years in ways that silently produce null columns if concatenated naively.

Solve, or openly concede, the exposure denominator. Should a future analysis on case-detail data observe a peak of injuries early in the shift, that peak is not a risk peak. Once exposure is modelled, risk rises approximately exponentially with time on shift, more than doubling by the twelfth hour relative to the first eight, on top of a slightly heightened early-shift period that has a name and a 25-year history as the 2-4 h shift phenomenon ^{[folkard2003, tucker2000]}. Hanecke and colleagues faced exactly this with 1.2 million German accidents and no working-time data, and built estimated exposure models rather than reporting counts ^{[hanecke1998]}. No public US dataset supplies hours at risk by shift-hour. Either that model gets built, or the claim gets restricted, in the paper's own words, to an unadjusted distribution of reported incidents.

Adjudicate the crosswalk and validate against outcomes. The ninety alignment rows need a second coder and a reported reliability figure. The screening bands the rule engine emits are ordinal labels from rules marked as convention and have never been compared against injury or incident rates. Until they are, they are a vocabulary, not a measurement.

Report prediction models the way prediction models are reported elsewhere. We found no externally validated, calibrated incident-risk prediction model in occupational or process safety, and no safety paper reporting a calibration curve or a temporal external validation in the TRIPOD sense ^{[collins2015tripod]}. Discrimination is the wrong headline at these base rates ^{[vancalster2019, steyerberg2010]}, class-imbalance corrections destroy calibration without buying AUC ^{[goorbergh2022, carriero2025]}, and net benefit at thresholds an EHS organisation would actually act on is the metric the field has not adopted. Bringing that reporting discipline across is a smaller contribution than a new model and a more useful one.

8.4 A standing tension we do not resolve#

The architecture here decomposes context into factors and evaluates rules over them. Rasmussen argued that task analysis focused on deviations should be replaced by modelling work-system constraints and boundaries ^{[rasmussen1997]}, and Leveson's STAMP treats safety as constraint enforcement across a control structure rather than as a sum of contributing factors ^{[leveson2004]}. We do not cite either as friendly authority, because they are not. A factor-based model inherits their objection, and the strongest defence available is a narrow one: the artifacts here are instruments for auditing what a system asserts and where it got it, not a general accident model. Whether an auditable factor ontology can be reconciled with a control-theoretic account of accidents is open, and we would rather leave it marked open than paper over it.

8.5 Status of the artifacts#

Four repositories accompany this paper: the grounding benchmark and harness, the human-factors ontology and rule engine, the OSHA ITA reanalysis pipeline, and the SEM estimator with its simulation studies. Each states in its own README what it does not establish. Synthetic material in every one lives under a synthetic/ directory with a seeded generator and a header declaring it synthetic, and no headline number in this paper derives from it. Two defects found during an independent audit and corrected before submission are recorded in the repositories rather than silently fixed: an item citing a clause whose anchor text lived in a different paragraph, and a README paragraph attributing to NUREG-2127 a conclusion the report partly contradicts. We expect the population of remaining subtle defects in 68 hand-built answer keys is not zero, and the keys have not been re-derived editorially by a second reviewer.

<!-- BEGIN GENERATED REFERENCES -->

References#

Every key cited in the body appears below, and every entry below is cited in the body. Each reference was checked against a primary or publisher-of-record source; the evidence URL used for each check is recorded in paper/citations_verified.json alongside the entry. 161 of the 197 entries carry a verification note recording what the check found - a correction to the reference as first drafted, a detail that still needs confirming against print pagination, or an explicit statement that the reference is correct as it stands. Those notes are reproduced here rather than silently folded into the reference strings, so a reader can see exactly what was checked and what was not.

One caveat that belongs in plain sight: page ranges for several conference papers are marked in their notes as needing confirmation against the printed proceedings. Venue, authorship and DOI are confirmed independently of pagination in every such case.