A pipeline gives an EHS team a table. An ontology gives them the site, its filings, the rule that fired, the right peers, and a governed way to record a judgement that changes the numbers and leaves a trail.
- Role
- Designer and author: spec, runtime, actions, functions, tests and documentation
- When
- October 2026
- Stack
- Python, pandas, a JSON ontology spec, SQLite and JSON export
- Status
- Merged to main of ehs-osha-analysis; demonstrated on a synthetic fixture
- Problem
The pipeline already shows that a small share of implausible filings can move the national injury rate by a large factor. An EHS team needs to act on that per site and per filing, not read it from a summary table.
- What I did
Modelled the data as linked object types, made analyst judgement a set of validated actions with an append-only log, and wired every calculation back to the pipeline code that the tests already check.
- Status
Runs on a synthetic fixture linked to real reference objects. The path for the real filings uses the same loader but has not been run, because the raw files are not in the repository.
- Why it matters
A benchmark built from public injury records is only as good as the governance over which records count. This design keeps that governance explicit, reviewable and reversible.
The decisions an EHS team needs to make
My public analysis of OSHA's Injury Tracking Application data covers 2,801,064 establishment filings from 2016 to 2024. It found that a small share of filings with implausible hours worked controls most of the national denominator. In 2019, filings that failed the plausibility screen held 99.6% of reported hours, and screening moved the aggregate total recordable incident rate (TRIR) from 0.016 to 4.018. A result like that is useful to a researcher as a table. An EHS team needs to work with it one site and one form at a time.
I started the design from the decisions, not from the columns. Each object type and link exists because one of these questions needs it:
- Should this form count? A filing can fail a screen rule, look wrong to an analyst for a reason the screen does not catch, or be correct but not comparable to peers.
- Which rule fired, and how much does that rule matter? Pooled across years, the rule for more than 4,500 hours per employee covers 96.65% of all reported hours. One rule can dominate a national figure.
- How far does this year's headline rate depend on its worst filings? A year-level object puts each filing beside the aggregate it moves.
- Which peers should this site be compared with? Peer groups are industry by employee-size band, and some cells are too small to publish.
- Who decided to exclude a filing, when, and why? A judgement that changes a benchmark has to be recorded and reversible.
Object model
The model is a single JSON spec. Filing is the centre: it is the unit an analyst judges and the only object with editable properties. Everything else gives a filing its context: the site that filed it, the year it belongs to, the industry and peer cell it is compared with, and the screen rules that fired on it.
Why each object type exists
| Object type | Backed by | Why it exists |
|---|---|---|
| Establishment | The harmonised filings frame (latest filing per site) | The unit an EHS manager is accountable for and the unit OSHA inspects. Company name lets a corporate team group its sites. |
| Filing | The harmonised frame after the pipeline's screen and peer keys | Answers "should this form count?". It holds the Form 300A case counts (columns G to L), the hours denominator, the six screen flags, the screen verdict as a read-only property, and the analyst writeback. |
| ReportingYear | quality_by_year.csv and metrics_by_year.csv | Shows how far a year's headline rate depends on its worst filings, so a reviewer sees the filing and the aggregate it moves side by side. |
| Sector | quality_by_naics2.csv | Shows how exposed a parent NAICS sector is to implausible hours. |
| Industry | The 209 NAICS 3-digit codes in peer_percentiles_pooled.csv | The industry level used for peer groups. It carries only the official NAICS 2022 title; codes that are not 2022 subsectors keep the bare code and are marked as having no official title, rather than getting an invented name. |
| PeerGroup | peer_percentiles_pooled.csv: 905 cells, 647 publishable | Answers "who are the right peers?". Size bands come from employee counts, not OSHA's size code, which changed meaning in 2023. Cells below the minimum size are kept and marked not publishable, so the model says why a comparison is unavailable. |
| QualityFlag | The pipeline's flag columns and quality_flag_prevalence.csv | Answers "which rule fired, and how much does it matter?". The no-injury contradiction flag is carried but marked as not counting toward the verdict, so it informs without excluding. |
Source: docs/ontology.md · materialise.py
Design choices in the model
- Every object carries a
sourceproperty, so a reader can tell a synthetic object from one built from committed outputs. - The spec is checked when it loads, and every error is reported at once: keys must exist, primary keys cannot be null or editable, links must point at real types with matching key types, and actions may edit only properties marked editable.
- Loading coerces each value to its declared type and rejects nulls in required properties and duplicate primary keys. A link integrity report counts resolved, null and dangling keys for every link.
- The Filing to QualityFlag link is backed by the boolean flag columns the pipeline already writes, rather than a second join table that could drift from them.
Actions and the audit log
The only way to change an object after it loads is to apply an action. Each action declares typed parameters, submission criteria that check the filing's current state, and the exact properties it edits. A rejected action changes nothing and writes nothing to the log.
| Action | Parameters | Submission criteria | Effect |
|---|---|---|---|
| flag_filing_implausible | filing; coded reason; note, required when the reason is "other", at most 500 characters | Not already excluded by the screen; not already flagged | Sets the analyst flag with its reason and note |
| clear_analyst_flag | filing; note of 10 to 500 characters, required | The filing is flagged | Clears the flag and records why |
| exclude_from_benchmark | filing; coded reason; note, required when the reason is "other" | Not already excluded by the screen; not already excluded | Takes the filing out of benchmark aggregates |
| restore_to_benchmark | filing; note of 10 to 500 characters, required | The filing is excluded | Puts the filing back and records why |
Source: spec.json · runtime.py
Why analysts flag rather than delete
Deleting a suspect filing would make the benchmark look cleaner and make the decision invisible. Here an analyst's judgement is a property on the filing, set by a named action, with a coded reason. The filing stays in the data, its screen flags stay visible, and the decision can be reversed by another action that must also say why. Reasons are a fixed list, such as hours inconsistent with operations, company-wide hours reported, or employee count mistyped, so decisions can be counted and reviewed by type. Free text is only required where the coded reason is "other".
I kept two separate judgements because they mean different things. A flag says "this form is wrong". A benchmark exclusion says "this form may be right, but this site is not comparable with its peers". Both take the filing out of the aggregates, and both are recorded separately from the automated screen.
The screen verdict itself cannot be edited by any action. If a reviewer disagrees with the screen, the right change is to the screen's bounds, which is a method decision, not a record edit. The plausibility_screen function lets a reviewer try other bounds and see the verdicts without editing any object.
The audit log
Every applied action appends an entry with an id, the action type, the actor, a UTC timestamp, the parameters, and the before and after value of every property it edited. Any one filing's edit history can be read back. The log is exported with the objects, so the record of who changed what travels with the data it changed.
Functions that reuse the validated pipeline
The pipeline behind this ontology is already tested, including tests that check its published figures against the committed outputs. A second copy of a rate formula inside the ontology would be a second thing to get wrong. So each function is a thin adapter that calls the pipeline module the spec names under reuses, and the spec check fails if that module path does not import.
| Function | Input | Reuses | What it returns |
|---|---|---|---|
| plausibility_screen | Set of filings, optional bounds | quality.apply_screen | A what-if verdict per filing; edits nothing |
| filing_rates | One filing | load.derive_fields | Establishment TRIR and DART rate re-derived from the stored counts and hours |
| aggregate_trir | Set of filings | quality.aggregate_trir | Ratio of sums over included filings |
| aggregate_rates | Set of filings | metrics.compute_rates | TRIR, DART, LTIR and severity, screened and unscreened |
| peer_position | One filing | peers.DEFAULT_PERCENTILES | The percentile band of the peer group the filing's TRIR falls in, or a status saying why there is no band: the filing is excluded, has no rate, or sits in a cell too small to publish |
"Included" means the filing passes the screen and has neither an analyst flag nor a benchmark exclusion, so actions change the numbers through the same path as the screen. The tests assert two things. With no actions applied, aggregate_rates on the fixture equals the pipeline's own rate computation on the screened fixture, pooled and for each year. After a flag, the aggregate equals the pipeline's computation with that filing removed.
Source: functions.py · tests/test_ontology.py
How it maps to Foundry concepts
I built it in plain Python so it runs anywhere and can be checked by tests, but each part follows a Foundry Ontology concept, so the same design could be moved onto a Foundry ontology.
| Foundry concept | In this build |
|---|---|
| Object Type | object_types in the spec: primary key, title property, typed properties with nullability and editability. Loading enforces the types and keys. |
| Link Type | link_types: cardinality, a foreign key or join rule, and a name for each direction. |
| Action Type | action_types: parameters with required, conditional, allowed-value and length rules; submission criteria with failure messages; edits to editable properties only. |
| Function | functions.py: functions on one object or an object set, each delegating to a pipeline module. |
| Object Set | ObjectSet with filters and Search Around (following a link from a whole set), plus link traversal from a single object. |
| Writeback | The editable Filing properties for the analyst flag and benchmark exclusion, changed only through actions. |
| Action log | An append-only log of every applied action with before and after values, and per-object edit history. |
| Backing datasets and export | Objects are built from the harmonised frame and committed output tables. Export writes SQLite (one typed table per object type with an enforced primary key, the many-to-many link, the action log and the spec) and strict JSON. |
Source: spec.py · runtime.py · export.py
What it shows and its limits
The build shows how I take a validated analysis and turn it into a model an EHS team can operate: objects named after the things they are accountable for, links that answer their questions, judgements that are governed and logged, and calculations that cannot drift from the tested method.
Synthetic demonstration. By default, Establishment and Filing objects come from a synthetic fixture. They link to real ReportingYear and PeerGroup objects so the joins are exercised, but no number computed from fixture objects is a finding. The figures quoted on this page come from the pipeline's committed outputs, not from the ontology demo.
Real-data path not run. The option to build from the real filings uses the same loader as the pipeline, but it has not been run on the full panel of 2,801,064 filings, because the raw files are not in the repository. The store is in memory.
No permission model. Actions are checked against the spec, but the actor is whatever name the caller passes. Peer groups are the pooled 2016 to 2024 cells; per-year cells are not modelled as objects.
Independent work. This is my own design, built on public OSHA data and the repository's own outputs. It is in the style of a Palantir Foundry Ontology but is not affiliated with or endorsed by Palantir, does not run on Foundry, and uses no employer data.
When a benchmark says a site is in the worst quartile, someone should be able to see which filings were counted, which were set aside, who set them aside and why, and undo it. This design makes those answers part of the data model rather than a spreadsheet beside it.
Source: docs/ontology.md (Limits) · synthetic/README.md · outputs/summary.json