Data-Estate X-Ray

Norwegian Petroleum Directorate — FactPages Estate

SAMPLE REPORT — PUBLIC DATA. This is a demonstration of the Data-Estate X-Ray deliverable, produced against the publicly available FactPages dataset of the Norwegian Petroleum Directorate (NPD), hosted in a PostgreSQL database. Nothing in this document is confidential. A paying client receives this exact report structure for their own estate, plus the machine-readable ontology file and a verification certificate.

1. Executive summary

Your database is the system of record for the Norwegian continental shelf: licences, companies, fields, discoveries, wellbores, facilities, seismic surveys, and their full production and ownership histories. Nobody has to tell our tooling this — it is recovered directly from the database itself, in minutes, and every statement in this report is traceable either to a declared database constraint or to a live query we ran against your data.

Headline numbers

The four findings we would walk you through first

1. F-1 (High) — 38 unresolved assets in your production history. Monthly production data references 89 distinct assets; only 51 resolve to a record in the field master table. 45% of monthly production rows (6,339 of 13,973) cannot be joined to a field master record. Any report, dashboard, or AI agent that joins production to fields silently drops nearly half the history.
2. F-2 (High) — the production link is informal. The relationship between production history and fields is not declared as a foreign key in the database at all. We recovered it by mining the data. It works — but nothing in your database enforces it, which is exactly how F-1 became possible.
3. F-9 (High) — 86% of your declared foreign keys were never validated. 83 of 96 FK constraints carry PostgreSQL's NOT VALID flag: they document intent, but the engine never checked existing rows against them and does not guarantee they hold. Your schema looks more enforced than it is — and the delivered map now records that distinction per link.
4. F-8 (Positive) — the declared-and-validated core is clean. Every declared relationship we probed live (discoveries→fields, cores→wellbores, drill-stem tests→wellbores) shows zero orphaned rows. The genuinely enforced backbone of this estate is in excellent health; the risk concentrates in the informal (F-1/F-2) and unvalidated (F-9) links.

2. Estate map

2.1 What the estate is

The recovered structure resolves into six domains that hub around five master concepts. Connectivity (number of distinct relationships) tells you where the gravity is:

Around these hubs, the remaining ~60 entities fall into recognisable families: history tables (*_hst — 15 tables of licensee, operator, owner, transfer, phase and area history), production and reserves (8 tables of monthly/yearly production and reserve figures), wellbore technical data (12 tables — cores, mud, casing, formation tops, lithostratigraphy, oil samples, documents, photos), seismic surveys (3 tables), facilities (fixed, moveable, points), and geography (area polygons for licences, fields, discoveries, and sea areas).

2.2 Largest tables (live row counts)

Estate total: ~244,700 rows across 72 entities. Wellbore technical data dominates the volume; licence and field administration dominates the structure.

2.3 Relationship map — core excerpt

The full recovered map contains 114 unique relationships. The excerpt below shows the top ten most-connected entities and the links between them (crow's-foot notation: one field has many discoveries, etc.):

erDiagram
    COMPANY ||--o{ FIELD : "operator"
    LICENCE ||--o{ FIELD : "owner licence"
    FIELD ||--o{ DISCOVERY : "included in"
    WELLBORE_NPDID_OVERVIEW ||--o{ DISCOVERY : "discovery wellbore"
    WELLBORE_NPDID_OVERVIEW ||--o{ FIELD : "discovery wellbore"
    WELLBORE ||--o{ WELLBORE_NPDID_OVERVIEW : "wellbore id"
    WELLBORE ||--o{ WELLBORE_DEVELOPMENT_ALL : "wellbore id"
    WELLBORE ||--o{ WELLBORE_EXPLORATION_ALL : "wellbore id"
    COMPANY ||--o{ WELLBORE_DEVELOPMENT_ALL : "drilling operator"
    COMPANY ||--o{ WELLBORE_EXPLORATION_ALL : "drilling operator"
    DISCOVERY ||--o{ WELLBORE_DEVELOPMENT_ALL : "target discovery"
    DISCOVERY ||--o{ WELLBORE_EXPLORATION_ALL : "target discovery"
    FIELD ||--o{ WELLBORE_DEVELOPMENT_ALL : "target field"
    FIELD ||--o{ WELLBORE_EXPLORATION_ALL : "target field"
    LICENCE ||--o{ WELLBORE_DEVELOPMENT_ALL : "production licence"
    LICENCE ||--o{ WELLBORE_EXPLORATION_ALL : "production licence"
    COMPANY ||--o{ SEIS_ACQUISITION : "reporting company"

The full machine-readable map — all 72 entities, 979 fields, 114 relationships, with per-relationship provenance (declared vs. mined, confidence, enforcement and validation status) plus the 798 recovered column descriptions — ships alongside this report as estate.ontology.yaml and can be rendered into complete ER documentation, a dbt project, OWL/SHACL, or a data dictionary on request.

2.4 Coverage statement

• 70 of 70 business tables captured automatically (100%). The first pass missed two (finding F-3); the gap was root-caused and fixed the same day, and this revision reflects the corrected engine.
• 3 GIS system catalogs (spatial_ref_sys, geometry_columns, geography_columns — PostGIS internals) were correctly recognised as infrastructure and excluded from the business map.
• 2 implicit concepts (wellbore, tuf) were recovered that have no table of their own — see F-5. The map contains more than the schema literally says.

3. Your estate, narrated

The following is an excerpt from the machine-generated narrative of the estate — produced deterministically from the recovered map, with zero manual diagramming. The full narrative (all 72 entities) is delivered as an appendix file, alongside a complete field-level data dictionary that carries your database's own column documentation (798 descriptions — F-10) with per-claim evidence stamps. In a full engagement both are further enriched with your team's business vocabulary during the curation session.

The estate at a glance

This estate describes the petroleum activity lifecycle on the Norwegian continental shelf. Each entity corresponds to a distinct concept and is queryable by field filters; all backend connections, table names, and column names are resolved internally — consumers of this map never touch the physical schema.

How the concepts connect (direct excerpt from the generated relationship narrative)

• discovery → field — a discovery may be developed into a field
• discovery → wellbore_npdid_overview — the wellbore that made the discovery
• field → company — the operating company
• field → licence — the owning production licence
• field_production_monthly → field — monthly production per field (mined link — see F-2)
• field_licensee_hst → company / field — licensee history joins both masters
• licence_licensee_hst → licence / company — who held which licence share, when
• wellbore_core, wellbore_mud, wellbore_formation_top, … → wellbore_npdid_overview — all technical measurements hang off the wellbore master
• fclpoint → facility_fixed / tuf_petreg_licence — facility locations tie into the transport-facility register

(105 further relationships in the full narrative.)

How it can be queried

Because the map is explicit, any consumer — human, BI tool, or AI agent — can submit precise structured queries ("list discoveries in field X", "monthly oil production for field Y in 2024") and the joins are constructed automatically from the recovered relationships. No SQL knowledge of the physical schema is required, and no query can silently invent a join that is not on the map.

4. Findings register

Each finding is traceable to an automated check or a live query. Severity reflects business impact, not technical pedantry.

F-1 — Production history references 38 assets with no master record — HIGH

Evidence (live query). field_production_monthly contains production for 89 distinct assets. Only 51 resolve to a row in the field master table (one further resolves to a discovery). 6,339 of 13,973 monthly production rows (45.4%) do not join to any field master record.

What it means for you. Any consumer that answers "total production by field" by joining these two tables silently excludes almost half the rows — and gives a confident, wrong answer. The likely cause is that the master table holds only current fields (60 rows) while the history references decommissioned, renamed, or merged assets. The full engagement resolves each of the 38 with your domain experts and encodes the resolution into the map.

F-2 — The production↔field link is informal (mined, not declared) — HIGH

Evidence. Of the 114 recovered links, 96 are declared as foreign keys in the database and the remainder were mined from the data or the naming structure with no declared constraint behind them. The production→field link — arguably the most business-critical join in the estate — is in the mined category: nothing in the database enforces it.

What it means for you. Informal links are where silent data drift lives (F-1 is the proof). Each mined link is flagged with its confidence and enforcement status in the delivered map; the full engagement verifies each one against the live data and issues a per-relationship verification certificate.

F-3 — Two populated tables were off the automated map — MEDIUM, RESOLVED same day

Evidence. tuf_operator_hst (46 rows) and tuf_owner_hst (5,063 rows) — operator and ownership history for transport facilities — existed in the database but were not captured by the first automated pass.

Resolution (same day). The gap was root-caused to a discovery-stage filter that misread these history tables' concatenated validity-column names. The engine now recognises temporal columns by data type as well as by name, and every excluded table is reported by name — silent exclusions no longer exist. This revision's map carries 100% business-table coverage, including a newly synthesised tuf master concept those tables' keys made visible (see F-5).

What it means for you. A structural map that misses tables is a map people will stop trusting; we flag such gaps loudly rather than hide them — and this one improved the engine for every future estate within hours.

F-4 — 63% of discoveries are not assigned to a field — MEDIUM

Evidence (live query). 209 of 329 rows in discovery (63.5%) have no value in the field-assignment column.

What it means for you. Partly legitimate — undeveloped discoveries have no field yet — but any consumer must know this, or every "discoveries per field" analysis quietly covers only a third of the data. The delivered map annotates the field as optional so downstream consumers handle the absence explicitly rather than by accident.

F-5 — Two implicit master concepts were recovered: wellbore and tuf — INFORMATIONAL / POSITIVE

Evidence. Eleven tables share a common wellbore key, and the introspection recovered a wellbore concept that has no physical table of its own — the role of wellbore master is de facto played by wellbore_npdid_overview. The same-day F-3 fix surfaced a second one: four transport-facility tables share a tuf key with no tuf master table behind it.

What it means for you. Your schema encodes concepts it never names. The recovered map makes them explicit, so consumers can say "wellbore" and get the right joins. In a full engagement we would recommend either promoting these to real master tables or formally designating their de-facto stand-ins.

F-6 — 192 field names are shared across entities without disambiguation — MEDIUM

Evidence (automated consistency check). 192 field names appear on two or more entities (e.g. the same date-validity columns on both a master and its history table); 21 names appear on up to six entities.

What it means for you. For humans with tribal knowledge this is harmless. For new hires, BI tools, and especially AI agents, a question like "show me baaname" is ambiguous six ways — this is the raw material of wrong joins and wrong answers. The curation phase attaches disambiguation rules to each shared name.

F-7 — Two entities are disconnected islands — LOW

Evidence (automated graph analysis). The estate graph has 3 components: one large component of 70 entities, plus apaareagros and apaareanet (APA licensing-round area geometries) standing alone with no relationships to anything.

What it means for you. Possibly intentional (pure GIS reference data), but disconnected tables are where undocumented knowledge hides. One conversation in the curation session settles whether these need links to the licence domain.

F-8 — Declared referential integrity is clean — POSITIVE

Evidence (live queries). Orphan checks on declared relationships:

What it means for you. Where your database declares and validates its structure, the data honours it perfectly. The estate's risk is concentrated in what is undeclared (F-1/F-2) and unvalidated (F-9), not in what is genuinely enforced — which is exactly the kind of prioritisation this audit exists to give you.

F-9 — 83 of 96 declared foreign keys were never validated by the engine — HIGH

Evidence (system catalog). 83 of the estate's 96 FK constraints carry PostgreSQL's NOT VALID flag (pg_constraint.convalidated = false): they were added without checking the rows that already existed, and the engine makes no guarantee those rows satisfy them. Only 13 constraints are fully validated.

What it means for you. Your schema documents far more integrity than the database guarantees. Most tools — ER diagrammers, ORMs, AI agents reading the schema — treat a declared FK as a promise; on this estate, 86% of those promises were never checked. The delivered map is honest about this: every link carries enforced and validated separately, so a consumer can distinguish "the database guarantees this join" from "somebody once declared it". The full engagement probes each unvalidated constraint against the live rows (as F-8 did for a sample — those probed came back clean) and either upgrades it to validated or raises the violating rows as findings.

F-10 — Your database documents itself: 798 column descriptions recovered — POSITIVE

Evidence (system catalog). The estate carries 798 human-authored column comments (COMMENT ON COLUMN) — e.g. prfnpdidinformationcarrier is documented in-database as "NPDID information carrier". All of them are now harvested into the delivered map as field descriptions with declared provenance — no AI guessed them; your own DBAs wrote them.

What it means for you. This estate is unusually well documented in-band, and that documentation now rides every artifact derived from the map — data dictionary, dbt docs, OWL export, and the narrated estate — instead of living only inside the database where nobody reads it. Where comments are missing, the full engagement's curation phase fills the gaps, and every added description is marked as curated, never silently mixed with the declared ones.

Additional observations (no action required)

• Every entity currently lacks a designated human-readable identifier field (the automated pass finds keys, not labels). Assigning display identifiers is a standard step of the curation session and materially improves every downstream consumer.
• 18 mined relationships are one-directional and unconfirmed; they are marked as such in the delivered map and verified in a full engagement.
• Null-rate spot checks on operationally important columns came back clean: mud type 0.0% missing, well type 0.0% missing, field completion date 0.0% missing. The estate is dense — absence concentrates in the specific columns called out in F-4.

5. What the full engagement adds beyond this sample

This sample is the raw X-Ray. The fixed-price engagement delivers, in five working days:

1. Curation session (½ day with your team). Business vocabulary, display identifiers, disambiguation rules for the 192 shared names, and resolution of every flagged island and mined link — encoded into the map, not into a Word document.
2. Verification certificate. Every relationship on the map is machine-verified against your live data and the result is issued as a signed certificate ("your declared relationships: N of M verified consistent"), re-runnable at any time.
3. 100% coverage reconciliation. Every table accounted for — mapped, explicitly excluded with a reason, or raised as a finding (this sample's F-3 was found and closed the same day).
4. Finding resolutions. Each High/Medium finding worked to a concrete recommendation with your domain experts (e.g. the 38 unresolved production assets in F-1 individually dispositioned).
5. Drift baseline. The certified map becomes a baseline; re-running the introspection later diffs your estate against it — new tables, changed columns, broken links — enabling an ongoing drift-watch subscription.
6. Executive readout. A one-hour walkthrough of this report with your engineering leadership, plus a one-page summary for non-technical stakeholders.

You keep everything: the report, the machine-readable ontology file, the narrative, and the certificate. They are yours, tool-independent, and readable by any team or system you choose.

Appendix A — Methodology

Deterministic introspection. The estate map (Sections 2–3) is recovered mechanically from the database's own metadata and data: declared keys and constraints, type information, and statistical evidence mined from the rows themselves. No large-language-model output is used to construct the map — every entity, field, and relationship is either declared by your database or backed by named evidence, and each relationship carries its provenance (declared vs. mined), confidence, and enforcement/validation status in the delivered file. Descriptions are likewise mechanical where possible: the 798 field descriptions in this map are your database's own COMMENT metadata, marked declared. AI assists only at the editorial layer (filling description gaps, naming) in the full engagement's curation phase, and every such suggestion is additive, human-approved, and marked as curated — it can never alter or silently blend into the mechanically recovered facts.

Automated consistency analysis. The recovered map is passed through a battery of 51 graph-theoretic and type-theoretic consistency checks (connectivity, identifier coverage, name collisions, relationship symmetry, cardinality sanity). Findings F-5 through F-7 and the additional observations derive from this battery.

Live data probes. Findings F-1, F-4, and F-8 come from live SQL executed against the subject database during the audit window: row counts on the six largest tables, null-rate checks on operationally significant columns, and orphan checks (child keys with no parent) on both declared and mined relationships. Probes are sampled and bounded; total probe runtime for this estate was under one minute.

Honesty rule. Anything the automated pass could not capture is reported as a finding (F-3), never silently omitted.

Appendix B — Provenance

This SAMPLE was produced on public data (Norwegian Petroleum Directorate FactPages). Client engagements are covered by NDA; the client's data never leaves the client's infrastructure — the introspection connects to the client's database, recovers structure, and discards row data beyond the bounded probes listed above.

Addendum — 2026-07-09 (same day)

Finding F-3 (two populated history tables absent from the recovered model) was root-caused and fixed the same day it was reported: a discovery-stage filter classified history-suffixed tables as audit noise when their validity columns used concatenated names the pattern-matcher could not recognise. The engine now recognises temporal columns by data type as well as by name, and every excluded table is reported by name — silent exclusions no longer exist.

The regenerated estate model (attached: estate.ontology.yaml) recovers 72 entities — all 70 business tables (100% coverage) — including both previously-missed tables (tuf_owner_hst, tuf_operator_hst, the licence ownership/operator history) and, as a direct consequence, a newly synthesised tuf master concept their foreign keys made visible.

This is the engagement model working as intended: the audit found a real gap, named it honestly instead of hiding it, and the toolchain was corrected within hours — every future estate benefits.

Addendum 2 — 2026-07-09 (revision 2)

Later the same day, an adversarial benchmark of the toolchain (an independent analysis of this estate's raw schema dump) surfaced two further engine gaps, both fixed within hours and both now reflected in this revised report and the attached map:

1. Validation state (new finding F-9). The engine previously recorded declared foreign keys as enforced without checking PostgreSQL's NOT VALID flag. It now reads the validation state per constraint: on this estate, 83 of 96 declared FKs were never validated — a headline finding that the first revision could not see.
2. In-database documentation (new finding F-10). The engine previously ignored COMMENT ON metadata. It now harvests it: 798 human-authored column descriptions ride the delivered map with declared provenance.

The report body, headline numbers, and estate.ontology.yaml were regenerated end-to-end with the corrected engine. The audit's honesty rule cuts both ways: when the tooling itself is caught short, that is reported and fixed, not hidden.