petrolyst-infra/stacks/fuel-monitoring/fuel-db/migrations/README.md

fuel-db migrations

Status: DRAFT, not applied. Nothing here runs automatically: deploy.sh never touches fuel-db, and the compose file mounts only ../init/01-roles.sh. A person applies these files by hand, once the owner gives the go-ahead (phase P3 of the design).

The design and its rationale live in the fuel-monitoring corpus: .context/plans/2026-10-24-fuel-db-baseline-dashboard.md. The SQL comments cite it as "design §n". This folder carries no site data.

Files

File	Creates	Readable by fuel_ro (Superset, Petrolyst server)
0001_fm_core.sql	schema fm, btree_gist in it, 25 tables, constraints, trigger-backed rules 1–13	no
0002_fm_derivations.sql	internal views in fm: claims → visits → stock intervals → L/h and L/day intervals, dependency hashes, baseline validity	no
0003_bi_views.sql	fm.v_site_baseline_status and the 18 public.v_* views Superset reads	yes, read-only

Each file is one transaction. Apply them in order.

Who depends on the views. Superset's dashboards list every column (petrolyst-insights/dashboards/atoma/columns.tsv; the import reports drift). The Petrolyst server reads public.v_site; its integration test (petrolyst-platform server/tests/test_integration.py) names the columns and types it needs. After a change to 0003_bi_views.sql, run both checks. The migrations stay in this repository (open point 8, closed 30 Oct 2026).

Applying (on the server)

1. Create a loader role as in the repository README, "Data loaders". Its sessions act as fuel_owner, so every object belongs to fuel_owner. fuel_ro can then read the public views through the existing default privileges, with no role or grant changes.

2. Open the tunnel and apply the three files as the loader:

   ssh -L 5433:127.0.0.1:5433 petrolyst
   for f in 0001_fm_core 0002_fm_derivations 0003_bi_views; do
     psql -h localhost -p 5433 -U <loader> -d fuel -v ON_ERROR_STOP=1 -f "$f.sql" || break
   done
   

3. Check as fuel_ro (docker exec -it petrolyst-fuel-db-1 psql -U postgres -d fuel, then SET ROLE fuel_ro;):

   • \dv public.* lists the 18 v_* views;
   • SELECT count(*) FROM fm.site; fails with "permission denied for schema fm".

The public views are owned by fuel_owner and run with its rights, so fuel_ro never needs, and never gets, read access to fm. PostgreSQL's catalog still shows every schema's table and column names to any role, so Superset's SQL Lab can list fm's names; names only, no rows.

Data

• Nobody edits fm.* by hand. Rows arrive only through the corpus loader (fmdb.py export / load, phase P1, not yet written), in one transaction per load.
• Every value is a claim:
  • a correction marks the old row superseded, inserts the new row under the same source_key, and points superseded_by at it;
  • the old row may point at itself until the new row exists;
  • a removed record is retracted, never deleted.
• Review status, coordinate acceptance and assumption acceptance come only from an fm.review row that matches the exact assertion_sha256 (rule 11).
• A baseline version is inserted as proposed. approved is a later update, refused unless the version meets its policy and its inputs are unchanged (rule 1).

Trigger-backed rules (in 0001)

1. Baseline versions start as draft or proposed. Approval needs meets_policy and unchanged inputs. Approved rows and their inputs are immutable; the only change is approved → superseded. An input never moves to another version.
2. An internal transfer's tanks are placed at the movement's site on the visit date.
3. A claim's site is its visit's site, and a reading's meter belongs to that site.
4. A reading's epoch belongs to its meter.
5. A runtime assumption derived from fuel cannot be accepted. This is a CHECK constraint.
6. A baseline input's dates equal its endpoint visits. Included inputs carry the version's population.
7. A source document supersedes only one of its own family.
8. A confirmed or proposed meter mapping names a generator placed at the meter's site.
9. Meter-coverage members:
   • valid when written, and written in the assumption's own transaction;
   • immutable afterwards;
   • complete at commit: as many counters as covered sets.
10. The live claims of one event_key share visit, kind and tanks. A superseded or retracted claim keeps its own, so a correction can change them.
11. Status comes only from a matching review. Values are never corrected in place.
12. An idle assumption names a generator placed at its own site during its dates.
13. Identity and classification never change in place: a visit's key, site, date and sequence; a source document's hash, family and kind (its rank); an asset's uid and kind; a meter's id and site; an epoch's id and meter. A wrong visit date is corrected by a new visit, which the superseding claims point to.

Rollback

Before any data matters, run DROP SCHEMA fm CASCADE as the loader. This also drops every public.v_* view, and leaves fuel-db as it was before 0001.

Once data has been loaded, this deletes it. The corpus remains the source, so a re-load restores it, except baseline versions and their approvals, which live only in fuel-db: export fm.baseline_version and fm.baseline_input first.

Verified locally (2026-10-24, scratch only)

The reads below ran through Trino, removed since on 29 Oct 2026, as today's fuel_ro.

Setup:

• timescale/timescaledb:2.31.0-pg17, with this repository's 01-roles.sh;
• a loader created by the README recipe;

Results:

• Apply: all three files applied as the loader, under the loader's default search_path.
• Fixture: the operator workbook of 21 Oct 2026, the five site records, and synthetic sites for the runtime path.
• Tests: all 150 checks of the design's §9 pass: 98 written with the migrations, and 52 regressions for the implementation inspection's findings. These cover:
  • figures and statuses;
  • claims, reviews and revisions;
  • movement events;
  • coverage, placement and idle scope;
  • baseline dependencies and approval;
  • the L/day path;
  • current periods;
  • flow-meter reconciliation (coverage, placement, visit times), runtime validity, modelled-L/h windows, disputed claims supplying no values, the L/day reference, S2's L/h rows following the version's validity, corrections under one event_key, fixed meter and epoch owners, and population keys that name the generators a coverage assumption covers;
  • every refusal listed above.
• Read-only role:
  • fm is unreadable: a query is refused, though its names are listed (see Applying);
  • writes are refused.
• Not run on the server: none of this has been run against the live fuel-db.