petrolyst-infra/stacks/fuel-monitoring/fuel-db/migrations/README.md

fuel-db migrations

Status: DRAFT, not applied. Nothing here runs automatically: deploy.sh never touches fuel-db, and the compose file mounts only ../init/01-roles.sh. A person applies these files by hand, once the owner gives the go-ahead (phase P3 of the design).

The design and its rationale live in the fuel-monitoring corpus: .context/plans/2026-10-24-fuel-db-baseline-dashboard.md. The SQL comments cite it as "design §n". This folder carries no site data.

Files

File	Creates	Readable by fuel_ro (Superset, Petrolyst server)
0001_fm_core.sql	schema fm, btree_gist in it, 29 tables (customers, assets, places and holders, claims, reviews, baselines), constraints, trigger-backed rules 1–15	no
0002_fm_derivations.sql	internal views in fm: claims → visits → stock intervals → L/h and L/day intervals, dependency hashes, baseline validity	no
0003_bi_views.sql	fm.v_site_baseline_status and the 19 public.v_* views Superset reads	yes, read-only
0004_licence.sql	schema licence: plans, prices, entitlements, customer licences, overrides and their change log (PD-19, PD-20). Holds no customer's licence	no

Each file is one transaction. Apply them in order.

The shape (PD-16, PD-18). Every row in fm carries its customer, operator_uid, except asset_kind. A child names its parent and the customer together in a composite foreign key, so it cannot point at another customer's parent. A site is a place with one fixed holder; holders keep their SITE-0001 ids and places get PLC-0001. Internal fm speaks of holder and place. The public views keep their site_* column names, because dashboards, the app and the server read them; each gains operator_uid. They get renamed, or holder-named siblings, with the first fleet customer.

Runtime is generator-only. An engine can be a placed asset, but rules 8, 9 and 12 refuse a meter or idle assumption on one: the runtime and L/h calculations have not been checked on engines.

Who depends on the views. Superset's dashboards list every column (petrolyst-insights/dashboards/atoma/columns.tsv; the import reports drift). The Petrolyst server reads public.v_site; its integration test (petrolyst-platform server/tests/test_integration.py) names the columns and types it needs. After a change to 0003_bi_views.sql, run both checks. The migrations stay in this repository (open point 8, closed 30 Oct 2026).

Applying (on the server)

1. Create a loader role as in the repository README, "Data loaders". Its sessions act as fuel_owner, so every object belongs to fuel_owner. fuel_ro can then read the public views through the existing default privileges, with no role or grant changes.

2. Open the tunnel and apply the four files as the loader:

   ssh -L 5433:127.0.0.1:5433 petrolyst
   for f in 0001_fm_core 0002_fm_derivations 0003_bi_views 0004_licence; do
     psql -h localhost -p 5433 -U <loader> -d fuel -v ON_ERROR_STOP=1 -f "$f.sql" || break
   done
   

3. Check as fuel_ro (docker exec -it petrolyst-fuel-db-1 psql -U postgres -d fuel, then SET ROLE fuel_ro;):

   • \dv public.* lists the 19 v_* views;
   • SELECT count(*) FROM fm.holder; fails with "permission denied for schema fm";
   • SELECT count(*) FROM licence.plan; fails with "permission denied for schema licence".

The public views are owned by fuel_owner and run with its rights, so fuel_ro never needs, and never gets, read access to fm. PostgreSQL's catalog still shows every schema's table and column names to any role, so Superset's SQL Lab can list fm's names; names only, no rows.

Data

• Nobody edits fm.* by hand. Rows arrive only through the corpus loader (fmdb.py export / load, phase P1, not yet written), in one transaction per load.
• Every value is a claim:
  • a correction marks the old row superseded, inserts the new row under the same source_key, and points superseded_by at it;
  • the old row may point at itself until the new row exists;
  • a removed record is retracted, never deleted.
• Review status, coordinate acceptance and assumption acceptance come only from an fm.review row that matches the exact assertion_sha256 (rule 11).
• A baseline version is inserted as proposed. approved is a later update, refused unless the version meets its policy and its inputs are unchanged (rule 1).

Trigger-backed rules (in 0001)

1. Baseline versions start as draft or proposed. Approval needs meets_policy and unchanged inputs. Approved rows and their inputs are immutable; the only change is approved → superseded. An input never moves to another version.
2. An internal transfer's tanks are placed on the movement's holder on the visit date.
3. A claim's holder is its visit's holder, and a reading's meter belongs to that holder.
4. A reading's epoch belongs to its meter.
5. A runtime assumption derived from fuel cannot be accepted. This is a CHECK constraint.
6. A baseline input's dates equal its endpoint visits. Included inputs carry the version's population.
7. A source document supersedes only one of its own family. The same document under a second customer is refused until document sharing exists.
8. A confirmed or proposed meter mapping names a generator placed on the meter's holder.
9. Meter-coverage members:
   • valid when written, and written in the assumption's own transaction;
   • immutable afterwards;
   • complete at commit: as many counters as covered sets.
10. The live claims of one event_key share visit, kind and tanks. A superseded or retracted claim keeps its own, so a correction can change them.
11. Status comes only from a matching review. Values are never corrected in place.
12. An idle assumption names a generator placed on its own holder during its dates.
13. Identity and classification never change in place: a visit's key, holder, date and sequence; a source document's hash, family and kind (its rank); an asset's uid and kind; a place's uid; a holder's uid, kind and asset; a meter's id and holder; an epoch's id and meter; any row's customer. A wrong visit date is corrected by a new visit, which the superseding claims point to. Naming a column a table does not have raises an error.
14. A time zone is one PostgreSQL knows.
15. A vehicle holder's asset is a vehicle; only a fixed installation stands at a place.

Rollback

Before any data matters, run DROP SCHEMA licence CASCADE; DROP SCHEMA fm CASCADE as the loader. This also drops every public.v_* view, and leaves fuel-db as it was before 0001.

Once data has been loaded, this deletes it. The corpus remains the source, so a re-load restores it, except baseline versions and their approvals, which live only in fuel-db: export fm.baseline_version and fm.baseline_input first.

Verified locally (2026-10-30, scratch only)

Not run against the live fuel-db. An earlier set of 150 checks was a scratch script, since removed; these replace it:

• Rules: ../tests/run.sh applies these files to a throw-away database and runs tests/rules.sql and tests/read_only.sql, with synthetic data only: each rule with an accepted and a refused case, two customers, a vehicle as asset and holder, licences, and what fuel_ro may read.
• Loader: fuel-monitoring's make selftest and bin/fmdb.py dbtest.
• Row comparison: before this rewrite, all 18 views were dumped from a database loaded from the corpus and a recorded review manifest; after it, the same corpus gave identical rows, leaving out operator_uid. That one-time comparison covers only data the corpus has; rules with no corpus data rest on rules.sql.