petrolyst-infra/stacks/fuel-monitoring/fuel-db/migrations/README.md

# fuel-db migrations

**Status: DRAFT, not applied.** Nothing here runs automatically: `deploy.sh` never touches
fuel-db, and the compose file mounts only `../init/01-roles.sh`. A person applies these files by
hand, once the owner gives the go-ahead (phase P3 of the design).

The design and its rationale live in the fuel-monitoring corpus:
`.context/plans/2026-10-24-fuel-db-baseline-dashboard.md`. The SQL comments cite it as
"design §n". This folder carries no site data.

## Files

| File | Creates | Readable by `fuel_ro` (Superset, Petrolyst server) |
|---|---|---|
| `0001_fm_core.sql` | schema `fm`, `btree_gist` in it, 29 tables (customers, assets, places and holders, claims, reviews, baselines), constraints, trigger-backed rules 1–15 | no |
| `0002_fm_derivations.sql` | internal views in `fm`: claims → visits → stock intervals → L/h and L/day intervals, dependency hashes, baseline validity | no |
| `0003_bi_views.sql` | `fm.v_site_baseline_status` and the 19 `public.v_*` views Superset reads | yes, read-only |
| `0004_licence.sql` | schema `licence`: plans, prices, entitlements, customer licences, overrides and their change log (PD-19, PD-20). Holds no customer's licence | no |

Each file is one transaction. Apply them in order.

**The shape (PD-16, PD-18).** Every row in `fm` carries its customer, `operator_uid`, except
`asset_kind`. A child names its parent and the customer together in a composite foreign key, so it
cannot point at another customer's parent. A site is a `place` with one fixed `holder`; holders keep
their `SITE-0001` ids and places get `PLC-0001`. Internal `fm` speaks of holder and place. The public
views keep their `site_*` column names, because dashboards, the app and the server read them; each
gains `operator_uid`. They get renamed, or holder-named siblings, with the first fleet customer.

**Runtime is generator-only.** An engine can be a placed asset, but rules 8, 9 and 12 refuse a
meter or idle assumption on one: the runtime and L/h calculations have not been checked on engines.

**Who depends on the views.** Superset's dashboards list every column (`petrolyst-insights/dashboards/atoma/columns.tsv`;
the import reports drift). The Petrolyst server reads `public.v_site`; its integration test
(petrolyst-platform `server/tests/test_integration.py`) names the columns and types it needs. After
a change to `0003_bi_views.sql`, run both checks. The migrations stay in this repository (open
point 8, closed 30 Oct 2026).

## Applying (on the server)

1. Create a loader role as in the repository README, "Data loaders". Its sessions act as
   `fuel_owner`, so every object belongs to `fuel_owner`. `fuel_ro` can then read the `public`
   views through the existing default privileges, with no role or grant changes.
2. Open the tunnel and apply the four files as the loader:

   ```sh
   ssh -L 5433:127.0.0.1:5433 petrolyst
   for f in 0001_fm_core 0002_fm_derivations 0003_bi_views 0004_licence; do
     psql -h localhost -p 5433 -U <loader> -d fuel -v ON_ERROR_STOP=1 -f "$f.sql" || break
   done
   ```

3. Check as `fuel_ro`
   (`docker exec -it petrolyst-fuel-db-1 psql -U postgres -d fuel`, then `SET ROLE fuel_ro;`):
   - `\dv public.*` lists the 19 `v_*` views;
   - `SELECT count(*) FROM fm.holder;` fails with "permission denied for schema fm";
   - `SELECT count(*) FROM licence.plan;` fails with "permission denied for schema licence".

The `public` views are owned by `fuel_owner` and run with its rights, so `fuel_ro` never needs,
and never gets, read access to `fm`. PostgreSQL's catalog still shows every schema's table and
column names to any role, so Superset's SQL Lab can list fm's names; names only, no rows.

## Data

- Nobody edits `fm.*` by hand. Rows arrive only through the corpus loader (`fmdb.py export` /
  `load`, phase P1, not yet written), in one transaction per load.
- Every value is a claim:
  - a correction marks the old row `superseded`, inserts the new row under the same `source_key`,
    and points `superseded_by` at it;
  - the old row may point at itself until the new row exists;
  - a removed record is `retracted`, never deleted.
- Review status, coordinate acceptance and assumption acceptance come only from an `fm.review` row
  that matches the exact `assertion_sha256` (rule 11).
- A baseline version is inserted as `proposed`. `approved` is a later update, refused unless the
  version meets its policy and its inputs are unchanged (rule 1).

## Trigger-backed rules (in 0001)

1. Baseline versions start as draft or proposed. Approval needs `meets_policy` and unchanged
   inputs. Approved rows and their inputs are immutable; the only change is approved →
   superseded. An input never moves to another version.
2. An internal transfer's tanks are placed on the movement's holder on the visit date.
3. A claim's holder is its visit's holder, and a reading's meter belongs to that holder.
4. A reading's epoch belongs to its meter.
5. A runtime assumption derived from fuel cannot be accepted. This is a CHECK constraint.
6. A baseline input's dates equal its endpoint visits. Included inputs carry the version's
   population.
7. A source document supersedes only one of its own family. The same document under a second
   customer is refused until document sharing exists.
8. A confirmed or proposed meter mapping names a generator placed on the meter's holder.
9. Meter-coverage members:
   - valid when written, and written in the assumption's own transaction;
   - immutable afterwards;
   - complete at commit: as many counters as covered sets.
10. The live claims of one `event_key` share visit, kind and tanks. A superseded or retracted
    claim keeps its own, so a correction can change them.
11. Status comes only from a matching review. Values are never corrected in place.
12. An idle assumption names a generator placed on its own holder during its dates.
13. Identity and classification never change in place: a visit's key, holder, date and sequence; a
    source document's hash, family and kind (its rank); an asset's uid and kind; a place's uid; a
    holder's uid, kind and asset; a meter's id and holder; an epoch's id and meter; any row's customer.
    A wrong visit date is corrected by a new visit, which the superseding claims point to. Naming a
    column a table does not have raises an error.
14. A time zone is one PostgreSQL knows.
15. A vehicle holder's asset is a vehicle; only a fixed installation stands at a place.

## Rollback

Before any data matters, run `DROP SCHEMA licence CASCADE; DROP SCHEMA fm CASCADE` as the loader. This
also drops every `public.v_*` view, and leaves fuel-db as it was before 0001.

Once data has been loaded, this deletes it. The corpus remains the source, so a re-load restores
it, except baseline versions and their approvals, which live only in fuel-db: export
`fm.baseline_version` and `fm.baseline_input` first.

## Verified locally (2026-10-30, scratch only)

Not run against the live fuel-db. An earlier set of 150 checks was a scratch script, since removed;
these replace it:

- **Rules:** `../tests/run.sh` applies these files to a throw-away database and runs `tests/rules.sql`
  and `tests/read_only.sql`, with synthetic data only: each rule with an accepted and a refused case,
  two customers, a vehicle as asset and holder, licences, and what `fuel_ro` may read.
- **Loader:** fuel-monitoring's `make selftest` and `bin/fmdb.py dbtest`.
- **Row comparison:** before this rewrite, all 18 views were dumped from a database loaded from the
  corpus and a recorded review manifest; after it, the same corpus gave identical rows, leaving out
  `operator_uid`. That one-time comparison covers only data the corpus has; rules with no corpus data
  rest on `rules.sql`.