Trino out of the code; commit messages name bands and rules

The rename step in init.sh and the guard in deploy/lib.sh are gone. README's
procedure is now "Removing the old project"; it keeps the old names until it
has run on the server. Migrations stay here (open point 8), and their README
says so. AGENTS.md adds the commit-message rule. Not deployed: live still runs
Trino.

Bands: 4 (Superset), persistence (docs). Rules: 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Nasim SadatandClaude Opus 5.5 committed 2026-10-30 13:50:57 +01:00
1 parent 8405706ed8
commit e4806f5799
9 files changed
+55 -67

No files matched your search

+38 -31
View File
@@ -6,7 +6,7 @@ Everything that runs on the petrolyst server (109.199.127.84, Ubuntu 24.04):
https://traefik.petrolyst.com, behind Keycloak.
- **fuel-monitoring** (ATOMA): Superset at https://atoma.petrolyst.com, reading the operational
store `fuel-db` (Postgres 17 with TimescaleDB) directly, as the read-only role `fuel_ro`
(Trino was removed on 29 Oct 2026). Sign-in goes through Keycloak
(no Trino since 29 Oct 2026). Sign-in goes through Keycloak
client `petrolyst`, and the UI carries the Petrolyst brand.
**Deploys are manual.** A person deploys a clean commit of `main` from the Mac ("Manual deploy"
@@ -245,18 +245,14 @@ Tables it creates, hypertables included, then belong to `fuel_owner`, and `fuel_
After `SET ROLE NONE`, it cannot create anything in `public`. To repair a table that another
role created, run `ALTER TABLE <t> OWNER TO fuel_owner`, then `GRANT SELECT ON <t> TO fuel_ro`.
**One-time steps after 29 Oct 2026**, at the first deploy that includes them. Two changes
meet there:
- Trino is removed: Superset reads fuel-db directly. The role `trino_ro` becomes `fuel_ro` and
keeps its password (a SCRAM password survives a rename); its secret file moves with it.
- The Compose project `fuel-monitoring` is renamed `petrolyst`. Containers are then called
`petrolyst-<service>-1`. The data volumes keep their names (`fuel-monitoring_superset_db_data`,
`fuel-monitoring_fuel_db_data`; pinned in `compose.yaml`), so no data moves. The old project's
containers must be gone first: two PostgreSQL servers on one data volume corrupt it.
Preflight refuses to deploy while any of them exists.
**Removing the old project**, once, right before the next deploy; delete this paragraph when it
is done. The server still runs the Compose project `fuel-monitoring`, with Trino. Afterwards it runs
`petrolyst`, without it: containers are `petrolyst-<service>-1`, and the data volumes keep their
names (pinned in `compose.yaml`), so no data moves. The old containers must be gone first: two
PostgreSQL servers on one data volume corrupt it, and preflight refuses to deploy while any exists.
As root on the server, right before the deploy (Superset is down from step 3 until the deploy
finishes):
As root on the server (Superset is down from step 3 until the deploy finishes). The role keeps its
password, because a SCRAM password survives a rename:
```bash
S=/opt/petrolyst/secrets/fuel-monitoring
@@ -267,20 +263,35 @@ docker exec -i fuel-monitoring-fuel-db-1 psql -U postgres -d fuel -v ON_ERROR_ST
docker rm -f $(docker ps -aq --filter label=com.docker.compose.project=fuel-monitoring) # 3
```
4. Deploy as usual ("Manual deploy"). Compose starts the `petrolyst` project on the same volumes
and no longer starts Trino. Compose warns once that the volumes belong to another project;
that is expected. `superset-init` renames the connection "Fuel (Trino)" to "Fuel" and points
it at fuel-db; the smoke test checks that it reads as `fuel_ro`, read-only.
5. Clean up: `rm /opt/petrolyst/env/fuel-monitoring.env`, `rmdir /opt/petrolyst/env`,
4. Deploy as usual ("Manual deploy"). Compose warns once that the volumes belong to another project;
that is expected. `superset-init` registers the connection "Fuel"; the smoke test checks that it
reads as `fuel_ro`, read-only.
5. Delete the old connection. It refuses if a dataset still uses it; then stop and look:
```bash
docker exec -i petrolyst-superset-1 /app/.venv/bin/python - <<'PY'
from superset.app import create_app
with create_app().app_context():
from superset import db
from superset.connectors.sqla.models import SqlaTable
from superset.models.core import Database
old = db.session.query(Database).filter_by(database_name="Fuel (Trino)").one_or_none()
if old:
used = db.session.query(SqlaTable).filter_by(database_id=old.id).count()
if used:
raise SystemExit(f"{used} datasets still use the old connection")
db.session.delete(old)
db.session.commit()
print("old connection deleted")
PY
```
6. Clean up: `rm /opt/petrolyst/env/fuel-monitoring.env`, `rmdir /opt/petrolyst/env`,
`docker image rm trinodb/trino:484`,
`docker network rm fuel-monitoring_backend fuel-monitoring_fuel-db-tunnel`.
**Rollback across the Trino removal** (not tested). `rollback.sh` stops for a release from before
it. To go back anyway: remove the `petrolyst` project's containers, reverse steps 1 and 2 above (rename `fuel_ro` back to `trino_ro` and
`ALTER ROLE trino_ro RESET default_transaction_read_only`, move the secret file back, write
`/opt/petrolyst/env/fuel-monitoring.env` with `FUEL_DB_TRINO_PASSWORD=<that secret>`), rename
the Superset connection "Fuel" back to "Fuel (Trino)" with `trino://superset@trino:8080/fuel`,
then run that release's own script, `/opt/petrolyst/releases/<sha>/deploy/rollback.sh <sha>`.
7. Releases from before this deploy can no longer run, as their role is gone, and `rollback.sh`
stops on them. Remove them from `/opt/petrolyst/releases/`.
**Adding a project**: create `stacks/<name>/compose.yaml`, attach the public service to the
external network `proxy`, and set `traefik.*` labels with a unique router name and
@@ -299,8 +310,6 @@ its own gate in its stack:
## Verified locally (2026-10-23)
Before Trino's removal (29 Oct 2026): its lines below are history.
An end-to-end run ran the real `deploy.sh` and `rollback.sh` in a Linux container against
Docker Desktop, with a scratch `/opt/petrolyst`. Production differences in that run:
- a test CA certificate stood in for Let's Encrypt;
@@ -310,7 +319,6 @@ Docker Desktop, with a scratch `/opt/petrolyst`. Production differences in that
Results:
- The fresh install took the init path.
- An identical re-run recreated nothing and left `ARTIFACTS` byte-identical.
- A Trino-only change recreated only `trino`.
- A Superset change stopped, backed up, migrated and recreated only `superset`.
- A change to `superset/init.sh` alone did the same, and the changed script ran.
- A same-SHA upload with different content was refused.
@@ -327,13 +335,12 @@ Results:
- Retention pruned read-only releases.
- Retention removed a queued upload that a plain `rsync -a` had given an old checkout's
directory times. It kept one uploaded with `--omit-dir-times`.
- In `fuel-db`, a loader that created a table as its own role left it unreadable for `trino_ro`.
With the "Data loaders" recipe, tables (hypertables too) belonged to `fuel_owner` and Trino
could read them. After `SET ROLE NONE`, the loader could not create anything.
- In `fuel-db`, a loader that created a table as its own role left it unreadable for the read-only
role. With the "Data loaders" recipe, tables (hypertables too) belonged to `fuel_owner` and that
role could read them. After `SET ROLE NONE`, the loader could not create anything.
- Smoke tests passed, including the redirects to the real Keycloak with `client_id=petrolyst`
and PKCE S256.
- TLS 1.1 and unknown SNI were refused, and HTTP redirected to HTTPS.
- Trino and `trino_ro` both refused writes.
Not verified locally:
- Let's Encrypt issuance;