Trino out of the code; commit messages name bands and rules
The rename step in init.sh and the guard in deploy/lib.sh are gone. README's procedure is now "Removing the old project"; it keeps the old names until it has run on the server. Migrations stay here (open point 8), and their README says so. AGENTS.md adds the commit-message rule. Not deployed: live still runs Trino. Bands: 4 (Superset), persistence (docs). Rules: 2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
8405706ed8
commit
e4806f5799
9 files changed
+55
-67
No files matched your search
@@ -6,7 +6,7 @@ Everything that runs on the petrolyst server (109.199.127.84, Ubuntu 24.04):
|
||||
https://traefik.petrolyst.com, behind Keycloak.
|
||||
- **fuel-monitoring** (ATOMA): Superset at https://atoma.petrolyst.com, reading the operational
|
||||
store `fuel-db` (Postgres 17 with TimescaleDB) directly, as the read-only role `fuel_ro`
|
||||
(Trino was removed on 29 Oct 2026). Sign-in goes through Keycloak
|
||||
(no Trino since 29 Oct 2026). Sign-in goes through Keycloak
|
||||
client `petrolyst`, and the UI carries the Petrolyst brand.
|
||||
|
||||
**Deploys are manual.** A person deploys a clean commit of `main` from the Mac ("Manual deploy"
|
||||
@@ -245,18 +245,14 @@ Tables it creates, hypertables included, then belong to `fuel_owner`, and `fuel_
|
||||
After `SET ROLE NONE`, it cannot create anything in `public`. To repair a table that another
|
||||
role created, run `ALTER TABLE <t> OWNER TO fuel_owner`, then `GRANT SELECT ON <t> TO fuel_ro`.
|
||||
|
||||
**One-time steps after 29 Oct 2026**, at the first deploy that includes them. Two changes
|
||||
meet there:
|
||||
- Trino is removed: Superset reads fuel-db directly. The role `trino_ro` becomes `fuel_ro` and
|
||||
keeps its password (a SCRAM password survives a rename); its secret file moves with it.
|
||||
- The Compose project `fuel-monitoring` is renamed `petrolyst`. Containers are then called
|
||||
`petrolyst-<service>-1`. The data volumes keep their names (`fuel-monitoring_superset_db_data`,
|
||||
`fuel-monitoring_fuel_db_data`; pinned in `compose.yaml`), so no data moves. The old project's
|
||||
containers must be gone first: two PostgreSQL servers on one data volume corrupt it.
|
||||
Preflight refuses to deploy while any of them exists.
|
||||
**Removing the old project**, once, right before the next deploy; delete this paragraph when it
|
||||
is done. The server still runs the Compose project `fuel-monitoring`, with Trino. Afterwards it runs
|
||||
`petrolyst`, without it: containers are `petrolyst-<service>-1`, and the data volumes keep their
|
||||
names (pinned in `compose.yaml`), so no data moves. The old containers must be gone first: two
|
||||
PostgreSQL servers on one data volume corrupt it, and preflight refuses to deploy while any exists.
|
||||
|
||||
As root on the server, right before the deploy (Superset is down from step 3 until the deploy
|
||||
finishes):
|
||||
As root on the server (Superset is down from step 3 until the deploy finishes). The role keeps its
|
||||
password, because a SCRAM password survives a rename:
|
||||
|
||||
```bash
|
||||
S=/opt/petrolyst/secrets/fuel-monitoring
|
||||
@@ -267,20 +263,35 @@ docker exec -i fuel-monitoring-fuel-db-1 psql -U postgres -d fuel -v ON_ERROR_ST
|
||||
docker rm -f $(docker ps -aq --filter label=com.docker.compose.project=fuel-monitoring) # 3
|
||||
```
|
||||
|
||||
4. Deploy as usual ("Manual deploy"). Compose starts the `petrolyst` project on the same volumes
|
||||
and no longer starts Trino. Compose warns once that the volumes belong to another project;
|
||||
that is expected. `superset-init` renames the connection "Fuel (Trino)" to "Fuel" and points
|
||||
it at fuel-db; the smoke test checks that it reads as `fuel_ro`, read-only.
|
||||
5. Clean up: `rm /opt/petrolyst/env/fuel-monitoring.env`, `rmdir /opt/petrolyst/env`,
|
||||
4. Deploy as usual ("Manual deploy"). Compose warns once that the volumes belong to another project;
|
||||
that is expected. `superset-init` registers the connection "Fuel"; the smoke test checks that it
|
||||
reads as `fuel_ro`, read-only.
|
||||
5. Delete the old connection. It refuses if a dataset still uses it; then stop and look:
|
||||
|
||||
```bash
|
||||
docker exec -i petrolyst-superset-1 /app/.venv/bin/python - <<'PY'
|
||||
from superset.app import create_app
|
||||
|
||||
with create_app().app_context():
|
||||
from superset import db
|
||||
from superset.connectors.sqla.models import SqlaTable
|
||||
from superset.models.core import Database
|
||||
|
||||
old = db.session.query(Database).filter_by(database_name="Fuel (Trino)").one_or_none()
|
||||
if old:
|
||||
used = db.session.query(SqlaTable).filter_by(database_id=old.id).count()
|
||||
if used:
|
||||
raise SystemExit(f"{used} datasets still use the old connection")
|
||||
db.session.delete(old)
|
||||
db.session.commit()
|
||||
print("old connection deleted")
|
||||
PY
|
||||
```
|
||||
6. Clean up: `rm /opt/petrolyst/env/fuel-monitoring.env`, `rmdir /opt/petrolyst/env`,
|
||||
`docker image rm trinodb/trino:484`,
|
||||
`docker network rm fuel-monitoring_backend fuel-monitoring_fuel-db-tunnel`.
|
||||
|
||||
**Rollback across the Trino removal** (not tested). `rollback.sh` stops for a release from before
|
||||
it. To go back anyway: remove the `petrolyst` project's containers, reverse steps 1 and 2 above (rename `fuel_ro` back to `trino_ro` and
|
||||
`ALTER ROLE trino_ro RESET default_transaction_read_only`, move the secret file back, write
|
||||
`/opt/petrolyst/env/fuel-monitoring.env` with `FUEL_DB_TRINO_PASSWORD=<that secret>`), rename
|
||||
the Superset connection "Fuel" back to "Fuel (Trino)" with `trino://superset@trino:8080/fuel`,
|
||||
then run that release's own script, `/opt/petrolyst/releases/<sha>/deploy/rollback.sh <sha>`.
|
||||
7. Releases from before this deploy can no longer run, as their role is gone, and `rollback.sh`
|
||||
stops on them. Remove them from `/opt/petrolyst/releases/`.
|
||||
|
||||
**Adding a project**: create `stacks/<name>/compose.yaml`, attach the public service to the
|
||||
external network `proxy`, and set `traefik.*` labels with a unique router name and
|
||||
@@ -299,8 +310,6 @@ its own gate in its stack:
|
||||
|
||||
## Verified locally (2026-10-23)
|
||||
|
||||
Before Trino's removal (29 Oct 2026): its lines below are history.
|
||||
|
||||
An end-to-end run ran the real `deploy.sh` and `rollback.sh` in a Linux container against
|
||||
Docker Desktop, with a scratch `/opt/petrolyst`. Production differences in that run:
|
||||
- a test CA certificate stood in for Let's Encrypt;
|
||||
@@ -310,7 +319,6 @@ Docker Desktop, with a scratch `/opt/petrolyst`. Production differences in that
|
||||
Results:
|
||||
- The fresh install took the init path.
|
||||
- An identical re-run recreated nothing and left `ARTIFACTS` byte-identical.
|
||||
- A Trino-only change recreated only `trino`.
|
||||
- A Superset change stopped, backed up, migrated and recreated only `superset`.
|
||||
- A change to `superset/init.sh` alone did the same, and the changed script ran.
|
||||
- A same-SHA upload with different content was refused.
|
||||
@@ -327,13 +335,12 @@ Results:
|
||||
- Retention pruned read-only releases.
|
||||
- Retention removed a queued upload that a plain `rsync -a` had given an old checkout's
|
||||
directory times. It kept one uploaded with `--omit-dir-times`.
|
||||
- In `fuel-db`, a loader that created a table as its own role left it unreadable for `trino_ro`.
|
||||
With the "Data loaders" recipe, tables (hypertables too) belonged to `fuel_owner` and Trino
|
||||
could read them. After `SET ROLE NONE`, the loader could not create anything.
|
||||
- In `fuel-db`, a loader that created a table as its own role left it unreadable for the read-only
|
||||
role. With the "Data loaders" recipe, tables (hypertables too) belonged to `fuel_owner` and that
|
||||
role could read them. After `SET ROLE NONE`, the loader could not create anything.
|
||||
- Smoke tests passed, including the redirects to the real Keycloak with `client_id=petrolyst`
|
||||
and PKCE S256.
|
||||
- TLS 1.1 and unknown SNI were refused, and HTTP redirected to HTTPS.
|
||||
- Trino and `trino_ro` both refused writes.
|
||||
|
||||
Not verified locally:
|
||||
- Let's Encrypt issuance;
|
||||
|
||||
Reference in new issue
Block a user