From 8405706ed8808c66f6e4749cb7f4688ad8c65637 Mon Sep 17 00:00:00 2001 From: Nasim Sadat Date: Fri, 30 Oct 2026 10:57:18 +0100 Subject: [PATCH] Remove Trino: Superset reads fuel-db directly as fuel_ro (PD-14) trino/ deleted; the role trino_ro becomes fuel_ro, its password read from a secret file at connect time; dashboards' SQL and column list for PostgreSQL; the Compose project is renamed petrolyst, with volumes pinned to their old names. AGENTS.md points to the architecture context. Not deployed: the next deploy needs README's one-time steps first. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 4 + README.md | 70 ++- deploy/deploy.sh | 2 +- deploy/lib.sh | 40 +- host/bootstrap.sh | 2 +- host/secrets-init.sh | 18 +- petrolyst-insights/dashboards/README.md | 14 +- petrolyst-insights/dashboards/atoma/SPEC.md | 11 +- .../dashboards/atoma/build_bundle.py | 30 +- .../dashboards/atoma/columns.tsv | 491 +++++++++--------- .../dashboards/atoma/import_bundle.py | 4 +- petrolyst-insights/superset_config.py | 15 + stacks/fuel-monitoring/compose.yaml | 42 +- .../fuel-monitoring/fuel-db/init/01-roles.sh | 19 +- .../migrations/0002_fm_derivations.sql | 2 +- .../fuel-db/migrations/0003_bi_views.sql | 10 +- .../fuel-db/migrations/README.md | 19 +- stacks/fuel-monitoring/superset/Dockerfile | 5 +- stacks/fuel-monitoring/superset/init.sh | 25 +- .../trino/etc/access-control.properties | 3 - .../trino/etc/catalog/fuel.properties | 5 - .../trino/etc/config.properties | 14 - stacks/fuel-monitoring/trino/etc/jvm.config | 14 - .../fuel-monitoring/trino/etc/log.properties | 1 - .../fuel-monitoring/trino/etc/node.properties | 3 - 25 files changed, 453 insertions(+), 410 deletions(-) delete mode 100644 stacks/fuel-monitoring/trino/etc/access-control.properties delete mode 100644 stacks/fuel-monitoring/trino/etc/catalog/fuel.properties delete mode 100644 stacks/fuel-monitoring/trino/etc/config.properties delete mode 100644 stacks/fuel-monitoring/trino/etc/jvm.config delete mode 100644 stacks/fuel-monitoring/trino/etc/log.properties delete mode 100644 stacks/fuel-monitoring/trino/etc/node.properties diff --git a/AGENTS.md b/AGENTS.md index e5cc0fc..d2d6756 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,6 +4,10 @@ Deployment of the petrolyst server (atoma.petrolyst.com) and the Superset custom Runbook: README.md. Workspace rules: `../AGENTS.md`; shared context: `../.context/` (server and identity facts: `deployment.md`). +**Architecture:** read `../.context/architecture.md` before planning or building. This repository +runs persistence (fuel-db and its migrations) and Superset; name the bands a change touches and +keep the rules there; a change against a rule or an open conflict goes to the owner first. + ## Layout - `stacks/`, `traefik/`, `deploy/`, `host/`: how and where things run. - `petrolyst-insights/`: what Superset looks like and does (config, security manager, the guest diff --git a/README.md b/README.md index 298fd5c..adddcf4 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,8 @@ Everything that runs on the petrolyst server (109.199.127.84, Ubuntu 24.04): - **Traefik** at the edge for every project on the server. Its dashboard is at https://traefik.petrolyst.com, behind Keycloak. - **fuel-monitoring** (ATOMA): Superset at https://atoma.petrolyst.com, reading the operational - store `fuel-db` (Postgres 17 with TimescaleDB) through Trino. Sign-in goes through Keycloak + store `fuel-db` (Postgres 17 with TimescaleDB) directly, as the read-only role `fuel_ro` + (Trino was removed on 29 Oct 2026). Sign-in goes through Keycloak client `petrolyst`, and the UI carries the Petrolyst brand. **Deploys are manual.** A person deploys a clean commit of `main` from the Mac ("Manual deploy" @@ -17,7 +18,7 @@ Customization and deployment are separate, as with media-insights and media-inte |---|---| | `petrolyst-insights/` | how Superset looks and behaves for Petrolyst: config, Keycloak login code, branding, patches, dashboards. See its README. | | `traefik/` | edge proxy, TLS, the `sso` middleware (oauth2-proxy) that guards the dashboard | -| `stacks/fuel-monitoring/` | Compose stack, Superset Dockerfile, Trino config and catalog, fuel-db roles | +| `stacks/fuel-monitoring/` | Compose stack, Superset Dockerfile and init, fuel-db roles and migrations | | `deploy/` | `deploy.sh` (run on the server by a manual deploy), `rollback.sh` (manual), `lib.sh` (shared) | | `host/` | one-time server setup: `bootstrap.sh`, `secrets-init.sh` | | `keycloak/` | the `petrolyst` client for Keycloak's partial import | @@ -38,10 +39,10 @@ No credential is in this repository. Secrets exist only on the server, under 4. It resolves the artifacts and writes them to `releases//ARTIFACTS`: - `SUPERSET_IMAGE_TAG` is a hash of exactly the Superset image's build inputs. They include `superset/init.sh`, so changing that file runs `superset-init` again. - - `TRINO_CONFIG_HASH` and `TRAEFIK_CONFIG_HASH` are hashes of the bind-mounted config. + - `TRAEFIK_CONFIG_HASH` is a hash of Traefik's bind-mounted static config. - Compose recreates a container only when its value changes. 5. Preflight, before anything live changes: - - every secret and env file exists; + - every secret file exists; - `petrolyst-insights/` holds no deployment files; - `docker compose config` passes for every stack; - the Superset image is built if it is new; @@ -52,13 +53,14 @@ No credential is in this repository. Secrets exist only on the server, under - An empty database is a fresh install: it runs `superset-init`. - If the image changed, or the database is not at the image's migration head: it stops `superset`, writes `backups/superset-meta___