Compare commits

..
5 Commits
Author SHA1 Message Date
Nasim SadatandClaude Opus 5.5 8405706ed8 Remove Trino: Superset reads fuel-db directly as fuel_ro (PD-14)
trino/ deleted; the role trino_ro becomes fuel_ro, its password read from a
secret file at connect time; dashboards' SQL and column list for PostgreSQL;
the Compose project is renamed petrolyst, with volumes pinned to their old
names. AGENTS.md points to the architecture context. Not deployed: the next
deploy needs README's one-time steps first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 10:57:18 +01:00
Nasim SadatandClaude Opus 5.5 686318dfac Remove the Keycloak client import
The file was already gone from the working tree; why is not recorded. README
and AGENTS.md still point to it. Revert this commit to bring it back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 10:57:18 +01:00
Nasim SadatandClaude Opus 5.5 c92c960025 Agent rules for the repository (AGENTS.md, CLAUDE.md)
Layout, deploy and dashboard rules, and checks. Correction to da3f803: the
shell lint did run afterwards, through the README's Docker shellcheck command
(koalaman/shellcheck:v0.10.0 -S warning): clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:17:07 +01:00
Nasim SadatandClaude Opus 5.5 da3f803947 Retire the Superset reader setup and the ATOMA dashboard (PD-13)
ATOMA's users use the Petrolyst app; Superset keeps admins, editors and the
app's embedded dashboards. Removed: the "ATOMA Reader" role and its rules
(petrolyst_reader.py, ReaderAccessMixin, reader.css, the head template, the
SQL-hiding patch), the atoma-reader mapping (it now maps to nothing in
Superset), the iframe allowance for the Map tab, and the "atoma" dashboard
with its Fleet/Map tab, D0 and dashboard.css. The bundle holds atoma-overview
and atoma-site only (byte-identical to before); the guest role keeps the
reader's 12 permissions in roles/atoma_guest.py. Brand files now name
petrolyst-web. SPEC marks the retired parts as history.

Checks (local): bundle checks pass; image rebuilt and healthy; import with
column drift 0 and unchanged embedding ids; the production config loads with
OAuth, embedding off, no reader hook; admin sees atoma-site with its tabs and
filter column; the app's browser tests pass 10/10. shellcheck not run (not
installed). Not deployed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:10:19 +01:00
Nasim SadatandClaude Opus 5.5 1b8e57adb4 Trial state of 29 Oct 2026: ATOMA reader setup, app dashboards, embedding notes
Baseline before the clean-up (PD-13): the PD-11 reader setup and the PD-12 app
dashboards as they ran on the local stack, uncommitted until now. Not deployed,
not independently reviewed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 20:50:26 +01:00
32 changed files with 2453 additions and 333 deletions

No files matched your search

+34
View File
@@ -0,0 +1,34 @@
# petrolyst-infra: agent rules
Deployment of the petrolyst server (atoma.petrolyst.com) and the Superset customization.
Runbook: README.md. Workspace rules: `../AGENTS.md`; shared context: `../.context/` (server and
identity facts: `deployment.md`).
**Architecture:** read `../.context/architecture.md` before planning or building. This repository
runs persistence (fuel-db and its migrations) and Superset; name the bands a change touches and
keep the rules there; a change against a rule or an open conflict goes to the owner first.
## Layout
- `stacks/`, `traefik/`, `deploy/`, `host/`: how and where things run.
- `petrolyst-insights/`: what Superset looks like and does (config, security manager, the guest
role, branding, patches, dashboards as code). Its README maps each file to the image.
- `keycloak/`: the `petrolyst` client import.
## Rules
- **Deploys:** only by README "Manual deploy", only when the owner asks. There is no CI. Commit on
`main` first; never push or deploy unprompted.
- **No secrets** in the repository; `deploy/deploy.sh` refuses if one appears.
- **The Superset image's inputs** (`SUPERSET_INPUTS` in `deploy/lib.sh`) make its tag: a change
there rebuilds Superset on the next deploy.
- **Dashboards are code:** change `build_bundle.py` and SPEC.md, never the UI followed by an
export. `import_bundle.py` never deletes; old objects go by hand (`dashboards/README.md`).
- **Embedding** stays off in production (`superset_config.py`) until PD-12's points are decided;
the local stack turns it on in its own override.
- **ATOMA's users** have no Superset role (PD-13): `atoma-reader` opens the Petrolyst app only.
- fuel-db migrations (`stacks/fuel-monitoring/fuel-db/migrations/`) are applied by hand (P3).
## Checks
- Shell: the README's Docker shellcheck command (warnings and errors must be zero).
- Dashboards: `build_bundle.py` checks the bundle as it builds; import locally and expect column
drift 0 (`../.context/local-stack.md`).
- Superset changes: rebuild the local image and check it turns healthy before calling it done.
+1
View File
@@ -0,0 +1 @@
@AGENTS.md
+61 -17
View File
@@ -5,7 +5,8 @@ Everything that runs on the petrolyst server (109.199.127.84, Ubuntu 24.04):
- **Traefik** at the edge for every project on the server. Its dashboard is at
https://traefik.petrolyst.com, behind Keycloak.
- **fuel-monitoring** (ATOMA): Superset at https://atoma.petrolyst.com, reading the operational
store `fuel-db` (Postgres 17 with TimescaleDB) through Trino. Sign-in goes through Keycloak
store `fuel-db` (Postgres 17 with TimescaleDB) directly, as the read-only role `fuel_ro`
(Trino was removed on 29 Oct 2026). Sign-in goes through Keycloak
client `petrolyst`, and the UI carries the Petrolyst brand.
**Deploys are manual.** A person deploys a clean commit of `main` from the Mac ("Manual deploy"
@@ -17,7 +18,7 @@ Customization and deployment are separate, as with media-insights and media-inte
|---|---|
| `petrolyst-insights/` | how Superset looks and behaves for Petrolyst: config, Keycloak login code, branding, patches, dashboards. See its README. |
| `traefik/` | edge proxy, TLS, the `sso` middleware (oauth2-proxy) that guards the dashboard |
| `stacks/fuel-monitoring/` | Compose stack, Superset Dockerfile, Trino config and catalog, fuel-db roles |
| `stacks/fuel-monitoring/` | Compose stack, Superset Dockerfile and init, fuel-db roles and migrations |
| `deploy/` | `deploy.sh` (run on the server by a manual deploy), `rollback.sh` (manual), `lib.sh` (shared) |
| `host/` | one-time server setup: `bootstrap.sh`, `secrets-init.sh` |
| `keycloak/` | the `petrolyst` client for Keycloak's partial import |
@@ -38,10 +39,10 @@ No credential is in this repository. Secrets exist only on the server, under
4. It resolves the artifacts and writes them to `releases/<sha>/ARTIFACTS`:
- `SUPERSET_IMAGE_TAG` is a hash of exactly the Superset image's build inputs. They include
`superset/init.sh`, so changing that file runs `superset-init` again.
- `TRINO_CONFIG_HASH` and `TRAEFIK_CONFIG_HASH` are hashes of the bind-mounted config.
- `TRAEFIK_CONFIG_HASH` is a hash of Traefik's bind-mounted static config.
- Compose recreates a container only when its value changes.
5. Preflight, before anything live changes:
- every secret and env file exists;
- every secret file exists;
- `petrolyst-insights/` holds no deployment files;
- `docker compose config` passes for every stack;
- the Superset image is built if it is new;
@@ -52,13 +53,14 @@ No credential is in this repository. Secrets exist only on the server, under
- An empty database is a fresh install: it runs `superset-init`.
- If the image changed, or the database is not at the image's migration head: it stops
`superset`, writes `backups/superset-meta_<old tag>_<revision>_<time>.dump`, then runs
`superset-init` (`db upgrade`, `init`, registers "Fuel (Trino)").
`superset-init` (`db upgrade`, `init`, registers "Fuel": fuel-db as `fuel_ro`, its password
read from `/run/secrets` at connect time, never stored in Superset).
- If the image is unchanged and the schema is at its head, it does none of this.
8. Smoke tests. A failure ends the deploy with an error and prints the stacks' state.
- `/health` answers through Traefik with a valid certificate.
- `/login/` leads to Keycloak with `client_id=petrolyst` and PKCE.
- The dashboard redirects to Keycloak.
- Trino reads `fuel`, and Superset reaches Trino.
- Superset's "Fuel" connection reads fuel-db as `fuel_ro`, and its transactions are read-only.
9. Retention:
- 5 releases;
- backups for 30 days;
@@ -75,7 +77,6 @@ The manual deploy then checks both hosts from outside.
| `/opt/petrolyst/incoming/<sha>-manual-<time>` | deploy | one upload; only `deploy.sh` turns it into a release |
| `/opt/petrolyst/releases/<sha>` | deploy, read-only | finalized release plus `ARTIFACTS`; the last 5 are kept |
| `/opt/petrolyst/secrets/{shared,traefik,fuel-monitoring}/` | root:deploy 0750, files 0444 | secrets, mounted as `/run/secrets/<name>` |
| `/opt/petrolyst/env/fuel-monitoring.env` | root:deploy 0440 | `FUEL_DB_TRINO_PASSWORD` for Trino's `${ENV:…}` |
| `/opt/petrolyst/backups/` | deploy 0750 | Superset metadata dumps |
| `/opt/petrolyst/state/` | deploy | last deployed SHA and Superset image tag; `rollback-in-progress` during a migration rollback |
@@ -93,7 +94,8 @@ deploy secrets.
and "Skip" if a resource exists. This creates:
- the confidential client `petrolyst`: standard flow only, PKCE S256, redirect URIs for both
hosts;
- its client roles `superset-admin`, `superset-editor`, `superset-viewer` and `traefik-admin`;
- its client roles `superset-admin`, `superset-editor`, `superset-viewer`, `atoma-reader`
and `traefik-admin`;
- a mapper that puts those roles into the claim `petrolyst_roles`.
2. Clients → petrolyst → Credentials: copy the client secret (step 3 stores it on the server).
3. **MFA**, which Exhibit B requires. A copy of the built-in `browser` flow is not enough, for two
@@ -109,7 +111,10 @@ deploy secrets.
enrols first. Other clients of the realm are unaffected. There is no silent SSO between
Superset and the dashboard.
4. Give people roles: Users → (user) → Role mapping → Assign role → filter by clients →
`petrolyst`. Group mappings work too.
`petrolyst`. Group mappings work too. ATOMA's users get `atoma-reader` and nothing else:
it opens the Petrolyst app (petrolyst-platform) and maps to no Superset role, so in
Superset they see nothing. A realm that already has the client gets the role by hand
(Clients → petrolyst → Roles → Create role) or by importing again with "Skip".
5. Users of the Traefik dashboard need **Email verified** switched on, because oauth2-proxy
refuses unverified addresses.
@@ -156,10 +161,10 @@ becomes the first Superset admin; there is no local admin account.
| Re-deploy a kept release | `sudo -u deploy /opt/petrolyst/infra/deploy/deploy.sh <sha>` |
| Roll back | see below |
| Traefik dashboard | https://traefik.petrolyst.com (role `traefik-admin`) |
| Trino shell | `docker exec -it fuel-monitoring-trino-1 trino` |
| fuel-db shell | `docker exec -it petrolyst-fuel-db-1 psql -U postgres -d fuel` |
| fuel-db from the Mac | `ssh -L 5433:127.0.0.1:5433 petrolyst`, then connect to `localhost:5433`, database `fuel` |
| Logs | `docker compose -p fuel-monitoring logs -f superset` (or `-p traefik`) |
| Superset break-glass | `docker exec -it fuel-monitoring-superset-1 superset fab ...` |
| Logs | `docker compose -p petrolyst logs -f superset` (or `-p traefik`) |
| Superset break-glass | `docker exec -it petrolyst-superset-1 superset fab ...` |
**Manual deploy**, the only way to deploy (there is no CI runner). On the Mac, in this
repository:
@@ -223,12 +228,12 @@ sudo -u deploy /opt/petrolyst/infra/deploy/rollback.sh <sha>
Afterwards, `git revert` the bad commit and push, so `main` matches. A plain revert without the
rollback script is stopped by the downgrade guard.
**Data loaders** (fuel-db). Every table in `public` must belong to `fuel_owner`. `trino_ro` can
**Data loaders** (fuel-db). Every table in `public` must belong to `fuel_owner`. `fuel_ro` can
read new tables only through `fuel_owner`'s default privileges, and PostgreSQL applies those
only to objects `fuel_owner` creates itself. A table that a loader creates as its own role stays
unreadable for Trino. Create each loader role so that its sessions act as `fuel_owner`, and it
unreadable for Superset and the Petrolyst server. Create each loader role so that its sessions act as `fuel_owner`, and it
cannot create anything as itself. Run this in
`docker exec -it fuel-monitoring-fuel-db-1 psql -U postgres -d fuel`:
`docker exec -it petrolyst-fuel-db-1 psql -U postgres -d fuel`:
```sql
CREATE ROLE <loader> LOGIN PASSWORD '<password>';
@@ -236,9 +241,46 @@ GRANT fuel_owner TO <loader> WITH INHERIT FALSE, SET TRUE;
ALTER ROLE <loader> SET role TO 'fuel_owner';
```
Tables it creates, hypertables included, then belong to `fuel_owner`, and Trino can read them.
Tables it creates, hypertables included, then belong to `fuel_owner`, and `fuel_ro` can read them.
After `SET ROLE NONE`, it cannot create anything in `public`. To repair a table that another
role created, run `ALTER TABLE <t> OWNER TO fuel_owner`, then `GRANT SELECT ON <t> TO trino_ro`.
role created, run `ALTER TABLE <t> OWNER TO fuel_owner`, then `GRANT SELECT ON <t> TO fuel_ro`.
**One-time steps after 29 Oct 2026**, at the first deploy that includes them. Two changes
meet there:
- Trino is removed: Superset reads fuel-db directly. The role `trino_ro` becomes `fuel_ro` and
keeps its password (a SCRAM password survives a rename); its secret file moves with it.
- The Compose project `fuel-monitoring` is renamed `petrolyst`. Containers are then called
`petrolyst-<service>-1`. The data volumes keep their names (`fuel-monitoring_superset_db_data`,
`fuel-monitoring_fuel_db_data`; pinned in `compose.yaml`), so no data moves. The old project's
containers must be gone first: two PostgreSQL servers on one data volume corrupt it.
Preflight refuses to deploy while any of them exists.
As root on the server, right before the deploy (Superset is down from step 3 until the deploy
finishes):
```bash
S=/opt/petrolyst/secrets/fuel-monitoring
mv "$S/fuel_db_trino_password" "$S/fuel_db_ro_password" # 1
docker exec -i fuel-monitoring-fuel-db-1 psql -U postgres -d fuel -v ON_ERROR_STOP=1 \
-c 'ALTER ROLE trino_ro RENAME TO fuel_ro' \
-c 'ALTER ROLE fuel_ro SET default_transaction_read_only = on' # 2
docker rm -f $(docker ps -aq --filter label=com.docker.compose.project=fuel-monitoring) # 3
```
4. Deploy as usual ("Manual deploy"). Compose starts the `petrolyst` project on the same volumes
and no longer starts Trino. Compose warns once that the volumes belong to another project;
that is expected. `superset-init` renames the connection "Fuel (Trino)" to "Fuel" and points
it at fuel-db; the smoke test checks that it reads as `fuel_ro`, read-only.
5. Clean up: `rm /opt/petrolyst/env/fuel-monitoring.env`, `rmdir /opt/petrolyst/env`,
`docker image rm trinodb/trino:484`,
`docker network rm fuel-monitoring_backend fuel-monitoring_fuel-db-tunnel`.
**Rollback across the Trino removal** (not tested). `rollback.sh` stops for a release from before
it. To go back anyway: remove the `petrolyst` project's containers, reverse steps 1 and 2 above (rename `fuel_ro` back to `trino_ro` and
`ALTER ROLE trino_ro RESET default_transaction_read_only`, move the secret file back, write
`/opt/petrolyst/env/fuel-monitoring.env` with `FUEL_DB_TRINO_PASSWORD=<that secret>`), rename
the Superset connection "Fuel" back to "Fuel (Trino)" with `trino://superset@trino:8080/fuel`,
then run that release's own script, `/opt/petrolyst/releases/<sha>/deploy/rollback.sh <sha>`.
**Adding a project**: create `stacks/<name>/compose.yaml`, attach the public service to the
external network `proxy`, and set `traefik.*` labels with a unique router name and
@@ -257,6 +299,8 @@ its own gate in its stack:
## Verified locally (2026-10-23)
Before Trino's removal (29 Oct 2026): its lines below are history.
An end-to-end run ran the real `deploy.sh` and `rollback.sh` in a Linux container against
Docker Desktop, with a scratch `/opt/petrolyst`. Production differences in that run:
- a test CA certificate stood in for Let's Encrypt;
+1 -1
View File
@@ -44,7 +44,7 @@ resolve_artifacts() {
fi
chmod -R a-w "$release"
load_artifacts "$release"
log "artifacts: superset $SUPERSET_IMAGE_TAG, trino config $TRINO_CONFIG_HASH, traefik config $TRAEFIK_CONFIG_HASH"
log "artifacts: superset $SUPERSET_IMAGE_TAG, traefik config $TRAEFIK_CONFIG_HASH"
}
# prepare_fuel_monitoring <release>: before anything live changes, build the
+32 -11
View File
@@ -21,7 +21,7 @@ KEYCLOAK_REALM_URL=${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/
PETROLYST_ROOT=$ROOT
export ATOMA_HOST TRAEFIK_HOST KEYCLOAK_REALM_URL PETROLYST_ROOT
# --host pins the daemon, as in petrolyst-ui/deploy/deploy.sh: a stale
# --host pins the daemon, as in petrolyst-web/deploy/deploy.sh: a stale
# `docker context use` must not send this deploy to another daemon.
DOCKER=(docker --host unix:///var/run/docker.sock)
@@ -33,6 +33,7 @@ SUPERSET_INPUTS=(
stacks/fuel-monitoring/superset/init.sh
petrolyst-insights/superset_config.py
petrolyst-insights/petrolyst_security.py
petrolyst-insights/roles
petrolyst-insights/assets/branding
petrolyst-insights/patches
)
@@ -44,8 +45,7 @@ REQUIRED_FILES=(
secrets/fuel-monitoring/superset_secret_key
secrets/fuel-monitoring/superset_db_password
secrets/fuel-monitoring/fuel_db_password
secrets/fuel-monitoring/fuel_db_trino_password
env/fuel-monitoring.env
secrets/fuel-monitoring/fuel_db_ro_password
)
log() { printf '%s %s\n' "$(date -u +%H:%M:%S)" "$*" >&2; }
@@ -82,7 +82,6 @@ hash_inputs() {
compute_artifacts() {
printf 'SUPERSET_IMAGE_TAG=%s\n' "$(hash_inputs "$1" "${SUPERSET_INPUTS[@]}")"
printf 'TRINO_CONFIG_HASH=%s\n' "$(hash_inputs "$1" stacks/fuel-monitoring/trino/etc)"
# Only the static config: dynamic/ is watched and applies without a restart.
printf 'TRAEFIK_CONFIG_HASH=%s\n' "$(hash_inputs "$1" traefik/static)"
}
@@ -91,16 +90,21 @@ compute_artifacts() {
load_artifacts() {
local file=$1/ARTIFACTS line key value
[ -s "$file" ] || die "$file is missing"
unset SUPERSET_IMAGE_TAG TRINO_CONFIG_HASH TRAEFIK_CONFIG_HASH
unset SUPERSET_IMAGE_TAG TRAEFIK_CONFIG_HASH
while IFS= read -r line; do
key=${line%%=*}
value=${line#*=}
[[ $key =~ ^(SUPERSET_IMAGE_TAG|TRINO_CONFIG_HASH|TRAEFIK_CONFIG_HASH)$ && $value =~ ^[0-9a-f]{16}$ ]] ||
# A release from before 29 Oct 2026 still runs Trino, which logs in as
# `trino_ro`, since renamed `fuel_ro`: it cannot start (README "Rollback
# across the Trino removal").
[ "$key" != TRINO_CONFIG_HASH ] ||
die "$file is from before Trino was removed; see README \"Rollback across the Trino removal\""
[[ $key =~ ^(SUPERSET_IMAGE_TAG|TRAEFIK_CONFIG_HASH)$ && $value =~ ^[0-9a-f]{16}$ ]] ||
die "unexpected line in $file: $line"
printf -v "$key" '%s' "$value"
export "${key?}"
done <"$file"
[ -n "${SUPERSET_IMAGE_TAG:-}" ] && [ -n "${TRINO_CONFIG_HASH:-}" ] && [ -n "${TRAEFIK_CONFIG_HASH:-}" ] ||
[ -n "${SUPERSET_IMAGE_TAG:-}" ] && [ -n "${TRAEFIK_CONFIG_HASH:-}" ] ||
die "$file is incomplete"
}
@@ -136,6 +140,10 @@ preflight() {
[ -z "$bad" ] || die "petrolyst-insights/ holds customization only; found: $bad"
compose "$release" traefik config -q
while IFS= read -r dir; do compose "$release" "$dir" config -q; done < <(stack_dirs "$release")
# The stack was the Compose project "fuel-monitoring" until 29 Oct 2026; "petrolyst" uses the
# same data volumes, so both must never run at once (README "One-time steps after 29 Oct 2026").
[ -z "$("${DOCKER[@]}" ps -aq --filter label=com.docker.compose.project=fuel-monitoring)" ] ||
die "containers of the old project fuel-monitoring exist; remove them first (README \"One-time steps after 29 Oct 2026\")"
}
# promote <release> <sha>: make the release the live tree. --chmod applies to
@@ -254,10 +262,23 @@ smoke_tests() {
[ "$code" = 302 ] && [[ $location == "$KEYCLOAK_REALM_URL/"* || $location == *"/oauth2/start"* ]] ||
die "https://$TRAEFIK_HOST/ should redirect to Keycloak, got $code $location"
fm exec -T trino trino --execute 'SELECT count(*) FROM fuel.information_schema.schemata' >/dev/null ||
die "Trino cannot read the fuel catalog"
fm exec -T superset curl -fsS --max-time 10 http://trino:8080/v1/info >/dev/null ||
die "Superset cannot reach Trino"
# Superset's own connection "Fuel" (fuel_ro, password from /run/secrets) reads fuel-db and
# cannot write to it.
fm exec -T superset /app/.venv/bin/python - <<'PY' || die "Superset cannot read fuel-db as fuel_ro, or fuel_ro can write"
import sys
from superset.app import create_app
with create_app().app_context():
from superset import db
from superset.models.core import Database
fuel = db.session.query(Database).filter_by(database_name="Fuel").one()
with fuel.get_sqla_engine() as engine, engine.connect() as conn:
user, read_only = conn.exec_driver_sql(
"SELECT current_user, current_setting('transaction_read_only')").one()
sys.exit(0 if (user, read_only) == ("fuel_ro", "on") else 1)
PY
log "smoke tests passed"
}
+1 -1
View File
@@ -40,7 +40,7 @@ if [ "$(cat /etc/docker/daemon.json 2>/dev/null)" != "$daemon_json" ]; then
fi
systemctl enable --now docker
echo "== swap (4 GB) for Trino, Superset and two Postgres in 7.8 GB"
echo "== swap (4 GB) for Superset and two Postgres in 7.8 GB"
if ! swapon --show=NAME --noheadings | grep -qx /swapfile; then
[ -e /swapfile ] || fallocate -l 4G /swapfile
chmod 600 /swapfile
+3 -15
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Server secrets and env files, run as root after bootstrap.sh.
# Server secrets, run as root after bootstrap.sh.
# secrets-init.sh generate every missing secret; never overwrites one
# secrets-init.sh keycloak-secret store the client secret of Keycloak client
# "petrolyst" (Clients > petrolyst > Credentials),
@@ -17,7 +17,7 @@ getent group deploy >/dev/null || die "group deploy is missing: run host/bootstr
ROOT=/opt/petrolyst
SECRETS=$ROOT/secrets
umask 077
install -d -o root -g deploy -m 0750 "$SECRETS" "$SECRETS/shared" "$SECRETS/traefik" "$SECRETS/fuel-monitoring" "$ROOT/env"
install -d -o root -g deploy -m 0750 "$SECRETS" "$SECRETS/shared" "$SECRETS/traefik" "$SECRETS/fuel-monitoring"
# write_secret <path> <value>: no trailing newline (oauth2-proxy reads the file
# byte for byte), replaced atomically.
@@ -62,22 +62,10 @@ esac
generate "$SECRETS/fuel-monitoring/superset_secret_key" 32
generate "$SECRETS/fuel-monitoring/superset_db_password" 32
generate "$SECRETS/fuel-monitoring/fuel_db_password" 32
generate "$SECRETS/fuel-monitoring/fuel_db_trino_password" 32
generate "$SECRETS/fuel-monitoring/fuel_db_ro_password" 32
# 32 hex characters = a 32-byte cookie key, as oauth2-proxy requires.
generate "$SECRETS/traefik/oauth2_proxy_cookie_secret" 16
# Trino reads the catalog password from its environment (${ENV:...}); keep the
# env file in step with the secret file that fuel-db's init script uses.
env_file=$ROOT/env/fuel-monitoring.env
line="FUEL_DB_TRINO_PASSWORD=$(cat "$SECRETS/fuel-monitoring/fuel_db_trino_password")"
if ! grep -qxF "$line" "$env_file" 2>/dev/null; then
printf '%s\n' "$line" >"$env_file.tmp"
chown root:deploy "$env_file.tmp"
chmod 0440 "$env_file.tmp"
mv "$env_file.tmp" "$env_file"
echo "wrote $env_file"
fi
if [ ! -s "$SECRETS/shared/keycloak_petrolyst_client_secret" ]; then
echo
echo "Still missing: the Keycloak client secret. Copy it from Keycloak"
-56
View File
@@ -1,56 +0,0 @@
{
"clients": [
{
"clientId": "petrolyst",
"name": "Petrolyst",
"description": "Petrolyst apps on the petrolyst server: Superset at atoma.petrolyst.com and the Traefik dashboard at traefik.petrolyst.com (oauth2-proxy). Managed from petrolyst-infra/keycloak/.",
"enabled": true,
"protocol": "openid-connect",
"publicClient": false,
"clientAuthenticatorType": "client-secret",
"standardFlowEnabled": true,
"implicitFlowEnabled": false,
"directAccessGrantsEnabled": false,
"serviceAccountsEnabled": false,
"frontchannelLogout": true,
"baseUrl": "https://atoma.petrolyst.com/",
"redirectUris": [
"https://atoma.petrolyst.com/oauth-authorized/keycloak",
"https://traefik.petrolyst.com/oauth2/callback"
],
"webOrigins": ["+"],
"attributes": {
"pkce.code.challenge.method": "S256",
"post.logout.redirect.uris": "https://atoma.petrolyst.com/*##https://traefik.petrolyst.com/*"
},
"protocolMappers": [
{
"name": "petrolyst client roles",
"protocol": "openid-connect",
"protocolMapper": "oidc-usermodel-client-role-mapper",
"consentRequired": false,
"config": {
"usermodel.clientRoleMapping.clientId": "petrolyst",
"multivalued": "true",
"claim.name": "petrolyst_roles",
"jsonType.label": "String",
"id.token.claim": "true",
"access.token.claim": "true",
"userinfo.token.claim": "true",
"introspection.token.claim": "true"
}
}
]
}
],
"roles": {
"client": {
"petrolyst": [
{ "name": "superset-admin", "description": "Superset Admin at atoma.petrolyst.com" },
{ "name": "superset-editor", "description": "Superset Alpha: build charts and dashboards" },
{ "name": "superset-viewer", "description": "Superset Gamma: view dashboards shared with the role" },
{ "name": "traefik-admin", "description": "Traefik dashboard at traefik.petrolyst.com" }
]
}
}
}
+29 -4
View File
@@ -15,8 +15,9 @@ and where it runs lives in `stacks/`, `traefik/`, `deploy/` and `host/`.
|---|---|---|
| `superset_config.py` | Keycloak sign-in, role mapping, proxy and cookie settings, Redis caches, theme and chart palettes | `/app/pythonpath/` |
| `petrolyst_security.py` | `PetrolystSecurityManager`: user info and roles from the Keycloak ID token, `/login/` straight to Keycloak, `/logout/` through Keycloak | `/app/pythonpath/` |
| `assets/branding/` | logos (full colour and white), favicons, app icons | `/app/superset/static/assets/branding/`, served at `/static/assets/branding/…` |
| `patches/` | runtime patches, `fix_*.py` with `--verify` (see its README); empty at start | `/app/patches/petrolyst/`, applied during the build |
| `roles/` | `atoma_guest.py`: the role "ATOMA Guest" for the Petrolyst app's embedded dashboards, with exactly its permissions; run by hand (see "Embedding") | `/app/petrolyst-roles/` |
| `assets/branding/` | logos (full colour and white): the wordmark alone for the navbar, and the stacked logo; favicons, app icons; ATOMA's logo, which the Petrolyst app shows in its map's corner (`PETROLYST_OPERATOR_LOGO`) | `/app/superset/static/assets/branding/`, served at `/static/assets/branding/…` |
| `patches/` | runtime patches, `fix_*.py` with `--verify` (see its README); none at present | `/app/patches/petrolyst/`, applied during the build |
| `dashboards/` | exported dashboards and import scripts (see its README) | not in the image |
How the image consumes this folder: `stacks/fuel-monitoring/compose.yaml` passes it to the build
@@ -51,15 +52,39 @@ On the server they come from `/opt/petrolyst/secrets/` (see the deployment READM
| `superset-admin` | Admin |
| `superset-editor` | Alpha |
| `superset-viewer` | Gamma |
| `atoma-reader` | none (Public): ATOMA's users use the Petrolyst app, not Superset (PD-13) |
| none | Public (no permissions) |
- `DASHBOARD_RBAC` is on: a viewer sees a dashboard once an admin adds the Gamma role (or a
custom one) to it.
- `FILTERBAR_CLOSED_BY_DEFAULT` is on: a dashboard's filter column opens collapsed, a narrow
strip with an expand button; `?expand_filters=true` opens it from a link.
## Embedding (local trial only)
The Petrolyst app (PD-12, petrolyst-platform `app/` and `server/`) shows dashboards inside
its Insights lens with Superset guest passes. This is switched on **only in the local scratch
stack** (`.local/fmdb-scratch`); `superset_config.py` here leaves it off, and loads without the
guest secret.
- The scratch stack's `superset_config_local.py` turns on `EMBEDDED_SUPERSET`, names the guest
role `GUEST_ROLE_NAME = "ATOMA Guest"`, reads the signing secret with
`_read_secret("superset_guest_token_secret")` and expects the audience `petrolyst-app`. The
Petrolyst server signs the guest passes itself with the same secret.
- `roles/atoma_guest.py` gives "ATOMA Guest" exactly what a dashboard needs to be read. Which
dashboard a guest sees comes from the pass, not the role. It is not in `init.sh`; run it once,
and again whenever its permissions change:
docker exec <superset> /app/.venv/bin/python /app/petrolyst-roles/atoma_guest.py
- The app's dashboards and their embedding ids: `dashboards/README.md`.
Production embedding is an open point of PD-12 (`.context/design-decisions.md`).
## Look and feel
Source: `petrolyst-ui/design/brand/` (tokens `07_UI_Tokens/petrolyst_tokens.css`, logos
`public/brand/`, favicons `03_Favicon_App_Icons/`), copied on 2026-10-23 from petrolyst-ui commit
Source: `petrolyst-web/design/brand/` (tokens `07_UI_Tokens/petrolyst_tokens.css`, logos
`public/brand/`, favicons `03_Favicon_App_Icons/`), copied on 2026-10-23 from petrolyst-web (then petrolyst-ui) commit
`2b2e694`. Copy again when the brand changes.
- Light theme: action teal `#00695C` for primary and links, navy `#0B1D33` text, `#F8FAFC`
@@ -0,0 +1,39 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="300 225 1900 520" role="img" aria-label="ATOMA">
<!-- ATOMA's logo as published on atoma.com.af (mtn-logo.svg, sha256 7a718cdc163c4c59...),
in its own colours; only the viewBox is cropped to the logo. Shown in the fleet map's
corner (dashboards/atoma/build_bundle.py, FLEET_MAP_SRC). -->
<path fill="#1229B3" d="M477.499,608.807h82.857l-82.857-248.571h-82.857l-82.857,248.571h82.857l5.524-16.571h71.809
L477.499,608.807z M422.261,525.95l13.81-41.428l13.809,41.428H422.261z"/>
<path fill="#1229B3" d="M1526.188,608.807h82.856l-82.856-248.571h-82.857l-82.856,248.571h82.856l5.524-16.571h71.809
L1526.188,608.807z M1470.95,525.95l13.81-41.428l13.81,41.428H1470.95z"/>
<polygon fill="#1229B3" points="792.355,360.236 709.498,360.236 626.641,360.236 543.784,360.236 543.784,443.093 626.641,443.093
626.641,608.808 709.498,608.808 709.498,443.093 792.355,443.093 "/>
<g>
<defs>
<rect id="SVGID_1_" width="2500" height="1000"/>
</defs>
<clipPath id="SVGID_2_">
<use xlink:href="#SVGID_1_" overflow="visible"/>
</clipPath>
<path clip-path="url(#SVGID_2_)" fill="#1229B3" d="M933.212,360.236c-68.641,0-124.285,55.645-124.285,124.286
c0,68.64,55.644,124.285,124.285,124.285s124.285-55.645,124.285-124.285C1057.497,415.881,1001.853,360.236,933.212,360.236
M933.212,525.95c-22.88,0-41.428-18.548-41.428-41.428c0-22.881,18.548-41.429,41.428-41.429s41.428,18.548,41.428,41.429
C974.64,507.402,956.092,525.95,933.212,525.95"/>
<polygon clip-path="url(#SVGID_2_)" fill="#1229B3" points="1248.068,360.236 1206.64,401.665 1165.211,360.236 1082.354,360.236
1082.354,608.808 1165.211,608.808 1165.211,484.521 1206.64,525.95 1248.068,484.521 1248.068,608.808 1330.925,608.808
1330.925,360.236 "/>
<path clip-path="url(#SVGID_2_)" fill="#00C8FF" d="M1929.197,240.505l-106.099,106.098c-5.857,5.858-5.857,15.356,0,21.214
l116.705,116.705l-116.705,116.705c-5.857,5.858-5.857,15.356,0,21.214l106.099,106.098c5.857,5.858,15.355,5.858,21.214,0
l116.705-116.704l116.704-116.705c5.858-5.858,5.858-15.356,0-21.214L2067.116,357.21l-116.705-116.705
C1944.553,234.647,1935.055,234.647,1929.197,240.505"/>
<path clip-path="url(#SVGID_2_)" fill="#2449FB" d="M2056.782,494.855l-116.978,116.978l-244.291-244.29
c-5.707-5.707-5.707-14.96,0-20.667l106.645-106.645c5.707-5.707,14.96-5.707,20.667,0l233.957,233.957
C2062.489,479.895,2062.489,489.148,2056.782,494.855"/>
<path clip-path="url(#SVGID_2_)" fill="#1229B3" d="M1802.158,728.812l-106.645-106.645c-5.707-5.707-5.707-14.96,0-20.667
l116.978-116.978l127.313,127.312l-116.978,116.978C1817.119,734.519,1807.865,734.519,1802.158,728.812"/>
<path clip-path="url(#SVGID_2_)" fill="#00AAFF" d="M1823.1,346.603c-5.858,5.858-5.858,15.356,0,21.214l116.705,116.705
L1823.1,601.227c-5.858,5.858-5.858,15.356,0,21.214l53.049,53.049l127.312-127.312l53.049-53.049
c5.858-5.858,5.858-15.356,0-21.214l-53.049-53.049l-127.312-127.312L1823.1,346.603z"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 2.9 KiB

@@ -0,0 +1,28 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="89 671 1268 103" role="img" aria-label="Petrolyst">
<!-- Petrolyst's wordmark alone: petrolyst-web/design/brand/02_Logo/Vector_Master/
petrolyst_logo_standard_fullcolor_no_tagline.svg without the Y mark above it, paths and
colours unchanged, the viewBox cropped to the letters. The navbar logo (superset_config.py). -->
<defs>
<linearGradient id="wordYGradient" x1="945" y1="660" x2="1090" y2="780" gradientUnits="userSpaceOnUse">
<stop offset="0%" stop-color="#009688"/>
<stop offset="100%" stop-color="#00B6E6"/>
</linearGradient>
</defs>
<g fill="#0B1D33">
<path d="M 250.0,753.0 L 250.0,767.0 L 339.0,767.0 L 339.0,764.0 L 340.0,763.0 L 340.0,757.0 L 339.0,756.0 L 339.0,752.0 L 251.0,752.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
<path d="M 250.0,715.0 L 250.0,729.0 L 266.0,729.0 L 267.0,730.0 L 301.0,730.0 L 302.0,729.0 L 307.0,729.0 L 308.0,730.0 L 325.0,730.0 L 326.0,729.0 L 340.0,729.0 L 340.0,716.0 L 339.0,715.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
<path d="M 1260.0,678.0 L 1260.0,692.0 L 1264.0,692.0 L 1265.0,693.0 L 1296.0,693.0 L 1297.0,694.0 L 1297.0,767.0 L 1313.0,767.0 L 1313.0,694.0 L 1314.0,693.0 L 1351.0,693.0 L 1351.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
<path d="M 867.0,678.0 L 867.0,767.0 L 951.0,767.0 L 951.0,753.0 L 950.0,752.0 L 883.0,752.0 L 882.0,751.0 L 882.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
<path d="M 551.0,678.0 L 551.0,767.0 L 566.0,767.0 L 566.0,743.0 L 567.0,742.0 L 567.0,739.0 L 566.0,738.0 L 567.0,737.0 L 604.0,737.0 L 614.0,747.0 L 614.0,748.0 L 622.0,756.0 L 622.0,757.0 L 630.0,765.0 L 630.0,766.0 L 631.0,767.0 L 651.0,767.0 L 651.0,766.0 L 649.0,764.0 L 649.0,763.0 L 642.0,756.0 L 642.0,755.0 L 628.0,741.0 L 628.0,740.0 L 626.0,738.0 L 628.0,736.0 L 630.0,736.0 L 631.0,735.0 L 633.0,735.0 L 634.0,734.0 L 636.0,734.0 L 638.0,732.0 L 639.0,732.0 L 645.0,726.0 L 645.0,725.0 L 648.0,720.0 L 648.0,718.0 L 649.0,717.0 L 649.0,713.0 L 650.0,712.0 L 650.0,703.0 L 649.0,702.0 L 649.0,698.0 L 648.0,697.0 L 648.0,695.0 L 647.0,694.0 L 646.0,691.0 L 644.0,689.0 L 644.0,688.0 L 640.0,684.0 L 639.0,684.0 L 637.0,682.0 L 636.0,682.0 L 633.0,680.0 L 631.0,680.0 L 630.0,679.0 L 626.0,679.0 L 625.0,678.0 Z M 567.0,694.0 L 568.0,693.0 L 625.0,693.0 L 626.0,694.0 L 629.0,695.0 L 632.0,698.0 L 632.0,699.0 L 633.0,700.0 L 633.0,713.0 L 632.0,714.0 L 632.0,716.0 L 630.0,718.0 L 630.0,719.0 L 629.0,720.0 L 628.0,720.0 L 626.0,722.0 L 622.0,722.0 L 621.0,723.0 L 568.0,723.0 L 566.0,721.0 L 566.0,703.0 L 567.0,702.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
<path d="M 398.0,678.0 L 398.0,693.0 L 434.0,693.0 L 435.0,694.0 L 435.0,767.0 L 451.0,767.0 L 451.0,694.0 L 452.0,693.0 L 489.0,693.0 L 489.0,679.0 L 488.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
<path d="M 250.0,678.0 L 250.0,692.0 L 251.0,692.0 L 252.0,693.0 L 339.0,693.0 L 339.0,691.0 L 340.0,690.0 L 340.0,679.0 L 339.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
<path d="M 1134.0,683.0 L 1129.0,688.0 L 1129.0,689.0 L 1127.0,691.0 L 1127.0,693.0 L 1126.0,694.0 L 1126.0,696.0 L 1125.0,697.0 L 1125.0,712.0 L 1126.0,713.0 L 1126.0,715.0 L 1127.0,716.0 L 1128.0,719.0 L 1134.0,725.0 L 1135.0,725.0 L 1137.0,727.0 L 1139.0,727.0 L 1140.0,728.0 L 1143.0,728.0 L 1144.0,729.0 L 1185.0,729.0 L 1186.0,730.0 L 1191.0,730.0 L 1192.0,731.0 L 1193.0,731.0 L 1197.0,735.0 L 1197.0,739.0 L 1198.0,740.0 L 1198.0,742.0 L 1197.0,743.0 L 1197.0,746.0 L 1196.0,747.0 L 1196.0,748.0 L 1193.0,751.0 L 1192.0,751.0 L 1189.0,753.0 L 1184.0,753.0 L 1183.0,754.0 L 1151.0,754.0 L 1150.0,753.0 L 1143.0,753.0 L 1142.0,752.0 L 1139.0,752.0 L 1138.0,751.0 L 1135.0,751.0 L 1134.0,750.0 L 1131.0,750.0 L 1130.0,749.0 L 1128.0,749.0 L 1127.0,748.0 L 1125.0,748.0 L 1125.0,760.0 L 1124.0,761.0 L 1126.0,763.0 L 1128.0,763.0 L 1129.0,764.0 L 1131.0,764.0 L 1132.0,765.0 L 1134.0,765.0 L 1135.0,766.0 L 1139.0,766.0 L 1140.0,767.0 L 1146.0,767.0 L 1147.0,768.0 L 1186.0,768.0 L 1187.0,767.0 L 1191.0,767.0 L 1192.0,766.0 L 1195.0,766.0 L 1196.0,765.0 L 1198.0,765.0 L 1199.0,764.0 L 1201.0,764.0 L 1203.0,762.0 L 1204.0,762.0 L 1209.0,757.0 L 1209.0,756.0 L 1211.0,754.0 L 1211.0,753.0 L 1212.0,752.0 L 1212.0,750.0 L 1213.0,749.0 L 1213.0,734.0 L 1212.0,733.0 L 1212.0,730.0 L 1211.0,729.0 L 1211.0,728.0 L 1209.0,726.0 L 1209.0,725.0 L 1204.0,720.0 L 1203.0,720.0 L 1202.0,719.0 L 1201.0,719.0 L 1196.0,716.0 L 1192.0,716.0 L 1191.0,715.0 L 1148.0,715.0 L 1147.0,714.0 L 1145.0,714.0 L 1142.0,711.0 L 1142.0,710.0 L 1141.0,709.0 L 1141.0,699.0 L 1142.0,698.0 L 1142.0,697.0 L 1145.0,694.0 L 1146.0,694.0 L 1147.0,693.0 L 1150.0,693.0 L 1151.0,692.0 L 1190.0,692.0 L 1191.0,693.0 L 1196.0,693.0 L 1197.0,694.0 L 1201.0,694.0 L 1202.0,695.0 L 1204.0,695.0 L 1205.0,696.0 L 1209.0,696.0 L 1209.0,683.0 L 1206.0,681.0 L 1203.0,681.0 L 1202.0,680.0 L 1198.0,680.0 L 1197.0,679.0 L 1193.0,679.0 L 1192.0,678.0 L 1182.0,678.0 L 1181.0,677.0 L 1156.0,677.0 L 1155.0,678.0 L 1146.0,678.0 L 1145.0,679.0 L 1143.0,679.0 L 1142.0,680.0 L 1140.0,680.0 L 1139.0,681.0 L 1138.0,681.0 L 1137.0,682.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
<path d="M 714.0,691.0 L 714.0,692.0 L 711.0,695.0 L 711.0,696.0 L 709.0,698.0 L 709.0,699.0 L 708.0,700.0 L 708.0,702.0 L 707.0,703.0 L 707.0,706.0 L 706.0,707.0 L 706.0,711.0 L 705.0,712.0 L 705.0,734.0 L 706.0,735.0 L 706.0,739.0 L 707.0,740.0 L 707.0,743.0 L 708.0,744.0 L 708.0,745.0 L 709.0,746.0 L 709.0,747.0 L 710.0,748.0 L 711.0,751.0 L 713.0,753.0 L 713.0,754.0 L 718.0,759.0 L 719.0,759.0 L 721.0,761.0 L 722.0,761.0 L 723.0,762.0 L 724.0,762.0 L 729.0,765.0 L 731.0,765.0 L 732.0,766.0 L 735.0,766.0 L 736.0,767.0 L 741.0,767.0 L 742.0,768.0 L 775.0,768.0 L 776.0,767.0 L 781.0,767.0 L 782.0,766.0 L 785.0,766.0 L 786.0,765.0 L 788.0,765.0 L 789.0,764.0 L 790.0,764.0 L 791.0,763.0 L 792.0,763.0 L 793.0,762.0 L 796.0,761.0 L 798.0,759.0 L 799.0,759.0 L 805.0,753.0 L 805.0,752.0 L 807.0,750.0 L 807.0,749.0 L 809.0,746.0 L 809.0,744.0 L 811.0,741.0 L 811.0,737.0 L 812.0,736.0 L 812.0,711.0 L 811.0,710.0 L 811.0,706.0 L 810.0,705.0 L 810.0,703.0 L 809.0,702.0 L 809.0,700.0 L 808.0,699.0 L 808.0,698.0 L 806.0,696.0 L 806.0,695.0 L 803.0,692.0 L 803.0,691.0 L 799.0,687.0 L 798.0,687.0 L 793.0,683.0 L 792.0,683.0 L 791.0,682.0 L 789.0,682.0 L 786.0,680.0 L 783.0,680.0 L 782.0,679.0 L 779.0,679.0 L 778.0,678.0 L 770.0,678.0 L 769.0,677.0 L 748.0,677.0 L 747.0,678.0 L 739.0,678.0 L 738.0,679.0 L 734.0,679.0 L 733.0,680.0 L 731.0,680.0 L 730.0,681.0 L 728.0,681.0 L 727.0,682.0 L 726.0,682.0 L 725.0,683.0 L 722.0,684.0 L 720.0,686.0 L 719.0,686.0 Z M 725.0,702.0 L 730.0,697.0 L 731.0,697.0 L 733.0,695.0 L 735.0,695.0 L 738.0,693.0 L 742.0,693.0 L 743.0,692.0 L 772.0,692.0 L 773.0,693.0 L 777.0,693.0 L 778.0,694.0 L 781.0,694.0 L 782.0,695.0 L 785.0,696.0 L 787.0,698.0 L 788.0,698.0 L 791.0,701.0 L 791.0,702.0 L 793.0,704.0 L 793.0,705.0 L 794.0,706.0 L 794.0,708.0 L 795.0,709.0 L 795.0,712.0 L 796.0,713.0 L 796.0,733.0 L 795.0,734.0 L 795.0,738.0 L 794.0,739.0 L 793.0,742.0 L 791.0,744.0 L 791.0,745.0 L 788.0,748.0 L 787.0,748.0 L 785.0,750.0 L 784.0,750.0 L 781.0,752.0 L 778.0,752.0 L 777.0,753.0 L 770.0,753.0 L 769.0,754.0 L 747.0,754.0 L 746.0,753.0 L 740.0,753.0 L 739.0,752.0 L 737.0,752.0 L 736.0,751.0 L 734.0,751.0 L 733.0,750.0 L 732.0,750.0 L 730.0,748.0 L 729.0,748.0 L 725.0,744.0 L 725.0,743.0 L 722.0,738.0 L 722.0,734.0 L 721.0,733.0 L 721.0,713.0 L 722.0,712.0 L 722.0,709.0 L 723.0,708.0 L 723.0,706.0 L 724.0,705.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
<path d="M 95.0,678.0 L 95.0,766.0 L 96.0,767.0 L 111.0,767.0 L 111.0,737.0 L 112.0,736.0 L 171.0,736.0 L 172.0,735.0 L 175.0,735.0 L 176.0,734.0 L 178.0,734.0 L 179.0,733.0 L 180.0,733.0 L 182.0,731.0 L 183.0,731.0 L 186.0,728.0 L 187.0,728.0 L 187.0,727.0 L 190.0,724.0 L 190.0,723.0 L 193.0,718.0 L 193.0,715.0 L 194.0,714.0 L 194.0,702.0 L 193.0,701.0 L 193.0,696.0 L 192.0,695.0 L 192.0,693.0 L 191.0,692.0 L 190.0,689.0 L 188.0,687.0 L 188.0,686.0 L 185.0,683.0 L 184.0,683.0 L 182.0,681.0 L 181.0,681.0 L 180.0,680.0 L 178.0,680.0 L 177.0,679.0 L 174.0,679.0 L 173.0,678.0 L 168.0,678.0 L 167.0,677.0 L 156.0,677.0 L 155.0,678.0 L 135.0,678.0 L 134.0,677.0 L 132.0,677.0 L 131.0,678.0 L 108.0,678.0 L 107.0,677.0 L 105.0,677.0 L 104.0,678.0 Z M 111.0,693.0 L 112.0,692.0 L 169.0,692.0 L 170.0,693.0 L 172.0,693.0 L 177.0,698.0 L 177.0,699.0 L 178.0,700.0 L 178.0,712.0 L 177.0,713.0 L 177.0,715.0 L 176.0,716.0 L 176.0,717.0 L 174.0,719.0 L 173.0,719.0 L 170.0,721.0 L 167.0,721.0 L 166.0,722.0 L 154.0,722.0 L 153.0,721.0 L 152.0,722.0 L 115.0,722.0 L 114.0,721.0 L 113.0,722.0 L 111.0,720.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
<rect x="250" y="678" width="16" height="90" fill="#0B1D33"/>
</g>
<path d="M 1084.0,678.0 L 1063.0,678.0 L 1056.0,685.0 L 1056.0,686.0 L 1049.0,693.0 L 1049.0,694.0 L 1042.0,701.0 L 1042.0,702.0 L 1036.0,708.0 L 1036.0,709.0 L 1035.0,710.0 L 1035.0,711.0 L 1033.0,713.0 L 1033.0,714.0 L 1032.0,715.0 L 1032.0,716.0 L 1031.0,717.0 L 1031.0,718.0 L 1030.0,719.0 L 1030.0,720.0 L 1028.0,723.0 L 1028.0,725.0 L 1026.0,728.0 L 1026.0,730.0 L 1025.0,731.0 L 1025.0,733.0 L 1024.0,734.0 L 1024.0,738.0 L 1023.0,739.0 L 1023.0,743.0 L 1022.0,744.0 L 1022.0,752.0 L 1021.0,753.0 L 1021.0,766.0 L 1022.0,767.0 L 1038.0,767.0 L 1038.0,737.0 L 1039.0,736.0 L 1039.0,734.0 L 1040.0,733.0 L 1040.0,732.0 L 1041.0,731.0 L 1041.0,730.0 L 1043.0,728.0 L 1043.0,727.0 L 1046.0,724.0 L 1046.0,723.0 L 1052.0,717.0 L 1052.0,716.0 L 1060.0,708.0 L 1060.0,707.0 L 1065.0,702.0 L 1065.0,701.0 L 1072.0,694.0 L 1072.0,693.0 L 1077.0,688.0 L 1077.0,687.0 L 1082.0,682.0 L 1082.0,681.0 L 1084.0,679.0 L 1085.0,679.0 Z" fill="url(#wordYGradient)" fill-rule="evenodd"/>
<path d="M 975.0,678.0 L 975.0,679.0 L 979.0,683.0 L 979.0,684.0 L 983.0,688.0 L 983.0,689.0 L 990.0,696.0 L 990.0,697.0 L 996.0,703.0 L 996.0,704.0 L 1002.0,710.0 L 1002.0,711.0 L 1010.0,719.0 L 1010.0,720.0 L 1014.0,724.0 L 1014.0,725.0 L 1017.0,728.0 L 1017.0,729.0 L 1018.0,730.0 L 1018.0,731.0 L 1019.0,732.0 L 1020.0,732.0 L 1020.0,729.0 L 1021.0,728.0 L 1021.0,726.0 L 1022.0,725.0 L 1022.0,722.0 L 1023.0,721.0 L 1023.0,715.0 L 1024.0,714.0 L 1023.0,713.0 L 1023.0,709.0 L 1022.0,708.0 L 1022.0,707.0 L 1021.0,706.0 L 1021.0,705.0 L 1016.0,700.0 L 1016.0,699.0 L 1010.0,693.0 L 1010.0,692.0 L 1003.0,685.0 L 1003.0,684.0 L 997.0,678.0 Z" fill="url(#wordYGradient)" fill-rule="evenodd"/>
</svg>

After

Width:  |  Height:  |  Size: 10 KiB

@@ -0,0 +1,18 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="89 671 1268 103" role="img" aria-label="Petrolyst">
<!-- Petrolyst's wordmark alone: petrolyst-web/design/brand/02_Logo/Vector_Master/
petrolyst_logo_standard_white_no_tagline.svg without the Y mark above it, paths and
colours unchanged, the viewBox cropped to the letters. The navbar logo (superset_config.py). -->
<g fill="#FFFFFF"><path d="M 250.0,753.0 L 250.0,767.0 L 339.0,767.0 L 339.0,764.0 L 340.0,763.0 L 340.0,757.0 L 339.0,756.0 L 339.0,752.0 L 251.0,752.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
<path d="M 250.0,715.0 L 250.0,729.0 L 266.0,729.0 L 267.0,730.0 L 301.0,730.0 L 302.0,729.0 L 307.0,729.0 L 308.0,730.0 L 325.0,730.0 L 326.0,729.0 L 340.0,729.0 L 340.0,716.0 L 339.0,715.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
<path d="M 1260.0,678.0 L 1260.0,692.0 L 1264.0,692.0 L 1265.0,693.0 L 1296.0,693.0 L 1297.0,694.0 L 1297.0,767.0 L 1313.0,767.0 L 1313.0,694.0 L 1314.0,693.0 L 1351.0,693.0 L 1351.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
<path d="M 867.0,678.0 L 867.0,767.0 L 951.0,767.0 L 951.0,753.0 L 950.0,752.0 L 883.0,752.0 L 882.0,751.0 L 882.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
<path d="M 551.0,678.0 L 551.0,767.0 L 566.0,767.0 L 566.0,743.0 L 567.0,742.0 L 567.0,739.0 L 566.0,738.0 L 567.0,737.0 L 604.0,737.0 L 614.0,747.0 L 614.0,748.0 L 622.0,756.0 L 622.0,757.0 L 630.0,765.0 L 630.0,766.0 L 631.0,767.0 L 651.0,767.0 L 651.0,766.0 L 649.0,764.0 L 649.0,763.0 L 642.0,756.0 L 642.0,755.0 L 628.0,741.0 L 628.0,740.0 L 626.0,738.0 L 628.0,736.0 L 630.0,736.0 L 631.0,735.0 L 633.0,735.0 L 634.0,734.0 L 636.0,734.0 L 638.0,732.0 L 639.0,732.0 L 645.0,726.0 L 645.0,725.0 L 648.0,720.0 L 648.0,718.0 L 649.0,717.0 L 649.0,713.0 L 650.0,712.0 L 650.0,703.0 L 649.0,702.0 L 649.0,698.0 L 648.0,697.0 L 648.0,695.0 L 647.0,694.0 L 646.0,691.0 L 644.0,689.0 L 644.0,688.0 L 640.0,684.0 L 639.0,684.0 L 637.0,682.0 L 636.0,682.0 L 633.0,680.0 L 631.0,680.0 L 630.0,679.0 L 626.0,679.0 L 625.0,678.0 Z M 567.0,694.0 L 568.0,693.0 L 625.0,693.0 L 626.0,694.0 L 629.0,695.0 L 632.0,698.0 L 632.0,699.0 L 633.0,700.0 L 633.0,713.0 L 632.0,714.0 L 632.0,716.0 L 630.0,718.0 L 630.0,719.0 L 629.0,720.0 L 628.0,720.0 L 626.0,722.0 L 622.0,722.0 L 621.0,723.0 L 568.0,723.0 L 566.0,721.0 L 566.0,703.0 L 567.0,702.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
<path d="M 398.0,678.0 L 398.0,693.0 L 434.0,693.0 L 435.0,694.0 L 435.0,767.0 L 451.0,767.0 L 451.0,694.0 L 452.0,693.0 L 489.0,693.0 L 489.0,679.0 L 488.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
<path d="M 250.0,678.0 L 250.0,692.0 L 251.0,692.0 L 252.0,693.0 L 339.0,693.0 L 339.0,691.0 L 340.0,690.0 L 340.0,679.0 L 339.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
<path d="M 1134.0,683.0 L 1129.0,688.0 L 1129.0,689.0 L 1127.0,691.0 L 1127.0,693.0 L 1126.0,694.0 L 1126.0,696.0 L 1125.0,697.0 L 1125.0,712.0 L 1126.0,713.0 L 1126.0,715.0 L 1127.0,716.0 L 1128.0,719.0 L 1134.0,725.0 L 1135.0,725.0 L 1137.0,727.0 L 1139.0,727.0 L 1140.0,728.0 L 1143.0,728.0 L 1144.0,729.0 L 1185.0,729.0 L 1186.0,730.0 L 1191.0,730.0 L 1192.0,731.0 L 1193.0,731.0 L 1197.0,735.0 L 1197.0,739.0 L 1198.0,740.0 L 1198.0,742.0 L 1197.0,743.0 L 1197.0,746.0 L 1196.0,747.0 L 1196.0,748.0 L 1193.0,751.0 L 1192.0,751.0 L 1189.0,753.0 L 1184.0,753.0 L 1183.0,754.0 L 1151.0,754.0 L 1150.0,753.0 L 1143.0,753.0 L 1142.0,752.0 L 1139.0,752.0 L 1138.0,751.0 L 1135.0,751.0 L 1134.0,750.0 L 1131.0,750.0 L 1130.0,749.0 L 1128.0,749.0 L 1127.0,748.0 L 1125.0,748.0 L 1125.0,760.0 L 1124.0,761.0 L 1126.0,763.0 L 1128.0,763.0 L 1129.0,764.0 L 1131.0,764.0 L 1132.0,765.0 L 1134.0,765.0 L 1135.0,766.0 L 1139.0,766.0 L 1140.0,767.0 L 1146.0,767.0 L 1147.0,768.0 L 1186.0,768.0 L 1187.0,767.0 L 1191.0,767.0 L 1192.0,766.0 L 1195.0,766.0 L 1196.0,765.0 L 1198.0,765.0 L 1199.0,764.0 L 1201.0,764.0 L 1203.0,762.0 L 1204.0,762.0 L 1209.0,757.0 L 1209.0,756.0 L 1211.0,754.0 L 1211.0,753.0 L 1212.0,752.0 L 1212.0,750.0 L 1213.0,749.0 L 1213.0,734.0 L 1212.0,733.0 L 1212.0,730.0 L 1211.0,729.0 L 1211.0,728.0 L 1209.0,726.0 L 1209.0,725.0 L 1204.0,720.0 L 1203.0,720.0 L 1202.0,719.0 L 1201.0,719.0 L 1196.0,716.0 L 1192.0,716.0 L 1191.0,715.0 L 1148.0,715.0 L 1147.0,714.0 L 1145.0,714.0 L 1142.0,711.0 L 1142.0,710.0 L 1141.0,709.0 L 1141.0,699.0 L 1142.0,698.0 L 1142.0,697.0 L 1145.0,694.0 L 1146.0,694.0 L 1147.0,693.0 L 1150.0,693.0 L 1151.0,692.0 L 1190.0,692.0 L 1191.0,693.0 L 1196.0,693.0 L 1197.0,694.0 L 1201.0,694.0 L 1202.0,695.0 L 1204.0,695.0 L 1205.0,696.0 L 1209.0,696.0 L 1209.0,683.0 L 1206.0,681.0 L 1203.0,681.0 L 1202.0,680.0 L 1198.0,680.0 L 1197.0,679.0 L 1193.0,679.0 L 1192.0,678.0 L 1182.0,678.0 L 1181.0,677.0 L 1156.0,677.0 L 1155.0,678.0 L 1146.0,678.0 L 1145.0,679.0 L 1143.0,679.0 L 1142.0,680.0 L 1140.0,680.0 L 1139.0,681.0 L 1138.0,681.0 L 1137.0,682.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
<path d="M 714.0,691.0 L 714.0,692.0 L 711.0,695.0 L 711.0,696.0 L 709.0,698.0 L 709.0,699.0 L 708.0,700.0 L 708.0,702.0 L 707.0,703.0 L 707.0,706.0 L 706.0,707.0 L 706.0,711.0 L 705.0,712.0 L 705.0,734.0 L 706.0,735.0 L 706.0,739.0 L 707.0,740.0 L 707.0,743.0 L 708.0,744.0 L 708.0,745.0 L 709.0,746.0 L 709.0,747.0 L 710.0,748.0 L 711.0,751.0 L 713.0,753.0 L 713.0,754.0 L 718.0,759.0 L 719.0,759.0 L 721.0,761.0 L 722.0,761.0 L 723.0,762.0 L 724.0,762.0 L 729.0,765.0 L 731.0,765.0 L 732.0,766.0 L 735.0,766.0 L 736.0,767.0 L 741.0,767.0 L 742.0,768.0 L 775.0,768.0 L 776.0,767.0 L 781.0,767.0 L 782.0,766.0 L 785.0,766.0 L 786.0,765.0 L 788.0,765.0 L 789.0,764.0 L 790.0,764.0 L 791.0,763.0 L 792.0,763.0 L 793.0,762.0 L 796.0,761.0 L 798.0,759.0 L 799.0,759.0 L 805.0,753.0 L 805.0,752.0 L 807.0,750.0 L 807.0,749.0 L 809.0,746.0 L 809.0,744.0 L 811.0,741.0 L 811.0,737.0 L 812.0,736.0 L 812.0,711.0 L 811.0,710.0 L 811.0,706.0 L 810.0,705.0 L 810.0,703.0 L 809.0,702.0 L 809.0,700.0 L 808.0,699.0 L 808.0,698.0 L 806.0,696.0 L 806.0,695.0 L 803.0,692.0 L 803.0,691.0 L 799.0,687.0 L 798.0,687.0 L 793.0,683.0 L 792.0,683.0 L 791.0,682.0 L 789.0,682.0 L 786.0,680.0 L 783.0,680.0 L 782.0,679.0 L 779.0,679.0 L 778.0,678.0 L 770.0,678.0 L 769.0,677.0 L 748.0,677.0 L 747.0,678.0 L 739.0,678.0 L 738.0,679.0 L 734.0,679.0 L 733.0,680.0 L 731.0,680.0 L 730.0,681.0 L 728.0,681.0 L 727.0,682.0 L 726.0,682.0 L 725.0,683.0 L 722.0,684.0 L 720.0,686.0 L 719.0,686.0 Z M 725.0,702.0 L 730.0,697.0 L 731.0,697.0 L 733.0,695.0 L 735.0,695.0 L 738.0,693.0 L 742.0,693.0 L 743.0,692.0 L 772.0,692.0 L 773.0,693.0 L 777.0,693.0 L 778.0,694.0 L 781.0,694.0 L 782.0,695.0 L 785.0,696.0 L 787.0,698.0 L 788.0,698.0 L 791.0,701.0 L 791.0,702.0 L 793.0,704.0 L 793.0,705.0 L 794.0,706.0 L 794.0,708.0 L 795.0,709.0 L 795.0,712.0 L 796.0,713.0 L 796.0,733.0 L 795.0,734.0 L 795.0,738.0 L 794.0,739.0 L 793.0,742.0 L 791.0,744.0 L 791.0,745.0 L 788.0,748.0 L 787.0,748.0 L 785.0,750.0 L 784.0,750.0 L 781.0,752.0 L 778.0,752.0 L 777.0,753.0 L 770.0,753.0 L 769.0,754.0 L 747.0,754.0 L 746.0,753.0 L 740.0,753.0 L 739.0,752.0 L 737.0,752.0 L 736.0,751.0 L 734.0,751.0 L 733.0,750.0 L 732.0,750.0 L 730.0,748.0 L 729.0,748.0 L 725.0,744.0 L 725.0,743.0 L 722.0,738.0 L 722.0,734.0 L 721.0,733.0 L 721.0,713.0 L 722.0,712.0 L 722.0,709.0 L 723.0,708.0 L 723.0,706.0 L 724.0,705.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
<path d="M 95.0,678.0 L 95.0,766.0 L 96.0,767.0 L 111.0,767.0 L 111.0,737.0 L 112.0,736.0 L 171.0,736.0 L 172.0,735.0 L 175.0,735.0 L 176.0,734.0 L 178.0,734.0 L 179.0,733.0 L 180.0,733.0 L 182.0,731.0 L 183.0,731.0 L 186.0,728.0 L 187.0,728.0 L 187.0,727.0 L 190.0,724.0 L 190.0,723.0 L 193.0,718.0 L 193.0,715.0 L 194.0,714.0 L 194.0,702.0 L 193.0,701.0 L 193.0,696.0 L 192.0,695.0 L 192.0,693.0 L 191.0,692.0 L 190.0,689.0 L 188.0,687.0 L 188.0,686.0 L 185.0,683.0 L 184.0,683.0 L 182.0,681.0 L 181.0,681.0 L 180.0,680.0 L 178.0,680.0 L 177.0,679.0 L 174.0,679.0 L 173.0,678.0 L 168.0,678.0 L 167.0,677.0 L 156.0,677.0 L 155.0,678.0 L 135.0,678.0 L 134.0,677.0 L 132.0,677.0 L 131.0,678.0 L 108.0,678.0 L 107.0,677.0 L 105.0,677.0 L 104.0,678.0 Z M 111.0,693.0 L 112.0,692.0 L 169.0,692.0 L 170.0,693.0 L 172.0,693.0 L 177.0,698.0 L 177.0,699.0 L 178.0,700.0 L 178.0,712.0 L 177.0,713.0 L 177.0,715.0 L 176.0,716.0 L 176.0,717.0 L 174.0,719.0 L 173.0,719.0 L 170.0,721.0 L 167.0,721.0 L 166.0,722.0 L 154.0,722.0 L 153.0,721.0 L 152.0,722.0 L 115.0,722.0 L 114.0,721.0 L 113.0,722.0 L 111.0,720.0 Z" fill="#FFFFFF" fill-rule="evenodd"/><rect x="250" y="678" width="16" height="90" fill="#FFFFFF"/></g>
<path d="M 1084.0,678.0 L 1063.0,678.0 L 1056.0,685.0 L 1056.0,686.0 L 1049.0,693.0 L 1049.0,694.0 L 1042.0,701.0 L 1042.0,702.0 L 1036.0,708.0 L 1036.0,709.0 L 1035.0,710.0 L 1035.0,711.0 L 1033.0,713.0 L 1033.0,714.0 L 1032.0,715.0 L 1032.0,716.0 L 1031.0,717.0 L 1031.0,718.0 L 1030.0,719.0 L 1030.0,720.0 L 1028.0,723.0 L 1028.0,725.0 L 1026.0,728.0 L 1026.0,730.0 L 1025.0,731.0 L 1025.0,733.0 L 1024.0,734.0 L 1024.0,738.0 L 1023.0,739.0 L 1023.0,743.0 L 1022.0,744.0 L 1022.0,752.0 L 1021.0,753.0 L 1021.0,766.0 L 1022.0,767.0 L 1038.0,767.0 L 1038.0,737.0 L 1039.0,736.0 L 1039.0,734.0 L 1040.0,733.0 L 1040.0,732.0 L 1041.0,731.0 L 1041.0,730.0 L 1043.0,728.0 L 1043.0,727.0 L 1046.0,724.0 L 1046.0,723.0 L 1052.0,717.0 L 1052.0,716.0 L 1060.0,708.0 L 1060.0,707.0 L 1065.0,702.0 L 1065.0,701.0 L 1072.0,694.0 L 1072.0,693.0 L 1077.0,688.0 L 1077.0,687.0 L 1082.0,682.0 L 1082.0,681.0 L 1084.0,679.0 L 1085.0,679.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
<path d="M 975.0,678.0 L 975.0,679.0 L 979.0,683.0 L 979.0,684.0 L 983.0,688.0 L 983.0,689.0 L 990.0,696.0 L 990.0,697.0 L 996.0,703.0 L 996.0,704.0 L 1002.0,710.0 L 1002.0,711.0 L 1010.0,719.0 L 1010.0,720.0 L 1014.0,724.0 L 1014.0,725.0 L 1017.0,728.0 L 1017.0,729.0 L 1018.0,730.0 L 1018.0,731.0 L 1019.0,732.0 L 1020.0,732.0 L 1020.0,729.0 L 1021.0,728.0 L 1021.0,726.0 L 1022.0,725.0 L 1022.0,722.0 L 1023.0,721.0 L 1023.0,715.0 L 1024.0,714.0 L 1023.0,713.0 L 1023.0,709.0 L 1022.0,708.0 L 1022.0,707.0 L 1021.0,706.0 L 1021.0,705.0 L 1016.0,700.0 L 1016.0,699.0 L 1010.0,693.0 L 1010.0,692.0 L 1003.0,685.0 L 1003.0,684.0 L 997.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
</svg>

After

Width:  |  Height:  |  Size: 9.9 KiB

+37 -5
View File
@@ -1,13 +1,45 @@
# dashboards/
Exported Superset dashboards and the operator scripts that import them. Not baked into the
image, so changes here do not rebuild or restart Superset. Empty until the first fuel
dashboards exist.
Superset dashboards as code and the operator scripts that import them. Not baked into the
image, so changes here do not rebuild or restart Superset.
- `atoma/`: ATOMA's two dashboards for the Petrolyst app (PD-12, PD-13; SPEC.md §10):
`atoma-overview` (fleet totals) and `atoma-site` (one site, its four sections as tabs). The
app embeds them; admins open them in Superset, with the filter column. `SPEC.md` is the
design; `build_bundle.py` generates the import bundle and `import_bundle.py` imports it
(overwriting the objects the bundle defines).
Build and import:
docker cp atoma/import_bundle.py petrolyst-superset-1:/tmp/
docker exec petrolyst-superset-1 /app/.venv/bin/python /tmp/import_bundle.py --show-database
python3 atoma/build_bundle.py --database-uuid <printed uuid> --out atoma-dashboards.zip
docker cp atoma-dashboards.zip petrolyst-superset-1:/tmp/
docker exec petrolyst-superset-1 /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip --user <username>
Exit status 2 means a dataset's columns differ from its view: regenerate `atoma/columns.tsv`
(command in its header), rebuild and import again.
An import never deletes. Where they exist, delete these by hand after importing: the
dashboards from before 28 Oct ("ATOMA - Sites", "ATOMA - Site Detail") and their charts M1,
M3, M4 and M5; the ATOMA dashboard (slug `atoma`, retired 29 Oct with PD-13) and its chart D0
"Site"; the one-tab dashboards `atoma-site-summary`, `-fuel`, `-runtime` and `-baseline`
(replaced by `atoma-site` on 29 Oct); and the role "ATOMA Reader".
The dashboards get no role; the app shows them with guest passes (see "Embedding" in the
parent README; local stack only). Add `--embedded-out FILE --embed-domain <app origin>` to the
import to make them embeddable from the app's origin; FILE then receives
`{slug: embedding id}` for the Petrolyst server. The ids stay the same across imports. Local
stack:
docker exec fmdb-scratch-superset /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip \
--user admin --embedded-out /tmp/embedded-dashboards.json --embed-domain http://127.0.0.1:8098
docker cp fmdb-scratch-superset:/tmp/embedded-dashboards.json <Petrolyst>/.local/fmdb-scratch/
Export from a running instance (inside the container, as the server's deploy user):
docker exec fuel-monitoring-superset-1 superset export-dashboards -f /tmp/dashboards.zip
docker cp fuel-monitoring-superset-1:/tmp/dashboards.zip .
docker exec petrolyst-superset-1 superset export-dashboards -f /tmp/dashboards.zip
docker cp petrolyst-superset-1:/tmp/dashboards.zip .
Import is the reverse with `superset import-dashboards -p <zip> -u <username>`. The
username must be an existing Superset user, which here means someone who has signed in
+437 -120
View File
@@ -1,13 +1,23 @@
# ATOMA dashboards: Sites Map and Site Detail
# ATOMA dashboards: Overview and Site, for the Petrolyst app
**Status: proposed, not built.**
- `build_bundle.py` (phase P2) will generate both dashboards, every dataset and every chart from
this specification, as one Superset import bundle.
**Status (29 Oct 2026, PD-13): two dashboards, `atoma-overview` and `atoma-site` (§10), built and
tested on a local Superset; not imported live.** The Petrolyst app embeds them, and admins open
them in Superset with the filter column.
- `build_bundle.py` generates the dashboards, every dataset and every chart from this
specification, as one Superset import bundle; `import_bundle.py` imports it (§6).
- **Retired 29 Oct (PD-13):** the "ATOMA" dashboard (slug `atoma`) with its Map tab (the React
fleet-map view of petrolyst-platform) and Insights tab, its D0 header row, and the Superset
reader setup ("ATOMA Reader", `reader.css`). ATOMA's users use the Petrolyst app. §2, §3, §9 and
the readers' part of §6 describe them and stay as history. §4's data rules and four tabs are
current: they are `atoma-site`'s tabs, without D0 (the app shows the site's header).
- Earlier changes are marked *(P2)* (local tests of 24 Oct, §8) and *(28 Oct)* (one dashboard
with the fleet-map view, §9).
- Nothing is imported into the live Superset without the owner's go-ahead (phase P3).
**Datasets.** The `public.v_*` views of
`stacks/fuel-monitoring/fuel-db/migrations/0003_bi_views.sql`, read through Trino as
`fuel.public`.
`stacks/fuel-monitoring/fuel-db/migrations/0003_bi_views.sql`, read from fuel-db directly (database
`fuel`, schema `public`) as the read-only role `fuel_ro`. *Until 29 Oct 2026 Superset read them
through Trino as `fuel.public`; Trino was removed then.*
**Design and rationale** live in the fuel-monitoring corpus, in
`.context/plans/2026-10-24-fuel-db-baseline-dashboard.md`:
@@ -15,72 +25,103 @@
- the site-specific facts left out here (positions, distances, pilot sites) are there too;
- Q-numbers refer to its §7, and phases P0–P5 to its §8.
Two dashboards replace the earlier tab-per-site design; nothing in them is specific to a site.
Nothing in the dashboards is specific to a site.
## 1. Capabilities of the installed Superset 6.1.0
Source: read-only inspection of `config.py`, `superset_config.py` and the frontend bundle on
24 Oct 2026. Items marked *(local test)* are confirmed in P2 before the design relies on them.
24 Oct 2026. The former *(local test)* items were tested in P2 (§8); rows marked *(P2)* changed as a result.
| Requirement | What 6.1.0 provides | Choice |
|---|---|---|
| Map chart | `deck_scatter` (deck.gl Scatterplot): a modern plugin with the InteractiveChart behaviour. Also: deck_multi, deck_screengrid / hex / grid, country_map, world_map, cartodiagram, legacy `mapbox` | **deck.gl Scatterplot** |
| Basemap | `DECKGL_BASE_MAP` unset → OSM raster (`tile.openstreetmap.org`), OSM Swiss topo, Mapbox styles. `MAPBOX_API_KEY` is empty, so the Mapbox styles fail. CSP `connect-src` already allows the OSM host; Traefik sets no CSP | set `DECKGL_BASE_MAP` to **OSM only**; "© OpenStreetMap contributors" in a Markdown under the map |
| OSM terms | no key, no cost; tile usage policy: light use, attribution, no bulk download, a valid Referer (ours sends the origin) | fine for an internal user base; viewers' browsers reveal the viewed area (tile coordinates) and their IP to OSMF, not our data. Self-hosted tiles later if needed |
| Tooltip | "Tooltip contents" (drag columns) plus "Customize tooltips template" (Handlebars). `js_tooltip` exists but `ENABLE_JAVASCRIPT_CONTROLS` is False | contents and template; **no JavaScript** |
| Marker colour | "Categorical color" by a dimension, from the colour scheme; a fixed colour per label through the dashboard's `label_colors` *(local test)* | colour by `baseline_status_label`. Fallback: `deck_multi` with one fixed-colour scatter layer per status |
| Marker size | fixed radius, with minimum = maximum pixel radius | uniform |
| Tooltip | "Tooltip contents" (drag columns) plus "Customize tooltips template" (Handlebars). `js_tooltip` exists but `ENABLE_JAVASCRIPT_CONTROLS` is False. The contents show column names, not labels, and leave out the colour column | **contents only** *(P2)*: short tooltip-only calculated columns (`code`, `name`, `l_per_h`, `method`, `status`, `validity`, `latest_record`); no template, no JavaScript |
| Marker colour | "Categorical color" by a dimension, from the colour scheme; a fixed colour per label through the dashboard's `label_colors` (confirmed in P2) | colour by `baseline_status_label` |
| Marker size | fixed radius, with minimum = maximum pixel radius. No outline: the layer is drawn with `stroked: false` | uniform, no outline *(P2)* |
| Text cards | The Handlebars chart compiles its template with `eval`; Superset's own CSP (`script-src 'self' 'strict-dynamic'` + nonce) blocks that, so the chart shows a CSP error instead *(P2)* | **no Handlebars**: one-row tables. The CSP stays as it is |
| Chart descriptions | "Show description" (`expanded_slices`) never renders in 6.1: the dashboard loads `description_markeddown`, the chart holder reads `description_markdown` *(P2)* | descriptions stay on the charts; the visible reading notes are a Markdown block at the top of each tab |
| Viewport | `autozoom` fits the filtered points | on |
| Clustering | none in deck.gl; only the legacy `mapbox` chart (`clustering_radius`, needs a Mapbox key) | **no clustering** (§3) |
| Marker click | emits a **cross-filter** on the latitude and longitude columns: the exact point, or a box around the click that shrinks as you zoom in. It applies to other charts on the same dashboard | cross-filters the Sites list |
| Navigation | **no chart-to-dashboard navigation.** A dashboard reads `?native_filters=<rison>` into preselected native filter values (also `native_filters_key`). Table cells render sanitized HTML (`allow_render_html`) | an "Open site detail" link column in the Sites list |
| Native filters | Select (single or multi, required, searchable, "first value by default"), Time range; scoping per chart and tab; cross-filter scoping per chart | §4, §5 |
| Markdown iframe *(28 Oct)* | `HTML_SANITIZATION` is on and strips an iframe from a Markdown block; `HTML_SANITIZATION_SCHEMA_EXTENSIONS` lets named tags and attributes through. The CSP (`default-src 'self'`, no `frame-src`) allows frames from this origin only | allow `iframe` with `src`, `title`, `loading`, `style`; the fleet-map view is served on the same origin under `/views` (§3) |
| Deep link into a tab *(28 Oct)* | `#TAB-<id>` after the URL opens that tab; it combines with `?native_filters=` (tested 28 Oct) | the "Site details" button opens the Site details tab with the site preset |
*(28 Oct)* The map moved out of Superset into the fleet-map view (§3). The rows Map chart,
Basemap, Tooltip, Marker colour, Marker size, Viewport, Clustering and Marker click describe the
deck.gl option the dashboard used until then; the view's choices are in §3. The Navigation row
still holds: the view builds the same `?native_filters=` link.
No custom frontend and no patch are needed.
## 2. Navigation flow
## 2. Navigation flow *(retired, PD-13: history)*
*(28 Oct)* One dashboard, two top-level tabs. *(29 Oct)* They are named **Map** (`TAB-fleet`) and
**Insights** (`TAB-site`), the dashboard's lenses; earlier text in this document calls them Fleet and
Site details. Readers see them as a lens switch, centred in the navbar (§6).
```
ATOMA — Sites (landing, slug atoma-sites)
filters: Site search · Zone · Province · Pilot status (default: active) · Baseline status
M1 map: markers fitted to the filtered sites ──hover──► tooltip
└─click marker──► cross-filter: M3 Sites list shows that site (or the few under the click)
M3 Sites list ─────────────"Open site detail"──► ATOMA — Site Detail ?native_filters=(site preset)
M4/M5 Sites without an accepted location ──────"Open site detail"──► same
ATOMA — Site Detail (slug atoma-site-detail)
filters: Site (exactly one, searchable) · Viewing period
tabs: Summary · Fuel & Deliveries · Runtime & Electricity · Baseline & Evidence
header link "← Sites map"
ATOMA (slug atoma)
filters, in Superset's left column: Site (exactly one, searchable) · Viewing period
— both out of scope on Map; readers have no column there (§6)
[Map] the fleet-map view (iframe, /views/fleet-map/), the page's full width
towers fitted to the placed sites ──hover or click──► popup
"Find a site" (code or name) ──► the map flies to it ──► its popup
└─"Site details"──► /superset/dashboard/atoma/?native_filters=(site preset)#TAB-site
sites without an accepted location: counted in the map's key ("Choose them in Insights",
the view's tab_label parameter); chosen in the Site filter
[Insights] D0 site header, then the tabs
Summary · Fuel & Deliveries · Runtime & Electricity · Baseline & Evidence
(a row of section buttons, not a second row of tabs: dashboard.css)
```
## 3. Sites Map (landing dashboard)
The filters sit in Superset's left column (`filter_bar_orientation: VERTICAL`) *(owner,
29 Oct; `build_bundle.py` checks it)*. A filter shows in the column on the tabs where it is in
scope; the others are listed under "Filters out of scope" (the Viewing period on Summary, for
example). No filter applies to the Fleet tab, so readers have no column there and the map takes
the full width (`reader.css`, §6); admins keep the column on both tabs. The column opens
collapsed, a 32 px strip with an expand button (`FILTERBAR_CLOSED_BY_DEFAULT`,
`superset_config.py`; owner, 29 Oct); the selected site shows in the Site table above the
sections. So for a reader the tabs move sideways by 32 px between Fleet and Site details, or by
the column's width once it is expanded; the owner accepted that.
From 28 to 29 Oct the filters sat in a row above the tabs (`HORIZONTAL`). On the Fleet tab that
row held only an empty "More filters", and hiding it there made the tabs jump down on Site
details. Superset 6.1 has no per-tab setting for the filter bar; it hides it only per link
(`?show_filters=0`).
Until 28 Oct there were two dashboards: "ATOMA - Sites" (slug `atoma-sites`: a deck.gl map, five
filters, a sites list) and "ATOMA - Site Detail" (slug `atoma-site-detail`), linked by "Open site
detail" and "← Sites map".
## 3. Fleet tab: the fleet-map view *(28 Oct; retired, PD-13: history)*
The Fleet tab holds one Markdown block with an iframe of the fleet-map view, a page of
petrolyst-platform (`views/src/fleet-map/`) served under `/views/` on Superset's own origin. The
view reads the read service's `site_status` operation (`/views/api/v1/site-status`, from
`public.v_site`). It is not a Superset chart: it has no dataset and no native filter, and both
filters are out of scope on this tab.
```
[Filter bar: Site search · Zone · Province · Pilot status · Baseline status]
[M1 Map (deck.gl Scatterplot) ................................ w8][M2 Status key + attribution w4]
[M3 Sites list (table with link column) ..................................................... w12]
[M4 Sites without accepted location: count w3][M5 Sites without accepted location (table) .. w9]
[Fleet tab: MARKDOWN-fleet-map, w12 h100]
<iframe src="/views/fleet-map/?detail=/superset/dashboard/atoma/&filter=NATIVE_FILTER-site&tab=TAB-site">
```
| # | Question | Type | Dataset | Content | Missing or insufficient |
|---|---|---|---|---|---|
| M1 | Where are the sites, and what is each one's baseline status? | deck.gl Scatterplot; lat/lon from `latitude`, `longitude`; uniform size; colour = `baseline_status_label`; autozoom | v_site | tooltip: `site_code`, `site_name`, `lph_estimate` ("not available" when NULL), `lph_method`, `baseline_status_label`, `baseline_validity_note` (when set), `latest_record_on` | sites without an accepted location have NULL lat/lon and are not plotted; M4/M5 show them |
| M2 | What do the colours mean? | Markdown (static) | — | the status definitions below; "Colours show evidence status, not performance"; OSM attribution | — |
| M3 | Which sites match, and how do I open one? | Table; search box; server pagination | v_site | code, name, province, pilot status, baseline status, L/h (+ method), L/day, latest record, location status, `detail_link` | marker click narrows it; "not available" shown as text |
| M4 | How many sites lack an accepted location? | Big Number Total, COUNT(*) with saved filter `location_status <> 'accepted'` | v_site | count; follows the dashboard filters | 0 is shown as 0 |
| M5 | Which sites are they? | Table; search box | v_site, same saved filter | code, name, province, `location_status` (awaiting acceptance · missing), baseline status, `detail_link` | — |
| Part | Content | Missing or insufficient |
|---|---|---|
| Map | MapLibre on OpenFreeMap Positron: OpenStreetMap data, English and Dari place names (chosen by the owner on 28 Oct after a side-by-side with OSM standard tiles, which carry the same data). Fitted to the placed sites, clear of the key | no placed site: Afghanistan, and the key says so |
| Towers | one per site with an accepted location: a pin holding a lattice tower, in the site's baseline status colour (below); a site that is not an active pilot site faded; uniform size. The site's identity is `site_uid`, since two sites can share a `site_code` (an alias cut to 20 characters) | sites without an accepted location are not drawn |
| Popup | opens 120 ms after hover; stays while the pointer is on the tower or the popup; closes 250 ms after it leaves both. A click, tap or keyboard focus pins it; Esc or a click on the map closes it. It springs out of the tower (a fade only with reduced motion) and takes the side with room; when neither side has room for all of it, the side with more room, where its figures scroll while the header, the status with any validity note, and **Site details** stay in view. Content: code, name; status as `baseline_status_text` ("not approved" for Eligible and Provisional); any `baseline_validity_note` right beside the status and the L/h; L/h with method ("not available" when NULL); L/day, "whole record, not per running hour"; latest record; pilot status by name (active, reference, candidate, withdrawn, or not in the pilot); other sites whose towers cover this one (the same or nearly the same position), each a button that switches the popup to it; **Site details** | an approved estimate whose inputs changed is never shown without its validity note |
| Site details | a link (`target="_top"`) to this dashboard with the site preset in the Site filter, on the Site details tab: the `?native_filters=` value of §5 plus `#TAB-site`. The iframe's address names the dashboard, the filter and the tab, and `build_bundle.py` checks them | no link when that address is missing or malformed: the view accepts only a dashboard path and filter and tab ids of a fixed shape |
| Search | "Find a site: code or name", top left. Matches ignore case: code prefixes first, then codes containing the text, then names containing it; at most 8, each with its status dot and status. Arrow keys and Enter, or the pointer, choose one; the map flies to it at zoom 14 or closer (past the clusters) and pins its popup. Esc clears the field | sites without an accepted location are not searched (owner's choice, 28 Oct): the key counts them and the Site filter finds them; "No site matches" when nothing does |
| Operator's logo | ATOMA's logo in its own colours, bottom right above the map's credits, on frosted glass (*owner, 29 Oct*). The dashboard passes it in the iframe's address (`logo=/static/assets/branding/atoma_logo.svg`, `logo_alt=ATOMA`; `build_bundle.py` checks the file exists); the view takes only an image from that folder on its own origin, so it carries no operator's branding | no `logo` in the address: no logo |
| Key | the four statuses; "Faded: not an active pilot site"; "Colours show evidence status, not performance"; the number of sites without an accepted location, with "Choose them in Site details" | — |
| Keyboard, screen readers | a hidden list of the sites with the same links; focusing one opens its popup | — |
**Filters** (all on `v_site`, scoped to M1, M3, M4 and M5):
- Site search: Select on `site_label`, multi, searchable. The label holds code, name and uid, so
code or name both match.
- Zone, Province: Select.
- Pilot status: default `active`.
- Baseline status: Select on `baseline_status_label`.
The cross-filter from M1 is scoped **to M3 only**. M4 and M5 hold the sites without coordinates and
must not be emptied by it.
**Baseline status and marker colours.** Colours are neutral categorical hues, with no red or green
**Baseline status and tower colours.** Colours are neutral categorical hues, with no red or green
and no reference to L/h:
| Status | Definition (`fm.v_site_baseline_status`) | Colour |
@@ -90,24 +131,26 @@ and no reference to L/h:
| Provisional | no approval; a proposed version below BP-1, **or** usable intervals with no version yet (no estimate shown, method "no baseline version proposed") | amber #c98500 |
| Insufficient evidence | no usable interval. **Every pilot site while no runtime evidence exists** | slate #64748B |
**Nearby sites.**
- Sites a few kilometres apart separate at zoom ≥ 11 (≈ 60 px per 2–4 km at zoom 12) and coincide
at country zoom (≈ 2 km per pixel at zoom 6). The pilot's own distances are in the design doc.
- Handling:
- autozoom to the filtered sites (the default "active" filter opens on the pilot area);
- small uniform markers (min = max radius, about 7 px) with an outline;
- a click near overlapping markers selects every site in the click box, and M3 lists them;
- no clustering.
- With hundreds of sites, a `deck_multi` density layer (screengrid) for low zoom is a later option.
**Nearby sites.** Towers that would overlap cluster into a teal circle with their count (radius
36 px, up to zoom 13); a click on it zooms in until they separate. The pilot's sites, a few
kilometres apart, separate at the zoom the map opens with. The towers are one symbol layer, so
thousands of sites stay fast.
## 4. Site Detail (one dashboard, any site)
**Sites without an accepted location** are not plotted. The key counts them, and the Site filter
in Site details lists every site, placed or not.
## 4. Site details: the four tabs of `atoma-site` (any site)
*(PD-13)* What follows about D0 (the site header row) and the Fleet tab is history: the app shows
the site's header, and its Map lens replaced the Fleet tab.
**Filters**
- **Site** (`NATIVE_FILTER-site`):
- Select on `v_site.site_label`, single value, **required**, searchable, no default;
- Select on `v_site.site_label`, single value, **required**, searchable,
**first value by default** with `requiredFirst` (the §4 fallback, adopted in P2);
- scope: every chart on every tab;
- every detail dataset carries `site_label` from `v_site`, so this one filter drives all of
them *(local test)*.
them (confirmed in P2: every chart request carries the selected site).
- **Viewing period** (`NATIVE_FILTER-period`):
- Time range, default no filter;
- scope: the time-series charts on Fuel & Deliveries and Runtime & Electricity only;
@@ -115,24 +158,31 @@ and no reference to L/h:
baseline's fixed window.
**Exactly one site, and never network totals:**
1. A required, single-value filter *(local test: with the value cleared, no chart queries
unfiltered data)*.
2. The header (D0) and KPI (S1) are Handlebars templates that render a site only when exactly one
row arrives. Otherwise they show "Select one site".
1. A required, single-value filter. *P2: "required" alone does not stop the charts: with no
value they queried every site.*
2. Therefore "Select first filter value by default", with `requiredFirst`, as the filter editor
writes it: the charts wait for the filter, the first site is applied without "Apply filters",
and "Clear all" stays disabled. Without `requiredFirst` the first value waits behind "Apply
filters" while the charts have already queried every site.
3. No Big Number or aggregate tile for site figures. They are computed per site in `v_site`,
never aggregated by Superset.
4. Fallback, if (1) fails its test: "Select first filter value by default", so the Site filter
is never empty.
never aggregated by Superset. *(29 Oct, PD-12: the app's Overview, §10, adds up recorded
amounts across sites; per-site rates are still never averaged.)*
4. The Handlebars "Select one site" guard is gone with Handlebars (§1); with (2) the filter is
never empty.
```
[D0 Site header: code + name (large) · province · pilot status · baseline status · "← Sites map"]
Summary [S1 L/h KPI card w6][S2 Supporting measures w6]
[D0 Site: one-row table, code · name · province · pilot status · baseline status · location] (28 Oct: inside Site details, no link)
every tab [Reading notes (Markdown) w12] (P2)
Summary [S1 Fuel per running hour: one-row table w12] (P2)
[S2 Supporting measures w12]
[S3 Evidence coverage timeline (Gantt) w12]
Fuel & Deliveries [F1 Deliveries w6][F2 Recorded stock w6]
[F5 Deliveries table w12] (P2)
[F3 Fuel per calendar day by interval w12]
[F4 Interval table w12]
Runtime & Electricity[R1 Runtime per day by meter interval w6][R2 Hour-meter evidence w6]
[R3 Grid energy, kWh w6][R4 Electricity cost, AFN w6]
[R1T Runtime pairs and their validity w12] (P2)
[R3 Grid energy, kWh w6][R4 Electricity cost, AFN: bar over table w6]
Baseline & Evidence [B1 Baseline versions w6][B2 Inputs of each version w6]
[B3 All intervals: eligibility and exclusions w12]
[B4 Interval L/h against baseline w12]
@@ -144,17 +194,17 @@ Baseline & Evidence [B1 Baseline versions w6][B2 Inputs of each version w6]
| # | Question it answers | Type | Dataset | Dims / measures (units) | Grain | When data are missing or insufficient |
|---|---|---|---|---|---|---|
| D0 | Which site am I looking at? | Handlebars | v_site | code and name (large), province, pilot status, baseline status, location status, link "← Sites map" | site | "Select one site" unless exactly one row |
| S1 | What is the site's L/h, by which method, over which window, how strong? | Handlebars | v_site | `lph_estimate`, `lph_method`, `baseline_status_label`, `baseline_validity_note`, window, decision; L/day whole record | site | Keeps its position. **In v1 every site shows "Insufficient runtime evidence: no hour-meter readings"**, plus L/day labelled "per calendar day, not per running hour". "Eligible" and "Provisional" carry "not approved" |
| D0 | Which site am I looking at? | Table, one row *(P2)* | v_site | code, name, province, pilot status, baseline status, location status *(28 Oct: the link "← Sites map" is gone; the Fleet tab is one click away)* | site | the Site filter is never empty (§4) |
| S1 | What is the site's L/h, by which method, over which window, how strong? | Table, one row *(P2)* | v_site | `lph_estimate`, `lph_method`, `baseline_status_label`, `baseline_validity_note`, window, decision; L/day whole record | site | Keeps its position. **In v1 every site shows "Insufficient runtime evidence: no hour-meter readings"**, plus L/day labelled "per calendar day, not per running hour". "Eligible" and "Provisional" carry "not approved" |
| S2 | What supports it? | Table | v_kpi_measures | measure, value, unit, `value_kind` (recorded · calculated · assumed · not established), window. Two L/day rows: the whole record (moves with new records) and the L/day reference version (fixed window, its status and any input drift in the note) | site × measure | Rows stay and show "not established" |
| S3 | Which periods are covered by which evidence, and where are the gaps? | Gantt | v_coverage | lane; start/end; colour = status | span | Empty lanes are visible; nothing is interpolated |
| F1 | When and how much was delivered? | Bar (time axis, **day** grain) | v_delivery (one row per delivery event) | `delivery_date`; **SUM(`delivered_l`)**, titled "Litres delivered per day (all deliveries that day)"; tooltip COUNT(deliveries) | day | Bars only on delivery dates; subtitle "deliveries are not consumption"; F4 lists the individual deliveries |
| F2 | What stock was recorded? | Scatter (no lines) | v_stock_reading | `observed_on`; litres; series = reading type × `value_kind` | observation | Points only; nothing between readings |
| F3 | How much fuel was used per calendar day in each observed interval? | Bar (categorical x, ordered by start) | v_interval | `interval_label`; MAX(`fuel_l_per_day`) (L/d); colour = `stock_status` | stock interval | Excluded intervals greyed, with the reason in the tooltip; no monthly spreading |
| S3 | Which periods are covered by which evidence, and where are the gaps? | Gantt | coverage_detail *(P2: virtual dataset over v_coverage)* | lane name without the view's sort number; start/end; colour = status; tooltip: litres delivered in a stock interval, or a bill's AFN amount | span | Nothing is interpolated. *(P2)* The view emits a lane only where spans exist, so the tab's notes name the four lanes and say that a missing lane means no span could be formed: no evidence of that kind, or too few readings (F2, R2 show single readings) |
| F1 | When and how much was delivered? | Bar (time axis, **day** grain) | v_delivery (one row per delivery event) | `delivery_date`; **SUM(`delivered_l`)**, titled "Litres delivered per day (all deliveries that day)" | day | Bars only on delivery dates; "deliveries are not consumption" in the tab's notes. *(P2)* The count and the individual deliveries are in **F5**, a table on v_delivery (date, visit of the day, event, litres, price, sources, review), which also lists a delivery of unknown quantity; its search box shows the number of records |
| F2 | What stock was recorded? | Scatter (no lines) | v_stock_reading | `observed_on`; litres; series = reading type × `value_kind`, split by visit when a day has several *(P2)* | observation | Points only; nothing between readings |
| F3 | How much fuel was used per calendar day in each observed interval? | Bar (categorical x, ordered by start) | v_interval | `interval_label`; MAX(`fuel_l_per_day`) (L/d); colour = `stock_status` | stock interval | Intervals that are not `ok` greyed (*P2*: fixed grey per status in Site Detail's `label_colors`); the status is the series name, so legend and tooltip give the reason; no monthly spreading |
| F4 | Interval detail | Table | v_interval | dates, opening, deliveries, closing, used, L/d, status | interval | Shows the reasons |
| R1 | How many hours did each generator run per day? | Bar (categorical) | v_runtime | interval × meter; MAX(h/day). h/day is NULL for an invalid pair (`pair_valid` false: two epochs, a decreasing timer or over 24 h/day); such pairs show no bar and are listed with their flags in the tooltip | meter interval | **v1: empty**, with "No hour-meter readings" in the description |
| R1 | How many hours did each generator run per day? | Bar (categorical) | v_runtime | interval × meter; MAX(h/day). h/day is NULL for an invalid pair (`pair_valid` false: two epochs, a decreasing timer or over 24 h/day); such pairs show no bar and are listed with their flags in **R1T**, a pair table with `pair_valid`, `same_epoch`, `monotonic`, `over_24h_per_day` *(P2: the bar tooltip cannot show them)* | meter interval | **v1: empty**, with "No hour-meter readings" in the tab's notes |
| R2 | What runtime evidence exists? | Table | v_hour_meter_evidence | meter, mapped set, date, hours, state, epoch | reading | **v1: no rows**, "No hour-meter readings" |
| R3 | How much grid energy did the site receive? | Handlebars → bar once kWh exist | v_kpi_measures (kWh row) | kWh per bill month | month | "No kWh records; bills are cost only and are not converted to fuel or hours" |
| R3 | How much grid energy did the site receive? | Table, the kWh row *(P2)*; a bar once kWh exist | v_kpi_measures (kWh row) | kWh per bill month | month | "No kWh records; bills are cost only and are not converted to fuel or hours" |
| R4 | What did grid electricity cost? | Bar (month) + table | v_electricity_cost | bill month; amount (AFN, inferred) | month | Own chart, never on a fuel axis. A disputed bill is listed without amount or kWh |
| B1 | Which baseline versions exist, and who approved them? | Table | v_baseline_version | code, status, value, window, grade, decision | version | "No baseline defined" |
| B2 | What went into each version? | Table | v_baseline_input | interval, included, reason, fuel, runtime, drift | version × interval | empty until a version exists |
@@ -164,24 +214,31 @@ Baseline & Evidence [B1 Baseline versions w6][B2 Inputs of each version w6]
| B6 | What is missing or inconsistent? | Table | v_data_quality | check, status, detail | check | Lists the gaps: no runtime, placeholder rows, no kWh, no transfer records, location awaiting acceptance |
| B7 | Which documents do the figures rest on? | Table | v_evidence | title, kind, issuer, date, used for | document | — |
| B8 | What did the operator workbook calculate, and is it usable? | Table | v_imported_calculation | label, formula, value, our reading, use, reason, `claim_status` | cell | Shows "excluded" with the reason for "actual L/h" and "CP L/h". A disputed transcription shows its status, without formula or value |
| B9 | Does stock depletion match engine-measured fuel? | Table | v_reconciliation | stock L, engine L, `covered_h` of `interval_h`, difference L and %. The window runs from the opening visit's departure to the closing visit's arrival. The difference is shown only for an `ok` stock interval with both visit times recorded, in which every generator is placed at the site throughout and covered for the whole window | interval × generator | "No flow-meter data yet"; partial coverage, unknown visit times or a mid-interval placement show the hours covered and no difference |
| B9 | Does stock depletion match engine-measured fuel? | Table | v_reconciliation | engine L of this generator, `covered_h` of `interval_h`; site-wide: stock L, engine L of all generators, `all_generators_metered`, difference L and % *(P2: both engine totals shown, so the difference reconciles)*. The window runs from the opening visit's departure to the closing visit's arrival. The difference is shown only for an `ok` stock interval with both visit times recorded, in which every generator is placed at the site throughout and covered for the whole window | interval × generator | "No flow-meter data yet"; partial coverage, unknown visit times or a mid-interval placement show the hours covered and no difference |
| B10 | What does the equipment specification say? | Table | v_spec_reference | attribute, value, unit, basis, applicability, `claim_status` | spec | Titled "Manufacturer and nameplate data, not site performance". A disputed specification shows its status, without value |
| B11 | What would L/h be under the stated runtime? | Table | v_modelled_lph | assumption code, status, basis; h/day with low–high; window, `matched_days`; L/day; modelled L/h with low–high; `value_kind` | site × assumption | Titled "Modelled from an assumed runtime, not measured". "No runtime assumption recorded" |
Modelled L/h (B11) is in v1 (Q9, decided 24 Oct 2026). It pools only the usable intervals inside
the runtime assumption's dates, and shows that window and its days. It never reaches S1, the map or
the M3 Sites list.
the runtime assumption's dates, and shows that window and its days. It never reaches S1 or the
fleet map.
## 5. Datasets and filter mapping
Physical datasets, one per `public.v_*` view on Trino `fuel.public`. Every dataset has the same
Physical datasets, one per `public.v_*` view in fuel-db's `public` schema, plus one virtual dataset
*(P2)*. `coverage_detail` adds two columns to `v_coverage`:
- `lane_name`, the lane without its sort number, which read as a count;
- `amount`, a stock interval's delivered litres (from its opening visit up to its closing visit,
so the opening refill is included) or a bill's AFN amount.
Both use only `v_coverage`, `v_delivery` and `v_electricity_cost`. Every dataset has the same
four site columns from `v_site`: `site_uid`, `site_code`, `site_name`, `site_label`.
| Dataset | Grain | Used by | Native filters that apply |
|---|---|---|---|
| v_site | site | M1–M5, D0, S1 | landing: all five filters, plus the M1 cross-filter (on M3); detail: Site |
| v_site | site | D0, S1; the fleet-map view through the read service *(28 Oct)* | Site |
| v_kpi_measures | site × measure | S2, R3 | Site |
| v_coverage | site × lane × span | S3 | Site |
| v_coverage | site × lane × span | — (S3 reads `coverage_detail`) | Site |
| coverage_detail *(P2)* | site × lane × span | S3 | Site |
| v_delivery | delivery | F1 | Site, Viewing period |
| v_stock_reading | observation | F2 | Site, Viewing period |
| v_interval | stock interval | F3, F4, B3, B4 | Site; Viewing period on F3 and F4 only |
@@ -198,79 +255,339 @@ is grouped or filtered by `metric` and `population_key` (both in `v_interval`),
across methods or generator sets. The site's L/h figure itself comes only from a baseline version
(`v_site`), never from this metric. There is no AVG metric on ratio columns.
**Link column.** `detail_link` is a calculated column on the `v_site` dataset, defined in the
bundle in Trino SQL. The native filter id stays inside Superset's own configuration:
**Site details link** *(28 Oct; history since PD-13)*. The fleet-map view built it
(petrolyst-platform `views/src/shared/deepLink.ts`, retired) from its iframe address: the
dashboard path, the Site filter's id and the tab. The Petrolyst app uses the same rison shape for
the Site filter's preset (`app/src/app/core/site-filter.ts`). The rison shape is the one the sites
list used until 28 Oct, confirmed in P2, and the view's tests compared it with that list's Trino
output:
```sql
'<a href="/superset/dashboard/atoma-site-detail/?native_filters='
|| url_encode('(NATIVE_FILTER-site:(id:NATIVE_FILTER-site,'
|| 'extraFormData:(filters:!((col:site_label,op:IN,val:!(' || q || ')))),'
|| 'filterState:(value:!(' || q || ')),ownState:()))')
|| '">Open site detail</a>'
-- q = '''' || replace(replace(site_label, '!', '!!'), '''', '!''') || '''' -- rison string quoting
```
/superset/dashboard/atoma/?native_filters=<url-encoded:>
(NATIVE_FILTER-site:(id:NATIVE_FILTER-site,extraFormData:(filters:!((col:site_label,op:IN,val:!(q)))),
filterState:(value:!(q)),ownState:()))
#TAB-site
-- q = the site label in rison quotes: 'label', with ! written !! and ' written !'
```
- The exact rison state shape is a *(local test)*.
- Fallback: the link opens Site Detail with the Site filter expanded (`expand_filters=1`) and the
site's label shown beside the link, to pick by search.
## 6. Build as code, scale, access
**Build**
- `petrolyst-insights/dashboards/atoma/build_bundle.py` writes one import bundle containing both
dashboards, every dataset and every chart:
- stable UUIDs (uuid5), so a re-import updates instead of duplicating;
- fixed native-filter ids (`NATIVE_FILTER-site`, `-period`, `-search`, …);
- `build_bundle.py --database-uuid <uuid>` writes one import bundle containing the dashboard,
every dataset and every chart:
- stable UUIDs (uuid5), so a re-import updates instead of duplicating; the zip is
byte-identical for the same inputs;
- fixed native-filter ids (`NATIVE_FILTER-site`, `NATIVE_FILTER-period`) and tab ids
(`TAB-fleet`, `TAB-site`, `TAB-summary`, …);
- `label_colors` for the four statuses;
- the existing "Fuel (Trino)" database UUID, read from an export first.
- Import: `superset import-dashboards`.
- every view column, from `columns.tsv` (a snapshot of PostgreSQL's catalog);
- the existing "Fuel" database UUID, which differs per instance:
`import_bundle.py --show-database` prints it.
- Before writing, it checks that every column a chart or filter uses exists in its dataset,
that every chart in a Site filter's scope has `site_label`, that the fleet map's iframe
address names this dashboard, its Site filter and the Site details tab *(28 Oct)*, and that
no row is wider than 12.
- Import: `import_bundle.py <zip> --user <username>`, inside the Superset container (*P2*: not
`superset import-dashboards`, which leaves existing datasets and charts unchanged on a
re-import). It runs the dataset, chart and dashboard importers with overwrite on, never creates
or changes the database, and reports columns that differ between a dataset and its view.
- A re-import replaces edits made in the Superset UI to these objects.
**Adding a site**
- Import its `site.yaml` and its records. Nothing in either dashboard names a site.
- New sites appear in M1 (once their location is accepted) or in M5, in M3, and in the Site filter.
- Import its `site.yaml` and its records. Nothing in the dashboard or the fleet-map view names a
site.
- A new site appears in the Site filter, and on the map once its location is accepted; until
then the map's key counts it.
**Hundreds of sites**
- `v_site` has one row per site, and the scatter layer handles thousands of points.
- The Select filters load the distinct labels; M3 and M5 paginate on the server.
- `v_site` has one row per site; the fleet map draws its towers as one symbol layer and
clusters overlapping ones, so thousands of sites stay fast.
- The Site filter loads the distinct labels; with thousands of sites, check its value limit
(open point 6 of the design decisions).
**Superset configuration change** (petrolyst-insights; image rebuild in P3):
`DECKGL_BASE_MAP = [["tile://https://tile.openstreetmap.org/{z}/{x}/{y}.png", "OpenStreetMap"]]`.
The CSP needs no change *(local test: no CSP violation in the browser console)*.
**Superset configuration changes** (in `superset_config.py`; image rebuild in P3):
- `DECKGL_BASE_MAP = [["tile://https://tile.openstreetmap.org/{z}/{x}/{y}.png", "OpenStreetMap"]]`
(no deck.gl chart since 28 Oct; kept for charts made by hand);
- *(28 Oct)* `HTML_SANITIZATION_SCHEMA_EXTENSIONS` lets an `iframe` (`src`, `title`, `loading`,
`style`) through a Markdown block. The fleet-map view must be served on the same origin under
`/views` (petrolyst-infra routing, not part of this step).
**Access, internal first.** Both dashboards for Admin and Alpha only. No Gamma role until the
The CSP needs no change. P2: the tiles load and the map causes no CSP violation. One violation
appears on every Superset page, including the stock dashboard list: a capability probe
(`Function("")`) in the 6.1 frontend bundle, caught by the bundle itself.
**Access, internal first.** The dashboard for Admin and Alpha only; the read service behind the
same sign-in *(28 Oct: not built yet; locally it has none)*. No Gamma role until the
ATOMA-facing edit.
**ATOMA's readers** *(owner, 29 Oct; retired the same day, PD-13: readers use the Petrolyst
app; kept as history)*. The Superset role **"ATOMA Reader"** (Keycloak client role
`atoma-reader`) reads the ATOMA dashboard and nothing else.
- **The role** (`petrolyst-insights/roles/atoma_reader.py`): exactly 12 permissions, set by
`superset-init` whenever the image changes, and reset by hand with the command in the README.
- Dashboard and Chart `can_read`;
- Superset `can_dashboard`, `can_dashboard_permalink`, `can_log`;
- Api `can_time_range`;
- the filter state (`can_read` and `can_write`), permalinks (`can_read`), the current user,
the CSRF token and the menu.
It has no menus, no write rights, no Explore, CSV, sharing, drill or query view. The bundle
import attaches the role to the dashboard (DASHBOARD_RBAC).
- **Rules a role cannot express** (`petrolyst_reader.py`, in both security managers):
- readers own nothing, and reach a dashboard only through its roles (published, role match);
- the dashboard list shows them only those;
- their chart-data responses carry no SQL (`patches/fix_redact_query_for_readers.py`;
Superset itself drops it only for embedded guests).
- **What they see:**
- every page they open leads to the ATOMA dashboard;
- a stylesheet loaded only for them (`assets/branding/reader.css`) hides the dashboard's
title row, the theme switcher, and Info and About in Settings, leaving Logout;
- the same stylesheet hides the filter column on the Fleet tab, and while the dashboard
loads. It sizes the map to the window: the full width at once (Superset measures widths
again only about 0.3 s after loading), and down to 32 px above the window's bottom edge
instead of the layout's 800 px, at least 560 px. `build_bundle.py` checks that it names the
Fleet tab and that the map spans all 12 columns;
- the same stylesheet turns the top-level tabs into a lens switch, centred in the navbar, with
an icon each; the map's block then grows by the tab row's height. A scroll-linked animation
moves the switch up with the navbar, which scrolls away. Without support for scroll-linked
animations, or narrower than 900 px, the tab row stays in the page. From 1100 px the switch
shows a third lens, AI, as a teaser: greyed, tagged PRO in green, drawn by the tab list
(not a tab, not clickable; the layout keeps Map and Insights). Narrower, it would cover the
logo. `build_bundle.py` checks that every layout id the stylesheet names exists;
- the stylesheet's address carries a hash of its content
(`petrolyst_reader.register`): Superset lets browsers keep static files for a year, so a
changed file would otherwise not reach readers who loaded the old one;
- Superset already hides the menus, "+" and editing from a role without them.
- **One refused request remains, harmless:** Superset stores a dashboard's label colours when
they changed (`PUT /api/v1/dashboard/<id>/colors`). A reader may not, so it is refused until
someone with edit rights opens the dashboard; the colours show either way.
- A reader is someone whose only roles are "ATOMA Reader" (and Public). Anyone with Admin, Alpha
or Gamma as well sees everything as before.
## 7. Acceptance checks
*(PD-13)* Written for the retired ATOMA dashboard. Where they name D0 or the fleet map, the app's
site header and Map lens take their place (petrolyst-platform's browser tests).
Run in P2 against a local Superset, and repeated as a smoke test in P3.
1. **New site, no new objects.** Count dashboards, charts and datasets in the metadata database.
Then import a synthetic fourth site with records and an accepted location, and a fifth with no
location. The counts are unchanged; site 4 appears in M1, M3 and the Site filter; site 5
appears in M5, and M4 goes up by 1.
location. The counts are unchanged; site 4 appears on the fleet map and in the Site filter;
site 5 appears in the Site filter, and the map's count of sites without an accepted location
goes up by 1 *(28 Oct)*.
2. **Site switching.** In Site Detail, select pilot site A, then site B. D0 shows the new code and
name; every chart's query (View query, or chart-data API) carries `site_label = …<site B>…`; no
chart keeps site A's rows.
3. **Clearing the selection.** With the Site filter cleared, no chart shows multi-site data; D0 and
S1 show "Select one site". If not, switch to the §4 fallback and re-test.
4. **Unavailable L/h.** For each pilot site:
- the tooltip and M3 say "not available" with the method "no runtime evidence";
- the fleet map's popup says "not available" with the method "no runtime evidence" *(28 Oct)*;
- S1 shows the insufficient-evidence state plus L/day;
- B3 shows every interval excluded, with its reason;
- B11 shows a modelled range labelled "not measured", and that figure appears nowhere else.
5. **Missing coordinates.** A site without an accepted location is not plotted, is listed in M5
and counted in M4, and its link opens its detail.
5. **Missing coordinates.** A site without an accepted location is not plotted, is counted in the
map's key, and can be chosen in the Site filter *(28 Oct)*.
6. **Filter propagation.**
- On the landing page, each filter changes M1, M3, M4 and M5, and the default "active" shows
exactly the active pilot sites.
- A marker click filters M3 only.
- In Site Detail, the Viewing period changes F1–F3 and R1, R2, R4, but not S1 or B1–B4.
7. **Detail links.** For each active pilot site, "Open site detail" in M3 opens Site Detail with
exactly that site selected, and D0 matches. Same from M5.
- *(28 Oct)* Both filters are out of scope on the Fleet tab; the fleet map shows every placed
site, sites that are not active pilot sites faded.
- In Site details, the Viewing period changes F1–F5, R1, R1T, R2, R4 and R4T, but not S1 or
B1–B4.
7. **Detail links** *(28 Oct)*. For each placed site, "Site details" in its popup opens the Site
details tab with exactly that site selected and Summary open; D0 matches, and every chart's
request carries that site.
8. **Colours.**
- Fixtures in each status show the four colours.
- "Eligible" never renders as "Approved" unless the fixture has an approved version with a
decision reference.
- Marker size is identical across sites.
9. **Basemap.** OSM tiles load, there are no CSP violations in the console, and the attribution
is visible.
- *(28 Oct)* An approved site whose inputs changed shows its validity note beside the status
and the L/h in the popup.
9. **Basemap.** The fleet map's tiles load, the attribution is visible, and neither the view nor
the dashboard adds a CSP violation (Superset's own probe, §6, stays) *(28 Oct)*.
## 8. P2 results (local, 24 Oct 2026)
Local Superset 6.1.0 image built from this repository, Trino 484, fuel-db with migrations
0001–0003 and the corpus data loaded; browser checks in headless Chrome.
| Check | Result |
|---|---|
| 1 New site | pass: with a synthetic site with an accepted location and one without, the object counts stay unchanged (2 dashboards, 28 charts before R1T was added, 18 datasets); the first is in M1, M3 and the Site filter, the second in M5, and M4 goes from 0 to 1. Fixtures removed afterwards |
| 2 Site switching | pass: for each active pilot site, every chart request on the four tabs carries that site's `site_label`, all HTTP 200 (25 per site; F5, added later, re-checked for one site) |
| 3 Clearing | pass with the fallback (§4): "required" alone let the charts query every site |
| 4 Unavailable L/h | pass: tooltip and M3 "not available", method "Not available: no runtime evidence"; S1 the same plus L/day; B3 every interval excluded with its reason; B11 the modelled range, labelled "not measured", nowhere else |
| 5 Missing coordinates | pass (synthetic site, check 1): not plotted, in M4 and M5, its link opens its detail |
| 6 Filter propagation | pass: the default "active" reaches M1, M3, M4, M5; a marker click re-queries M3 only, with a latitude/longitude box; the Viewing period reaches F1–F5, R1, R1T, R2, R4 and the bill table, and no Summary or Baseline & Evidence chart |
| 7 Detail links | pass: from M3 for each active pilot site, and from M5 |
| 8 Colours | pass for the mapping: a throwaway fixture with one point per status shows the four colours and equal marker sizes. Not tested by fixture: that the database never labels an unapproved version "Approved baseline" |
| 9 Basemap | pass: OSM tiles load, attribution visible, no CSP violation from the map |
Changes forced by the tests and by two Codex inspection rounds of 24 Oct (F2 visits, R1T, B9
totals, coverage lanes; then F5, grey stock statuses, lane wording), all marked *(P2)* above.
The second round's fixes were not inspected again. The grey colour for a non-ok stock interval
is configured but not shown by the current data, which has no such interval.
- Handlebars charts (D0, S1, R3) became one-row tables.
- The Site filter uses "first value by default" with `requiredFirst`.
- Reading notes moved into a Markdown block per tab.
- The map tooltip uses short tooltip-only columns.
- The markers have no outline.
The marker palette is unchanged from §3. The dataviz palette check flags slate as reading
grey, which is intended for "insufficient evidence", and amber at 2.99:1 contrast. The label is
always also shown as text, in the legend, the tooltip and M3.
## 9. Fleet map results (local, 28 Oct 2026) *(history)*
Local Superset 6.1.0 rebuilt with the iframe setting, behind a local Traefik that serves Superset
and the petrolyst-platform read service on one origin (as on the server); the fuel-db of P2.
Browser checks with Playwright in Chrome (`petrolyst-platform/views/e2e/`).
**Spike, before building**
- A Markdown iframe survives the sanitiser with the setting of §6.
- A `target="_top"` link inside it navigates the whole dashboard page.
- `?native_filters=…#TAB-…` selects the site and opens that tab.
- Basemaps compared at zoom 11, 14 and 16 over the pilot area: OpenFreeMap Positron and OSM
standard tiles show the same OpenStreetMap data; the owner chose Positron.
**Bundle:** 1 dashboard, 26 charts, 19 datasets; two builds are byte-identical; the import reports
column drift 0. The two earlier dashboards and the charts M1, M3, M4 and M5 were deleted from the
local Superset through its API. The live Superset never had them (P3 not done).
| Check | Result |
|---|---|
| 2 Site switching | pass: choosing a second site in the Site filter moves every chart request to it |
| 3 Clearing | pass: after "Clear all" every chart request still carries exactly one site |
| 4 Unavailable L/h | pass for the popup ("not available" and the method); S1, B3 and B11 are unchanged by this step and were not re-run |
| 5 Missing coordinates | pass for the count: the map's key shows the 2 sites without an accepted location |
| 7 Detail links | pass: for each of the 3 placed sites, hover, then "Site details", opens the Site details tab with Summary active, D0 shows the site, and every chart request carries exactly its `site_label` |
| 8 Colours | the validity note beside an approved status and its L/h, and "(not approved)" for Provisional, pass as component tests on fixtures. Live, every site is "Insufficient evidence", so only slate was seen; the other three colours were checked by reading the code against §3, not on screen |
| 9 Basemap | pass: tiles load, attribution visible; no CSP violation from the view. Superset's own `script-src eval` probe appears on every Superset page, as in P2 (§6), and nothing else. The test's watcher was checked with a deliberately disallowed image inside the view |
Also passed: the popup opens on hover, stays while the pointer moves onto it, closes after the
pointer leaves; a click pins it; Esc and a click on the map close it; with reduced motion it only
fades and nothing pulses. The view's minimum size is 480 × 560 px (`catalogue.json`): there, and
also at 480 × 360, every site's popup stays inside the map, and an approved estimate whose inputs
changed is never in view without its warning, checked by scrolling the card. At 480 × 560 every
site's L/h can be read. A single placed site sits clear of the key. Two sites at the same
position can each be opened with the pointer. The states the scratch data lacks (approved with a
changed input, one site, two at one spot) are made in the browser by changing the read service's
answer; the database is unchanged.
**Filters on top and the map search** *(owner, later on 28 Oct)*: the filter row was checked on
screen first: the Fleet tab shows only "More filters", and Site details shows Site and Viewing
period in the row. Browser checks:
- the map takes the page's width;
- the search finds a site by part of its name and by its code, and its popup's "Site details"
opens exactly that site;
- a site inside a cluster is taken out of it;
- site switching and clearing (checks 2 and 3) still pass with the filters in the row.
**ATOMA readers and the operator's logo** *(owner, 29 Oct)*: the plan was reviewed by Codex
(approved after three rounds: deploy inputs, ownership and no-dataset paths, SQL in responses).
Browser checks as the local user `reader`:
- every page leads to the dashboard;
- no title row and no menus; Settings shows only Logout;
- the whole fleet-map suite passes as the reader;
- reading needs nothing the role lacks (only the colours PUT above is refused);
- no chart-data response carries SQL, fresh, cached or as `result_type=query` (an admin's
does);
- CSV and editing answer 403;
- two probe dashboards (Markdown only without roles, and one the reader owns) stay out of
reach: page, API and permalink.
Also checked:
- the role script removes a permission added by hand;
- every new file changes the image tag;
- the patch's `--verify` fails on unpatched Superset and on a tampered line.
Found on the way: under many parallel requests the local proxy returned 502s. gunicorn closes
idle connections after 2 s (`--keep-alive 2`) and Traefik reused them. The local proxy now drops
idle connections after 1 s. **The server has the same pairing** (Traefik in front of gunicorn
with the default keep-alive): open item.
**Final inspection:** two fresh Codex sessions reviewed the build.
- The first found four defects, all fixed and re-checked:
- the popup could leave a small map;
- two sites sharing a `site_code` could open each other's details;
- every non-active pilot status was called "reference";
- a failed database connection gave a 500 instead of the generic 503.
- The second confirmed those fixes and found three more:
- in a short card the validity warning could scroll away from the L/h;
- a single site was centred under the key;
- sites at one position hid each other.
All three are fixed and tested. Those last fixes, and the filter row and the map search, have
not had an independent inspection.
**Filters back on the left, the readers' Fleet tab, the navbar logo, section buttons** *(owner,
29 Oct)*:
checked in the browser as `reader` and `admin` at 1400 × 1000, 1920 × 1080, 1440 × 760,
1280 × 600 and 1024 wide:
- a reader's Fleet tab has no filter column and no resize handle, and the map is as wide as
the page and reaches 32 px above the window's bottom edge (no page scroll), at once rather
than after Superset measures again; at 600 px high it keeps 560 px and the page scrolls;
- Site details shows the column collapsed (32 px, for readers and admins, also after the
map's "Site details" link); expanded, it has the Site filter; switching back hides it again;
- an admin has the column on both tabs;
- the column no longer flashes up while the Fleet tab loads (one frame, about 20 ms, remains:
the grid exists before its tabs do);
- after a switch to Site details its charts are too wide for one or two frames before
Superset measures again; not changed;
- a reader's page links `reader.css?v=<hash>`, an admin's none;
- the navbar shows the wordmark at 24 px for readers, admins and the login page, white in
dark mode;
- Site details' inner tabs are a row of section buttons (`dashboard.css`), lined up with the
content, all four visible and switching, for readers and admins, light and dark; a
vertical list was tried first and dropped: Superset sizes the charts for the full width, so
they were cut off on the right;
- `build_bundle.py` stops when the orientation is not VERTICAL, when reader.css does not name
the Fleet tab, when the map spans fewer than 12 columns, or when `dashboard.css` names an
id that is not in the layout.
- *(lenses)* a reader at 1400 px: the switch centred in the navbar (Map | Insights), the map
735 px high in an 860 px window and 32 px above its bottom edge; scrolled by 30, 61 and
400 px the switch moves up by exactly as much and then stays out of view; at 820 px the tab
row stays in the page; an admin keeps the tab row, named Map and Insights; the map's key
says "Choose them in Insights".
- *(AI teaser)* a reader at 1400 px: the switch reads Map | Insights | AI PRO, still centred,
44 px high; a click on AI changes nothing; at 900 px the switch keeps Map | Insights.
These changes have not had an independent inspection (the owner skipped Codex).
**Not run:** check 1 (a synthetic fourth and fifth site), check 6 (the Viewing period's scope is
unchanged), sign-in through Keycloak, the server's routing of `/views`, sign-in for the read
service, thousands of sites, and dark mode.
## 10. The Petrolyst app's dashboards *(PD-12, PD-13; local, 29 Oct 2026)*
The Petrolyst app (petrolyst-platform `app/`, PD-12) shows Superset dashboards inside its
Insights lens, with guest passes from the Petrolyst server (petrolyst-platform `server/`).
`build_bundle.py` builds them (since PD-13 the only dashboards in the bundle):
| Slug | Content | Filters (set by the app; Superset's filter column is hidden) |
|---|---|---|
| `atoma-overview` | O1–O6: sites by pilot status and by baseline status, litres delivered, fuel cost by currency, electricity billed by currency and basis, litres delivered per month | Viewing period on O3–O6 |
| `atoma-site` | the four §4 detail tabs as Superset's own tabs, the same rows and charts; no D0 (the app shows the site's header) | Site (all charts), Viewing period (as §4: fuel and runtime charts only) |
- **One site dashboard with tabs** *(owner, 29 Oct: "move it back inside the dashboard and make
the tab as they were")*: replaces the four one-tab dashboards `atoma-site-summary`, `-fuel`,
`-runtime`, `-baseline` and the app's per-section pages. Switching tabs needs no reload. The
app opens a site on its first tab; a new viewing period reloads the dashboard on the tab that
was open (the app reads it from Superset and names it in the address, `#TAB-fuel`). Delete
the four old dashboards by hand on an instance that has them.
- **Overview rule** (changes §4 rule 3 for the Overview only): only sums and counts of
recorded values across sites (`OVERVIEW_METRICS`); per-site rates (L/h, L/day) are never
averaged; money is grouped by `currency_text`, a missing currency is its own row, and no
total crosses currencies. `build_bundle.py` checks both.
- They carry no CSS.
- `import_bundle.py --embedded-out FILE --embed-domain URL` makes them embeddable from the app's
origin and writes `{slug: embedding id}` for the Petrolyst server.
- Embedding is switched on in the local stack only (`.local/fmdb-scratch`,
`superset_config_local.py`); production keeps it off until PD-12's open points are settled.
- **Checked (local):** a self-signed guest pass opens its own dashboard and nothing else (another
embedded dashboard and ATOMA: 404; no pass or a wrong audience: 401); the embedded page answers
only to the app's referrer (403 otherwise); chart-data responses to guests carry no SQL.
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,310 @@
# Columns of the public.v_* views (migrations 0001-0003), as PostgreSQL reports them.
# build_bundle.py lists every one in its dataset; import_bundle.py reports drift.
# Regenerate after a migration that changes a view (psql -d fuel -AtF $'\t' -c "..."):
# select c.relname, a.attname, format_type(a.atttypid, a.atttypmod)
# from pg_attribute a join pg_class c on c.oid = a.attrelid join pg_namespace n on n.oid = c.relnamespace
# where n.nspname = 'public' and c.relkind = 'v' and c.relname like 'v\_%' and a.attnum > 0
# and not a.attisdropped order by c.relname, a.attnum
v_assumption site_uid character varying(12)
v_assumption site_code character varying(20)
v_assumption site_name character varying(120)
v_assumption site_label character varying(170)
v_assumption code character varying(60)
v_assumption topic character varying(30)
v_assumption asset_uid character varying(40)
v_assumption value numeric(10,3)
v_assumption value_low numeric(10,3)
v_assumption value_high numeric(10,3)
v_assumption unit character varying(20)
v_assumption applies_from date
v_assumption applies_to date
v_assumption basis character varying(20)
v_assumption derived_from_fuel boolean
v_assumption status character varying(12)
v_assumption accepted_on date
v_assumption decision_ref character varying(12)
v_assumption rationale character varying(400)
v_assumption contradicted_by character varying(200)
v_assumption members character varying(400)
v_baseline_input site_uid character varying(12)
v_baseline_input site_code character varying(20)
v_baseline_input site_name character varying(120)
v_baseline_input site_label character varying(170)
v_baseline_input baseline_code character varying(60)
v_baseline_input baseline_status character varying(12)
v_baseline_input interval_label character varying(30)
v_baseline_input start_on date
v_baseline_input end_on date
v_baseline_input included boolean
v_baseline_input exclusion_reason character varying(300)
v_baseline_input fuel_l numeric(10,1)
v_baseline_input runtime_h numeric(10,1)
v_baseline_input days integer
v_baseline_input eligibility_at_computation character varying(12)
v_baseline_input evidence_grade_at_computation character varying(2)
v_baseline_input dependencies_changed boolean
v_baseline_version site_uid character varying(12)
v_baseline_version site_code character varying(20)
v_baseline_version site_name character varying(120)
v_baseline_version site_label character varying(170)
v_baseline_version code character varying(60)
v_baseline_version metric character varying(40)
v_baseline_version status character varying(12)
v_baseline_version population character varying(200)
v_baseline_version value numeric(8,3)
v_baseline_version range_low numeric(8,3)
v_baseline_version range_high numeric(8,3)
v_baseline_version unit character varying(8)
v_baseline_version window_start date
v_baseline_version window_end date
v_baseline_version fuel_l numeric(10,1)
v_baseline_version runtime_h numeric(10,1)
v_baseline_version matched_days integer
v_baseline_version policy_version character varying(10)
v_baseline_version meets_policy boolean
v_baseline_version policy_check character varying(300)
v_baseline_version evidence_grade character varying(2)
v_baseline_version conditions character varying(200)
v_baseline_version season_note character varying(200)
v_baseline_version approved_on date
v_baseline_version approved_by_role character varying(40)
v_baseline_version decision_ref character varying(12)
v_baseline_version inputs_unchanged boolean
v_baseline_version computed_at timestamp(0) without time zone
v_coverage site_uid character varying(12)
v_coverage site_code character varying(20)
v_coverage site_name character varying(120)
v_coverage site_label character varying(170)
v_coverage lane character varying(60)
v_coverage status character varying(40)
v_coverage start_on date
v_coverage end_on date
v_coverage start_ts timestamp(0) without time zone
v_coverage end_ts timestamp(0) without time zone
v_data_quality site_uid character varying(12)
v_data_quality site_code character varying(20)
v_data_quality site_name character varying(120)
v_data_quality site_label character varying(170)
v_data_quality check_order integer
v_data_quality check_code character varying(40)
v_data_quality label character varying(80)
v_data_quality status character varying(10)
v_data_quality detail character varying(300)
v_delivery site_uid character varying(12)
v_delivery site_code character varying(20)
v_delivery site_name character varying(120)
v_delivery site_label character varying(170)
v_delivery event_key character varying(80)
v_delivery delivery_date date
v_delivery seq_in_day integer
v_delivery delivered_l numeric(8,1)
v_delivery unit_price numeric(8,2)
v_delivery currency character varying(8)
v_delivery source_ref character varying(60)
v_delivery n_sources integer
v_delivery sources_disagree boolean
v_delivery reviewed boolean
v_electricity_cost site_uid character varying(12)
v_electricity_cost site_code character varying(20)
v_electricity_cost site_name character varying(120)
v_electricity_cost site_label character varying(170)
v_electricity_cost bill_month date
v_electricity_cost period_start date
v_electricity_cost period_end date
v_electricity_cost period_known boolean
v_electricity_cost amount numeric(12,2)
v_electricity_cost currency character varying(8)
v_electricity_cost currency_basis character varying(10)
v_electricity_cost kwh numeric(10,1)
v_electricity_cost source_ref character varying(60)
v_electricity_cost claim_status character varying(12)
v_evidence site_uid character varying(12)
v_evidence site_code character varying(20)
v_evidence site_name character varying(120)
v_evidence site_label character varying(170)
v_evidence document_key character varying(16)
v_evidence title character varying(200)
v_evidence kind character varying(20)
v_evidence issuer character varying(80)
v_evidence document_date date
v_evidence used_for character varying(200)
v_evidence contains_personal_data boolean
v_hour_meter_evidence site_uid character varying(12)
v_hour_meter_evidence site_code character varying(20)
v_hour_meter_evidence site_name character varying(120)
v_hour_meter_evidence site_label character varying(170)
v_hour_meter_evidence meter_label character varying(40)
v_hour_meter_evidence mapped_set character varying(60)
v_hour_meter_evidence reading_date date
v_hour_meter_evidence date_basis character varying(30)
v_hour_meter_evidence hours numeric(9,1)
v_hour_meter_evidence reading_state character varying(16)
v_hour_meter_evidence source_field character varying(16)
v_hour_meter_evidence epoch_no integer
v_hour_meter_evidence claim_status character varying(12)
v_imported_calculation site_uid character varying(12)
v_imported_calculation site_code character varying(20)
v_imported_calculation site_name character varying(120)
v_imported_calculation site_label character varying(170)
v_imported_calculation cell_ref character varying(40)
v_imported_calculation source_label character varying(80)
v_imported_calculation formula character varying(200)
v_imported_calculation cached_value numeric(14,6)
v_imported_calculation unit_as_labelled character varying(30)
v_imported_calculation our_reading character varying(300)
v_imported_calculation baseline_use character varying(12)
v_imported_calculation exclusion_reason character varying(300)
v_imported_calculation document_key character varying(16)
v_imported_calculation claim_status character varying(12)
v_interval site_uid character varying(12)
v_interval site_code character varying(20)
v_interval site_name character varying(120)
v_interval site_label character varying(170)
v_interval interval_label character varying(30)
v_interval start_on date
v_interval end_on date
v_interval days integer
v_interval opening_l numeric(9,1)
v_interval opening_basis character varying(8)
v_interval deliveries_l numeric(9,1)
v_interval withdrawals_l numeric(9,1)
v_interval closing_l numeric(9,1)
v_interval closing_basis character varying(8)
v_interval fuel_used_l numeric(9,1)
v_interval fuel_used_unc_l numeric(9,1)
v_interval fuel_l_per_day numeric(8,1)
v_interval stock_status character varying(40)
v_interval runtime_h numeric(9,1)
v_interval runtime_h_per_day numeric(6,2)
v_interval n_meters integer
v_interval metric character varying(40)
v_interval population_key character varying(200)
v_interval l_per_h numeric(8,3)
v_interval l_per_h_low numeric(8,3)
v_interval l_per_h_high numeric(8,3)
v_interval eligibility character varying(12)
v_interval exclusion_reasons character varying(600)
v_interval evidence_grade character varying(2)
v_interval conditions character varying(200)
v_interval in_current_baseline boolean
v_interval baseline_comparable boolean
v_interval current_baseline_l_per_h numeric(8,3)
v_kpi_measures site_uid character varying(12)
v_kpi_measures site_code character varying(20)
v_kpi_measures site_name character varying(120)
v_kpi_measures site_label character varying(170)
v_kpi_measures measure_order integer
v_kpi_measures measure character varying(60)
v_kpi_measures value numeric(12,2)
v_kpi_measures unit character varying(12)
v_kpi_measures value_kind character varying(16)
v_kpi_measures window_start date
v_kpi_measures window_end date
v_kpi_measures note character varying(200)
v_modelled_lph site_uid character varying(12)
v_modelled_lph site_code character varying(20)
v_modelled_lph site_name character varying(120)
v_modelled_lph site_label character varying(170)
v_modelled_lph assumption_code character varying(60)
v_modelled_lph status character varying(12)
v_modelled_lph basis character varying(20)
v_modelled_lph h_per_day numeric(5,2)
v_modelled_lph h_per_day_low numeric(5,2)
v_modelled_lph h_per_day_high numeric(5,2)
v_modelled_lph window_start date
v_modelled_lph window_end date
v_modelled_lph matched_days integer
v_modelled_lph l_per_day numeric(7,1)
v_modelled_lph modelled_l_per_h numeric(6,2)
v_modelled_lph modelled_l_per_h_low numeric(6,2)
v_modelled_lph modelled_l_per_h_high numeric(6,2)
v_modelled_lph value_kind character varying(70)
v_reconciliation site_uid character varying(12)
v_reconciliation site_code character varying(20)
v_reconciliation site_name character varying(120)
v_reconciliation site_label character varying(170)
v_reconciliation interval_label character varying(30)
v_reconciliation start_on date
v_reconciliation end_on date
v_reconciliation stock_status character varying(40)
v_reconciliation endpoint_times_known boolean
v_reconciliation asset_uid character varying(40)
v_reconciliation placed_throughout boolean
v_reconciliation stock_balance_l numeric(9,1)
v_reconciliation engine_l numeric(9,1)
v_reconciliation covered_h numeric(9,1)
v_reconciliation interval_h numeric(9,1)
v_reconciliation engine_all_generators_l numeric(9,1)
v_reconciliation all_generators_metered boolean
v_reconciliation difference_l numeric(9,1)
v_reconciliation difference_pct numeric(7,1)
v_runtime site_uid character varying(12)
v_runtime site_code character varying(20)
v_runtime site_name character varying(120)
v_runtime site_label character varying(170)
v_runtime meter_label character varying(40)
v_runtime mapped_set character varying(60)
v_runtime interval_label character varying(80)
v_runtime start_on date
v_runtime end_on date
v_runtime days integer
v_runtime runtime_h numeric(9,1)
v_runtime h_per_day numeric(6,2)
v_runtime pair_valid boolean
v_runtime same_epoch boolean
v_runtime monotonic boolean
v_runtime over_24h_per_day boolean
v_runtime reviewed boolean
v_site site_uid character varying(12)
v_site site_code character varying(20)
v_site site_name character varying(120)
v_site site_label character varying(170)
v_site zone character varying(40)
v_site province character varying(40)
v_site district character varying(60)
v_site pilot_status character varying(12)
v_site latitude numeric(9,6)
v_site longitude numeric(9,6)
v_site location_status character varying(20)
v_site location_basis character varying(20)
v_site location_precision_m integer
v_site baseline_status character varying(24)
v_site baseline_status_label character varying(40)
v_site baseline_validity_note character varying(60)
v_site lph_estimate numeric(6,2)
v_site lph_method character varying(60)
v_site lph_window_start date
v_site lph_window_end date
v_site baseline_conditions character varying(200)
v_site baseline_decision_ref character varying(12)
v_site fuel_l_per_calendar_day numeric(7,1)
v_site latest_record_on date
v_spec_reference site_uid character varying(12)
v_spec_reference site_code character varying(20)
v_spec_reference site_name character varying(120)
v_spec_reference site_label character varying(170)
v_spec_reference slot character varying(20)
v_spec_reference asset_uid character varying(40)
v_spec_reference asset_kind character varying(10)
v_spec_reference make_model character varying(120)
v_spec_reference attribute character varying(40)
v_spec_reference load_pct numeric(5,1)
v_spec_reference value numeric(12,3)
v_spec_reference value_text character varying(120)
v_spec_reference unit character varying(20)
v_spec_reference basis character varying(30)
v_spec_reference applicability character varying(12)
v_spec_reference applicability_note character varying(300)
v_spec_reference value_kind character varying(30)
v_spec_reference claim_status character varying(12)
v_stock_reading site_uid character varying(12)
v_stock_reading site_code character varying(20)
v_stock_reading site_name character varying(120)
v_stock_reading site_label character varying(170)
v_stock_reading observed_on date
v_stock_reading seq_in_day integer
v_stock_reading reading_type character varying(20)
v_stock_reading litres numeric(8,1)
v_stock_reading value_kind character varying(12)
v_stock_reading spread_l numeric(8,1)
v_stock_reading reviewed boolean
Can't render this file because it contains an unexpected character in line 3 and column 80.
@@ -0,0 +1,136 @@
"""Import the ATOMA dashboards bundle into Superset, overwriting what it defines.
Runs inside the Superset container, with the image's Python:
docker cp import_bundle.py <container>:/tmp/ && docker cp atoma-dashboards.zip <container>:/tmp/
docker exec <container> /app/.venv/bin/python /tmp/import_bundle.py --show-database
docker exec <container> /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip --user <username>
`superset import-dashboards` imports a dashboard's datasets and charts with
overwrite off, so an existing chart or dataset keeps its old definition. This
script runs the dataset, chart and dashboard importers in turn, each with
overwrite on (the bundle's metadata.yaml type is set to what each importer
expects). A dataset import replaces the column list, so columns missing from
the bundle are dropped; afterwards each dataset's columns are compared with the
live view and any drift is reported (exit status 2).
The database is never created or changed: the bundle's database uuid must
already exist under the name "Fuel", otherwise nothing is imported.
The dashboards (atoma-overview, atoma-site) get no role: admins see them in Superset, and
the Petrolyst app shows them with guest passes (PD-12). With --embedded-out FILE
--embed-domain URL, each of them is made embeddable from that origin (its embedding id stays
the same across imports), and FILE receives {slug: embedding id} for the Petrolyst server:
docker exec <container> /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip \
--user <username> --embedded-out /tmp/embedded-dashboards.json --embed-domain http://127.0.0.1:8098
"""
from __future__ import annotations
import argparse
import json
import sys
import zipfile
import yaml
DATABASE_NAME = "Fuel"
EMBEDDED_PREFIX = "atoma-"
STEPS = [("datasets", "SqlaTable"), ("charts", "Slice"), ("dashboards", "Dashboard")]
def main() -> int:
parser = argparse.ArgumentParser(description="Import the ATOMA dashboards bundle.")
parser.add_argument("bundle", nargs="?")
parser.add_argument("--user", help="existing Superset user who will own new objects")
parser.add_argument("--show-database", action="store_true",
help=f'print the uuid of "{DATABASE_NAME}" and stop')
parser.add_argument("--embedded-out", help="write {slug: embedding id} of the app's dashboards")
parser.add_argument("--embed-domain", help="the app's origin, e.g. http://127.0.0.1:8098")
args = parser.parse_args()
if not args.show_database and not (args.bundle and args.user):
parser.error("give a bundle and --user, or --show-database")
if bool(args.embedded_out) != bool(args.embed_domain):
parser.error("--embedded-out and --embed-domain go together")
from superset.app import create_app
app = create_app()
with app.app_context():
from superset import db, security_manager
from superset.commands.chart.importers.v1 import ImportChartsCommand
from superset.commands.dashboard.importers.v1 import ImportDashboardsCommand
from superset.commands.dataset.importers.v1 import ImportDatasetsCommand
from superset.commands.importers.v1.utils import get_contents_from_bundle
from superset.connectors.sqla.models import SqlaTable
from superset.models.core import Database
from superset.models.dashboard import Dashboard
from superset.utils.core import override_user
if args.show_database:
found = db.session.query(Database).filter_by(database_name=DATABASE_NAME).all()
for database in found:
print(database.uuid)
return 0 if found else 1
user = security_manager.find_user(username=args.user)
if user is None:
print(f"no Superset user {args.user!r}", file=sys.stderr)
return 1
with zipfile.ZipFile(args.bundle) as bundle:
contents = get_contents_from_bundle(bundle)
databases = [yaml.safe_load(t) for p, t in contents.items() if p.startswith("databases/")]
for config in databases:
database = db.session.query(Database).filter_by(uuid=config["uuid"]).one_or_none()
if database is None or database.database_name != DATABASE_NAME:
print(f'database {config["uuid"]} is not "{DATABASE_NAME}" here; '
"rebuild the bundle with the uuid from --show-database", file=sys.stderr)
return 1
metadata = yaml.safe_load(contents["metadata.yaml"])
commands = {"datasets": ImportDatasetsCommand, "charts": ImportChartsCommand,
"dashboards": ImportDashboardsCommand}
with override_user(user):
for step, model in STEPS:
step_contents = dict(contents)
step_contents["metadata.yaml"] = yaml.safe_dump({**metadata, "type": model})
commands[step](step_contents, overwrite=True).run()
count = sum(p.startswith(step + "/") for p in contents)
print(f"{step}: {count} imported (overwrite)")
embedded: dict[str, str] = {}
for path, text in contents.items():
if path.startswith("dashboards/"):
dashboard = db.session.query(Dashboard).filter_by(
uuid=yaml.safe_load(text)["uuid"]).one()
if dashboard.slug.startswith(EMBEDDED_PREFIX) and args.embedded_out:
from superset.daos.dashboard import EmbeddedDashboardDAO
config = EmbeddedDashboardDAO.upsert(dashboard, [args.embed_domain])
db.session.flush()
embedded[dashboard.slug] = str(config.uuid)
db.session.commit()
if args.embedded_out:
with open(args.embedded_out, "w") as out:
json.dump(dict(sorted(embedded.items())), out, indent=2)
print(f"embedded: {len(embedded)} dashboard(s) for {args.embed_domain}, "
f"ids in {args.embedded_out}")
drift = 0
dataset_uuids = [yaml.safe_load(t)["uuid"] for p, t in contents.items()
if p.startswith("datasets/")]
for dataset_uuid in dataset_uuids:
dataset = db.session.query(SqlaTable).filter_by(uuid=dataset_uuid).one()
physical = {c.column_name for c in dataset.columns if not c.expression}
live = {c["column_name"] for c in dataset.external_metadata()}
if physical != live:
drift += 1
print(f"drift {dataset.table_name}: only in bundle {sorted(physical - live)}, "
f"only in view {sorted(live - physical)}")
print(f"column drift: {drift} dataset(s)")
return 2 if drift else 0
if __name__ == "__main__":
sys.exit(main())
+80
View File
@@ -0,0 +1,80 @@
"""The role "ATOMA Guest": what the Petrolyst app's embedded dashboards may do (PD-12).
Superset gives every embedded viewer (a guest pass signed for one dashboard) the role named by
GUEST_ROLE_NAME. Which dashboard a guest sees comes from the pass, not from the role. Superset
leaves the SQL out of chart-data responses for guests.
Embedding is switched on in the local stack only (.local/fmdb-scratch, superset_config_local.py),
so this script is not part of init.sh; run it by hand inside the Superset container, once, and
again whenever PERMISSIONS change:
docker exec <superset> /app/.venv/bin/python /app/petrolyst-roles/atoma_guest.py
Creates the role if needed and gives it exactly PERMISSIONS: missing ones are added, any other
is removed (also one added by hand in the UI). A second run changes nothing.
The list was established on the local stack by reading the ATOMA dashboard as a reader (the
former "ATOMA Reader" role, retired with PD-13): everything a dashboard asks the server for, and
nothing more. No menu access, nothing to write, no Explore, CSV, sharing, drill or query view.
"""
from __future__ import annotations
import sys
ROLE = "ATOMA Guest"
PERMISSIONS = [
("can_read", "Dashboard"), # the dashboard and its charts' definitions
("can_dashboard", "Superset"), # the dashboard page
("can_dashboard_permalink", "Superset"), # a dashboard permalink
("can_read", "Chart"), # chart data
("can_time_range", "Api"), # the Viewing period filter
("can_read", "DashboardFilterStateRestApi"), # native filter state
("can_write", "DashboardFilterStateRestApi"),
("can_read", "DashboardPermalinkRestApi"),
("can_read", "CurrentUserRestApi"), # who is signed in
("can_read", "SecurityRestApi"), # the CSRF token
("can_get", "MenuApi"),
("can_log", "Superset"), # Superset's event log
]
def sync(name: str, permissions: list[tuple[str, str]]) -> int:
"""Give the role `name` exactly `permissions`, creating it if needed."""
from superset.app import create_app
app = create_app()
with app.app_context():
from superset import db, security_manager as sm
wanted = []
for permission, view in permissions:
pvm = sm.find_permission_view_menu(permission, view)
if pvm is None:
print(f"no permission {permission} on {view}: run `superset init` first?",
file=sys.stderr)
return 1
wanted.append(pvm)
role = sm.find_role(name) or sm.add_role(name)
added = [pvm for pvm in wanted if pvm not in role.permissions]
removed = [pvm for pvm in role.permissions if pvm not in wanted]
for pvm in added:
sm.add_permission_role(role, pvm)
for pvm in removed:
sm.del_permission_role(role, pvm)
db.session.commit()
print(f'role "{name}": {len(wanted)} permissions, {len(added)} added, '
f"{len(removed)} removed")
for pvm in removed:
print(f" removed {pvm.permission.name} on {pvm.view_menu.name}")
return 0
def main() -> int:
return sync(ROLE, PERMISSIONS)
if __name__ == "__main__":
sys.exit(main())
+37 -8
View File
@@ -45,6 +45,21 @@ SQLALCHEMY_DATABASE_URI = URL.create(
database="superset",
).render_as_string(hide_password=False)
# --- The data: fuel-db ----------------------------------------------------------
# Superset reads fuel-db directly as the read-only role fuel_ro (no Trino since 29 Oct 2026);
# superset-init registers that connection, "Fuel", without a password. Passwords of data
# connections come from secret files at connect time, so none is kept in Superset's own
# database. A connection added in the UI gets no password this way.
_DATABASE_PASSWORD_SECRETS = {"fuel_ro": "fuel_db_ro_password"}
def _database_password(url: URL) -> str | None:
secret = _DATABASE_PASSWORD_SECRETS.get(url.username or "")
return _read_secret(secret) if secret else None
SQLALCHEMY_CUSTOM_PASSWORD_STORE = _database_password
# --- Sign-in through Keycloak (client "petrolyst") ----------------------------
# Users and roles are managed in Keycloak; there is no local admin account.
# Break-glass access: `superset fab ...` inside the container on the server.
@@ -79,6 +94,7 @@ AUTH_ROLES_MAPPING = {
"superset-admin": ["Admin"],
"superset-editor": ["Alpha"],
"superset-viewer": ["Gamma"],
# "atoma-reader" opens the Petrolyst app only; here it maps to nothing (Public).
}
AUTH_ROLES_SYNC_AT_LOGIN = True
AUTH_USER_REGISTRATION = True
@@ -87,7 +103,17 @@ AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = "Public"
# Viewers (Gamma) see a dashboard once it lists their role.
FEATURE_FLAGS = {"DASHBOARD_RBAC": True}
# FILTERBAR_CLOSED_BY_DEFAULT: a dashboard's filter column opens collapsed, a
# narrow strip with an expand button (owner, 29 Oct); ?expand_filters=true
# opens it from a link.
FEATURE_FLAGS = {"DASHBOARD_RBAC": True, "FILTERBAR_CLOSED_BY_DEFAULT": True}
# deck.gl maps: OpenStreetMap raster tiles only. The default list adds Mapbox
# styles, which fail without MAPBOX_API_KEY. The default CSP already allows the
# tile host.
DECKGL_BASE_MAP = [
["tile://https://tile.openstreetmap.org/{z}/{x}/{y}.png", "OpenStreetMap"],
]
# --- Behind Traefik -----------------------------------------------------------
ENABLE_PROXY_FIX = True
@@ -128,12 +154,12 @@ EXPLORE_FORM_DATA_CACHE_CONFIG = {
RATELIMIT_STORAGE_URI = f"{REDIS_URL}/1"
# --- Petrolyst look and feel --------------------------------------------------
# Values from petrolyst-ui/design/brand/07_UI_Tokens/petrolyst_tokens.css.
# Values from petrolyst-web/design/brand/07_UI_Tokens/petrolyst_tokens.css.
# Assets live in assets/branding/ and are served from /static/assets/branding/.
BRANDING = "/static/assets/branding"
APP_NAME = "Petrolyst"
APP_ICON = f"{BRANDING}/petrolyst_logo_standard_fullcolor_no_tagline.svg"
APP_ICON = f"{BRANDING}/petrolyst_wordmark_fullcolor.svg"
FAVICONS = [
{"href": f"{BRANDING}/favicon.svg", "type": "image/svg+xml"},
{"href": f"{BRANDING}/favicon.ico", "type": "image/x-icon"},
@@ -147,10 +173,13 @@ THEME_DEFAULT["token"].update(
"brandLogoAlt": "Petrolyst",
"brandLogoUrl": APP_ICON,
"brandLogoHref": "/",
# The standard logo is stacked (mark over wordmark); 40px keeps the Y
# legible and, with the margin, the 60px navbar height.
"brandLogoHeight": "40px",
"brandLogoMargin": "10px 0",
# The wordmark alone (owner, 29 Oct): in the stacked logo, at 40px, its
# letters were about 5px high. Its own Y carries the brand accent. 24px
# (letters about 20px) and the margin fill the 60px navbar height. On the
# left, 15px more than the navbar's 16px: the P (1.4px inside the image)
# lines up with the dashboard's content at 32px.
"brandLogoHeight": "24px",
"brandLogoMargin": "18px 0 18px 15px",
# Action teal, not the raw brand teal #009688: the brand guide keeps
# raw teal off small text and controls.
"colorPrimary": "#00695C",
@@ -171,7 +200,7 @@ THEME_DARK = deepcopy(THEME_DEFAULT)
THEME_DARK["algorithm"] = "dark"
THEME_DARK["token"].update(
{
"brandLogoUrl": f"{BRANDING}/petrolyst_logo_standard_white_no_tagline.svg",
"brandLogoUrl": f"{BRANDING}/petrolyst_wordmark_white.svg",
"colorPrimary": "#35C5B6",
"colorLink": "#35C5B6",
"colorSuccess": "#35C5B6",
+15 -27
View File
@@ -1,9 +1,9 @@
# Fuel-monitoring analytics: Superset on atoma.petrolyst.com, reading the
# operational store (fuel-db) through Trino. Run only through deploy/deploy.sh
# or deploy/rollback.sh: they export SUPERSET_IMAGE_TAG and TRINO_CONFIG_HASH
# from the release's ARTIFACTS file, and the :? guards below stop Compose from
# running with either unset.
name: fuel-monitoring
# operational store (fuel-db) directly as the read-only role fuel_ro (Trino was
# removed on 29 Oct 2026). Run only through deploy/deploy.sh or
# deploy/rollback.sh: they export SUPERSET_IMAGE_TAG from the release's
# ARTIFACTS file, and the :? guard below stops Compose from running without it.
name: petrolyst
x-superset-env: &superset-env
KEYCLOAK_REALM_URL: ${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/thedigitalbridge}
@@ -15,6 +15,8 @@ x-superset: &superset
- superset_secret_key
- superset_db_password
- keycloak_petrolyst_client_secret
# fuel_ro's password, read at connect time (superset_config.py)
- fuel_db_ro_password
services:
superset:
@@ -100,25 +102,6 @@ services:
timeout: 5s
retries: 10
trino:
image: trinodb/trino:484
env_file:
- ${PETROLYST_ROOT:-/opt/petrolyst}/env/fuel-monitoring.env
volumes:
- ./trino/etc:/etc/trino:ro
labels:
# Compose cannot see content changes in a bind mount; this label changes
# with trino/etc, so exactly a config change recreates the container.
petrolyst.config-hash: ${TRINO_CONFIG_HASH:?TRINO_CONFIG_HASH unset - run through deploy/deploy.sh}
depends_on:
fuel-db:
condition: service_healthy
networks:
- backend
restart: unless-stopped
mem_limit: 3g
# The image's own healthcheck (/usr/lib/trino/bin/health-check) applies.
fuel-db:
image: timescale/timescaledb:2.31.0-pg17
environment:
@@ -128,7 +111,7 @@ services:
TS_TUNE_NUM_CPUS: "2"
secrets:
- fuel_db_password
- fuel_db_trino_password
- fuel_db_ro_password
volumes:
- fuel_db_data:/var/lib/postgresql/data
# A single file, not the folder: the image's own TimescaleDB init scripts
@@ -157,9 +140,14 @@ networks:
# so fuel-db's loopback port works.
fuel-db-tunnel: {}
# Created under the project's first name, fuel-monitoring; named explicitly so the
# data stayed when the project became petrolyst (29 Oct 2026). Compose warns once
# that they belong to another project; that is expected.
volumes:
superset_db_data:
name: fuel-monitoring_superset_db_data
fuel_db_data:
name: fuel-monitoring_fuel_db_data
secrets:
superset_secret_key:
@@ -168,7 +156,7 @@ secrets:
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/superset_db_password
fuel_db_password:
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/fuel_db_password
fuel_db_trino_password:
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/fuel_db_trino_password
fuel_db_ro_password:
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/fuel_db_ro_password
keycloak_petrolyst_client_secret:
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/shared/keycloak_petrolyst_client_secret
@@ -5,21 +5,24 @@
# Loaders log in as their own role but act as fuel_owner (README
# "Data loaders"): the default privileges below cover only what
# fuel_owner itself creates, not what its members create.
# trino_ro what Trino connects as: read-only, also on tables created later.
# fuel_ro what Superset and the Petrolyst server read as: SELECT on public,
# also on tables created later. Read-only by default as well: a
# second lock behind the grants.
set -euo pipefail
trino_password=$(cat /run/secrets/fuel_db_trino_password)
ro_password=$(cat /run/secrets/fuel_db_ro_password)
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" \
-v trino_password="$trino_password" <<'SQL'
-v ro_password="$ro_password" <<'SQL'
CREATE ROLE fuel_owner NOLOGIN;
ALTER DATABASE fuel OWNER TO fuel_owner;
REVOKE CREATE ON SCHEMA public FROM PUBLIC;
CREATE ROLE trino_ro LOGIN PASSWORD :'trino_password';
GRANT CONNECT ON DATABASE fuel TO trino_ro;
GRANT USAGE ON SCHEMA public TO trino_ro;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO trino_ro;
CREATE ROLE fuel_ro LOGIN PASSWORD :'ro_password';
ALTER ROLE fuel_ro SET default_transaction_read_only = on;
GRANT CONNECT ON DATABASE fuel TO fuel_ro;
GRANT USAGE ON SCHEMA public TO fuel_ro;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO fuel_ro;
ALTER DEFAULT PRIVILEGES FOR ROLE fuel_owner IN SCHEMA public
GRANT SELECT ON TABLES TO trino_ro;
GRANT SELECT ON TABLES TO fuel_ro;
SQL
@@ -1,4 +1,4 @@
-- 0002_fm_derivations.sql: internal derivations in schema `fm` (not visible to Trino).
-- 0002_fm_derivations.sql: internal derivations in schema `fm` (not readable by fuel_ro).
-- DRAFT, not applied. Applied after 0001 by a loader acting as fuel_owner (README in this folder).
-- Design: fuel-monitoring .context/plans/2026-10-24-fuel-db-baseline-dashboard.md, §4 and §5.1.
BEGIN;
@@ -1,7 +1,7 @@
-- 0003_bi_views.sql: the views Superset reads through Trino (schema `public`), plus
-- fm.v_site_baseline_status. Owned by fuel_owner, so trino_ro can read them through the existing
-- default privileges; they run with the owner's rights, so Trino never reads `fm` (it lists fm's
-- table names, as it lists every schema's, but any query on them is refused).
-- 0003_bi_views.sql: the views Superset reads (schema `public`), plus
-- fm.v_site_baseline_status. Owned by fuel_owner, so fuel_ro can read them through the existing
-- default privileges; they run with the owner's rights, so fuel_ro never reads `fm` (the catalog
-- shows fm's table names, as it shows every schema's, but any query on them is refused).
-- DRAFT, not applied. Applied after 0002 by a loader acting as fuel_owner (README in this folder).
-- Design: fuel-monitoring .context/plans/2026-10-24-fuel-db-baseline-dashboard.md, §5.2 and §6.
BEGIN;
@@ -97,7 +97,7 @@ LEFT JOIN pp USING (site_uid) LEFT JOIN fm.v_site_baseline_status bs USING (site
LEFT JOIN cal USING (site_uid) LEFT JOIN latest USING (site_uid);
-- Every view below joins public.v_site for site_uid, site_code, site_name and site_label, so one
-- Site filter drives every dataset, and casts to Trino-safe types (varchar(n), integer,
-- Site filter drives every dataset, and casts to plain types (varchar(n), integer,
-- numeric(p,s), date, timestamp(0), boolean). Live = asserted, reviewed or disputed.
-- S2, R3: one row per site and measure; the rows stay when a value is not established.
@@ -10,7 +10,7 @@ The design and its rationale live in the fuel-monitoring corpus:
## Files
| File | Creates | Visible to Trino |
| File | Creates | Readable by `fuel_ro` (Superset, Petrolyst server) |
|---|---|---|
| `0001_fm_core.sql` | schema `fm`, `btree_gist` in it, 25 tables, constraints, trigger-backed rules 1–13 | no |
| `0002_fm_derivations.sql` | internal views in `fm`: claims → visits → stock intervals → L/h and L/day intervals, dependency hashes, baseline validity | no |
@@ -21,7 +21,7 @@ Each file is one transaction. Apply them in order.
## Applying (on the server)
1. Create a loader role as in the repository README, "Data loaders". Its sessions act as
`fuel_owner`, so every object belongs to `fuel_owner`. `trino_ro` can then read the `public`
`fuel_owner`, so every object belongs to `fuel_owner`. `fuel_ro` can then read the `public`
views through the existing default privileges, with no role or grant changes.
2. Open the tunnel and apply the three files as the loader:
@@ -32,13 +32,14 @@ Each file is one transaction. Apply them in order.
done
```
3. Check through Trino (`docker exec -it fuel-monitoring-trino-1 trino`):
- `SHOW TABLES FROM fuel.public` lists the 18 `v_*` views;
- `SELECT count(*) FROM fuel.fm.site` fails with "permission denied for schema fm".
3. Check as `fuel_ro`
(`docker exec -it petrolyst-fuel-db-1 psql -U postgres -d fuel`, then `SET ROLE fuel_ro;`):
- `\dv public.*` lists the 18 `v_*` views;
- `SELECT count(*) FROM fm.site;` fails with "permission denied for schema fm".
The `public` views are owned by `fuel_owner` and run with its rights, so Trino never needs, and
never gets, read access to `fm`. Trino's metadata still lists fm's table and column names, as it
lists every schema in fuel-db (TimescaleDB's internal ones included); names only, no rows.
The `public` views are owned by `fuel_owner` and run with its rights, so `fuel_ro` never needs,
and never gets, read access to `fm`. PostgreSQL's catalog still shows every schema's table and
column names to any role, so Superset's SQL Lab can list fm's names; names only, no rows.
## Data
@@ -91,6 +92,8 @@ it, except baseline versions and their approvals, which live only in fuel-db: ex
## Verified locally (2026-10-24, scratch only)
With Trino, before its removal on 29 Oct 2026; `trino_ro` is now `fuel_ro`.
Setup:
- `timescale/timescaledb:2.31.0-pg17`, with this repository's `01-roles.sh`;
- a loader created by the README recipe;
+3 -3
View File
@@ -6,12 +6,12 @@
FROM apache/superset:6.1.0@sha256:16b50bbef6648912a79e3293d418fefa743ce555d36c5af6b85d859119ed7f88
USER root
# The 6.1.0 image ships neither an OAuth client nor the Postgres and Trino
# drivers.
# The 6.1.0 image ships neither an OAuth client nor the Postgres driver.
RUN uv pip install --no-cache --python /app/.venv/bin/python \
authlib==1.9.1 psycopg2-binary==2.9.13 "trino[sqlalchemy]==0.340.0"
authlib==1.9.1 psycopg2-binary==2.9.13
COPY --from=insights --chown=superset:superset superset_config.py petrolyst_security.py /app/pythonpath/
COPY --from=insights --chown=superset:superset roles/ /app/petrolyst-roles/
COPY --from=insights --chown=superset:superset assets/branding/ /app/superset/static/assets/branding/
COPY --from=insights patches/ /app/patches/petrolyst/
# Stop at the first patch or verify that fails, so a later success cannot hide
+22 -3
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# superset-init: migrate the metadata database, sync roles and permissions, and
# register the Trino connection. It is copied into the Superset image, so a
# register the connection to fuel-db. It is copied into the Superset image, so a
# change here changes the image tag; deploy/deploy.sh runs it on a fresh install
# and whenever that tag changes. Every step is idempotent. Unlike the
# upstream docker-init.sh it never creates a local admin: users come from
@@ -9,6 +9,25 @@ set -euo pipefail
superset db upgrade
superset init
# Trino was removed on 29 Oct 2026. An instance still holding its connection
# "Fuel (Trino)" gets it renamed, so its datasets and dashboards keep their ids;
# the next step points it at fuel-db. Drop this once no instance has it.
/app/.venv/bin/python - <<'PY'
from superset.app import create_app
with create_app().app_context():
from superset import db
from superset.models.core import Database
old = db.session.query(Database).filter_by(database_name="Fuel (Trino)").one_or_none()
if old and not db.session.query(Database).filter_by(database_name="Fuel").count():
old.database_name = "Fuel"
db.session.commit()
print('renamed the connection "Fuel (Trino)" to "Fuel"')
PY
# No password here: superset_config.py reads it from /run/secrets at connect time.
superset set-database-uri \
--database_name "Fuel (Trino)" \
--uri "trino://superset@trino:8080/fuel"
--database_name "Fuel" \
--uri "postgresql+psycopg2://fuel_ro@fuel-db:5432/fuel"
@@ -1,3 +0,0 @@
# Analytics never writes through Trino. fuel-db's trino_ro role is read-only as
# well; this is the second lock.
access-control.name=read-only
@@ -1,5 +0,0 @@
# The fuel-monitoring operational store (service fuel-db), read-only.
connector.name=postgresql
connection-url=jdbc:postgresql://fuel-db:5432/fuel
connection-user=trino_ro
connection-password=${ENV:FUEL_DB_TRINO_PASSWORD}
@@ -1,14 +0,0 @@
# Single node: coordinator and worker in one process. Reachable only on the
# backend network (Superset); no port is published and Traefik has no route.
coordinator=true
node-scheduler.include-coordinator=true
http-server.http.port=8080
discovery.uri=http://localhost:8080
catalog.management=static
# Memory bounds. The container is capped at 3g (compose.yaml) and jvm.config
# gives the heap 75 % of it (~2.25 GB); 1 GB for queries plus 512 MB headroom
# leaves room for the rest of the heap.
query.max-memory=1GB
query.max-memory-per-node=1GB
memory.heap-headroom-per-node=512MB
@@ -1,14 +0,0 @@
-server
-agentpath:/usr/lib/trino/bin/libjvmkill.so
-XX:InitialRAMPercentage=75
-XX:MaxRAMPercentage=75
-XX:G1HeapRegionSize=32M
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:+ExitOnOutOfMemoryError
-XX:-OmitStackTraceInFastThrow
-XX:ReservedCodeCacheSize=256M
-XX:PerMethodRecompilationCutoff=10000
-XX:PerBytecodeRecompilationCutoff=10000
-Djdk.attach.allowAttachSelf=true
-Djdk.nio.maxCachedBufferSize=2000000
@@ -1 +0,0 @@
io.trino=INFO
@@ -1,3 +0,0 @@
node.environment=petrolyst
node.data-dir=/data/trino
node.id=${ENV:HOSTNAME}