Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8405706ed8 | ||
|
|
686318dfac | ||
|
|
c92c960025 | ||
|
|
da3f803947 | ||
|
|
1b8e57adb4 |
No files matched your search
@@ -0,0 +1,34 @@
|
||||
# petrolyst-infra: agent rules
|
||||
|
||||
Deployment of the petrolyst server (atoma.petrolyst.com) and the Superset customization.
|
||||
Runbook: README.md. Workspace rules: `../AGENTS.md`; shared context: `../.context/` (server and
|
||||
identity facts: `deployment.md`).
|
||||
|
||||
**Architecture:** read `../.context/architecture.md` before planning or building. This repository
|
||||
runs persistence (fuel-db and its migrations) and Superset; name the bands a change touches and
|
||||
keep the rules there; a change against a rule or an open conflict goes to the owner first.
|
||||
|
||||
## Layout
|
||||
- `stacks/`, `traefik/`, `deploy/`, `host/`: how and where things run.
|
||||
- `petrolyst-insights/`: what Superset looks like and does (config, security manager, the guest
|
||||
role, branding, patches, dashboards as code). Its README maps each file to the image.
|
||||
- `keycloak/`: the `petrolyst` client import.
|
||||
|
||||
## Rules
|
||||
- **Deploys:** only by README "Manual deploy", only when the owner asks. There is no CI. Commit on
|
||||
`main` first; never push or deploy unprompted.
|
||||
- **No secrets** in the repository; `deploy/deploy.sh` refuses if one appears.
|
||||
- **The Superset image's inputs** (`SUPERSET_INPUTS` in `deploy/lib.sh`) make its tag: a change
|
||||
there rebuilds Superset on the next deploy.
|
||||
- **Dashboards are code:** change `build_bundle.py` and SPEC.md, never the UI followed by an
|
||||
export. `import_bundle.py` never deletes; old objects go by hand (`dashboards/README.md`).
|
||||
- **Embedding** stays off in production (`superset_config.py`) until PD-12's points are decided;
|
||||
the local stack turns it on in its own override.
|
||||
- **ATOMA's users** have no Superset role (PD-13): `atoma-reader` opens the Petrolyst app only.
|
||||
- fuel-db migrations (`stacks/fuel-monitoring/fuel-db/migrations/`) are applied by hand (P3).
|
||||
|
||||
## Checks
|
||||
- Shell: the README's Docker shellcheck command (warnings and errors must be zero).
|
||||
- Dashboards: `build_bundle.py` checks the bundle as it builds; import locally and expect column
|
||||
drift 0 (`../.context/local-stack.md`).
|
||||
- Superset changes: rebuild the local image and check it turns healthy before calling it done.
|
||||
@@ -5,7 +5,8 @@ Everything that runs on the petrolyst server (109.199.127.84, Ubuntu 24.04):
|
||||
- **Traefik** at the edge for every project on the server. Its dashboard is at
|
||||
https://traefik.petrolyst.com, behind Keycloak.
|
||||
- **fuel-monitoring** (ATOMA): Superset at https://atoma.petrolyst.com, reading the operational
|
||||
store `fuel-db` (Postgres 17 with TimescaleDB) through Trino. Sign-in goes through Keycloak
|
||||
store `fuel-db` (Postgres 17 with TimescaleDB) directly, as the read-only role `fuel_ro`
|
||||
(Trino was removed on 29 Oct 2026). Sign-in goes through Keycloak
|
||||
client `petrolyst`, and the UI carries the Petrolyst brand.
|
||||
|
||||
**Deploys are manual.** A person deploys a clean commit of `main` from the Mac ("Manual deploy"
|
||||
@@ -17,7 +18,7 @@ Customization and deployment are separate, as with media-insights and media-inte
|
||||
|---|---|
|
||||
| `petrolyst-insights/` | how Superset looks and behaves for Petrolyst: config, Keycloak login code, branding, patches, dashboards. See its README. |
|
||||
| `traefik/` | edge proxy, TLS, the `sso` middleware (oauth2-proxy) that guards the dashboard |
|
||||
| `stacks/fuel-monitoring/` | Compose stack, Superset Dockerfile, Trino config and catalog, fuel-db roles |
|
||||
| `stacks/fuel-monitoring/` | Compose stack, Superset Dockerfile and init, fuel-db roles and migrations |
|
||||
| `deploy/` | `deploy.sh` (run on the server by a manual deploy), `rollback.sh` (manual), `lib.sh` (shared) |
|
||||
| `host/` | one-time server setup: `bootstrap.sh`, `secrets-init.sh` |
|
||||
| `keycloak/` | the `petrolyst` client for Keycloak's partial import |
|
||||
@@ -38,10 +39,10 @@ No credential is in this repository. Secrets exist only on the server, under
|
||||
4. It resolves the artifacts and writes them to `releases/<sha>/ARTIFACTS`:
|
||||
- `SUPERSET_IMAGE_TAG` is a hash of exactly the Superset image's build inputs. They include
|
||||
`superset/init.sh`, so changing that file runs `superset-init` again.
|
||||
- `TRINO_CONFIG_HASH` and `TRAEFIK_CONFIG_HASH` are hashes of the bind-mounted config.
|
||||
- `TRAEFIK_CONFIG_HASH` is a hash of Traefik's bind-mounted static config.
|
||||
- Compose recreates a container only when its value changes.
|
||||
5. Preflight, before anything live changes:
|
||||
- every secret and env file exists;
|
||||
- every secret file exists;
|
||||
- `petrolyst-insights/` holds no deployment files;
|
||||
- `docker compose config` passes for every stack;
|
||||
- the Superset image is built if it is new;
|
||||
@@ -52,13 +53,14 @@ No credential is in this repository. Secrets exist only on the server, under
|
||||
- An empty database is a fresh install: it runs `superset-init`.
|
||||
- If the image changed, or the database is not at the image's migration head: it stops
|
||||
`superset`, writes `backups/superset-meta_<old tag>_<revision>_<time>.dump`, then runs
|
||||
`superset-init` (`db upgrade`, `init`, registers "Fuel (Trino)").
|
||||
`superset-init` (`db upgrade`, `init`, registers "Fuel": fuel-db as `fuel_ro`, its password
|
||||
read from `/run/secrets` at connect time, never stored in Superset).
|
||||
- If the image is unchanged and the schema is at its head, it does none of this.
|
||||
8. Smoke tests. A failure ends the deploy with an error and prints the stacks' state.
|
||||
- `/health` answers through Traefik with a valid certificate.
|
||||
- `/login/` leads to Keycloak with `client_id=petrolyst` and PKCE.
|
||||
- The dashboard redirects to Keycloak.
|
||||
- Trino reads `fuel`, and Superset reaches Trino.
|
||||
- Superset's "Fuel" connection reads fuel-db as `fuel_ro`, and its transactions are read-only.
|
||||
9. Retention:
|
||||
- 5 releases;
|
||||
- backups for 30 days;
|
||||
@@ -75,7 +77,6 @@ The manual deploy then checks both hosts from outside.
|
||||
| `/opt/petrolyst/incoming/<sha>-manual-<time>` | deploy | one upload; only `deploy.sh` turns it into a release |
|
||||
| `/opt/petrolyst/releases/<sha>` | deploy, read-only | finalized release plus `ARTIFACTS`; the last 5 are kept |
|
||||
| `/opt/petrolyst/secrets/{shared,traefik,fuel-monitoring}/` | root:deploy 0750, files 0444 | secrets, mounted as `/run/secrets/<name>` |
|
||||
| `/opt/petrolyst/env/fuel-monitoring.env` | root:deploy 0440 | `FUEL_DB_TRINO_PASSWORD` for Trino's `${ENV:…}` |
|
||||
| `/opt/petrolyst/backups/` | deploy 0750 | Superset metadata dumps |
|
||||
| `/opt/petrolyst/state/` | deploy | last deployed SHA and Superset image tag; `rollback-in-progress` during a migration rollback |
|
||||
|
||||
@@ -93,7 +94,8 @@ deploy secrets.
|
||||
and "Skip" if a resource exists. This creates:
|
||||
- the confidential client `petrolyst`: standard flow only, PKCE S256, redirect URIs for both
|
||||
hosts;
|
||||
- its client roles `superset-admin`, `superset-editor`, `superset-viewer` and `traefik-admin`;
|
||||
- its client roles `superset-admin`, `superset-editor`, `superset-viewer`, `atoma-reader`
|
||||
and `traefik-admin`;
|
||||
- a mapper that puts those roles into the claim `petrolyst_roles`.
|
||||
2. Clients → petrolyst → Credentials: copy the client secret (step 3 stores it on the server).
|
||||
3. **MFA**, which Exhibit B requires. A copy of the built-in `browser` flow is not enough, for two
|
||||
@@ -109,7 +111,10 @@ deploy secrets.
|
||||
enrols first. Other clients of the realm are unaffected. There is no silent SSO between
|
||||
Superset and the dashboard.
|
||||
4. Give people roles: Users → (user) → Role mapping → Assign role → filter by clients →
|
||||
`petrolyst`. Group mappings work too.
|
||||
`petrolyst`. Group mappings work too. ATOMA's users get `atoma-reader` and nothing else:
|
||||
it opens the Petrolyst app (petrolyst-platform) and maps to no Superset role, so in
|
||||
Superset they see nothing. A realm that already has the client gets the role by hand
|
||||
(Clients → petrolyst → Roles → Create role) or by importing again with "Skip".
|
||||
5. Users of the Traefik dashboard need **Email verified** switched on, because oauth2-proxy
|
||||
refuses unverified addresses.
|
||||
|
||||
@@ -156,10 +161,10 @@ becomes the first Superset admin; there is no local admin account.
|
||||
| Re-deploy a kept release | `sudo -u deploy /opt/petrolyst/infra/deploy/deploy.sh <sha>` |
|
||||
| Roll back | see below |
|
||||
| Traefik dashboard | https://traefik.petrolyst.com (role `traefik-admin`) |
|
||||
| Trino shell | `docker exec -it fuel-monitoring-trino-1 trino` |
|
||||
| fuel-db shell | `docker exec -it petrolyst-fuel-db-1 psql -U postgres -d fuel` |
|
||||
| fuel-db from the Mac | `ssh -L 5433:127.0.0.1:5433 petrolyst`, then connect to `localhost:5433`, database `fuel` |
|
||||
| Logs | `docker compose -p fuel-monitoring logs -f superset` (or `-p traefik`) |
|
||||
| Superset break-glass | `docker exec -it fuel-monitoring-superset-1 superset fab ...` |
|
||||
| Logs | `docker compose -p petrolyst logs -f superset` (or `-p traefik`) |
|
||||
| Superset break-glass | `docker exec -it petrolyst-superset-1 superset fab ...` |
|
||||
|
||||
**Manual deploy**, the only way to deploy (there is no CI runner). On the Mac, in this
|
||||
repository:
|
||||
@@ -223,12 +228,12 @@ sudo -u deploy /opt/petrolyst/infra/deploy/rollback.sh <sha>
|
||||
Afterwards, `git revert` the bad commit and push, so `main` matches. A plain revert without the
|
||||
rollback script is stopped by the downgrade guard.
|
||||
|
||||
**Data loaders** (fuel-db). Every table in `public` must belong to `fuel_owner`. `trino_ro` can
|
||||
**Data loaders** (fuel-db). Every table in `public` must belong to `fuel_owner`. `fuel_ro` can
|
||||
read new tables only through `fuel_owner`'s default privileges, and PostgreSQL applies those
|
||||
only to objects `fuel_owner` creates itself. A table that a loader creates as its own role stays
|
||||
unreadable for Trino. Create each loader role so that its sessions act as `fuel_owner`, and it
|
||||
unreadable for Superset and the Petrolyst server. Create each loader role so that its sessions act as `fuel_owner`, and it
|
||||
cannot create anything as itself. Run this in
|
||||
`docker exec -it fuel-monitoring-fuel-db-1 psql -U postgres -d fuel`:
|
||||
`docker exec -it petrolyst-fuel-db-1 psql -U postgres -d fuel`:
|
||||
|
||||
```sql
|
||||
CREATE ROLE <loader> LOGIN PASSWORD '<password>';
|
||||
@@ -236,9 +241,46 @@ GRANT fuel_owner TO <loader> WITH INHERIT FALSE, SET TRUE;
|
||||
ALTER ROLE <loader> SET role TO 'fuel_owner';
|
||||
```
|
||||
|
||||
Tables it creates, hypertables included, then belong to `fuel_owner`, and Trino can read them.
|
||||
Tables it creates, hypertables included, then belong to `fuel_owner`, and `fuel_ro` can read them.
|
||||
After `SET ROLE NONE`, it cannot create anything in `public`. To repair a table that another
|
||||
role created, run `ALTER TABLE <t> OWNER TO fuel_owner`, then `GRANT SELECT ON <t> TO trino_ro`.
|
||||
role created, run `ALTER TABLE <t> OWNER TO fuel_owner`, then `GRANT SELECT ON <t> TO fuel_ro`.
|
||||
|
||||
**One-time steps after 29 Oct 2026**, at the first deploy that includes them. Two changes
|
||||
meet there:
|
||||
- Trino is removed: Superset reads fuel-db directly. The role `trino_ro` becomes `fuel_ro` and
|
||||
keeps its password (a SCRAM password survives a rename); its secret file moves with it.
|
||||
- The Compose project `fuel-monitoring` is renamed `petrolyst`. Containers are then called
|
||||
`petrolyst-<service>-1`. The data volumes keep their names (`fuel-monitoring_superset_db_data`,
|
||||
`fuel-monitoring_fuel_db_data`; pinned in `compose.yaml`), so no data moves. The old project's
|
||||
containers must be gone first: two PostgreSQL servers on one data volume corrupt it.
|
||||
Preflight refuses to deploy while any of them exists.
|
||||
|
||||
As root on the server, right before the deploy (Superset is down from step 3 until the deploy
|
||||
finishes):
|
||||
|
||||
```bash
|
||||
S=/opt/petrolyst/secrets/fuel-monitoring
|
||||
mv "$S/fuel_db_trino_password" "$S/fuel_db_ro_password" # 1
|
||||
docker exec -i fuel-monitoring-fuel-db-1 psql -U postgres -d fuel -v ON_ERROR_STOP=1 \
|
||||
-c 'ALTER ROLE trino_ro RENAME TO fuel_ro' \
|
||||
-c 'ALTER ROLE fuel_ro SET default_transaction_read_only = on' # 2
|
||||
docker rm -f $(docker ps -aq --filter label=com.docker.compose.project=fuel-monitoring) # 3
|
||||
```
|
||||
|
||||
4. Deploy as usual ("Manual deploy"). Compose starts the `petrolyst` project on the same volumes
|
||||
and no longer starts Trino. Compose warns once that the volumes belong to another project;
|
||||
that is expected. `superset-init` renames the connection "Fuel (Trino)" to "Fuel" and points
|
||||
it at fuel-db; the smoke test checks that it reads as `fuel_ro`, read-only.
|
||||
5. Clean up: `rm /opt/petrolyst/env/fuel-monitoring.env`, `rmdir /opt/petrolyst/env`,
|
||||
`docker image rm trinodb/trino:484`,
|
||||
`docker network rm fuel-monitoring_backend fuel-monitoring_fuel-db-tunnel`.
|
||||
|
||||
**Rollback across the Trino removal** (not tested). `rollback.sh` stops for a release from before
|
||||
it. To go back anyway: remove the `petrolyst` project's containers, reverse steps 1 and 2 above (rename `fuel_ro` back to `trino_ro` and
|
||||
`ALTER ROLE trino_ro RESET default_transaction_read_only`, move the secret file back, write
|
||||
`/opt/petrolyst/env/fuel-monitoring.env` with `FUEL_DB_TRINO_PASSWORD=<that secret>`), rename
|
||||
the Superset connection "Fuel" back to "Fuel (Trino)" with `trino://superset@trino:8080/fuel`,
|
||||
then run that release's own script, `/opt/petrolyst/releases/<sha>/deploy/rollback.sh <sha>`.
|
||||
|
||||
**Adding a project**: create `stacks/<name>/compose.yaml`, attach the public service to the
|
||||
external network `proxy`, and set `traefik.*` labels with a unique router name and
|
||||
@@ -257,6 +299,8 @@ its own gate in its stack:
|
||||
|
||||
## Verified locally (2026-10-23)
|
||||
|
||||
Before Trino's removal (29 Oct 2026): its lines below are history.
|
||||
|
||||
An end-to-end run ran the real `deploy.sh` and `rollback.sh` in a Linux container against
|
||||
Docker Desktop, with a scratch `/opt/petrolyst`. Production differences in that run:
|
||||
- a test CA certificate stood in for Let's Encrypt;
|
||||
|
||||
+1
-1
@@ -44,7 +44,7 @@ resolve_artifacts() {
|
||||
fi
|
||||
chmod -R a-w "$release"
|
||||
load_artifacts "$release"
|
||||
log "artifacts: superset $SUPERSET_IMAGE_TAG, trino config $TRINO_CONFIG_HASH, traefik config $TRAEFIK_CONFIG_HASH"
|
||||
log "artifacts: superset $SUPERSET_IMAGE_TAG, traefik config $TRAEFIK_CONFIG_HASH"
|
||||
}
|
||||
|
||||
# prepare_fuel_monitoring <release>: before anything live changes, build the
|
||||
|
||||
+32
-11
@@ -21,7 +21,7 @@ KEYCLOAK_REALM_URL=${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/
|
||||
PETROLYST_ROOT=$ROOT
|
||||
export ATOMA_HOST TRAEFIK_HOST KEYCLOAK_REALM_URL PETROLYST_ROOT
|
||||
|
||||
# --host pins the daemon, as in petrolyst-ui/deploy/deploy.sh: a stale
|
||||
# --host pins the daemon, as in petrolyst-web/deploy/deploy.sh: a stale
|
||||
# `docker context use` must not send this deploy to another daemon.
|
||||
DOCKER=(docker --host unix:///var/run/docker.sock)
|
||||
|
||||
@@ -33,6 +33,7 @@ SUPERSET_INPUTS=(
|
||||
stacks/fuel-monitoring/superset/init.sh
|
||||
petrolyst-insights/superset_config.py
|
||||
petrolyst-insights/petrolyst_security.py
|
||||
petrolyst-insights/roles
|
||||
petrolyst-insights/assets/branding
|
||||
petrolyst-insights/patches
|
||||
)
|
||||
@@ -44,8 +45,7 @@ REQUIRED_FILES=(
|
||||
secrets/fuel-monitoring/superset_secret_key
|
||||
secrets/fuel-monitoring/superset_db_password
|
||||
secrets/fuel-monitoring/fuel_db_password
|
||||
secrets/fuel-monitoring/fuel_db_trino_password
|
||||
env/fuel-monitoring.env
|
||||
secrets/fuel-monitoring/fuel_db_ro_password
|
||||
)
|
||||
|
||||
log() { printf '%s %s\n' "$(date -u +%H:%M:%S)" "$*" >&2; }
|
||||
@@ -82,7 +82,6 @@ hash_inputs() {
|
||||
|
||||
compute_artifacts() {
|
||||
printf 'SUPERSET_IMAGE_TAG=%s\n' "$(hash_inputs "$1" "${SUPERSET_INPUTS[@]}")"
|
||||
printf 'TRINO_CONFIG_HASH=%s\n' "$(hash_inputs "$1" stacks/fuel-monitoring/trino/etc)"
|
||||
# Only the static config: dynamic/ is watched and applies without a restart.
|
||||
printf 'TRAEFIK_CONFIG_HASH=%s\n' "$(hash_inputs "$1" traefik/static)"
|
||||
}
|
||||
@@ -91,16 +90,21 @@ compute_artifacts() {
|
||||
load_artifacts() {
|
||||
local file=$1/ARTIFACTS line key value
|
||||
[ -s "$file" ] || die "$file is missing"
|
||||
unset SUPERSET_IMAGE_TAG TRINO_CONFIG_HASH TRAEFIK_CONFIG_HASH
|
||||
unset SUPERSET_IMAGE_TAG TRAEFIK_CONFIG_HASH
|
||||
while IFS= read -r line; do
|
||||
key=${line%%=*}
|
||||
value=${line#*=}
|
||||
[[ $key =~ ^(SUPERSET_IMAGE_TAG|TRINO_CONFIG_HASH|TRAEFIK_CONFIG_HASH)$ && $value =~ ^[0-9a-f]{16}$ ]] ||
|
||||
# A release from before 29 Oct 2026 still runs Trino, which logs in as
|
||||
# `trino_ro`, since renamed `fuel_ro`: it cannot start (README "Rollback
|
||||
# across the Trino removal").
|
||||
[ "$key" != TRINO_CONFIG_HASH ] ||
|
||||
die "$file is from before Trino was removed; see README \"Rollback across the Trino removal\""
|
||||
[[ $key =~ ^(SUPERSET_IMAGE_TAG|TRAEFIK_CONFIG_HASH)$ && $value =~ ^[0-9a-f]{16}$ ]] ||
|
||||
die "unexpected line in $file: $line"
|
||||
printf -v "$key" '%s' "$value"
|
||||
export "${key?}"
|
||||
done <"$file"
|
||||
[ -n "${SUPERSET_IMAGE_TAG:-}" ] && [ -n "${TRINO_CONFIG_HASH:-}" ] && [ -n "${TRAEFIK_CONFIG_HASH:-}" ] ||
|
||||
[ -n "${SUPERSET_IMAGE_TAG:-}" ] && [ -n "${TRAEFIK_CONFIG_HASH:-}" ] ||
|
||||
die "$file is incomplete"
|
||||
}
|
||||
|
||||
@@ -136,6 +140,10 @@ preflight() {
|
||||
[ -z "$bad" ] || die "petrolyst-insights/ holds customization only; found: $bad"
|
||||
compose "$release" traefik config -q
|
||||
while IFS= read -r dir; do compose "$release" "$dir" config -q; done < <(stack_dirs "$release")
|
||||
# The stack was the Compose project "fuel-monitoring" until 29 Oct 2026; "petrolyst" uses the
|
||||
# same data volumes, so both must never run at once (README "One-time steps after 29 Oct 2026").
|
||||
[ -z "$("${DOCKER[@]}" ps -aq --filter label=com.docker.compose.project=fuel-monitoring)" ] ||
|
||||
die "containers of the old project fuel-monitoring exist; remove them first (README \"One-time steps after 29 Oct 2026\")"
|
||||
}
|
||||
|
||||
# promote <release> <sha>: make the release the live tree. --chmod applies to
|
||||
@@ -254,10 +262,23 @@ smoke_tests() {
|
||||
[ "$code" = 302 ] && [[ $location == "$KEYCLOAK_REALM_URL/"* || $location == *"/oauth2/start"* ]] ||
|
||||
die "https://$TRAEFIK_HOST/ should redirect to Keycloak, got $code $location"
|
||||
|
||||
fm exec -T trino trino --execute 'SELECT count(*) FROM fuel.information_schema.schemata' >/dev/null ||
|
||||
die "Trino cannot read the fuel catalog"
|
||||
fm exec -T superset curl -fsS --max-time 10 http://trino:8080/v1/info >/dev/null ||
|
||||
die "Superset cannot reach Trino"
|
||||
# Superset's own connection "Fuel" (fuel_ro, password from /run/secrets) reads fuel-db and
|
||||
# cannot write to it.
|
||||
fm exec -T superset /app/.venv/bin/python - <<'PY' || die "Superset cannot read fuel-db as fuel_ro, or fuel_ro can write"
|
||||
import sys
|
||||
|
||||
from superset.app import create_app
|
||||
|
||||
with create_app().app_context():
|
||||
from superset import db
|
||||
from superset.models.core import Database
|
||||
|
||||
fuel = db.session.query(Database).filter_by(database_name="Fuel").one()
|
||||
with fuel.get_sqla_engine() as engine, engine.connect() as conn:
|
||||
user, read_only = conn.exec_driver_sql(
|
||||
"SELECT current_user, current_setting('transaction_read_only')").one()
|
||||
sys.exit(0 if (user, read_only) == ("fuel_ro", "on") else 1)
|
||||
PY
|
||||
log "smoke tests passed"
|
||||
}
|
||||
|
||||
|
||||
+1
-1
@@ -40,7 +40,7 @@ if [ "$(cat /etc/docker/daemon.json 2>/dev/null)" != "$daemon_json" ]; then
|
||||
fi
|
||||
systemctl enable --now docker
|
||||
|
||||
echo "== swap (4 GB) for Trino, Superset and two Postgres in 7.8 GB"
|
||||
echo "== swap (4 GB) for Superset and two Postgres in 7.8 GB"
|
||||
if ! swapon --show=NAME --noheadings | grep -qx /swapfile; then
|
||||
[ -e /swapfile ] || fallocate -l 4G /swapfile
|
||||
chmod 600 /swapfile
|
||||
|
||||
+3
-15
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Server secrets and env files, run as root after bootstrap.sh.
|
||||
# Server secrets, run as root after bootstrap.sh.
|
||||
# secrets-init.sh generate every missing secret; never overwrites one
|
||||
# secrets-init.sh keycloak-secret store the client secret of Keycloak client
|
||||
# "petrolyst" (Clients > petrolyst > Credentials),
|
||||
@@ -17,7 +17,7 @@ getent group deploy >/dev/null || die "group deploy is missing: run host/bootstr
|
||||
ROOT=/opt/petrolyst
|
||||
SECRETS=$ROOT/secrets
|
||||
umask 077
|
||||
install -d -o root -g deploy -m 0750 "$SECRETS" "$SECRETS/shared" "$SECRETS/traefik" "$SECRETS/fuel-monitoring" "$ROOT/env"
|
||||
install -d -o root -g deploy -m 0750 "$SECRETS" "$SECRETS/shared" "$SECRETS/traefik" "$SECRETS/fuel-monitoring"
|
||||
|
||||
# write_secret <path> <value>: no trailing newline (oauth2-proxy reads the file
|
||||
# byte for byte), replaced atomically.
|
||||
@@ -62,22 +62,10 @@ esac
|
||||
generate "$SECRETS/fuel-monitoring/superset_secret_key" 32
|
||||
generate "$SECRETS/fuel-monitoring/superset_db_password" 32
|
||||
generate "$SECRETS/fuel-monitoring/fuel_db_password" 32
|
||||
generate "$SECRETS/fuel-monitoring/fuel_db_trino_password" 32
|
||||
generate "$SECRETS/fuel-monitoring/fuel_db_ro_password" 32
|
||||
# 32 hex characters = a 32-byte cookie key, as oauth2-proxy requires.
|
||||
generate "$SECRETS/traefik/oauth2_proxy_cookie_secret" 16
|
||||
|
||||
# Trino reads the catalog password from its environment (${ENV:...}); keep the
|
||||
# env file in step with the secret file that fuel-db's init script uses.
|
||||
env_file=$ROOT/env/fuel-monitoring.env
|
||||
line="FUEL_DB_TRINO_PASSWORD=$(cat "$SECRETS/fuel-monitoring/fuel_db_trino_password")"
|
||||
if ! grep -qxF "$line" "$env_file" 2>/dev/null; then
|
||||
printf '%s\n' "$line" >"$env_file.tmp"
|
||||
chown root:deploy "$env_file.tmp"
|
||||
chmod 0440 "$env_file.tmp"
|
||||
mv "$env_file.tmp" "$env_file"
|
||||
echo "wrote $env_file"
|
||||
fi
|
||||
|
||||
if [ ! -s "$SECRETS/shared/keycloak_petrolyst_client_secret" ]; then
|
||||
echo
|
||||
echo "Still missing: the Keycloak client secret. Copy it from Keycloak"
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
{
|
||||
"clients": [
|
||||
{
|
||||
"clientId": "petrolyst",
|
||||
"name": "Petrolyst",
|
||||
"description": "Petrolyst apps on the petrolyst server: Superset at atoma.petrolyst.com and the Traefik dashboard at traefik.petrolyst.com (oauth2-proxy). Managed from petrolyst-infra/keycloak/.",
|
||||
"enabled": true,
|
||||
"protocol": "openid-connect",
|
||||
"publicClient": false,
|
||||
"clientAuthenticatorType": "client-secret",
|
||||
"standardFlowEnabled": true,
|
||||
"implicitFlowEnabled": false,
|
||||
"directAccessGrantsEnabled": false,
|
||||
"serviceAccountsEnabled": false,
|
||||
"frontchannelLogout": true,
|
||||
"baseUrl": "https://atoma.petrolyst.com/",
|
||||
"redirectUris": [
|
||||
"https://atoma.petrolyst.com/oauth-authorized/keycloak",
|
||||
"https://traefik.petrolyst.com/oauth2/callback"
|
||||
],
|
||||
"webOrigins": ["+"],
|
||||
"attributes": {
|
||||
"pkce.code.challenge.method": "S256",
|
||||
"post.logout.redirect.uris": "https://atoma.petrolyst.com/*##https://traefik.petrolyst.com/*"
|
||||
},
|
||||
"protocolMappers": [
|
||||
{
|
||||
"name": "petrolyst client roles",
|
||||
"protocol": "openid-connect",
|
||||
"protocolMapper": "oidc-usermodel-client-role-mapper",
|
||||
"consentRequired": false,
|
||||
"config": {
|
||||
"usermodel.clientRoleMapping.clientId": "petrolyst",
|
||||
"multivalued": "true",
|
||||
"claim.name": "petrolyst_roles",
|
||||
"jsonType.label": "String",
|
||||
"id.token.claim": "true",
|
||||
"access.token.claim": "true",
|
||||
"userinfo.token.claim": "true",
|
||||
"introspection.token.claim": "true"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"roles": {
|
||||
"client": {
|
||||
"petrolyst": [
|
||||
{ "name": "superset-admin", "description": "Superset Admin at atoma.petrolyst.com" },
|
||||
{ "name": "superset-editor", "description": "Superset Alpha: build charts and dashboards" },
|
||||
{ "name": "superset-viewer", "description": "Superset Gamma: view dashboards shared with the role" },
|
||||
{ "name": "traefik-admin", "description": "Traefik dashboard at traefik.petrolyst.com" }
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -15,8 +15,9 @@ and where it runs lives in `stacks/`, `traefik/`, `deploy/` and `host/`.
|
||||
|---|---|---|
|
||||
| `superset_config.py` | Keycloak sign-in, role mapping, proxy and cookie settings, Redis caches, theme and chart palettes | `/app/pythonpath/` |
|
||||
| `petrolyst_security.py` | `PetrolystSecurityManager`: user info and roles from the Keycloak ID token, `/login/` straight to Keycloak, `/logout/` through Keycloak | `/app/pythonpath/` |
|
||||
| `assets/branding/` | logos (full colour and white), favicons, app icons | `/app/superset/static/assets/branding/`, served at `/static/assets/branding/…` |
|
||||
| `patches/` | runtime patches, `fix_*.py` with `--verify` (see its README); empty at start | `/app/patches/petrolyst/`, applied during the build |
|
||||
| `roles/` | `atoma_guest.py`: the role "ATOMA Guest" for the Petrolyst app's embedded dashboards, with exactly its permissions; run by hand (see "Embedding") | `/app/petrolyst-roles/` |
|
||||
| `assets/branding/` | logos (full colour and white): the wordmark alone for the navbar, and the stacked logo; favicons, app icons; ATOMA's logo, which the Petrolyst app shows in its map's corner (`PETROLYST_OPERATOR_LOGO`) | `/app/superset/static/assets/branding/`, served at `/static/assets/branding/…` |
|
||||
| `patches/` | runtime patches, `fix_*.py` with `--verify` (see its README); none at present | `/app/patches/petrolyst/`, applied during the build |
|
||||
| `dashboards/` | exported dashboards and import scripts (see its README) | not in the image |
|
||||
|
||||
How the image consumes this folder: `stacks/fuel-monitoring/compose.yaml` passes it to the build
|
||||
@@ -51,15 +52,39 @@ On the server they come from `/opt/petrolyst/secrets/` (see the deployment READM
|
||||
| `superset-admin` | Admin |
|
||||
| `superset-editor` | Alpha |
|
||||
| `superset-viewer` | Gamma |
|
||||
| `atoma-reader` | none (Public): ATOMA's users use the Petrolyst app, not Superset (PD-13) |
|
||||
| none | Public (no permissions) |
|
||||
|
||||
- `DASHBOARD_RBAC` is on: a viewer sees a dashboard once an admin adds the Gamma role (or a
|
||||
custom one) to it.
|
||||
- `FILTERBAR_CLOSED_BY_DEFAULT` is on: a dashboard's filter column opens collapsed, a narrow
|
||||
strip with an expand button; `?expand_filters=true` opens it from a link.
|
||||
|
||||
## Embedding (local trial only)
|
||||
|
||||
The Petrolyst app (PD-12, petrolyst-platform `app/` and `server/`) shows dashboards inside
|
||||
its Insights lens with Superset guest passes. This is switched on **only in the local scratch
|
||||
stack** (`.local/fmdb-scratch`); `superset_config.py` here leaves it off, and loads without the
|
||||
guest secret.
|
||||
|
||||
- The scratch stack's `superset_config_local.py` turns on `EMBEDDED_SUPERSET`, names the guest
|
||||
role `GUEST_ROLE_NAME = "ATOMA Guest"`, reads the signing secret with
|
||||
`_read_secret("superset_guest_token_secret")` and expects the audience `petrolyst-app`. The
|
||||
Petrolyst server signs the guest passes itself with the same secret.
|
||||
- `roles/atoma_guest.py` gives "ATOMA Guest" exactly what a dashboard needs to be read. Which
|
||||
dashboard a guest sees comes from the pass, not the role. It is not in `init.sh`; run it once,
|
||||
and again whenever its permissions change:
|
||||
|
||||
docker exec <superset> /app/.venv/bin/python /app/petrolyst-roles/atoma_guest.py
|
||||
|
||||
- The app's dashboards and their embedding ids: `dashboards/README.md`.
|
||||
|
||||
Production embedding is an open point of PD-12 (`.context/design-decisions.md`).
|
||||
|
||||
## Look and feel
|
||||
|
||||
Source: `petrolyst-ui/design/brand/` (tokens `07_UI_Tokens/petrolyst_tokens.css`, logos
|
||||
`public/brand/`, favicons `03_Favicon_App_Icons/`), copied on 2026-10-23 from petrolyst-ui commit
|
||||
Source: `petrolyst-web/design/brand/` (tokens `07_UI_Tokens/petrolyst_tokens.css`, logos
|
||||
`public/brand/`, favicons `03_Favicon_App_Icons/`), copied on 2026-10-23 from petrolyst-web (then petrolyst-ui) commit
|
||||
`2b2e694`. Copy again when the brand changes.
|
||||
|
||||
- Light theme: action teal `#00695C` for primary and links, navy `#0B1D33` text, `#F8FAFC`
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="300 225 1900 520" role="img" aria-label="ATOMA">
|
||||
<!-- ATOMA's logo as published on atoma.com.af (mtn-logo.svg, sha256 7a718cdc163c4c59...),
|
||||
in its own colours; only the viewBox is cropped to the logo. Shown in the fleet map's
|
||||
corner (dashboards/atoma/build_bundle.py, FLEET_MAP_SRC). -->
|
||||
<path fill="#1229B3" d="M477.499,608.807h82.857l-82.857-248.571h-82.857l-82.857,248.571h82.857l5.524-16.571h71.809
|
||||
L477.499,608.807z M422.261,525.95l13.81-41.428l13.809,41.428H422.261z"/>
|
||||
<path fill="#1229B3" d="M1526.188,608.807h82.856l-82.856-248.571h-82.857l-82.856,248.571h82.856l5.524-16.571h71.809
|
||||
L1526.188,608.807z M1470.95,525.95l13.81-41.428l13.81,41.428H1470.95z"/>
|
||||
<polygon fill="#1229B3" points="792.355,360.236 709.498,360.236 626.641,360.236 543.784,360.236 543.784,443.093 626.641,443.093
|
||||
626.641,608.808 709.498,608.808 709.498,443.093 792.355,443.093 "/>
|
||||
<g>
|
||||
<defs>
|
||||
<rect id="SVGID_1_" width="2500" height="1000"/>
|
||||
</defs>
|
||||
<clipPath id="SVGID_2_">
|
||||
<use xlink:href="#SVGID_1_" overflow="visible"/>
|
||||
</clipPath>
|
||||
<path clip-path="url(#SVGID_2_)" fill="#1229B3" d="M933.212,360.236c-68.641,0-124.285,55.645-124.285,124.286
|
||||
c0,68.64,55.644,124.285,124.285,124.285s124.285-55.645,124.285-124.285C1057.497,415.881,1001.853,360.236,933.212,360.236
|
||||
M933.212,525.95c-22.88,0-41.428-18.548-41.428-41.428c0-22.881,18.548-41.429,41.428-41.429s41.428,18.548,41.428,41.429
|
||||
C974.64,507.402,956.092,525.95,933.212,525.95"/>
|
||||
<polygon clip-path="url(#SVGID_2_)" fill="#1229B3" points="1248.068,360.236 1206.64,401.665 1165.211,360.236 1082.354,360.236
|
||||
1082.354,608.808 1165.211,608.808 1165.211,484.521 1206.64,525.95 1248.068,484.521 1248.068,608.808 1330.925,608.808
|
||||
1330.925,360.236 "/>
|
||||
<path clip-path="url(#SVGID_2_)" fill="#00C8FF" d="M1929.197,240.505l-106.099,106.098c-5.857,5.858-5.857,15.356,0,21.214
|
||||
l116.705,116.705l-116.705,116.705c-5.857,5.858-5.857,15.356,0,21.214l106.099,106.098c5.857,5.858,15.355,5.858,21.214,0
|
||||
l116.705-116.704l116.704-116.705c5.858-5.858,5.858-15.356,0-21.214L2067.116,357.21l-116.705-116.705
|
||||
C1944.553,234.647,1935.055,234.647,1929.197,240.505"/>
|
||||
<path clip-path="url(#SVGID_2_)" fill="#2449FB" d="M2056.782,494.855l-116.978,116.978l-244.291-244.29
|
||||
c-5.707-5.707-5.707-14.96,0-20.667l106.645-106.645c5.707-5.707,14.96-5.707,20.667,0l233.957,233.957
|
||||
C2062.489,479.895,2062.489,489.148,2056.782,494.855"/>
|
||||
<path clip-path="url(#SVGID_2_)" fill="#1229B3" d="M1802.158,728.812l-106.645-106.645c-5.707-5.707-5.707-14.96,0-20.667
|
||||
l116.978-116.978l127.313,127.312l-116.978,116.978C1817.119,734.519,1807.865,734.519,1802.158,728.812"/>
|
||||
<path clip-path="url(#SVGID_2_)" fill="#00AAFF" d="M1823.1,346.603c-5.858,5.858-5.858,15.356,0,21.214l116.705,116.705
|
||||
L1823.1,601.227c-5.858,5.858-5.858,15.356,0,21.214l53.049,53.049l127.312-127.312l53.049-53.049
|
||||
c5.858-5.858,5.858-15.356,0-21.214l-53.049-53.049l-127.312-127.312L1823.1,346.603z"/>
|
||||
</g>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 2.9 KiB |
@@ -0,0 +1,28 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="89 671 1268 103" role="img" aria-label="Petrolyst">
|
||||
<!-- Petrolyst's wordmark alone: petrolyst-web/design/brand/02_Logo/Vector_Master/
|
||||
petrolyst_logo_standard_fullcolor_no_tagline.svg without the Y mark above it, paths and
|
||||
colours unchanged, the viewBox cropped to the letters. The navbar logo (superset_config.py). -->
|
||||
|
||||
<defs>
|
||||
<linearGradient id="wordYGradient" x1="945" y1="660" x2="1090" y2="780" gradientUnits="userSpaceOnUse">
|
||||
<stop offset="0%" stop-color="#009688"/>
|
||||
<stop offset="100%" stop-color="#00B6E6"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
|
||||
<g fill="#0B1D33">
|
||||
<path d="M 250.0,753.0 L 250.0,767.0 L 339.0,767.0 L 339.0,764.0 L 340.0,763.0 L 340.0,757.0 L 339.0,756.0 L 339.0,752.0 L 251.0,752.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 250.0,715.0 L 250.0,729.0 L 266.0,729.0 L 267.0,730.0 L 301.0,730.0 L 302.0,729.0 L 307.0,729.0 L 308.0,730.0 L 325.0,730.0 L 326.0,729.0 L 340.0,729.0 L 340.0,716.0 L 339.0,715.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 1260.0,678.0 L 1260.0,692.0 L 1264.0,692.0 L 1265.0,693.0 L 1296.0,693.0 L 1297.0,694.0 L 1297.0,767.0 L 1313.0,767.0 L 1313.0,694.0 L 1314.0,693.0 L 1351.0,693.0 L 1351.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 867.0,678.0 L 867.0,767.0 L 951.0,767.0 L 951.0,753.0 L 950.0,752.0 L 883.0,752.0 L 882.0,751.0 L 882.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 551.0,678.0 L 551.0,767.0 L 566.0,767.0 L 566.0,743.0 L 567.0,742.0 L 567.0,739.0 L 566.0,738.0 L 567.0,737.0 L 604.0,737.0 L 614.0,747.0 L 614.0,748.0 L 622.0,756.0 L 622.0,757.0 L 630.0,765.0 L 630.0,766.0 L 631.0,767.0 L 651.0,767.0 L 651.0,766.0 L 649.0,764.0 L 649.0,763.0 L 642.0,756.0 L 642.0,755.0 L 628.0,741.0 L 628.0,740.0 L 626.0,738.0 L 628.0,736.0 L 630.0,736.0 L 631.0,735.0 L 633.0,735.0 L 634.0,734.0 L 636.0,734.0 L 638.0,732.0 L 639.0,732.0 L 645.0,726.0 L 645.0,725.0 L 648.0,720.0 L 648.0,718.0 L 649.0,717.0 L 649.0,713.0 L 650.0,712.0 L 650.0,703.0 L 649.0,702.0 L 649.0,698.0 L 648.0,697.0 L 648.0,695.0 L 647.0,694.0 L 646.0,691.0 L 644.0,689.0 L 644.0,688.0 L 640.0,684.0 L 639.0,684.0 L 637.0,682.0 L 636.0,682.0 L 633.0,680.0 L 631.0,680.0 L 630.0,679.0 L 626.0,679.0 L 625.0,678.0 Z M 567.0,694.0 L 568.0,693.0 L 625.0,693.0 L 626.0,694.0 L 629.0,695.0 L 632.0,698.0 L 632.0,699.0 L 633.0,700.0 L 633.0,713.0 L 632.0,714.0 L 632.0,716.0 L 630.0,718.0 L 630.0,719.0 L 629.0,720.0 L 628.0,720.0 L 626.0,722.0 L 622.0,722.0 L 621.0,723.0 L 568.0,723.0 L 566.0,721.0 L 566.0,703.0 L 567.0,702.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 398.0,678.0 L 398.0,693.0 L 434.0,693.0 L 435.0,694.0 L 435.0,767.0 L 451.0,767.0 L 451.0,694.0 L 452.0,693.0 L 489.0,693.0 L 489.0,679.0 L 488.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 250.0,678.0 L 250.0,692.0 L 251.0,692.0 L 252.0,693.0 L 339.0,693.0 L 339.0,691.0 L 340.0,690.0 L 340.0,679.0 L 339.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 1134.0,683.0 L 1129.0,688.0 L 1129.0,689.0 L 1127.0,691.0 L 1127.0,693.0 L 1126.0,694.0 L 1126.0,696.0 L 1125.0,697.0 L 1125.0,712.0 L 1126.0,713.0 L 1126.0,715.0 L 1127.0,716.0 L 1128.0,719.0 L 1134.0,725.0 L 1135.0,725.0 L 1137.0,727.0 L 1139.0,727.0 L 1140.0,728.0 L 1143.0,728.0 L 1144.0,729.0 L 1185.0,729.0 L 1186.0,730.0 L 1191.0,730.0 L 1192.0,731.0 L 1193.0,731.0 L 1197.0,735.0 L 1197.0,739.0 L 1198.0,740.0 L 1198.0,742.0 L 1197.0,743.0 L 1197.0,746.0 L 1196.0,747.0 L 1196.0,748.0 L 1193.0,751.0 L 1192.0,751.0 L 1189.0,753.0 L 1184.0,753.0 L 1183.0,754.0 L 1151.0,754.0 L 1150.0,753.0 L 1143.0,753.0 L 1142.0,752.0 L 1139.0,752.0 L 1138.0,751.0 L 1135.0,751.0 L 1134.0,750.0 L 1131.0,750.0 L 1130.0,749.0 L 1128.0,749.0 L 1127.0,748.0 L 1125.0,748.0 L 1125.0,760.0 L 1124.0,761.0 L 1126.0,763.0 L 1128.0,763.0 L 1129.0,764.0 L 1131.0,764.0 L 1132.0,765.0 L 1134.0,765.0 L 1135.0,766.0 L 1139.0,766.0 L 1140.0,767.0 L 1146.0,767.0 L 1147.0,768.0 L 1186.0,768.0 L 1187.0,767.0 L 1191.0,767.0 L 1192.0,766.0 L 1195.0,766.0 L 1196.0,765.0 L 1198.0,765.0 L 1199.0,764.0 L 1201.0,764.0 L 1203.0,762.0 L 1204.0,762.0 L 1209.0,757.0 L 1209.0,756.0 L 1211.0,754.0 L 1211.0,753.0 L 1212.0,752.0 L 1212.0,750.0 L 1213.0,749.0 L 1213.0,734.0 L 1212.0,733.0 L 1212.0,730.0 L 1211.0,729.0 L 1211.0,728.0 L 1209.0,726.0 L 1209.0,725.0 L 1204.0,720.0 L 1203.0,720.0 L 1202.0,719.0 L 1201.0,719.0 L 1196.0,716.0 L 1192.0,716.0 L 1191.0,715.0 L 1148.0,715.0 L 1147.0,714.0 L 1145.0,714.0 L 1142.0,711.0 L 1142.0,710.0 L 1141.0,709.0 L 1141.0,699.0 L 1142.0,698.0 L 1142.0,697.0 L 1145.0,694.0 L 1146.0,694.0 L 1147.0,693.0 L 1150.0,693.0 L 1151.0,692.0 L 1190.0,692.0 L 1191.0,693.0 L 1196.0,693.0 L 1197.0,694.0 L 1201.0,694.0 L 1202.0,695.0 L 1204.0,695.0 L 1205.0,696.0 L 1209.0,696.0 L 1209.0,683.0 L 1206.0,681.0 L 1203.0,681.0 L 1202.0,680.0 L 1198.0,680.0 L 1197.0,679.0 L 1193.0,679.0 L 1192.0,678.0 L 1182.0,678.0 L 1181.0,677.0 L 1156.0,677.0 L 1155.0,678.0 L 1146.0,678.0 L 1145.0,679.0 L 1143.0,679.0 L 1142.0,680.0 L 1140.0,680.0 L 1139.0,681.0 L 1138.0,681.0 L 1137.0,682.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 714.0,691.0 L 714.0,692.0 L 711.0,695.0 L 711.0,696.0 L 709.0,698.0 L 709.0,699.0 L 708.0,700.0 L 708.0,702.0 L 707.0,703.0 L 707.0,706.0 L 706.0,707.0 L 706.0,711.0 L 705.0,712.0 L 705.0,734.0 L 706.0,735.0 L 706.0,739.0 L 707.0,740.0 L 707.0,743.0 L 708.0,744.0 L 708.0,745.0 L 709.0,746.0 L 709.0,747.0 L 710.0,748.0 L 711.0,751.0 L 713.0,753.0 L 713.0,754.0 L 718.0,759.0 L 719.0,759.0 L 721.0,761.0 L 722.0,761.0 L 723.0,762.0 L 724.0,762.0 L 729.0,765.0 L 731.0,765.0 L 732.0,766.0 L 735.0,766.0 L 736.0,767.0 L 741.0,767.0 L 742.0,768.0 L 775.0,768.0 L 776.0,767.0 L 781.0,767.0 L 782.0,766.0 L 785.0,766.0 L 786.0,765.0 L 788.0,765.0 L 789.0,764.0 L 790.0,764.0 L 791.0,763.0 L 792.0,763.0 L 793.0,762.0 L 796.0,761.0 L 798.0,759.0 L 799.0,759.0 L 805.0,753.0 L 805.0,752.0 L 807.0,750.0 L 807.0,749.0 L 809.0,746.0 L 809.0,744.0 L 811.0,741.0 L 811.0,737.0 L 812.0,736.0 L 812.0,711.0 L 811.0,710.0 L 811.0,706.0 L 810.0,705.0 L 810.0,703.0 L 809.0,702.0 L 809.0,700.0 L 808.0,699.0 L 808.0,698.0 L 806.0,696.0 L 806.0,695.0 L 803.0,692.0 L 803.0,691.0 L 799.0,687.0 L 798.0,687.0 L 793.0,683.0 L 792.0,683.0 L 791.0,682.0 L 789.0,682.0 L 786.0,680.0 L 783.0,680.0 L 782.0,679.0 L 779.0,679.0 L 778.0,678.0 L 770.0,678.0 L 769.0,677.0 L 748.0,677.0 L 747.0,678.0 L 739.0,678.0 L 738.0,679.0 L 734.0,679.0 L 733.0,680.0 L 731.0,680.0 L 730.0,681.0 L 728.0,681.0 L 727.0,682.0 L 726.0,682.0 L 725.0,683.0 L 722.0,684.0 L 720.0,686.0 L 719.0,686.0 Z M 725.0,702.0 L 730.0,697.0 L 731.0,697.0 L 733.0,695.0 L 735.0,695.0 L 738.0,693.0 L 742.0,693.0 L 743.0,692.0 L 772.0,692.0 L 773.0,693.0 L 777.0,693.0 L 778.0,694.0 L 781.0,694.0 L 782.0,695.0 L 785.0,696.0 L 787.0,698.0 L 788.0,698.0 L 791.0,701.0 L 791.0,702.0 L 793.0,704.0 L 793.0,705.0 L 794.0,706.0 L 794.0,708.0 L 795.0,709.0 L 795.0,712.0 L 796.0,713.0 L 796.0,733.0 L 795.0,734.0 L 795.0,738.0 L 794.0,739.0 L 793.0,742.0 L 791.0,744.0 L 791.0,745.0 L 788.0,748.0 L 787.0,748.0 L 785.0,750.0 L 784.0,750.0 L 781.0,752.0 L 778.0,752.0 L 777.0,753.0 L 770.0,753.0 L 769.0,754.0 L 747.0,754.0 L 746.0,753.0 L 740.0,753.0 L 739.0,752.0 L 737.0,752.0 L 736.0,751.0 L 734.0,751.0 L 733.0,750.0 L 732.0,750.0 L 730.0,748.0 L 729.0,748.0 L 725.0,744.0 L 725.0,743.0 L 722.0,738.0 L 722.0,734.0 L 721.0,733.0 L 721.0,713.0 L 722.0,712.0 L 722.0,709.0 L 723.0,708.0 L 723.0,706.0 L 724.0,705.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 95.0,678.0 L 95.0,766.0 L 96.0,767.0 L 111.0,767.0 L 111.0,737.0 L 112.0,736.0 L 171.0,736.0 L 172.0,735.0 L 175.0,735.0 L 176.0,734.0 L 178.0,734.0 L 179.0,733.0 L 180.0,733.0 L 182.0,731.0 L 183.0,731.0 L 186.0,728.0 L 187.0,728.0 L 187.0,727.0 L 190.0,724.0 L 190.0,723.0 L 193.0,718.0 L 193.0,715.0 L 194.0,714.0 L 194.0,702.0 L 193.0,701.0 L 193.0,696.0 L 192.0,695.0 L 192.0,693.0 L 191.0,692.0 L 190.0,689.0 L 188.0,687.0 L 188.0,686.0 L 185.0,683.0 L 184.0,683.0 L 182.0,681.0 L 181.0,681.0 L 180.0,680.0 L 178.0,680.0 L 177.0,679.0 L 174.0,679.0 L 173.0,678.0 L 168.0,678.0 L 167.0,677.0 L 156.0,677.0 L 155.0,678.0 L 135.0,678.0 L 134.0,677.0 L 132.0,677.0 L 131.0,678.0 L 108.0,678.0 L 107.0,677.0 L 105.0,677.0 L 104.0,678.0 Z M 111.0,693.0 L 112.0,692.0 L 169.0,692.0 L 170.0,693.0 L 172.0,693.0 L 177.0,698.0 L 177.0,699.0 L 178.0,700.0 L 178.0,712.0 L 177.0,713.0 L 177.0,715.0 L 176.0,716.0 L 176.0,717.0 L 174.0,719.0 L 173.0,719.0 L 170.0,721.0 L 167.0,721.0 L 166.0,722.0 L 154.0,722.0 L 153.0,721.0 L 152.0,722.0 L 115.0,722.0 L 114.0,721.0 L 113.0,722.0 L 111.0,720.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<rect x="250" y="678" width="16" height="90" fill="#0B1D33"/>
|
||||
</g>
|
||||
<path d="M 1084.0,678.0 L 1063.0,678.0 L 1056.0,685.0 L 1056.0,686.0 L 1049.0,693.0 L 1049.0,694.0 L 1042.0,701.0 L 1042.0,702.0 L 1036.0,708.0 L 1036.0,709.0 L 1035.0,710.0 L 1035.0,711.0 L 1033.0,713.0 L 1033.0,714.0 L 1032.0,715.0 L 1032.0,716.0 L 1031.0,717.0 L 1031.0,718.0 L 1030.0,719.0 L 1030.0,720.0 L 1028.0,723.0 L 1028.0,725.0 L 1026.0,728.0 L 1026.0,730.0 L 1025.0,731.0 L 1025.0,733.0 L 1024.0,734.0 L 1024.0,738.0 L 1023.0,739.0 L 1023.0,743.0 L 1022.0,744.0 L 1022.0,752.0 L 1021.0,753.0 L 1021.0,766.0 L 1022.0,767.0 L 1038.0,767.0 L 1038.0,737.0 L 1039.0,736.0 L 1039.0,734.0 L 1040.0,733.0 L 1040.0,732.0 L 1041.0,731.0 L 1041.0,730.0 L 1043.0,728.0 L 1043.0,727.0 L 1046.0,724.0 L 1046.0,723.0 L 1052.0,717.0 L 1052.0,716.0 L 1060.0,708.0 L 1060.0,707.0 L 1065.0,702.0 L 1065.0,701.0 L 1072.0,694.0 L 1072.0,693.0 L 1077.0,688.0 L 1077.0,687.0 L 1082.0,682.0 L 1082.0,681.0 L 1084.0,679.0 L 1085.0,679.0 Z" fill="url(#wordYGradient)" fill-rule="evenodd"/>
|
||||
<path d="M 975.0,678.0 L 975.0,679.0 L 979.0,683.0 L 979.0,684.0 L 983.0,688.0 L 983.0,689.0 L 990.0,696.0 L 990.0,697.0 L 996.0,703.0 L 996.0,704.0 L 1002.0,710.0 L 1002.0,711.0 L 1010.0,719.0 L 1010.0,720.0 L 1014.0,724.0 L 1014.0,725.0 L 1017.0,728.0 L 1017.0,729.0 L 1018.0,730.0 L 1018.0,731.0 L 1019.0,732.0 L 1020.0,732.0 L 1020.0,729.0 L 1021.0,728.0 L 1021.0,726.0 L 1022.0,725.0 L 1022.0,722.0 L 1023.0,721.0 L 1023.0,715.0 L 1024.0,714.0 L 1023.0,713.0 L 1023.0,709.0 L 1022.0,708.0 L 1022.0,707.0 L 1021.0,706.0 L 1021.0,705.0 L 1016.0,700.0 L 1016.0,699.0 L 1010.0,693.0 L 1010.0,692.0 L 1003.0,685.0 L 1003.0,684.0 L 997.0,678.0 Z" fill="url(#wordYGradient)" fill-rule="evenodd"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 10 KiB |
@@ -0,0 +1,18 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="89 671 1268 103" role="img" aria-label="Petrolyst">
|
||||
<!-- Petrolyst's wordmark alone: petrolyst-web/design/brand/02_Logo/Vector_Master/
|
||||
petrolyst_logo_standard_white_no_tagline.svg without the Y mark above it, paths and
|
||||
colours unchanged, the viewBox cropped to the letters. The navbar logo (superset_config.py). -->
|
||||
<g fill="#FFFFFF"><path d="M 250.0,753.0 L 250.0,767.0 L 339.0,767.0 L 339.0,764.0 L 340.0,763.0 L 340.0,757.0 L 339.0,756.0 L 339.0,752.0 L 251.0,752.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 250.0,715.0 L 250.0,729.0 L 266.0,729.0 L 267.0,730.0 L 301.0,730.0 L 302.0,729.0 L 307.0,729.0 L 308.0,730.0 L 325.0,730.0 L 326.0,729.0 L 340.0,729.0 L 340.0,716.0 L 339.0,715.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 1260.0,678.0 L 1260.0,692.0 L 1264.0,692.0 L 1265.0,693.0 L 1296.0,693.0 L 1297.0,694.0 L 1297.0,767.0 L 1313.0,767.0 L 1313.0,694.0 L 1314.0,693.0 L 1351.0,693.0 L 1351.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 867.0,678.0 L 867.0,767.0 L 951.0,767.0 L 951.0,753.0 L 950.0,752.0 L 883.0,752.0 L 882.0,751.0 L 882.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 551.0,678.0 L 551.0,767.0 L 566.0,767.0 L 566.0,743.0 L 567.0,742.0 L 567.0,739.0 L 566.0,738.0 L 567.0,737.0 L 604.0,737.0 L 614.0,747.0 L 614.0,748.0 L 622.0,756.0 L 622.0,757.0 L 630.0,765.0 L 630.0,766.0 L 631.0,767.0 L 651.0,767.0 L 651.0,766.0 L 649.0,764.0 L 649.0,763.0 L 642.0,756.0 L 642.0,755.0 L 628.0,741.0 L 628.0,740.0 L 626.0,738.0 L 628.0,736.0 L 630.0,736.0 L 631.0,735.0 L 633.0,735.0 L 634.0,734.0 L 636.0,734.0 L 638.0,732.0 L 639.0,732.0 L 645.0,726.0 L 645.0,725.0 L 648.0,720.0 L 648.0,718.0 L 649.0,717.0 L 649.0,713.0 L 650.0,712.0 L 650.0,703.0 L 649.0,702.0 L 649.0,698.0 L 648.0,697.0 L 648.0,695.0 L 647.0,694.0 L 646.0,691.0 L 644.0,689.0 L 644.0,688.0 L 640.0,684.0 L 639.0,684.0 L 637.0,682.0 L 636.0,682.0 L 633.0,680.0 L 631.0,680.0 L 630.0,679.0 L 626.0,679.0 L 625.0,678.0 Z M 567.0,694.0 L 568.0,693.0 L 625.0,693.0 L 626.0,694.0 L 629.0,695.0 L 632.0,698.0 L 632.0,699.0 L 633.0,700.0 L 633.0,713.0 L 632.0,714.0 L 632.0,716.0 L 630.0,718.0 L 630.0,719.0 L 629.0,720.0 L 628.0,720.0 L 626.0,722.0 L 622.0,722.0 L 621.0,723.0 L 568.0,723.0 L 566.0,721.0 L 566.0,703.0 L 567.0,702.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 398.0,678.0 L 398.0,693.0 L 434.0,693.0 L 435.0,694.0 L 435.0,767.0 L 451.0,767.0 L 451.0,694.0 L 452.0,693.0 L 489.0,693.0 L 489.0,679.0 L 488.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 250.0,678.0 L 250.0,692.0 L 251.0,692.0 L 252.0,693.0 L 339.0,693.0 L 339.0,691.0 L 340.0,690.0 L 340.0,679.0 L 339.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 1134.0,683.0 L 1129.0,688.0 L 1129.0,689.0 L 1127.0,691.0 L 1127.0,693.0 L 1126.0,694.0 L 1126.0,696.0 L 1125.0,697.0 L 1125.0,712.0 L 1126.0,713.0 L 1126.0,715.0 L 1127.0,716.0 L 1128.0,719.0 L 1134.0,725.0 L 1135.0,725.0 L 1137.0,727.0 L 1139.0,727.0 L 1140.0,728.0 L 1143.0,728.0 L 1144.0,729.0 L 1185.0,729.0 L 1186.0,730.0 L 1191.0,730.0 L 1192.0,731.0 L 1193.0,731.0 L 1197.0,735.0 L 1197.0,739.0 L 1198.0,740.0 L 1198.0,742.0 L 1197.0,743.0 L 1197.0,746.0 L 1196.0,747.0 L 1196.0,748.0 L 1193.0,751.0 L 1192.0,751.0 L 1189.0,753.0 L 1184.0,753.0 L 1183.0,754.0 L 1151.0,754.0 L 1150.0,753.0 L 1143.0,753.0 L 1142.0,752.0 L 1139.0,752.0 L 1138.0,751.0 L 1135.0,751.0 L 1134.0,750.0 L 1131.0,750.0 L 1130.0,749.0 L 1128.0,749.0 L 1127.0,748.0 L 1125.0,748.0 L 1125.0,760.0 L 1124.0,761.0 L 1126.0,763.0 L 1128.0,763.0 L 1129.0,764.0 L 1131.0,764.0 L 1132.0,765.0 L 1134.0,765.0 L 1135.0,766.0 L 1139.0,766.0 L 1140.0,767.0 L 1146.0,767.0 L 1147.0,768.0 L 1186.0,768.0 L 1187.0,767.0 L 1191.0,767.0 L 1192.0,766.0 L 1195.0,766.0 L 1196.0,765.0 L 1198.0,765.0 L 1199.0,764.0 L 1201.0,764.0 L 1203.0,762.0 L 1204.0,762.0 L 1209.0,757.0 L 1209.0,756.0 L 1211.0,754.0 L 1211.0,753.0 L 1212.0,752.0 L 1212.0,750.0 L 1213.0,749.0 L 1213.0,734.0 L 1212.0,733.0 L 1212.0,730.0 L 1211.0,729.0 L 1211.0,728.0 L 1209.0,726.0 L 1209.0,725.0 L 1204.0,720.0 L 1203.0,720.0 L 1202.0,719.0 L 1201.0,719.0 L 1196.0,716.0 L 1192.0,716.0 L 1191.0,715.0 L 1148.0,715.0 L 1147.0,714.0 L 1145.0,714.0 L 1142.0,711.0 L 1142.0,710.0 L 1141.0,709.0 L 1141.0,699.0 L 1142.0,698.0 L 1142.0,697.0 L 1145.0,694.0 L 1146.0,694.0 L 1147.0,693.0 L 1150.0,693.0 L 1151.0,692.0 L 1190.0,692.0 L 1191.0,693.0 L 1196.0,693.0 L 1197.0,694.0 L 1201.0,694.0 L 1202.0,695.0 L 1204.0,695.0 L 1205.0,696.0 L 1209.0,696.0 L 1209.0,683.0 L 1206.0,681.0 L 1203.0,681.0 L 1202.0,680.0 L 1198.0,680.0 L 1197.0,679.0 L 1193.0,679.0 L 1192.0,678.0 L 1182.0,678.0 L 1181.0,677.0 L 1156.0,677.0 L 1155.0,678.0 L 1146.0,678.0 L 1145.0,679.0 L 1143.0,679.0 L 1142.0,680.0 L 1140.0,680.0 L 1139.0,681.0 L 1138.0,681.0 L 1137.0,682.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 714.0,691.0 L 714.0,692.0 L 711.0,695.0 L 711.0,696.0 L 709.0,698.0 L 709.0,699.0 L 708.0,700.0 L 708.0,702.0 L 707.0,703.0 L 707.0,706.0 L 706.0,707.0 L 706.0,711.0 L 705.0,712.0 L 705.0,734.0 L 706.0,735.0 L 706.0,739.0 L 707.0,740.0 L 707.0,743.0 L 708.0,744.0 L 708.0,745.0 L 709.0,746.0 L 709.0,747.0 L 710.0,748.0 L 711.0,751.0 L 713.0,753.0 L 713.0,754.0 L 718.0,759.0 L 719.0,759.0 L 721.0,761.0 L 722.0,761.0 L 723.0,762.0 L 724.0,762.0 L 729.0,765.0 L 731.0,765.0 L 732.0,766.0 L 735.0,766.0 L 736.0,767.0 L 741.0,767.0 L 742.0,768.0 L 775.0,768.0 L 776.0,767.0 L 781.0,767.0 L 782.0,766.0 L 785.0,766.0 L 786.0,765.0 L 788.0,765.0 L 789.0,764.0 L 790.0,764.0 L 791.0,763.0 L 792.0,763.0 L 793.0,762.0 L 796.0,761.0 L 798.0,759.0 L 799.0,759.0 L 805.0,753.0 L 805.0,752.0 L 807.0,750.0 L 807.0,749.0 L 809.0,746.0 L 809.0,744.0 L 811.0,741.0 L 811.0,737.0 L 812.0,736.0 L 812.0,711.0 L 811.0,710.0 L 811.0,706.0 L 810.0,705.0 L 810.0,703.0 L 809.0,702.0 L 809.0,700.0 L 808.0,699.0 L 808.0,698.0 L 806.0,696.0 L 806.0,695.0 L 803.0,692.0 L 803.0,691.0 L 799.0,687.0 L 798.0,687.0 L 793.0,683.0 L 792.0,683.0 L 791.0,682.0 L 789.0,682.0 L 786.0,680.0 L 783.0,680.0 L 782.0,679.0 L 779.0,679.0 L 778.0,678.0 L 770.0,678.0 L 769.0,677.0 L 748.0,677.0 L 747.0,678.0 L 739.0,678.0 L 738.0,679.0 L 734.0,679.0 L 733.0,680.0 L 731.0,680.0 L 730.0,681.0 L 728.0,681.0 L 727.0,682.0 L 726.0,682.0 L 725.0,683.0 L 722.0,684.0 L 720.0,686.0 L 719.0,686.0 Z M 725.0,702.0 L 730.0,697.0 L 731.0,697.0 L 733.0,695.0 L 735.0,695.0 L 738.0,693.0 L 742.0,693.0 L 743.0,692.0 L 772.0,692.0 L 773.0,693.0 L 777.0,693.0 L 778.0,694.0 L 781.0,694.0 L 782.0,695.0 L 785.0,696.0 L 787.0,698.0 L 788.0,698.0 L 791.0,701.0 L 791.0,702.0 L 793.0,704.0 L 793.0,705.0 L 794.0,706.0 L 794.0,708.0 L 795.0,709.0 L 795.0,712.0 L 796.0,713.0 L 796.0,733.0 L 795.0,734.0 L 795.0,738.0 L 794.0,739.0 L 793.0,742.0 L 791.0,744.0 L 791.0,745.0 L 788.0,748.0 L 787.0,748.0 L 785.0,750.0 L 784.0,750.0 L 781.0,752.0 L 778.0,752.0 L 777.0,753.0 L 770.0,753.0 L 769.0,754.0 L 747.0,754.0 L 746.0,753.0 L 740.0,753.0 L 739.0,752.0 L 737.0,752.0 L 736.0,751.0 L 734.0,751.0 L 733.0,750.0 L 732.0,750.0 L 730.0,748.0 L 729.0,748.0 L 725.0,744.0 L 725.0,743.0 L 722.0,738.0 L 722.0,734.0 L 721.0,733.0 L 721.0,713.0 L 722.0,712.0 L 722.0,709.0 L 723.0,708.0 L 723.0,706.0 L 724.0,705.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 95.0,678.0 L 95.0,766.0 L 96.0,767.0 L 111.0,767.0 L 111.0,737.0 L 112.0,736.0 L 171.0,736.0 L 172.0,735.0 L 175.0,735.0 L 176.0,734.0 L 178.0,734.0 L 179.0,733.0 L 180.0,733.0 L 182.0,731.0 L 183.0,731.0 L 186.0,728.0 L 187.0,728.0 L 187.0,727.0 L 190.0,724.0 L 190.0,723.0 L 193.0,718.0 L 193.0,715.0 L 194.0,714.0 L 194.0,702.0 L 193.0,701.0 L 193.0,696.0 L 192.0,695.0 L 192.0,693.0 L 191.0,692.0 L 190.0,689.0 L 188.0,687.0 L 188.0,686.0 L 185.0,683.0 L 184.0,683.0 L 182.0,681.0 L 181.0,681.0 L 180.0,680.0 L 178.0,680.0 L 177.0,679.0 L 174.0,679.0 L 173.0,678.0 L 168.0,678.0 L 167.0,677.0 L 156.0,677.0 L 155.0,678.0 L 135.0,678.0 L 134.0,677.0 L 132.0,677.0 L 131.0,678.0 L 108.0,678.0 L 107.0,677.0 L 105.0,677.0 L 104.0,678.0 Z M 111.0,693.0 L 112.0,692.0 L 169.0,692.0 L 170.0,693.0 L 172.0,693.0 L 177.0,698.0 L 177.0,699.0 L 178.0,700.0 L 178.0,712.0 L 177.0,713.0 L 177.0,715.0 L 176.0,716.0 L 176.0,717.0 L 174.0,719.0 L 173.0,719.0 L 170.0,721.0 L 167.0,721.0 L 166.0,722.0 L 154.0,722.0 L 153.0,721.0 L 152.0,722.0 L 115.0,722.0 L 114.0,721.0 L 113.0,722.0 L 111.0,720.0 Z" fill="#FFFFFF" fill-rule="evenodd"/><rect x="250" y="678" width="16" height="90" fill="#FFFFFF"/></g>
|
||||
<path d="M 1084.0,678.0 L 1063.0,678.0 L 1056.0,685.0 L 1056.0,686.0 L 1049.0,693.0 L 1049.0,694.0 L 1042.0,701.0 L 1042.0,702.0 L 1036.0,708.0 L 1036.0,709.0 L 1035.0,710.0 L 1035.0,711.0 L 1033.0,713.0 L 1033.0,714.0 L 1032.0,715.0 L 1032.0,716.0 L 1031.0,717.0 L 1031.0,718.0 L 1030.0,719.0 L 1030.0,720.0 L 1028.0,723.0 L 1028.0,725.0 L 1026.0,728.0 L 1026.0,730.0 L 1025.0,731.0 L 1025.0,733.0 L 1024.0,734.0 L 1024.0,738.0 L 1023.0,739.0 L 1023.0,743.0 L 1022.0,744.0 L 1022.0,752.0 L 1021.0,753.0 L 1021.0,766.0 L 1022.0,767.0 L 1038.0,767.0 L 1038.0,737.0 L 1039.0,736.0 L 1039.0,734.0 L 1040.0,733.0 L 1040.0,732.0 L 1041.0,731.0 L 1041.0,730.0 L 1043.0,728.0 L 1043.0,727.0 L 1046.0,724.0 L 1046.0,723.0 L 1052.0,717.0 L 1052.0,716.0 L 1060.0,708.0 L 1060.0,707.0 L 1065.0,702.0 L 1065.0,701.0 L 1072.0,694.0 L 1072.0,693.0 L 1077.0,688.0 L 1077.0,687.0 L 1082.0,682.0 L 1082.0,681.0 L 1084.0,679.0 L 1085.0,679.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 975.0,678.0 L 975.0,679.0 L 979.0,683.0 L 979.0,684.0 L 983.0,688.0 L 983.0,689.0 L 990.0,696.0 L 990.0,697.0 L 996.0,703.0 L 996.0,704.0 L 1002.0,710.0 L 1002.0,711.0 L 1010.0,719.0 L 1010.0,720.0 L 1014.0,724.0 L 1014.0,725.0 L 1017.0,728.0 L 1017.0,729.0 L 1018.0,730.0 L 1018.0,731.0 L 1019.0,732.0 L 1020.0,732.0 L 1020.0,729.0 L 1021.0,728.0 L 1021.0,726.0 L 1022.0,725.0 L 1022.0,722.0 L 1023.0,721.0 L 1023.0,715.0 L 1024.0,714.0 L 1023.0,713.0 L 1023.0,709.0 L 1022.0,708.0 L 1022.0,707.0 L 1021.0,706.0 L 1021.0,705.0 L 1016.0,700.0 L 1016.0,699.0 L 1010.0,693.0 L 1010.0,692.0 L 1003.0,685.0 L 1003.0,684.0 L 997.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 9.9 KiB |
@@ -1,13 +1,45 @@
|
||||
# dashboards/
|
||||
|
||||
Exported Superset dashboards and the operator scripts that import them. Not baked into the
|
||||
image, so changes here do not rebuild or restart Superset. Empty until the first fuel
|
||||
dashboards exist.
|
||||
Superset dashboards as code and the operator scripts that import them. Not baked into the
|
||||
image, so changes here do not rebuild or restart Superset.
|
||||
|
||||
- `atoma/`: ATOMA's two dashboards for the Petrolyst app (PD-12, PD-13; SPEC.md §10):
|
||||
`atoma-overview` (fleet totals) and `atoma-site` (one site, its four sections as tabs). The
|
||||
app embeds them; admins open them in Superset, with the filter column. `SPEC.md` is the
|
||||
design; `build_bundle.py` generates the import bundle and `import_bundle.py` imports it
|
||||
(overwriting the objects the bundle defines).
|
||||
|
||||
Build and import:
|
||||
|
||||
docker cp atoma/import_bundle.py petrolyst-superset-1:/tmp/
|
||||
docker exec petrolyst-superset-1 /app/.venv/bin/python /tmp/import_bundle.py --show-database
|
||||
python3 atoma/build_bundle.py --database-uuid <printed uuid> --out atoma-dashboards.zip
|
||||
docker cp atoma-dashboards.zip petrolyst-superset-1:/tmp/
|
||||
docker exec petrolyst-superset-1 /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip --user <username>
|
||||
|
||||
Exit status 2 means a dataset's columns differ from its view: regenerate `atoma/columns.tsv`
|
||||
(command in its header), rebuild and import again.
|
||||
|
||||
An import never deletes. Where they exist, delete these by hand after importing: the
|
||||
dashboards from before 28 Oct ("ATOMA - Sites", "ATOMA - Site Detail") and their charts M1,
|
||||
M3, M4 and M5; the ATOMA dashboard (slug `atoma`, retired 29 Oct with PD-13) and its chart D0
|
||||
"Site"; the one-tab dashboards `atoma-site-summary`, `-fuel`, `-runtime` and `-baseline`
|
||||
(replaced by `atoma-site` on 29 Oct); and the role "ATOMA Reader".
|
||||
|
||||
The dashboards get no role; the app shows them with guest passes (see "Embedding" in the
|
||||
parent README; local stack only). Add `--embedded-out FILE --embed-domain <app origin>` to the
|
||||
import to make them embeddable from the app's origin; FILE then receives
|
||||
`{slug: embedding id}` for the Petrolyst server. The ids stay the same across imports. Local
|
||||
stack:
|
||||
|
||||
docker exec fmdb-scratch-superset /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip \
|
||||
--user admin --embedded-out /tmp/embedded-dashboards.json --embed-domain http://127.0.0.1:8098
|
||||
docker cp fmdb-scratch-superset:/tmp/embedded-dashboards.json <Petrolyst>/.local/fmdb-scratch/
|
||||
|
||||
Export from a running instance (inside the container, as the server's deploy user):
|
||||
|
||||
docker exec fuel-monitoring-superset-1 superset export-dashboards -f /tmp/dashboards.zip
|
||||
docker cp fuel-monitoring-superset-1:/tmp/dashboards.zip .
|
||||
docker exec petrolyst-superset-1 superset export-dashboards -f /tmp/dashboards.zip
|
||||
docker cp petrolyst-superset-1:/tmp/dashboards.zip .
|
||||
|
||||
Import is the reverse with `superset import-dashboards -p <zip> -u <username>`. The
|
||||
username must be an existing Superset user, which here means someone who has signed in
|
||||
|
||||
@@ -1,13 +1,23 @@
|
||||
# ATOMA dashboards: Sites Map and Site Detail
|
||||
# ATOMA dashboards: Overview and Site, for the Petrolyst app
|
||||
|
||||
**Status: proposed, not built.**
|
||||
- `build_bundle.py` (phase P2) will generate both dashboards, every dataset and every chart from
|
||||
this specification, as one Superset import bundle.
|
||||
**Status (29 Oct 2026, PD-13): two dashboards, `atoma-overview` and `atoma-site` (§10), built and
|
||||
tested on a local Superset; not imported live.** The Petrolyst app embeds them, and admins open
|
||||
them in Superset with the filter column.
|
||||
- `build_bundle.py` generates the dashboards, every dataset and every chart from this
|
||||
specification, as one Superset import bundle; `import_bundle.py` imports it (§6).
|
||||
- **Retired 29 Oct (PD-13):** the "ATOMA" dashboard (slug `atoma`) with its Map tab (the React
|
||||
fleet-map view of petrolyst-platform) and Insights tab, its D0 header row, and the Superset
|
||||
reader setup ("ATOMA Reader", `reader.css`). ATOMA's users use the Petrolyst app. §2, §3, §9 and
|
||||
the readers' part of §6 describe them and stay as history. §4's data rules and four tabs are
|
||||
current: they are `atoma-site`'s tabs, without D0 (the app shows the site's header).
|
||||
- Earlier changes are marked *(P2)* (local tests of 24 Oct, §8) and *(28 Oct)* (one dashboard
|
||||
with the fleet-map view, §9).
|
||||
- Nothing is imported into the live Superset without the owner's go-ahead (phase P3).
|
||||
|
||||
**Datasets.** The `public.v_*` views of
|
||||
`stacks/fuel-monitoring/fuel-db/migrations/0003_bi_views.sql`, read through Trino as
|
||||
`fuel.public`.
|
||||
`stacks/fuel-monitoring/fuel-db/migrations/0003_bi_views.sql`, read from fuel-db directly (database
|
||||
`fuel`, schema `public`) as the read-only role `fuel_ro`. *Until 29 Oct 2026 Superset read them
|
||||
through Trino as `fuel.public`; Trino was removed then.*
|
||||
|
||||
**Design and rationale** live in the fuel-monitoring corpus, in
|
||||
`.context/plans/2026-10-24-fuel-db-baseline-dashboard.md`:
|
||||
@@ -15,72 +25,103 @@
|
||||
- the site-specific facts left out here (positions, distances, pilot sites) are there too;
|
||||
- Q-numbers refer to its §7, and phases P0–P5 to its §8.
|
||||
|
||||
Two dashboards replace the earlier tab-per-site design; nothing in them is specific to a site.
|
||||
Nothing in the dashboards is specific to a site.
|
||||
|
||||
## 1. Capabilities of the installed Superset 6.1.0
|
||||
|
||||
Source: read-only inspection of `config.py`, `superset_config.py` and the frontend bundle on
|
||||
24 Oct 2026. Items marked *(local test)* are confirmed in P2 before the design relies on them.
|
||||
24 Oct 2026. The former *(local test)* items were tested in P2 (§8); rows marked *(P2)* changed as a result.
|
||||
|
||||
| Requirement | What 6.1.0 provides | Choice |
|
||||
|---|---|---|
|
||||
| Map chart | `deck_scatter` (deck.gl Scatterplot): a modern plugin with the InteractiveChart behaviour. Also: deck_multi, deck_screengrid / hex / grid, country_map, world_map, cartodiagram, legacy `mapbox` | **deck.gl Scatterplot** |
|
||||
| Basemap | `DECKGL_BASE_MAP` unset → OSM raster (`tile.openstreetmap.org`), OSM Swiss topo, Mapbox styles. `MAPBOX_API_KEY` is empty, so the Mapbox styles fail. CSP `connect-src` already allows the OSM host; Traefik sets no CSP | set `DECKGL_BASE_MAP` to **OSM only**; "© OpenStreetMap contributors" in a Markdown under the map |
|
||||
| OSM terms | no key, no cost; tile usage policy: light use, attribution, no bulk download, a valid Referer (ours sends the origin) | fine for an internal user base; viewers' browsers reveal the viewed area (tile coordinates) and their IP to OSMF, not our data. Self-hosted tiles later if needed |
|
||||
| Tooltip | "Tooltip contents" (drag columns) plus "Customize tooltips template" (Handlebars). `js_tooltip` exists but `ENABLE_JAVASCRIPT_CONTROLS` is False | contents and template; **no JavaScript** |
|
||||
| Marker colour | "Categorical color" by a dimension, from the colour scheme; a fixed colour per label through the dashboard's `label_colors` *(local test)* | colour by `baseline_status_label`. Fallback: `deck_multi` with one fixed-colour scatter layer per status |
|
||||
| Marker size | fixed radius, with minimum = maximum pixel radius | uniform |
|
||||
| Tooltip | "Tooltip contents" (drag columns) plus "Customize tooltips template" (Handlebars). `js_tooltip` exists but `ENABLE_JAVASCRIPT_CONTROLS` is False. The contents show column names, not labels, and leave out the colour column | **contents only** *(P2)*: short tooltip-only calculated columns (`code`, `name`, `l_per_h`, `method`, `status`, `validity`, `latest_record`); no template, no JavaScript |
|
||||
| Marker colour | "Categorical color" by a dimension, from the colour scheme; a fixed colour per label through the dashboard's `label_colors` (confirmed in P2) | colour by `baseline_status_label` |
|
||||
| Marker size | fixed radius, with minimum = maximum pixel radius. No outline: the layer is drawn with `stroked: false` | uniform, no outline *(P2)* |
|
||||
| Text cards | The Handlebars chart compiles its template with `eval`; Superset's own CSP (`script-src 'self' 'strict-dynamic'` + nonce) blocks that, so the chart shows a CSP error instead *(P2)* | **no Handlebars**: one-row tables. The CSP stays as it is |
|
||||
| Chart descriptions | "Show description" (`expanded_slices`) never renders in 6.1: the dashboard loads `description_markeddown`, the chart holder reads `description_markdown` *(P2)* | descriptions stay on the charts; the visible reading notes are a Markdown block at the top of each tab |
|
||||
| Viewport | `autozoom` fits the filtered points | on |
|
||||
| Clustering | none in deck.gl; only the legacy `mapbox` chart (`clustering_radius`, needs a Mapbox key) | **no clustering** (§3) |
|
||||
| Marker click | emits a **cross-filter** on the latitude and longitude columns: the exact point, or a box around the click that shrinks as you zoom in. It applies to other charts on the same dashboard | cross-filters the Sites list |
|
||||
| Navigation | **no chart-to-dashboard navigation.** A dashboard reads `?native_filters=<rison>` into preselected native filter values (also `native_filters_key`). Table cells render sanitized HTML (`allow_render_html`) | an "Open site detail" link column in the Sites list |
|
||||
| Native filters | Select (single or multi, required, searchable, "first value by default"), Time range; scoping per chart and tab; cross-filter scoping per chart | §4, §5 |
|
||||
| Markdown iframe *(28 Oct)* | `HTML_SANITIZATION` is on and strips an iframe from a Markdown block; `HTML_SANITIZATION_SCHEMA_EXTENSIONS` lets named tags and attributes through. The CSP (`default-src 'self'`, no `frame-src`) allows frames from this origin only | allow `iframe` with `src`, `title`, `loading`, `style`; the fleet-map view is served on the same origin under `/views` (§3) |
|
||||
| Deep link into a tab *(28 Oct)* | `#TAB-<id>` after the URL opens that tab; it combines with `?native_filters=` (tested 28 Oct) | the "Site details" button opens the Site details tab with the site preset |
|
||||
|
||||
*(28 Oct)* The map moved out of Superset into the fleet-map view (§3). The rows Map chart,
|
||||
Basemap, Tooltip, Marker colour, Marker size, Viewport, Clustering and Marker click describe the
|
||||
deck.gl option the dashboard used until then; the view's choices are in §3. The Navigation row
|
||||
still holds: the view builds the same `?native_filters=` link.
|
||||
|
||||
No custom frontend and no patch are needed.
|
||||
|
||||
## 2. Navigation flow
|
||||
## 2. Navigation flow *(retired, PD-13: history)*
|
||||
|
||||
*(28 Oct)* One dashboard, two top-level tabs. *(29 Oct)* They are named **Map** (`TAB-fleet`) and
|
||||
**Insights** (`TAB-site`), the dashboard's lenses; earlier text in this document calls them Fleet and
|
||||
Site details. Readers see them as a lens switch, centred in the navbar (§6).
|
||||
|
||||
```
|
||||
ATOMA — Sites (landing, slug atoma-sites)
|
||||
filters: Site search · Zone · Province · Pilot status (default: active) · Baseline status
|
||||
M1 map: markers fitted to the filtered sites ──hover──► tooltip
|
||||
└─click marker──► cross-filter: M3 Sites list shows that site (or the few under the click)
|
||||
M3 Sites list ─────────────"Open site detail"──► ATOMA — Site Detail ?native_filters=(site preset)
|
||||
M4/M5 Sites without an accepted location ──────"Open site detail"──► same
|
||||
ATOMA — Site Detail (slug atoma-site-detail)
|
||||
filters: Site (exactly one, searchable) · Viewing period
|
||||
tabs: Summary · Fuel & Deliveries · Runtime & Electricity · Baseline & Evidence
|
||||
header link "← Sites map"
|
||||
ATOMA (slug atoma)
|
||||
filters, in Superset's left column: Site (exactly one, searchable) · Viewing period
|
||||
— both out of scope on Map; readers have no column there (§6)
|
||||
[Map] the fleet-map view (iframe, /views/fleet-map/), the page's full width
|
||||
towers fitted to the placed sites ──hover or click──► popup
|
||||
"Find a site" (code or name) ──► the map flies to it ──► its popup
|
||||
└─"Site details"──► /superset/dashboard/atoma/?native_filters=(site preset)#TAB-site
|
||||
sites without an accepted location: counted in the map's key ("Choose them in Insights",
|
||||
the view's tab_label parameter); chosen in the Site filter
|
||||
[Insights] D0 site header, then the tabs
|
||||
Summary · Fuel & Deliveries · Runtime & Electricity · Baseline & Evidence
|
||||
(a row of section buttons, not a second row of tabs: dashboard.css)
|
||||
```
|
||||
|
||||
## 3. Sites Map (landing dashboard)
|
||||
The filters sit in Superset's left column (`filter_bar_orientation: VERTICAL`) *(owner,
|
||||
29 Oct; `build_bundle.py` checks it)*. A filter shows in the column on the tabs where it is in
|
||||
scope; the others are listed under "Filters out of scope" (the Viewing period on Summary, for
|
||||
example). No filter applies to the Fleet tab, so readers have no column there and the map takes
|
||||
the full width (`reader.css`, §6); admins keep the column on both tabs. The column opens
|
||||
collapsed, a 32 px strip with an expand button (`FILTERBAR_CLOSED_BY_DEFAULT`,
|
||||
`superset_config.py`; owner, 29 Oct); the selected site shows in the Site table above the
|
||||
sections. So for a reader the tabs move sideways by 32 px between Fleet and Site details, or by
|
||||
the column's width once it is expanded; the owner accepted that.
|
||||
|
||||
From 28 to 29 Oct the filters sat in a row above the tabs (`HORIZONTAL`). On the Fleet tab that
|
||||
row held only an empty "More filters", and hiding it there made the tabs jump down on Site
|
||||
details. Superset 6.1 has no per-tab setting for the filter bar; it hides it only per link
|
||||
(`?show_filters=0`).
|
||||
|
||||
Until 28 Oct there were two dashboards: "ATOMA - Sites" (slug `atoma-sites`: a deck.gl map, five
|
||||
filters, a sites list) and "ATOMA - Site Detail" (slug `atoma-site-detail`), linked by "Open site
|
||||
detail" and "← Sites map".
|
||||
|
||||
## 3. Fleet tab: the fleet-map view *(28 Oct; retired, PD-13: history)*
|
||||
|
||||
The Fleet tab holds one Markdown block with an iframe of the fleet-map view, a page of
|
||||
petrolyst-platform (`views/src/fleet-map/`) served under `/views/` on Superset's own origin. The
|
||||
view reads the read service's `site_status` operation (`/views/api/v1/site-status`, from
|
||||
`public.v_site`). It is not a Superset chart: it has no dataset and no native filter, and both
|
||||
filters are out of scope on this tab.
|
||||
|
||||
```
|
||||
[Filter bar: Site search · Zone · Province · Pilot status · Baseline status]
|
||||
[M1 Map (deck.gl Scatterplot) ................................ w8][M2 Status key + attribution w4]
|
||||
[M3 Sites list (table with link column) ..................................................... w12]
|
||||
[M4 Sites without accepted location: count w3][M5 Sites without accepted location (table) .. w9]
|
||||
[Fleet tab: MARKDOWN-fleet-map, w12 h100]
|
||||
<iframe src="/views/fleet-map/?detail=/superset/dashboard/atoma/&filter=NATIVE_FILTER-site&tab=TAB-site">
|
||||
```
|
||||
|
||||
| # | Question | Type | Dataset | Content | Missing or insufficient |
|
||||
|---|---|---|---|---|---|
|
||||
| M1 | Where are the sites, and what is each one's baseline status? | deck.gl Scatterplot; lat/lon from `latitude`, `longitude`; uniform size; colour = `baseline_status_label`; autozoom | v_site | tooltip: `site_code`, `site_name`, `lph_estimate` ("not available" when NULL), `lph_method`, `baseline_status_label`, `baseline_validity_note` (when set), `latest_record_on` | sites without an accepted location have NULL lat/lon and are not plotted; M4/M5 show them |
|
||||
| M2 | What do the colours mean? | Markdown (static) | — | the status definitions below; "Colours show evidence status, not performance"; OSM attribution | — |
|
||||
| M3 | Which sites match, and how do I open one? | Table; search box; server pagination | v_site | code, name, province, pilot status, baseline status, L/h (+ method), L/day, latest record, location status, `detail_link` | marker click narrows it; "not available" shown as text |
|
||||
| M4 | How many sites lack an accepted location? | Big Number Total, COUNT(*) with saved filter `location_status <> 'accepted'` | v_site | count; follows the dashboard filters | 0 is shown as 0 |
|
||||
| M5 | Which sites are they? | Table; search box | v_site, same saved filter | code, name, province, `location_status` (awaiting acceptance · missing), baseline status, `detail_link` | — |
|
||||
| Part | Content | Missing or insufficient |
|
||||
|---|---|---|
|
||||
| Map | MapLibre on OpenFreeMap Positron: OpenStreetMap data, English and Dari place names (chosen by the owner on 28 Oct after a side-by-side with OSM standard tiles, which carry the same data). Fitted to the placed sites, clear of the key | no placed site: Afghanistan, and the key says so |
|
||||
| Towers | one per site with an accepted location: a pin holding a lattice tower, in the site's baseline status colour (below); a site that is not an active pilot site faded; uniform size. The site's identity is `site_uid`, since two sites can share a `site_code` (an alias cut to 20 characters) | sites without an accepted location are not drawn |
|
||||
| Popup | opens 120 ms after hover; stays while the pointer is on the tower or the popup; closes 250 ms after it leaves both. A click, tap or keyboard focus pins it; Esc or a click on the map closes it. It springs out of the tower (a fade only with reduced motion) and takes the side with room; when neither side has room for all of it, the side with more room, where its figures scroll while the header, the status with any validity note, and **Site details** stay in view. Content: code, name; status as `baseline_status_text` ("not approved" for Eligible and Provisional); any `baseline_validity_note` right beside the status and the L/h; L/h with method ("not available" when NULL); L/day, "whole record, not per running hour"; latest record; pilot status by name (active, reference, candidate, withdrawn, or not in the pilot); other sites whose towers cover this one (the same or nearly the same position), each a button that switches the popup to it; **Site details** | an approved estimate whose inputs changed is never shown without its validity note |
|
||||
| Site details | a link (`target="_top"`) to this dashboard with the site preset in the Site filter, on the Site details tab: the `?native_filters=` value of §5 plus `#TAB-site`. The iframe's address names the dashboard, the filter and the tab, and `build_bundle.py` checks them | no link when that address is missing or malformed: the view accepts only a dashboard path and filter and tab ids of a fixed shape |
|
||||
| Search | "Find a site: code or name", top left. Matches ignore case: code prefixes first, then codes containing the text, then names containing it; at most 8, each with its status dot and status. Arrow keys and Enter, or the pointer, choose one; the map flies to it at zoom 14 or closer (past the clusters) and pins its popup. Esc clears the field | sites without an accepted location are not searched (owner's choice, 28 Oct): the key counts them and the Site filter finds them; "No site matches" when nothing does |
|
||||
| Operator's logo | ATOMA's logo in its own colours, bottom right above the map's credits, on frosted glass (*owner, 29 Oct*). The dashboard passes it in the iframe's address (`logo=/static/assets/branding/atoma_logo.svg`, `logo_alt=ATOMA`; `build_bundle.py` checks the file exists); the view takes only an image from that folder on its own origin, so it carries no operator's branding | no `logo` in the address: no logo |
|
||||
| Key | the four statuses; "Faded: not an active pilot site"; "Colours show evidence status, not performance"; the number of sites without an accepted location, with "Choose them in Site details" | — |
|
||||
| Keyboard, screen readers | a hidden list of the sites with the same links; focusing one opens its popup | — |
|
||||
|
||||
**Filters** (all on `v_site`, scoped to M1, M3, M4 and M5):
|
||||
- Site search: Select on `site_label`, multi, searchable. The label holds code, name and uid, so
|
||||
code or name both match.
|
||||
- Zone, Province: Select.
|
||||
- Pilot status: default `active`.
|
||||
- Baseline status: Select on `baseline_status_label`.
|
||||
|
||||
The cross-filter from M1 is scoped **to M3 only**. M4 and M5 hold the sites without coordinates and
|
||||
must not be emptied by it.
|
||||
|
||||
**Baseline status and marker colours.** Colours are neutral categorical hues, with no red or green
|
||||
**Baseline status and tower colours.** Colours are neutral categorical hues, with no red or green
|
||||
and no reference to L/h:
|
||||
|
||||
| Status | Definition (`fm.v_site_baseline_status`) | Colour |
|
||||
@@ -90,24 +131,26 @@ and no reference to L/h:
|
||||
| Provisional | no approval; a proposed version below BP-1, **or** usable intervals with no version yet (no estimate shown, method "no baseline version proposed") | amber #c98500 |
|
||||
| Insufficient evidence | no usable interval. **Every pilot site while no runtime evidence exists** | slate #64748B |
|
||||
|
||||
**Nearby sites.**
|
||||
- Sites a few kilometres apart separate at zoom ≥ 11 (≈ 60 px per 2–4 km at zoom 12) and coincide
|
||||
at country zoom (≈ 2 km per pixel at zoom 6). The pilot's own distances are in the design doc.
|
||||
- Handling:
|
||||
- autozoom to the filtered sites (the default "active" filter opens on the pilot area);
|
||||
- small uniform markers (min = max radius, about 7 px) with an outline;
|
||||
- a click near overlapping markers selects every site in the click box, and M3 lists them;
|
||||
- no clustering.
|
||||
- With hundreds of sites, a `deck_multi` density layer (screengrid) for low zoom is a later option.
|
||||
**Nearby sites.** Towers that would overlap cluster into a teal circle with their count (radius
|
||||
36 px, up to zoom 13); a click on it zooms in until they separate. The pilot's sites, a few
|
||||
kilometres apart, separate at the zoom the map opens with. The towers are one symbol layer, so
|
||||
thousands of sites stay fast.
|
||||
|
||||
## 4. Site Detail (one dashboard, any site)
|
||||
**Sites without an accepted location** are not plotted. The key counts them, and the Site filter
|
||||
in Site details lists every site, placed or not.
|
||||
|
||||
## 4. Site details: the four tabs of `atoma-site` (any site)
|
||||
|
||||
*(PD-13)* What follows about D0 (the site header row) and the Fleet tab is history: the app shows
|
||||
the site's header, and its Map lens replaced the Fleet tab.
|
||||
|
||||
**Filters**
|
||||
- **Site** (`NATIVE_FILTER-site`):
|
||||
- Select on `v_site.site_label`, single value, **required**, searchable, no default;
|
||||
- Select on `v_site.site_label`, single value, **required**, searchable,
|
||||
**first value by default** with `requiredFirst` (the §4 fallback, adopted in P2);
|
||||
- scope: every chart on every tab;
|
||||
- every detail dataset carries `site_label` from `v_site`, so this one filter drives all of
|
||||
them *(local test)*.
|
||||
them (confirmed in P2: every chart request carries the selected site).
|
||||
- **Viewing period** (`NATIVE_FILTER-period`):
|
||||
- Time range, default no filter;
|
||||
- scope: the time-series charts on Fuel & Deliveries and Runtime & Electricity only;
|
||||
@@ -115,24 +158,31 @@ and no reference to L/h:
|
||||
baseline's fixed window.
|
||||
|
||||
**Exactly one site, and never network totals:**
|
||||
1. A required, single-value filter *(local test: with the value cleared, no chart queries
|
||||
unfiltered data)*.
|
||||
2. The header (D0) and KPI (S1) are Handlebars templates that render a site only when exactly one
|
||||
row arrives. Otherwise they show "Select one site".
|
||||
1. A required, single-value filter. *P2: "required" alone does not stop the charts: with no
|
||||
value they queried every site.*
|
||||
2. Therefore "Select first filter value by default", with `requiredFirst`, as the filter editor
|
||||
writes it: the charts wait for the filter, the first site is applied without "Apply filters",
|
||||
and "Clear all" stays disabled. Without `requiredFirst` the first value waits behind "Apply
|
||||
filters" while the charts have already queried every site.
|
||||
3. No Big Number or aggregate tile for site figures. They are computed per site in `v_site`,
|
||||
never aggregated by Superset.
|
||||
4. Fallback, if (1) fails its test: "Select first filter value by default", so the Site filter
|
||||
is never empty.
|
||||
never aggregated by Superset. *(29 Oct, PD-12: the app's Overview, §10, adds up recorded
|
||||
amounts across sites; per-site rates are still never averaged.)*
|
||||
4. The Handlebars "Select one site" guard is gone with Handlebars (§1); with (2) the filter is
|
||||
never empty.
|
||||
|
||||
```
|
||||
[D0 Site header: code + name (large) · province · pilot status · baseline status · "← Sites map"]
|
||||
Summary [S1 L/h KPI card w6][S2 Supporting measures w6]
|
||||
[D0 Site: one-row table, code · name · province · pilot status · baseline status · location] (28 Oct: inside Site details, no link)
|
||||
every tab [Reading notes (Markdown) w12] (P2)
|
||||
Summary [S1 Fuel per running hour: one-row table w12] (P2)
|
||||
[S2 Supporting measures w12]
|
||||
[S3 Evidence coverage timeline (Gantt) w12]
|
||||
Fuel & Deliveries [F1 Deliveries w6][F2 Recorded stock w6]
|
||||
[F5 Deliveries table w12] (P2)
|
||||
[F3 Fuel per calendar day by interval w12]
|
||||
[F4 Interval table w12]
|
||||
Runtime & Electricity[R1 Runtime per day by meter interval w6][R2 Hour-meter evidence w6]
|
||||
[R3 Grid energy, kWh w6][R4 Electricity cost, AFN w6]
|
||||
[R1T Runtime pairs and their validity w12] (P2)
|
||||
[R3 Grid energy, kWh w6][R4 Electricity cost, AFN: bar over table w6]
|
||||
Baseline & Evidence [B1 Baseline versions w6][B2 Inputs of each version w6]
|
||||
[B3 All intervals: eligibility and exclusions w12]
|
||||
[B4 Interval L/h against baseline w12]
|
||||
@@ -144,17 +194,17 @@ Baseline & Evidence [B1 Baseline versions w6][B2 Inputs of each version w6]
|
||||
|
||||
| # | Question it answers | Type | Dataset | Dims / measures (units) | Grain | When data are missing or insufficient |
|
||||
|---|---|---|---|---|---|---|
|
||||
| D0 | Which site am I looking at? | Handlebars | v_site | code and name (large), province, pilot status, baseline status, location status, link "← Sites map" | site | "Select one site" unless exactly one row |
|
||||
| S1 | What is the site's L/h, by which method, over which window, how strong? | Handlebars | v_site | `lph_estimate`, `lph_method`, `baseline_status_label`, `baseline_validity_note`, window, decision; L/day whole record | site | Keeps its position. **In v1 every site shows "Insufficient runtime evidence: no hour-meter readings"**, plus L/day labelled "per calendar day, not per running hour". "Eligible" and "Provisional" carry "not approved" |
|
||||
| D0 | Which site am I looking at? | Table, one row *(P2)* | v_site | code, name, province, pilot status, baseline status, location status *(28 Oct: the link "← Sites map" is gone; the Fleet tab is one click away)* | site | the Site filter is never empty (§4) |
|
||||
| S1 | What is the site's L/h, by which method, over which window, how strong? | Table, one row *(P2)* | v_site | `lph_estimate`, `lph_method`, `baseline_status_label`, `baseline_validity_note`, window, decision; L/day whole record | site | Keeps its position. **In v1 every site shows "Insufficient runtime evidence: no hour-meter readings"**, plus L/day labelled "per calendar day, not per running hour". "Eligible" and "Provisional" carry "not approved" |
|
||||
| S2 | What supports it? | Table | v_kpi_measures | measure, value, unit, `value_kind` (recorded · calculated · assumed · not established), window. Two L/day rows: the whole record (moves with new records) and the L/day reference version (fixed window, its status and any input drift in the note) | site × measure | Rows stay and show "not established" |
|
||||
| S3 | Which periods are covered by which evidence, and where are the gaps? | Gantt | v_coverage | lane; start/end; colour = status | span | Empty lanes are visible; nothing is interpolated |
|
||||
| F1 | When and how much was delivered? | Bar (time axis, **day** grain) | v_delivery (one row per delivery event) | `delivery_date`; **SUM(`delivered_l`)**, titled "Litres delivered per day (all deliveries that day)"; tooltip COUNT(deliveries) | day | Bars only on delivery dates; subtitle "deliveries are not consumption"; F4 lists the individual deliveries |
|
||||
| F2 | What stock was recorded? | Scatter (no lines) | v_stock_reading | `observed_on`; litres; series = reading type × `value_kind` | observation | Points only; nothing between readings |
|
||||
| F3 | How much fuel was used per calendar day in each observed interval? | Bar (categorical x, ordered by start) | v_interval | `interval_label`; MAX(`fuel_l_per_day`) (L/d); colour = `stock_status` | stock interval | Excluded intervals greyed, with the reason in the tooltip; no monthly spreading |
|
||||
| S3 | Which periods are covered by which evidence, and where are the gaps? | Gantt | coverage_detail *(P2: virtual dataset over v_coverage)* | lane name without the view's sort number; start/end; colour = status; tooltip: litres delivered in a stock interval, or a bill's AFN amount | span | Nothing is interpolated. *(P2)* The view emits a lane only where spans exist, so the tab's notes name the four lanes and say that a missing lane means no span could be formed: no evidence of that kind, or too few readings (F2, R2 show single readings) |
|
||||
| F1 | When and how much was delivered? | Bar (time axis, **day** grain) | v_delivery (one row per delivery event) | `delivery_date`; **SUM(`delivered_l`)**, titled "Litres delivered per day (all deliveries that day)" | day | Bars only on delivery dates; "deliveries are not consumption" in the tab's notes. *(P2)* The count and the individual deliveries are in **F5**, a table on v_delivery (date, visit of the day, event, litres, price, sources, review), which also lists a delivery of unknown quantity; its search box shows the number of records |
|
||||
| F2 | What stock was recorded? | Scatter (no lines) | v_stock_reading | `observed_on`; litres; series = reading type × `value_kind`, split by visit when a day has several *(P2)* | observation | Points only; nothing between readings |
|
||||
| F3 | How much fuel was used per calendar day in each observed interval? | Bar (categorical x, ordered by start) | v_interval | `interval_label`; MAX(`fuel_l_per_day`) (L/d); colour = `stock_status` | stock interval | Intervals that are not `ok` greyed (*P2*: fixed grey per status in Site Detail's `label_colors`); the status is the series name, so legend and tooltip give the reason; no monthly spreading |
|
||||
| F4 | Interval detail | Table | v_interval | dates, opening, deliveries, closing, used, L/d, status | interval | Shows the reasons |
|
||||
| R1 | How many hours did each generator run per day? | Bar (categorical) | v_runtime | interval × meter; MAX(h/day). h/day is NULL for an invalid pair (`pair_valid` false: two epochs, a decreasing timer or over 24 h/day); such pairs show no bar and are listed with their flags in the tooltip | meter interval | **v1: empty**, with "No hour-meter readings" in the description |
|
||||
| R1 | How many hours did each generator run per day? | Bar (categorical) | v_runtime | interval × meter; MAX(h/day). h/day is NULL for an invalid pair (`pair_valid` false: two epochs, a decreasing timer or over 24 h/day); such pairs show no bar and are listed with their flags in **R1T**, a pair table with `pair_valid`, `same_epoch`, `monotonic`, `over_24h_per_day` *(P2: the bar tooltip cannot show them)* | meter interval | **v1: empty**, with "No hour-meter readings" in the tab's notes |
|
||||
| R2 | What runtime evidence exists? | Table | v_hour_meter_evidence | meter, mapped set, date, hours, state, epoch | reading | **v1: no rows**, "No hour-meter readings" |
|
||||
| R3 | How much grid energy did the site receive? | Handlebars → bar once kWh exist | v_kpi_measures (kWh row) | kWh per bill month | month | "No kWh records; bills are cost only and are not converted to fuel or hours" |
|
||||
| R3 | How much grid energy did the site receive? | Table, the kWh row *(P2)*; a bar once kWh exist | v_kpi_measures (kWh row) | kWh per bill month | month | "No kWh records; bills are cost only and are not converted to fuel or hours" |
|
||||
| R4 | What did grid electricity cost? | Bar (month) + table | v_electricity_cost | bill month; amount (AFN, inferred) | month | Own chart, never on a fuel axis. A disputed bill is listed without amount or kWh |
|
||||
| B1 | Which baseline versions exist, and who approved them? | Table | v_baseline_version | code, status, value, window, grade, decision | version | "No baseline defined" |
|
||||
| B2 | What went into each version? | Table | v_baseline_input | interval, included, reason, fuel, runtime, drift | version × interval | empty until a version exists |
|
||||
@@ -164,24 +214,31 @@ Baseline & Evidence [B1 Baseline versions w6][B2 Inputs of each version w6]
|
||||
| B6 | What is missing or inconsistent? | Table | v_data_quality | check, status, detail | check | Lists the gaps: no runtime, placeholder rows, no kWh, no transfer records, location awaiting acceptance |
|
||||
| B7 | Which documents do the figures rest on? | Table | v_evidence | title, kind, issuer, date, used for | document | — |
|
||||
| B8 | What did the operator workbook calculate, and is it usable? | Table | v_imported_calculation | label, formula, value, our reading, use, reason, `claim_status` | cell | Shows "excluded" with the reason for "actual L/h" and "CP L/h". A disputed transcription shows its status, without formula or value |
|
||||
| B9 | Does stock depletion match engine-measured fuel? | Table | v_reconciliation | stock L, engine L, `covered_h` of `interval_h`, difference L and %. The window runs from the opening visit's departure to the closing visit's arrival. The difference is shown only for an `ok` stock interval with both visit times recorded, in which every generator is placed at the site throughout and covered for the whole window | interval × generator | "No flow-meter data yet"; partial coverage, unknown visit times or a mid-interval placement show the hours covered and no difference |
|
||||
| B9 | Does stock depletion match engine-measured fuel? | Table | v_reconciliation | engine L of this generator, `covered_h` of `interval_h`; site-wide: stock L, engine L of all generators, `all_generators_metered`, difference L and % *(P2: both engine totals shown, so the difference reconciles)*. The window runs from the opening visit's departure to the closing visit's arrival. The difference is shown only for an `ok` stock interval with both visit times recorded, in which every generator is placed at the site throughout and covered for the whole window | interval × generator | "No flow-meter data yet"; partial coverage, unknown visit times or a mid-interval placement show the hours covered and no difference |
|
||||
| B10 | What does the equipment specification say? | Table | v_spec_reference | attribute, value, unit, basis, applicability, `claim_status` | spec | Titled "Manufacturer and nameplate data, not site performance". A disputed specification shows its status, without value |
|
||||
| B11 | What would L/h be under the stated runtime? | Table | v_modelled_lph | assumption code, status, basis; h/day with low–high; window, `matched_days`; L/day; modelled L/h with low–high; `value_kind` | site × assumption | Titled "Modelled from an assumed runtime, not measured". "No runtime assumption recorded" |
|
||||
|
||||
Modelled L/h (B11) is in v1 (Q9, decided 24 Oct 2026). It pools only the usable intervals inside
|
||||
the runtime assumption's dates, and shows that window and its days. It never reaches S1, the map or
|
||||
the M3 Sites list.
|
||||
the runtime assumption's dates, and shows that window and its days. It never reaches S1 or the
|
||||
fleet map.
|
||||
|
||||
## 5. Datasets and filter mapping
|
||||
|
||||
Physical datasets, one per `public.v_*` view on Trino `fuel.public`. Every dataset has the same
|
||||
Physical datasets, one per `public.v_*` view in fuel-db's `public` schema, plus one virtual dataset
|
||||
*(P2)*. `coverage_detail` adds two columns to `v_coverage`:
|
||||
- `lane_name`, the lane without its sort number, which read as a count;
|
||||
- `amount`, a stock interval's delivered litres (from its opening visit up to its closing visit,
|
||||
so the opening refill is included) or a bill's AFN amount.
|
||||
|
||||
Both use only `v_coverage`, `v_delivery` and `v_electricity_cost`. Every dataset has the same
|
||||
four site columns from `v_site`: `site_uid`, `site_code`, `site_name`, `site_label`.
|
||||
|
||||
| Dataset | Grain | Used by | Native filters that apply |
|
||||
|---|---|---|---|
|
||||
| v_site | site | M1–M5, D0, S1 | landing: all five filters, plus the M1 cross-filter (on M3); detail: Site |
|
||||
| v_site | site | D0, S1; the fleet-map view through the read service *(28 Oct)* | Site |
|
||||
| v_kpi_measures | site × measure | S2, R3 | Site |
|
||||
| v_coverage | site × lane × span | S3 | Site |
|
||||
| v_coverage | site × lane × span | — (S3 reads `coverage_detail`) | Site |
|
||||
| coverage_detail *(P2)* | site × lane × span | S3 | Site |
|
||||
| v_delivery | delivery | F1 | Site, Viewing period |
|
||||
| v_stock_reading | observation | F2 | Site, Viewing period |
|
||||
| v_interval | stock interval | F3, F4, B3, B4 | Site; Viewing period on F3 and F4 only |
|
||||
@@ -198,79 +255,339 @@ is grouped or filtered by `metric` and `population_key` (both in `v_interval`),
|
||||
across methods or generator sets. The site's L/h figure itself comes only from a baseline version
|
||||
(`v_site`), never from this metric. There is no AVG metric on ratio columns.
|
||||
|
||||
**Link column.** `detail_link` is a calculated column on the `v_site` dataset, defined in the
|
||||
bundle in Trino SQL. The native filter id stays inside Superset's own configuration:
|
||||
**Site details link** *(28 Oct; history since PD-13)*. The fleet-map view built it
|
||||
(petrolyst-platform `views/src/shared/deepLink.ts`, retired) from its iframe address: the
|
||||
dashboard path, the Site filter's id and the tab. The Petrolyst app uses the same rison shape for
|
||||
the Site filter's preset (`app/src/app/core/site-filter.ts`). The rison shape is the one the sites
|
||||
list used until 28 Oct, confirmed in P2, and the view's tests compared it with that list's Trino
|
||||
output:
|
||||
|
||||
```sql
|
||||
'<a href="/superset/dashboard/atoma-site-detail/?native_filters='
|
||||
|| url_encode('(NATIVE_FILTER-site:(id:NATIVE_FILTER-site,'
|
||||
|| 'extraFormData:(filters:!((col:site_label,op:IN,val:!(' || q || ')))),'
|
||||
|| 'filterState:(value:!(' || q || ')),ownState:()))')
|
||||
|| '">Open site detail</a>'
|
||||
-- q = '''' || replace(replace(site_label, '!', '!!'), '''', '!''') || '''' -- rison string quoting
|
||||
```
|
||||
/superset/dashboard/atoma/?native_filters=<url-encoded:>
|
||||
(NATIVE_FILTER-site:(id:NATIVE_FILTER-site,extraFormData:(filters:!((col:site_label,op:IN,val:!(q)))),
|
||||
filterState:(value:!(q)),ownState:()))
|
||||
#TAB-site
|
||||
-- q = the site label in rison quotes: 'label', with ! written !! and ' written !'
|
||||
```
|
||||
|
||||
- The exact rison state shape is a *(local test)*.
|
||||
- Fallback: the link opens Site Detail with the Site filter expanded (`expand_filters=1`) and the
|
||||
site's label shown beside the link, to pick by search.
|
||||
|
||||
## 6. Build as code, scale, access
|
||||
|
||||
**Build**
|
||||
- `petrolyst-insights/dashboards/atoma/build_bundle.py` writes one import bundle containing both
|
||||
dashboards, every dataset and every chart:
|
||||
- stable UUIDs (uuid5), so a re-import updates instead of duplicating;
|
||||
- fixed native-filter ids (`NATIVE_FILTER-site`, `-period`, `-search`, …);
|
||||
- `build_bundle.py --database-uuid <uuid>` writes one import bundle containing the dashboard,
|
||||
every dataset and every chart:
|
||||
- stable UUIDs (uuid5), so a re-import updates instead of duplicating; the zip is
|
||||
byte-identical for the same inputs;
|
||||
- fixed native-filter ids (`NATIVE_FILTER-site`, `NATIVE_FILTER-period`) and tab ids
|
||||
(`TAB-fleet`, `TAB-site`, `TAB-summary`, …);
|
||||
- `label_colors` for the four statuses;
|
||||
- the existing "Fuel (Trino)" database UUID, read from an export first.
|
||||
- Import: `superset import-dashboards`.
|
||||
- every view column, from `columns.tsv` (a snapshot of PostgreSQL's catalog);
|
||||
- the existing "Fuel" database UUID, which differs per instance:
|
||||
`import_bundle.py --show-database` prints it.
|
||||
- Before writing, it checks that every column a chart or filter uses exists in its dataset,
|
||||
that every chart in a Site filter's scope has `site_label`, that the fleet map's iframe
|
||||
address names this dashboard, its Site filter and the Site details tab *(28 Oct)*, and that
|
||||
no row is wider than 12.
|
||||
- Import: `import_bundle.py <zip> --user <username>`, inside the Superset container (*P2*: not
|
||||
`superset import-dashboards`, which leaves existing datasets and charts unchanged on a
|
||||
re-import). It runs the dataset, chart and dashboard importers with overwrite on, never creates
|
||||
or changes the database, and reports columns that differ between a dataset and its view.
|
||||
- A re-import replaces edits made in the Superset UI to these objects.
|
||||
|
||||
**Adding a site**
|
||||
- Import its `site.yaml` and its records. Nothing in either dashboard names a site.
|
||||
- New sites appear in M1 (once their location is accepted) or in M5, in M3, and in the Site filter.
|
||||
- Import its `site.yaml` and its records. Nothing in the dashboard or the fleet-map view names a
|
||||
site.
|
||||
- A new site appears in the Site filter, and on the map once its location is accepted; until
|
||||
then the map's key counts it.
|
||||
|
||||
**Hundreds of sites**
|
||||
- `v_site` has one row per site, and the scatter layer handles thousands of points.
|
||||
- The Select filters load the distinct labels; M3 and M5 paginate on the server.
|
||||
- `v_site` has one row per site; the fleet map draws its towers as one symbol layer and
|
||||
clusters overlapping ones, so thousands of sites stay fast.
|
||||
- The Site filter loads the distinct labels; with thousands of sites, check its value limit
|
||||
(open point 6 of the design decisions).
|
||||
|
||||
**Superset configuration change** (petrolyst-insights; image rebuild in P3):
|
||||
`DECKGL_BASE_MAP = [["tile://https://tile.openstreetmap.org/{z}/{x}/{y}.png", "OpenStreetMap"]]`.
|
||||
The CSP needs no change *(local test: no CSP violation in the browser console)*.
|
||||
**Superset configuration changes** (in `superset_config.py`; image rebuild in P3):
|
||||
- `DECKGL_BASE_MAP = [["tile://https://tile.openstreetmap.org/{z}/{x}/{y}.png", "OpenStreetMap"]]`
|
||||
(no deck.gl chart since 28 Oct; kept for charts made by hand);
|
||||
- *(28 Oct)* `HTML_SANITIZATION_SCHEMA_EXTENSIONS` lets an `iframe` (`src`, `title`, `loading`,
|
||||
`style`) through a Markdown block. The fleet-map view must be served on the same origin under
|
||||
`/views` (petrolyst-infra routing, not part of this step).
|
||||
|
||||
**Access, internal first.** Both dashboards for Admin and Alpha only. No Gamma role until the
|
||||
The CSP needs no change. P2: the tiles load and the map causes no CSP violation. One violation
|
||||
appears on every Superset page, including the stock dashboard list: a capability probe
|
||||
(`Function("")`) in the 6.1 frontend bundle, caught by the bundle itself.
|
||||
|
||||
**Access, internal first.** The dashboard for Admin and Alpha only; the read service behind the
|
||||
same sign-in *(28 Oct: not built yet; locally it has none)*. No Gamma role until the
|
||||
ATOMA-facing edit.
|
||||
|
||||
**ATOMA's readers** *(owner, 29 Oct; retired the same day, PD-13: readers use the Petrolyst
|
||||
app; kept as history)*. The Superset role **"ATOMA Reader"** (Keycloak client role
|
||||
`atoma-reader`) reads the ATOMA dashboard and nothing else.
|
||||
- **The role** (`petrolyst-insights/roles/atoma_reader.py`): exactly 12 permissions, set by
|
||||
`superset-init` whenever the image changes, and reset by hand with the command in the README.
|
||||
- Dashboard and Chart `can_read`;
|
||||
- Superset `can_dashboard`, `can_dashboard_permalink`, `can_log`;
|
||||
- Api `can_time_range`;
|
||||
- the filter state (`can_read` and `can_write`), permalinks (`can_read`), the current user,
|
||||
the CSRF token and the menu.
|
||||
|
||||
It has no menus, no write rights, no Explore, CSV, sharing, drill or query view. The bundle
|
||||
import attaches the role to the dashboard (DASHBOARD_RBAC).
|
||||
- **Rules a role cannot express** (`petrolyst_reader.py`, in both security managers):
|
||||
- readers own nothing, and reach a dashboard only through its roles (published, role match);
|
||||
- the dashboard list shows them only those;
|
||||
- their chart-data responses carry no SQL (`patches/fix_redact_query_for_readers.py`;
|
||||
Superset itself drops it only for embedded guests).
|
||||
- **What they see:**
|
||||
- every page they open leads to the ATOMA dashboard;
|
||||
- a stylesheet loaded only for them (`assets/branding/reader.css`) hides the dashboard's
|
||||
title row, the theme switcher, and Info and About in Settings, leaving Logout;
|
||||
- the same stylesheet hides the filter column on the Fleet tab, and while the dashboard
|
||||
loads. It sizes the map to the window: the full width at once (Superset measures widths
|
||||
again only about 0.3 s after loading), and down to 32 px above the window's bottom edge
|
||||
instead of the layout's 800 px, at least 560 px. `build_bundle.py` checks that it names the
|
||||
Fleet tab and that the map spans all 12 columns;
|
||||
- the same stylesheet turns the top-level tabs into a lens switch, centred in the navbar, with
|
||||
an icon each; the map's block then grows by the tab row's height. A scroll-linked animation
|
||||
moves the switch up with the navbar, which scrolls away. Without support for scroll-linked
|
||||
animations, or narrower than 900 px, the tab row stays in the page. From 1100 px the switch
|
||||
shows a third lens, AI, as a teaser: greyed, tagged PRO in green, drawn by the tab list
|
||||
(not a tab, not clickable; the layout keeps Map and Insights). Narrower, it would cover the
|
||||
logo. `build_bundle.py` checks that every layout id the stylesheet names exists;
|
||||
- the stylesheet's address carries a hash of its content
|
||||
(`petrolyst_reader.register`): Superset lets browsers keep static files for a year, so a
|
||||
changed file would otherwise not reach readers who loaded the old one;
|
||||
- Superset already hides the menus, "+" and editing from a role without them.
|
||||
- **One refused request remains, harmless:** Superset stores a dashboard's label colours when
|
||||
they changed (`PUT /api/v1/dashboard/<id>/colors`). A reader may not, so it is refused until
|
||||
someone with edit rights opens the dashboard; the colours show either way.
|
||||
- A reader is someone whose only roles are "ATOMA Reader" (and Public). Anyone with Admin, Alpha
|
||||
or Gamma as well sees everything as before.
|
||||
|
||||
## 7. Acceptance checks
|
||||
|
||||
*(PD-13)* Written for the retired ATOMA dashboard. Where they name D0 or the fleet map, the app's
|
||||
site header and Map lens take their place (petrolyst-platform's browser tests).
|
||||
|
||||
Run in P2 against a local Superset, and repeated as a smoke test in P3.
|
||||
|
||||
1. **New site, no new objects.** Count dashboards, charts and datasets in the metadata database.
|
||||
Then import a synthetic fourth site with records and an accepted location, and a fifth with no
|
||||
location. The counts are unchanged; site 4 appears in M1, M3 and the Site filter; site 5
|
||||
appears in M5, and M4 goes up by 1.
|
||||
location. The counts are unchanged; site 4 appears on the fleet map and in the Site filter;
|
||||
site 5 appears in the Site filter, and the map's count of sites without an accepted location
|
||||
goes up by 1 *(28 Oct)*.
|
||||
2. **Site switching.** In Site Detail, select pilot site A, then site B. D0 shows the new code and
|
||||
name; every chart's query (View query, or chart-data API) carries `site_label = …<site B>…`; no
|
||||
chart keeps site A's rows.
|
||||
3. **Clearing the selection.** With the Site filter cleared, no chart shows multi-site data; D0 and
|
||||
S1 show "Select one site". If not, switch to the §4 fallback and re-test.
|
||||
4. **Unavailable L/h.** For each pilot site:
|
||||
- the tooltip and M3 say "not available" with the method "no runtime evidence";
|
||||
- the fleet map's popup says "not available" with the method "no runtime evidence" *(28 Oct)*;
|
||||
- S1 shows the insufficient-evidence state plus L/day;
|
||||
- B3 shows every interval excluded, with its reason;
|
||||
- B11 shows a modelled range labelled "not measured", and that figure appears nowhere else.
|
||||
5. **Missing coordinates.** A site without an accepted location is not plotted, is listed in M5
|
||||
and counted in M4, and its link opens its detail.
|
||||
5. **Missing coordinates.** A site without an accepted location is not plotted, is counted in the
|
||||
map's key, and can be chosen in the Site filter *(28 Oct)*.
|
||||
6. **Filter propagation.**
|
||||
- On the landing page, each filter changes M1, M3, M4 and M5, and the default "active" shows
|
||||
exactly the active pilot sites.
|
||||
- A marker click filters M3 only.
|
||||
- In Site Detail, the Viewing period changes F1–F3 and R1, R2, R4, but not S1 or B1–B4.
|
||||
7. **Detail links.** For each active pilot site, "Open site detail" in M3 opens Site Detail with
|
||||
exactly that site selected, and D0 matches. Same from M5.
|
||||
- *(28 Oct)* Both filters are out of scope on the Fleet tab; the fleet map shows every placed
|
||||
site, sites that are not active pilot sites faded.
|
||||
- In Site details, the Viewing period changes F1–F5, R1, R1T, R2, R4 and R4T, but not S1 or
|
||||
B1–B4.
|
||||
7. **Detail links** *(28 Oct)*. For each placed site, "Site details" in its popup opens the Site
|
||||
details tab with exactly that site selected and Summary open; D0 matches, and every chart's
|
||||
request carries that site.
|
||||
8. **Colours.**
|
||||
- Fixtures in each status show the four colours.
|
||||
- "Eligible" never renders as "Approved" unless the fixture has an approved version with a
|
||||
decision reference.
|
||||
- Marker size is identical across sites.
|
||||
9. **Basemap.** OSM tiles load, there are no CSP violations in the console, and the attribution
|
||||
is visible.
|
||||
- *(28 Oct)* An approved site whose inputs changed shows its validity note beside the status
|
||||
and the L/h in the popup.
|
||||
9. **Basemap.** The fleet map's tiles load, the attribution is visible, and neither the view nor
|
||||
the dashboard adds a CSP violation (Superset's own probe, §6, stays) *(28 Oct)*.
|
||||
|
||||
## 8. P2 results (local, 24 Oct 2026)
|
||||
|
||||
Local Superset 6.1.0 image built from this repository, Trino 484, fuel-db with migrations
|
||||
0001–0003 and the corpus data loaded; browser checks in headless Chrome.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| 1 New site | pass: with a synthetic site with an accepted location and one without, the object counts stay unchanged (2 dashboards, 28 charts before R1T was added, 18 datasets); the first is in M1, M3 and the Site filter, the second in M5, and M4 goes from 0 to 1. Fixtures removed afterwards |
|
||||
| 2 Site switching | pass: for each active pilot site, every chart request on the four tabs carries that site's `site_label`, all HTTP 200 (25 per site; F5, added later, re-checked for one site) |
|
||||
| 3 Clearing | pass with the fallback (§4): "required" alone let the charts query every site |
|
||||
| 4 Unavailable L/h | pass: tooltip and M3 "not available", method "Not available: no runtime evidence"; S1 the same plus L/day; B3 every interval excluded with its reason; B11 the modelled range, labelled "not measured", nowhere else |
|
||||
| 5 Missing coordinates | pass (synthetic site, check 1): not plotted, in M4 and M5, its link opens its detail |
|
||||
| 6 Filter propagation | pass: the default "active" reaches M1, M3, M4, M5; a marker click re-queries M3 only, with a latitude/longitude box; the Viewing period reaches F1–F5, R1, R1T, R2, R4 and the bill table, and no Summary or Baseline & Evidence chart |
|
||||
| 7 Detail links | pass: from M3 for each active pilot site, and from M5 |
|
||||
| 8 Colours | pass for the mapping: a throwaway fixture with one point per status shows the four colours and equal marker sizes. Not tested by fixture: that the database never labels an unapproved version "Approved baseline" |
|
||||
| 9 Basemap | pass: OSM tiles load, attribution visible, no CSP violation from the map |
|
||||
|
||||
Changes forced by the tests and by two Codex inspection rounds of 24 Oct (F2 visits, R1T, B9
|
||||
totals, coverage lanes; then F5, grey stock statuses, lane wording), all marked *(P2)* above.
|
||||
The second round's fixes were not inspected again. The grey colour for a non-ok stock interval
|
||||
is configured but not shown by the current data, which has no such interval.
|
||||
- Handlebars charts (D0, S1, R3) became one-row tables.
|
||||
- The Site filter uses "first value by default" with `requiredFirst`.
|
||||
- Reading notes moved into a Markdown block per tab.
|
||||
- The map tooltip uses short tooltip-only columns.
|
||||
- The markers have no outline.
|
||||
|
||||
The marker palette is unchanged from §3. The dataviz palette check flags slate as reading
|
||||
grey, which is intended for "insufficient evidence", and amber at 2.99:1 contrast. The label is
|
||||
always also shown as text, in the legend, the tooltip and M3.
|
||||
|
||||
## 9. Fleet map results (local, 28 Oct 2026) *(history)*
|
||||
|
||||
Local Superset 6.1.0 rebuilt with the iframe setting, behind a local Traefik that serves Superset
|
||||
and the petrolyst-platform read service on one origin (as on the server); the fuel-db of P2.
|
||||
Browser checks with Playwright in Chrome (`petrolyst-platform/views/e2e/`).
|
||||
|
||||
**Spike, before building**
|
||||
- A Markdown iframe survives the sanitiser with the setting of §6.
|
||||
- A `target="_top"` link inside it navigates the whole dashboard page.
|
||||
- `?native_filters=…#TAB-…` selects the site and opens that tab.
|
||||
- Basemaps compared at zoom 11, 14 and 16 over the pilot area: OpenFreeMap Positron and OSM
|
||||
standard tiles show the same OpenStreetMap data; the owner chose Positron.
|
||||
|
||||
**Bundle:** 1 dashboard, 26 charts, 19 datasets; two builds are byte-identical; the import reports
|
||||
column drift 0. The two earlier dashboards and the charts M1, M3, M4 and M5 were deleted from the
|
||||
local Superset through its API. The live Superset never had them (P3 not done).
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| 2 Site switching | pass: choosing a second site in the Site filter moves every chart request to it |
|
||||
| 3 Clearing | pass: after "Clear all" every chart request still carries exactly one site |
|
||||
| 4 Unavailable L/h | pass for the popup ("not available" and the method); S1, B3 and B11 are unchanged by this step and were not re-run |
|
||||
| 5 Missing coordinates | pass for the count: the map's key shows the 2 sites without an accepted location |
|
||||
| 7 Detail links | pass: for each of the 3 placed sites, hover, then "Site details", opens the Site details tab with Summary active, D0 shows the site, and every chart request carries exactly its `site_label` |
|
||||
| 8 Colours | the validity note beside an approved status and its L/h, and "(not approved)" for Provisional, pass as component tests on fixtures. Live, every site is "Insufficient evidence", so only slate was seen; the other three colours were checked by reading the code against §3, not on screen |
|
||||
| 9 Basemap | pass: tiles load, attribution visible; no CSP violation from the view. Superset's own `script-src eval` probe appears on every Superset page, as in P2 (§6), and nothing else. The test's watcher was checked with a deliberately disallowed image inside the view |
|
||||
|
||||
Also passed: the popup opens on hover, stays while the pointer moves onto it, closes after the
|
||||
pointer leaves; a click pins it; Esc and a click on the map close it; with reduced motion it only
|
||||
fades and nothing pulses. The view's minimum size is 480 × 560 px (`catalogue.json`): there, and
|
||||
also at 480 × 360, every site's popup stays inside the map, and an approved estimate whose inputs
|
||||
changed is never in view without its warning, checked by scrolling the card. At 480 × 560 every
|
||||
site's L/h can be read. A single placed site sits clear of the key. Two sites at the same
|
||||
position can each be opened with the pointer. The states the scratch data lacks (approved with a
|
||||
changed input, one site, two at one spot) are made in the browser by changing the read service's
|
||||
answer; the database is unchanged.
|
||||
|
||||
**Filters on top and the map search** *(owner, later on 28 Oct)*: the filter row was checked on
|
||||
screen first: the Fleet tab shows only "More filters", and Site details shows Site and Viewing
|
||||
period in the row. Browser checks:
|
||||
- the map takes the page's width;
|
||||
- the search finds a site by part of its name and by its code, and its popup's "Site details"
|
||||
opens exactly that site;
|
||||
- a site inside a cluster is taken out of it;
|
||||
- site switching and clearing (checks 2 and 3) still pass with the filters in the row.
|
||||
|
||||
**ATOMA readers and the operator's logo** *(owner, 29 Oct)*: the plan was reviewed by Codex
|
||||
(approved after three rounds: deploy inputs, ownership and no-dataset paths, SQL in responses).
|
||||
Browser checks as the local user `reader`:
|
||||
- every page leads to the dashboard;
|
||||
- no title row and no menus; Settings shows only Logout;
|
||||
- the whole fleet-map suite passes as the reader;
|
||||
- reading needs nothing the role lacks (only the colours PUT above is refused);
|
||||
- no chart-data response carries SQL, fresh, cached or as `result_type=query` (an admin's
|
||||
does);
|
||||
- CSV and editing answer 403;
|
||||
- two probe dashboards (Markdown only without roles, and one the reader owns) stay out of
|
||||
reach: page, API and permalink.
|
||||
|
||||
Also checked:
|
||||
- the role script removes a permission added by hand;
|
||||
- every new file changes the image tag;
|
||||
- the patch's `--verify` fails on unpatched Superset and on a tampered line.
|
||||
|
||||
Found on the way: under many parallel requests the local proxy returned 502s. gunicorn closes
|
||||
idle connections after 2 s (`--keep-alive 2`) and Traefik reused them. The local proxy now drops
|
||||
idle connections after 1 s. **The server has the same pairing** (Traefik in front of gunicorn
|
||||
with the default keep-alive): open item.
|
||||
|
||||
**Final inspection:** two fresh Codex sessions reviewed the build.
|
||||
- The first found four defects, all fixed and re-checked:
|
||||
- the popup could leave a small map;
|
||||
- two sites sharing a `site_code` could open each other's details;
|
||||
- every non-active pilot status was called "reference";
|
||||
- a failed database connection gave a 500 instead of the generic 503.
|
||||
- The second confirmed those fixes and found three more:
|
||||
- in a short card the validity warning could scroll away from the L/h;
|
||||
- a single site was centred under the key;
|
||||
- sites at one position hid each other.
|
||||
|
||||
All three are fixed and tested. Those last fixes, and the filter row and the map search, have
|
||||
not had an independent inspection.
|
||||
|
||||
**Filters back on the left, the readers' Fleet tab, the navbar logo, section buttons** *(owner,
|
||||
29 Oct)*:
|
||||
checked in the browser as `reader` and `admin` at 1400 × 1000, 1920 × 1080, 1440 × 760,
|
||||
1280 × 600 and 1024 wide:
|
||||
- a reader's Fleet tab has no filter column and no resize handle, and the map is as wide as
|
||||
the page and reaches 32 px above the window's bottom edge (no page scroll), at once rather
|
||||
than after Superset measures again; at 600 px high it keeps 560 px and the page scrolls;
|
||||
- Site details shows the column collapsed (32 px, for readers and admins, also after the
|
||||
map's "Site details" link); expanded, it has the Site filter; switching back hides it again;
|
||||
- an admin has the column on both tabs;
|
||||
- the column no longer flashes up while the Fleet tab loads (one frame, about 20 ms, remains:
|
||||
the grid exists before its tabs do);
|
||||
- after a switch to Site details its charts are too wide for one or two frames before
|
||||
Superset measures again; not changed;
|
||||
- a reader's page links `reader.css?v=<hash>`, an admin's none;
|
||||
- the navbar shows the wordmark at 24 px for readers, admins and the login page, white in
|
||||
dark mode;
|
||||
- Site details' inner tabs are a row of section buttons (`dashboard.css`), lined up with the
|
||||
content, all four visible and switching, for readers and admins, light and dark; a
|
||||
vertical list was tried first and dropped: Superset sizes the charts for the full width, so
|
||||
they were cut off on the right;
|
||||
- `build_bundle.py` stops when the orientation is not VERTICAL, when reader.css does not name
|
||||
the Fleet tab, when the map spans fewer than 12 columns, or when `dashboard.css` names an
|
||||
id that is not in the layout.
|
||||
|
||||
- *(lenses)* a reader at 1400 px: the switch centred in the navbar (Map | Insights), the map
|
||||
735 px high in an 860 px window and 32 px above its bottom edge; scrolled by 30, 61 and
|
||||
400 px the switch moves up by exactly as much and then stays out of view; at 820 px the tab
|
||||
row stays in the page; an admin keeps the tab row, named Map and Insights; the map's key
|
||||
says "Choose them in Insights".
|
||||
- *(AI teaser)* a reader at 1400 px: the switch reads Map | Insights | AI PRO, still centred,
|
||||
44 px high; a click on AI changes nothing; at 900 px the switch keeps Map | Insights.
|
||||
|
||||
These changes have not had an independent inspection (the owner skipped Codex).
|
||||
|
||||
**Not run:** check 1 (a synthetic fourth and fifth site), check 6 (the Viewing period's scope is
|
||||
unchanged), sign-in through Keycloak, the server's routing of `/views`, sign-in for the read
|
||||
service, thousands of sites, and dark mode.
|
||||
|
||||
## 10. The Petrolyst app's dashboards *(PD-12, PD-13; local, 29 Oct 2026)*
|
||||
|
||||
The Petrolyst app (petrolyst-platform `app/`, PD-12) shows Superset dashboards inside its
|
||||
Insights lens, with guest passes from the Petrolyst server (petrolyst-platform `server/`).
|
||||
`build_bundle.py` builds them (since PD-13 the only dashboards in the bundle):
|
||||
|
||||
| Slug | Content | Filters (set by the app; Superset's filter column is hidden) |
|
||||
|---|---|---|
|
||||
| `atoma-overview` | O1–O6: sites by pilot status and by baseline status, litres delivered, fuel cost by currency, electricity billed by currency and basis, litres delivered per month | Viewing period on O3–O6 |
|
||||
| `atoma-site` | the four §4 detail tabs as Superset's own tabs, the same rows and charts; no D0 (the app shows the site's header) | Site (all charts), Viewing period (as §4: fuel and runtime charts only) |
|
||||
|
||||
- **One site dashboard with tabs** *(owner, 29 Oct: "move it back inside the dashboard and make
|
||||
the tab as they were")*: replaces the four one-tab dashboards `atoma-site-summary`, `-fuel`,
|
||||
`-runtime`, `-baseline` and the app's per-section pages. Switching tabs needs no reload. The
|
||||
app opens a site on its first tab; a new viewing period reloads the dashboard on the tab that
|
||||
was open (the app reads it from Superset and names it in the address, `#TAB-fuel`). Delete
|
||||
the four old dashboards by hand on an instance that has them.
|
||||
|
||||
- **Overview rule** (changes §4 rule 3 for the Overview only): only sums and counts of
|
||||
recorded values across sites (`OVERVIEW_METRICS`); per-site rates (L/h, L/day) are never
|
||||
averaged; money is grouped by `currency_text`, a missing currency is its own row, and no
|
||||
total crosses currencies. `build_bundle.py` checks both.
|
||||
- They carry no CSS.
|
||||
- `import_bundle.py --embedded-out FILE --embed-domain URL` makes them embeddable from the app's
|
||||
origin and writes `{slug: embedding id}` for the Petrolyst server.
|
||||
- Embedding is switched on in the local stack only (`.local/fmdb-scratch`,
|
||||
`superset_config_local.py`); production keeps it off until PD-12's open points are settled.
|
||||
- **Checked (local):** a self-signed guest pass opens its own dashboard and nothing else (another
|
||||
embedded dashboard and ATOMA: 404; no pass or a wrong audience: 401); the embedded page answers
|
||||
only to the app's referrer (403 otherwise); chart-data responses to guests carry no SQL.
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,310 @@
|
||||
# Columns of the public.v_* views (migrations 0001-0003), as PostgreSQL reports them.
|
||||
# build_bundle.py lists every one in its dataset; import_bundle.py reports drift.
|
||||
# Regenerate after a migration that changes a view (psql -d fuel -AtF $'\t' -c "..."):
|
||||
# select c.relname, a.attname, format_type(a.atttypid, a.atttypmod)
|
||||
# from pg_attribute a join pg_class c on c.oid = a.attrelid join pg_namespace n on n.oid = c.relnamespace
|
||||
# where n.nspname = 'public' and c.relkind = 'v' and c.relname like 'v\_%' and a.attnum > 0
|
||||
# and not a.attisdropped order by c.relname, a.attnum
|
||||
v_assumption site_uid character varying(12)
|
||||
v_assumption site_code character varying(20)
|
||||
v_assumption site_name character varying(120)
|
||||
v_assumption site_label character varying(170)
|
||||
v_assumption code character varying(60)
|
||||
v_assumption topic character varying(30)
|
||||
v_assumption asset_uid character varying(40)
|
||||
v_assumption value numeric(10,3)
|
||||
v_assumption value_low numeric(10,3)
|
||||
v_assumption value_high numeric(10,3)
|
||||
v_assumption unit character varying(20)
|
||||
v_assumption applies_from date
|
||||
v_assumption applies_to date
|
||||
v_assumption basis character varying(20)
|
||||
v_assumption derived_from_fuel boolean
|
||||
v_assumption status character varying(12)
|
||||
v_assumption accepted_on date
|
||||
v_assumption decision_ref character varying(12)
|
||||
v_assumption rationale character varying(400)
|
||||
v_assumption contradicted_by character varying(200)
|
||||
v_assumption members character varying(400)
|
||||
v_baseline_input site_uid character varying(12)
|
||||
v_baseline_input site_code character varying(20)
|
||||
v_baseline_input site_name character varying(120)
|
||||
v_baseline_input site_label character varying(170)
|
||||
v_baseline_input baseline_code character varying(60)
|
||||
v_baseline_input baseline_status character varying(12)
|
||||
v_baseline_input interval_label character varying(30)
|
||||
v_baseline_input start_on date
|
||||
v_baseline_input end_on date
|
||||
v_baseline_input included boolean
|
||||
v_baseline_input exclusion_reason character varying(300)
|
||||
v_baseline_input fuel_l numeric(10,1)
|
||||
v_baseline_input runtime_h numeric(10,1)
|
||||
v_baseline_input days integer
|
||||
v_baseline_input eligibility_at_computation character varying(12)
|
||||
v_baseline_input evidence_grade_at_computation character varying(2)
|
||||
v_baseline_input dependencies_changed boolean
|
||||
v_baseline_version site_uid character varying(12)
|
||||
v_baseline_version site_code character varying(20)
|
||||
v_baseline_version site_name character varying(120)
|
||||
v_baseline_version site_label character varying(170)
|
||||
v_baseline_version code character varying(60)
|
||||
v_baseline_version metric character varying(40)
|
||||
v_baseline_version status character varying(12)
|
||||
v_baseline_version population character varying(200)
|
||||
v_baseline_version value numeric(8,3)
|
||||
v_baseline_version range_low numeric(8,3)
|
||||
v_baseline_version range_high numeric(8,3)
|
||||
v_baseline_version unit character varying(8)
|
||||
v_baseline_version window_start date
|
||||
v_baseline_version window_end date
|
||||
v_baseline_version fuel_l numeric(10,1)
|
||||
v_baseline_version runtime_h numeric(10,1)
|
||||
v_baseline_version matched_days integer
|
||||
v_baseline_version policy_version character varying(10)
|
||||
v_baseline_version meets_policy boolean
|
||||
v_baseline_version policy_check character varying(300)
|
||||
v_baseline_version evidence_grade character varying(2)
|
||||
v_baseline_version conditions character varying(200)
|
||||
v_baseline_version season_note character varying(200)
|
||||
v_baseline_version approved_on date
|
||||
v_baseline_version approved_by_role character varying(40)
|
||||
v_baseline_version decision_ref character varying(12)
|
||||
v_baseline_version inputs_unchanged boolean
|
||||
v_baseline_version computed_at timestamp(0) without time zone
|
||||
v_coverage site_uid character varying(12)
|
||||
v_coverage site_code character varying(20)
|
||||
v_coverage site_name character varying(120)
|
||||
v_coverage site_label character varying(170)
|
||||
v_coverage lane character varying(60)
|
||||
v_coverage status character varying(40)
|
||||
v_coverage start_on date
|
||||
v_coverage end_on date
|
||||
v_coverage start_ts timestamp(0) without time zone
|
||||
v_coverage end_ts timestamp(0) without time zone
|
||||
v_data_quality site_uid character varying(12)
|
||||
v_data_quality site_code character varying(20)
|
||||
v_data_quality site_name character varying(120)
|
||||
v_data_quality site_label character varying(170)
|
||||
v_data_quality check_order integer
|
||||
v_data_quality check_code character varying(40)
|
||||
v_data_quality label character varying(80)
|
||||
v_data_quality status character varying(10)
|
||||
v_data_quality detail character varying(300)
|
||||
v_delivery site_uid character varying(12)
|
||||
v_delivery site_code character varying(20)
|
||||
v_delivery site_name character varying(120)
|
||||
v_delivery site_label character varying(170)
|
||||
v_delivery event_key character varying(80)
|
||||
v_delivery delivery_date date
|
||||
v_delivery seq_in_day integer
|
||||
v_delivery delivered_l numeric(8,1)
|
||||
v_delivery unit_price numeric(8,2)
|
||||
v_delivery currency character varying(8)
|
||||
v_delivery source_ref character varying(60)
|
||||
v_delivery n_sources integer
|
||||
v_delivery sources_disagree boolean
|
||||
v_delivery reviewed boolean
|
||||
v_electricity_cost site_uid character varying(12)
|
||||
v_electricity_cost site_code character varying(20)
|
||||
v_electricity_cost site_name character varying(120)
|
||||
v_electricity_cost site_label character varying(170)
|
||||
v_electricity_cost bill_month date
|
||||
v_electricity_cost period_start date
|
||||
v_electricity_cost period_end date
|
||||
v_electricity_cost period_known boolean
|
||||
v_electricity_cost amount numeric(12,2)
|
||||
v_electricity_cost currency character varying(8)
|
||||
v_electricity_cost currency_basis character varying(10)
|
||||
v_electricity_cost kwh numeric(10,1)
|
||||
v_electricity_cost source_ref character varying(60)
|
||||
v_electricity_cost claim_status character varying(12)
|
||||
v_evidence site_uid character varying(12)
|
||||
v_evidence site_code character varying(20)
|
||||
v_evidence site_name character varying(120)
|
||||
v_evidence site_label character varying(170)
|
||||
v_evidence document_key character varying(16)
|
||||
v_evidence title character varying(200)
|
||||
v_evidence kind character varying(20)
|
||||
v_evidence issuer character varying(80)
|
||||
v_evidence document_date date
|
||||
v_evidence used_for character varying(200)
|
||||
v_evidence contains_personal_data boolean
|
||||
v_hour_meter_evidence site_uid character varying(12)
|
||||
v_hour_meter_evidence site_code character varying(20)
|
||||
v_hour_meter_evidence site_name character varying(120)
|
||||
v_hour_meter_evidence site_label character varying(170)
|
||||
v_hour_meter_evidence meter_label character varying(40)
|
||||
v_hour_meter_evidence mapped_set character varying(60)
|
||||
v_hour_meter_evidence reading_date date
|
||||
v_hour_meter_evidence date_basis character varying(30)
|
||||
v_hour_meter_evidence hours numeric(9,1)
|
||||
v_hour_meter_evidence reading_state character varying(16)
|
||||
v_hour_meter_evidence source_field character varying(16)
|
||||
v_hour_meter_evidence epoch_no integer
|
||||
v_hour_meter_evidence claim_status character varying(12)
|
||||
v_imported_calculation site_uid character varying(12)
|
||||
v_imported_calculation site_code character varying(20)
|
||||
v_imported_calculation site_name character varying(120)
|
||||
v_imported_calculation site_label character varying(170)
|
||||
v_imported_calculation cell_ref character varying(40)
|
||||
v_imported_calculation source_label character varying(80)
|
||||
v_imported_calculation formula character varying(200)
|
||||
v_imported_calculation cached_value numeric(14,6)
|
||||
v_imported_calculation unit_as_labelled character varying(30)
|
||||
v_imported_calculation our_reading character varying(300)
|
||||
v_imported_calculation baseline_use character varying(12)
|
||||
v_imported_calculation exclusion_reason character varying(300)
|
||||
v_imported_calculation document_key character varying(16)
|
||||
v_imported_calculation claim_status character varying(12)
|
||||
v_interval site_uid character varying(12)
|
||||
v_interval site_code character varying(20)
|
||||
v_interval site_name character varying(120)
|
||||
v_interval site_label character varying(170)
|
||||
v_interval interval_label character varying(30)
|
||||
v_interval start_on date
|
||||
v_interval end_on date
|
||||
v_interval days integer
|
||||
v_interval opening_l numeric(9,1)
|
||||
v_interval opening_basis character varying(8)
|
||||
v_interval deliveries_l numeric(9,1)
|
||||
v_interval withdrawals_l numeric(9,1)
|
||||
v_interval closing_l numeric(9,1)
|
||||
v_interval closing_basis character varying(8)
|
||||
v_interval fuel_used_l numeric(9,1)
|
||||
v_interval fuel_used_unc_l numeric(9,1)
|
||||
v_interval fuel_l_per_day numeric(8,1)
|
||||
v_interval stock_status character varying(40)
|
||||
v_interval runtime_h numeric(9,1)
|
||||
v_interval runtime_h_per_day numeric(6,2)
|
||||
v_interval n_meters integer
|
||||
v_interval metric character varying(40)
|
||||
v_interval population_key character varying(200)
|
||||
v_interval l_per_h numeric(8,3)
|
||||
v_interval l_per_h_low numeric(8,3)
|
||||
v_interval l_per_h_high numeric(8,3)
|
||||
v_interval eligibility character varying(12)
|
||||
v_interval exclusion_reasons character varying(600)
|
||||
v_interval evidence_grade character varying(2)
|
||||
v_interval conditions character varying(200)
|
||||
v_interval in_current_baseline boolean
|
||||
v_interval baseline_comparable boolean
|
||||
v_interval current_baseline_l_per_h numeric(8,3)
|
||||
v_kpi_measures site_uid character varying(12)
|
||||
v_kpi_measures site_code character varying(20)
|
||||
v_kpi_measures site_name character varying(120)
|
||||
v_kpi_measures site_label character varying(170)
|
||||
v_kpi_measures measure_order integer
|
||||
v_kpi_measures measure character varying(60)
|
||||
v_kpi_measures value numeric(12,2)
|
||||
v_kpi_measures unit character varying(12)
|
||||
v_kpi_measures value_kind character varying(16)
|
||||
v_kpi_measures window_start date
|
||||
v_kpi_measures window_end date
|
||||
v_kpi_measures note character varying(200)
|
||||
v_modelled_lph site_uid character varying(12)
|
||||
v_modelled_lph site_code character varying(20)
|
||||
v_modelled_lph site_name character varying(120)
|
||||
v_modelled_lph site_label character varying(170)
|
||||
v_modelled_lph assumption_code character varying(60)
|
||||
v_modelled_lph status character varying(12)
|
||||
v_modelled_lph basis character varying(20)
|
||||
v_modelled_lph h_per_day numeric(5,2)
|
||||
v_modelled_lph h_per_day_low numeric(5,2)
|
||||
v_modelled_lph h_per_day_high numeric(5,2)
|
||||
v_modelled_lph window_start date
|
||||
v_modelled_lph window_end date
|
||||
v_modelled_lph matched_days integer
|
||||
v_modelled_lph l_per_day numeric(7,1)
|
||||
v_modelled_lph modelled_l_per_h numeric(6,2)
|
||||
v_modelled_lph modelled_l_per_h_low numeric(6,2)
|
||||
v_modelled_lph modelled_l_per_h_high numeric(6,2)
|
||||
v_modelled_lph value_kind character varying(70)
|
||||
v_reconciliation site_uid character varying(12)
|
||||
v_reconciliation site_code character varying(20)
|
||||
v_reconciliation site_name character varying(120)
|
||||
v_reconciliation site_label character varying(170)
|
||||
v_reconciliation interval_label character varying(30)
|
||||
v_reconciliation start_on date
|
||||
v_reconciliation end_on date
|
||||
v_reconciliation stock_status character varying(40)
|
||||
v_reconciliation endpoint_times_known boolean
|
||||
v_reconciliation asset_uid character varying(40)
|
||||
v_reconciliation placed_throughout boolean
|
||||
v_reconciliation stock_balance_l numeric(9,1)
|
||||
v_reconciliation engine_l numeric(9,1)
|
||||
v_reconciliation covered_h numeric(9,1)
|
||||
v_reconciliation interval_h numeric(9,1)
|
||||
v_reconciliation engine_all_generators_l numeric(9,1)
|
||||
v_reconciliation all_generators_metered boolean
|
||||
v_reconciliation difference_l numeric(9,1)
|
||||
v_reconciliation difference_pct numeric(7,1)
|
||||
v_runtime site_uid character varying(12)
|
||||
v_runtime site_code character varying(20)
|
||||
v_runtime site_name character varying(120)
|
||||
v_runtime site_label character varying(170)
|
||||
v_runtime meter_label character varying(40)
|
||||
v_runtime mapped_set character varying(60)
|
||||
v_runtime interval_label character varying(80)
|
||||
v_runtime start_on date
|
||||
v_runtime end_on date
|
||||
v_runtime days integer
|
||||
v_runtime runtime_h numeric(9,1)
|
||||
v_runtime h_per_day numeric(6,2)
|
||||
v_runtime pair_valid boolean
|
||||
v_runtime same_epoch boolean
|
||||
v_runtime monotonic boolean
|
||||
v_runtime over_24h_per_day boolean
|
||||
v_runtime reviewed boolean
|
||||
v_site site_uid character varying(12)
|
||||
v_site site_code character varying(20)
|
||||
v_site site_name character varying(120)
|
||||
v_site site_label character varying(170)
|
||||
v_site zone character varying(40)
|
||||
v_site province character varying(40)
|
||||
v_site district character varying(60)
|
||||
v_site pilot_status character varying(12)
|
||||
v_site latitude numeric(9,6)
|
||||
v_site longitude numeric(9,6)
|
||||
v_site location_status character varying(20)
|
||||
v_site location_basis character varying(20)
|
||||
v_site location_precision_m integer
|
||||
v_site baseline_status character varying(24)
|
||||
v_site baseline_status_label character varying(40)
|
||||
v_site baseline_validity_note character varying(60)
|
||||
v_site lph_estimate numeric(6,2)
|
||||
v_site lph_method character varying(60)
|
||||
v_site lph_window_start date
|
||||
v_site lph_window_end date
|
||||
v_site baseline_conditions character varying(200)
|
||||
v_site baseline_decision_ref character varying(12)
|
||||
v_site fuel_l_per_calendar_day numeric(7,1)
|
||||
v_site latest_record_on date
|
||||
v_spec_reference site_uid character varying(12)
|
||||
v_spec_reference site_code character varying(20)
|
||||
v_spec_reference site_name character varying(120)
|
||||
v_spec_reference site_label character varying(170)
|
||||
v_spec_reference slot character varying(20)
|
||||
v_spec_reference asset_uid character varying(40)
|
||||
v_spec_reference asset_kind character varying(10)
|
||||
v_spec_reference make_model character varying(120)
|
||||
v_spec_reference attribute character varying(40)
|
||||
v_spec_reference load_pct numeric(5,1)
|
||||
v_spec_reference value numeric(12,3)
|
||||
v_spec_reference value_text character varying(120)
|
||||
v_spec_reference unit character varying(20)
|
||||
v_spec_reference basis character varying(30)
|
||||
v_spec_reference applicability character varying(12)
|
||||
v_spec_reference applicability_note character varying(300)
|
||||
v_spec_reference value_kind character varying(30)
|
||||
v_spec_reference claim_status character varying(12)
|
||||
v_stock_reading site_uid character varying(12)
|
||||
v_stock_reading site_code character varying(20)
|
||||
v_stock_reading site_name character varying(120)
|
||||
v_stock_reading site_label character varying(170)
|
||||
v_stock_reading observed_on date
|
||||
v_stock_reading seq_in_day integer
|
||||
v_stock_reading reading_type character varying(20)
|
||||
v_stock_reading litres numeric(8,1)
|
||||
v_stock_reading value_kind character varying(12)
|
||||
v_stock_reading spread_l numeric(8,1)
|
||||
v_stock_reading reviewed boolean
|
||||
|
Can't render this file because it contains an unexpected character in line 3 and column 80.
|
@@ -0,0 +1,136 @@
|
||||
"""Import the ATOMA dashboards bundle into Superset, overwriting what it defines.
|
||||
|
||||
Runs inside the Superset container, with the image's Python:
|
||||
|
||||
docker cp import_bundle.py <container>:/tmp/ && docker cp atoma-dashboards.zip <container>:/tmp/
|
||||
docker exec <container> /app/.venv/bin/python /tmp/import_bundle.py --show-database
|
||||
docker exec <container> /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip --user <username>
|
||||
|
||||
`superset import-dashboards` imports a dashboard's datasets and charts with
|
||||
overwrite off, so an existing chart or dataset keeps its old definition. This
|
||||
script runs the dataset, chart and dashboard importers in turn, each with
|
||||
overwrite on (the bundle's metadata.yaml type is set to what each importer
|
||||
expects). A dataset import replaces the column list, so columns missing from
|
||||
the bundle are dropped; afterwards each dataset's columns are compared with the
|
||||
live view and any drift is reported (exit status 2).
|
||||
|
||||
The database is never created or changed: the bundle's database uuid must
|
||||
already exist under the name "Fuel", otherwise nothing is imported.
|
||||
|
||||
The dashboards (atoma-overview, atoma-site) get no role: admins see them in Superset, and
|
||||
the Petrolyst app shows them with guest passes (PD-12). With --embedded-out FILE
|
||||
--embed-domain URL, each of them is made embeddable from that origin (its embedding id stays
|
||||
the same across imports), and FILE receives {slug: embedding id} for the Petrolyst server:
|
||||
|
||||
docker exec <container> /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip \
|
||||
--user <username> --embedded-out /tmp/embedded-dashboards.json --embed-domain http://127.0.0.1:8098
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
import zipfile
|
||||
|
||||
import yaml
|
||||
|
||||
DATABASE_NAME = "Fuel"
|
||||
EMBEDDED_PREFIX = "atoma-"
|
||||
STEPS = [("datasets", "SqlaTable"), ("charts", "Slice"), ("dashboards", "Dashboard")]
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Import the ATOMA dashboards bundle.")
|
||||
parser.add_argument("bundle", nargs="?")
|
||||
parser.add_argument("--user", help="existing Superset user who will own new objects")
|
||||
parser.add_argument("--show-database", action="store_true",
|
||||
help=f'print the uuid of "{DATABASE_NAME}" and stop')
|
||||
parser.add_argument("--embedded-out", help="write {slug: embedding id} of the app's dashboards")
|
||||
parser.add_argument("--embed-domain", help="the app's origin, e.g. http://127.0.0.1:8098")
|
||||
args = parser.parse_args()
|
||||
if not args.show_database and not (args.bundle and args.user):
|
||||
parser.error("give a bundle and --user, or --show-database")
|
||||
if bool(args.embedded_out) != bool(args.embed_domain):
|
||||
parser.error("--embedded-out and --embed-domain go together")
|
||||
|
||||
from superset.app import create_app
|
||||
|
||||
app = create_app()
|
||||
with app.app_context():
|
||||
from superset import db, security_manager
|
||||
from superset.commands.chart.importers.v1 import ImportChartsCommand
|
||||
from superset.commands.dashboard.importers.v1 import ImportDashboardsCommand
|
||||
from superset.commands.dataset.importers.v1 import ImportDatasetsCommand
|
||||
from superset.commands.importers.v1.utils import get_contents_from_bundle
|
||||
from superset.connectors.sqla.models import SqlaTable
|
||||
from superset.models.core import Database
|
||||
from superset.models.dashboard import Dashboard
|
||||
from superset.utils.core import override_user
|
||||
|
||||
if args.show_database:
|
||||
found = db.session.query(Database).filter_by(database_name=DATABASE_NAME).all()
|
||||
for database in found:
|
||||
print(database.uuid)
|
||||
return 0 if found else 1
|
||||
|
||||
user = security_manager.find_user(username=args.user)
|
||||
if user is None:
|
||||
print(f"no Superset user {args.user!r}", file=sys.stderr)
|
||||
return 1
|
||||
with zipfile.ZipFile(args.bundle) as bundle:
|
||||
contents = get_contents_from_bundle(bundle)
|
||||
databases = [yaml.safe_load(t) for p, t in contents.items() if p.startswith("databases/")]
|
||||
for config in databases:
|
||||
database = db.session.query(Database).filter_by(uuid=config["uuid"]).one_or_none()
|
||||
if database is None or database.database_name != DATABASE_NAME:
|
||||
print(f'database {config["uuid"]} is not "{DATABASE_NAME}" here; '
|
||||
"rebuild the bundle with the uuid from --show-database", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
metadata = yaml.safe_load(contents["metadata.yaml"])
|
||||
commands = {"datasets": ImportDatasetsCommand, "charts": ImportChartsCommand,
|
||||
"dashboards": ImportDashboardsCommand}
|
||||
with override_user(user):
|
||||
for step, model in STEPS:
|
||||
step_contents = dict(contents)
|
||||
step_contents["metadata.yaml"] = yaml.safe_dump({**metadata, "type": model})
|
||||
commands[step](step_contents, overwrite=True).run()
|
||||
count = sum(p.startswith(step + "/") for p in contents)
|
||||
print(f"{step}: {count} imported (overwrite)")
|
||||
|
||||
embedded: dict[str, str] = {}
|
||||
for path, text in contents.items():
|
||||
if path.startswith("dashboards/"):
|
||||
dashboard = db.session.query(Dashboard).filter_by(
|
||||
uuid=yaml.safe_load(text)["uuid"]).one()
|
||||
if dashboard.slug.startswith(EMBEDDED_PREFIX) and args.embedded_out:
|
||||
from superset.daos.dashboard import EmbeddedDashboardDAO
|
||||
|
||||
config = EmbeddedDashboardDAO.upsert(dashboard, [args.embed_domain])
|
||||
db.session.flush()
|
||||
embedded[dashboard.slug] = str(config.uuid)
|
||||
db.session.commit()
|
||||
if args.embedded_out:
|
||||
with open(args.embedded_out, "w") as out:
|
||||
json.dump(dict(sorted(embedded.items())), out, indent=2)
|
||||
print(f"embedded: {len(embedded)} dashboard(s) for {args.embed_domain}, "
|
||||
f"ids in {args.embedded_out}")
|
||||
|
||||
drift = 0
|
||||
dataset_uuids = [yaml.safe_load(t)["uuid"] for p, t in contents.items()
|
||||
if p.startswith("datasets/")]
|
||||
for dataset_uuid in dataset_uuids:
|
||||
dataset = db.session.query(SqlaTable).filter_by(uuid=dataset_uuid).one()
|
||||
physical = {c.column_name for c in dataset.columns if not c.expression}
|
||||
live = {c["column_name"] for c in dataset.external_metadata()}
|
||||
if physical != live:
|
||||
drift += 1
|
||||
print(f"drift {dataset.table_name}: only in bundle {sorted(physical - live)}, "
|
||||
f"only in view {sorted(live - physical)}")
|
||||
print(f"column drift: {drift} dataset(s)")
|
||||
return 2 if drift else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,80 @@
|
||||
"""The role "ATOMA Guest": what the Petrolyst app's embedded dashboards may do (PD-12).
|
||||
|
||||
Superset gives every embedded viewer (a guest pass signed for one dashboard) the role named by
|
||||
GUEST_ROLE_NAME. Which dashboard a guest sees comes from the pass, not from the role. Superset
|
||||
leaves the SQL out of chart-data responses for guests.
|
||||
|
||||
Embedding is switched on in the local stack only (.local/fmdb-scratch, superset_config_local.py),
|
||||
so this script is not part of init.sh; run it by hand inside the Superset container, once, and
|
||||
again whenever PERMISSIONS change:
|
||||
|
||||
docker exec <superset> /app/.venv/bin/python /app/petrolyst-roles/atoma_guest.py
|
||||
|
||||
Creates the role if needed and gives it exactly PERMISSIONS: missing ones are added, any other
|
||||
is removed (also one added by hand in the UI). A second run changes nothing.
|
||||
|
||||
The list was established on the local stack by reading the ATOMA dashboard as a reader (the
|
||||
former "ATOMA Reader" role, retired with PD-13): everything a dashboard asks the server for, and
|
||||
nothing more. No menu access, nothing to write, no Explore, CSV, sharing, drill or query view.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
|
||||
ROLE = "ATOMA Guest"
|
||||
PERMISSIONS = [
|
||||
("can_read", "Dashboard"), # the dashboard and its charts' definitions
|
||||
("can_dashboard", "Superset"), # the dashboard page
|
||||
("can_dashboard_permalink", "Superset"), # a dashboard permalink
|
||||
("can_read", "Chart"), # chart data
|
||||
("can_time_range", "Api"), # the Viewing period filter
|
||||
("can_read", "DashboardFilterStateRestApi"), # native filter state
|
||||
("can_write", "DashboardFilterStateRestApi"),
|
||||
("can_read", "DashboardPermalinkRestApi"),
|
||||
("can_read", "CurrentUserRestApi"), # who is signed in
|
||||
("can_read", "SecurityRestApi"), # the CSRF token
|
||||
("can_get", "MenuApi"),
|
||||
("can_log", "Superset"), # Superset's event log
|
||||
]
|
||||
|
||||
|
||||
def sync(name: str, permissions: list[tuple[str, str]]) -> int:
|
||||
"""Give the role `name` exactly `permissions`, creating it if needed."""
|
||||
from superset.app import create_app
|
||||
|
||||
app = create_app()
|
||||
with app.app_context():
|
||||
from superset import db, security_manager as sm
|
||||
|
||||
wanted = []
|
||||
for permission, view in permissions:
|
||||
pvm = sm.find_permission_view_menu(permission, view)
|
||||
if pvm is None:
|
||||
print(f"no permission {permission} on {view}: run `superset init` first?",
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
wanted.append(pvm)
|
||||
|
||||
role = sm.find_role(name) or sm.add_role(name)
|
||||
added = [pvm for pvm in wanted if pvm not in role.permissions]
|
||||
removed = [pvm for pvm in role.permissions if pvm not in wanted]
|
||||
for pvm in added:
|
||||
sm.add_permission_role(role, pvm)
|
||||
for pvm in removed:
|
||||
sm.del_permission_role(role, pvm)
|
||||
db.session.commit()
|
||||
|
||||
print(f'role "{name}": {len(wanted)} permissions, {len(added)} added, '
|
||||
f"{len(removed)} removed")
|
||||
for pvm in removed:
|
||||
print(f" removed {pvm.permission.name} on {pvm.view_menu.name}")
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
return sync(ROLE, PERMISSIONS)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -45,6 +45,21 @@ SQLALCHEMY_DATABASE_URI = URL.create(
|
||||
database="superset",
|
||||
).render_as_string(hide_password=False)
|
||||
|
||||
# --- The data: fuel-db ----------------------------------------------------------
|
||||
# Superset reads fuel-db directly as the read-only role fuel_ro (no Trino since 29 Oct 2026);
|
||||
# superset-init registers that connection, "Fuel", without a password. Passwords of data
|
||||
# connections come from secret files at connect time, so none is kept in Superset's own
|
||||
# database. A connection added in the UI gets no password this way.
|
||||
_DATABASE_PASSWORD_SECRETS = {"fuel_ro": "fuel_db_ro_password"}
|
||||
|
||||
|
||||
def _database_password(url: URL) -> str | None:
|
||||
secret = _DATABASE_PASSWORD_SECRETS.get(url.username or "")
|
||||
return _read_secret(secret) if secret else None
|
||||
|
||||
|
||||
SQLALCHEMY_CUSTOM_PASSWORD_STORE = _database_password
|
||||
|
||||
# --- Sign-in through Keycloak (client "petrolyst") ----------------------------
|
||||
# Users and roles are managed in Keycloak; there is no local admin account.
|
||||
# Break-glass access: `superset fab ...` inside the container on the server.
|
||||
@@ -79,6 +94,7 @@ AUTH_ROLES_MAPPING = {
|
||||
"superset-admin": ["Admin"],
|
||||
"superset-editor": ["Alpha"],
|
||||
"superset-viewer": ["Gamma"],
|
||||
# "atoma-reader" opens the Petrolyst app only; here it maps to nothing (Public).
|
||||
}
|
||||
AUTH_ROLES_SYNC_AT_LOGIN = True
|
||||
AUTH_USER_REGISTRATION = True
|
||||
@@ -87,7 +103,17 @@ AUTH_USER_REGISTRATION = True
|
||||
AUTH_USER_REGISTRATION_ROLE = "Public"
|
||||
|
||||
# Viewers (Gamma) see a dashboard once it lists their role.
|
||||
FEATURE_FLAGS = {"DASHBOARD_RBAC": True}
|
||||
# FILTERBAR_CLOSED_BY_DEFAULT: a dashboard's filter column opens collapsed, a
|
||||
# narrow strip with an expand button (owner, 29 Oct); ?expand_filters=true
|
||||
# opens it from a link.
|
||||
FEATURE_FLAGS = {"DASHBOARD_RBAC": True, "FILTERBAR_CLOSED_BY_DEFAULT": True}
|
||||
|
||||
# deck.gl maps: OpenStreetMap raster tiles only. The default list adds Mapbox
|
||||
# styles, which fail without MAPBOX_API_KEY. The default CSP already allows the
|
||||
# tile host.
|
||||
DECKGL_BASE_MAP = [
|
||||
["tile://https://tile.openstreetmap.org/{z}/{x}/{y}.png", "OpenStreetMap"],
|
||||
]
|
||||
|
||||
# --- Behind Traefik -----------------------------------------------------------
|
||||
ENABLE_PROXY_FIX = True
|
||||
@@ -128,12 +154,12 @@ EXPLORE_FORM_DATA_CACHE_CONFIG = {
|
||||
RATELIMIT_STORAGE_URI = f"{REDIS_URL}/1"
|
||||
|
||||
# --- Petrolyst look and feel --------------------------------------------------
|
||||
# Values from petrolyst-ui/design/brand/07_UI_Tokens/petrolyst_tokens.css.
|
||||
# Values from petrolyst-web/design/brand/07_UI_Tokens/petrolyst_tokens.css.
|
||||
# Assets live in assets/branding/ and are served from /static/assets/branding/.
|
||||
BRANDING = "/static/assets/branding"
|
||||
|
||||
APP_NAME = "Petrolyst"
|
||||
APP_ICON = f"{BRANDING}/petrolyst_logo_standard_fullcolor_no_tagline.svg"
|
||||
APP_ICON = f"{BRANDING}/petrolyst_wordmark_fullcolor.svg"
|
||||
FAVICONS = [
|
||||
{"href": f"{BRANDING}/favicon.svg", "type": "image/svg+xml"},
|
||||
{"href": f"{BRANDING}/favicon.ico", "type": "image/x-icon"},
|
||||
@@ -147,10 +173,13 @@ THEME_DEFAULT["token"].update(
|
||||
"brandLogoAlt": "Petrolyst",
|
||||
"brandLogoUrl": APP_ICON,
|
||||
"brandLogoHref": "/",
|
||||
# The standard logo is stacked (mark over wordmark); 40px keeps the Y
|
||||
# legible and, with the margin, the 60px navbar height.
|
||||
"brandLogoHeight": "40px",
|
||||
"brandLogoMargin": "10px 0",
|
||||
# The wordmark alone (owner, 29 Oct): in the stacked logo, at 40px, its
|
||||
# letters were about 5px high. Its own Y carries the brand accent. 24px
|
||||
# (letters about 20px) and the margin fill the 60px navbar height. On the
|
||||
# left, 15px more than the navbar's 16px: the P (1.4px inside the image)
|
||||
# lines up with the dashboard's content at 32px.
|
||||
"brandLogoHeight": "24px",
|
||||
"brandLogoMargin": "18px 0 18px 15px",
|
||||
# Action teal, not the raw brand teal #009688: the brand guide keeps
|
||||
# raw teal off small text and controls.
|
||||
"colorPrimary": "#00695C",
|
||||
@@ -171,7 +200,7 @@ THEME_DARK = deepcopy(THEME_DEFAULT)
|
||||
THEME_DARK["algorithm"] = "dark"
|
||||
THEME_DARK["token"].update(
|
||||
{
|
||||
"brandLogoUrl": f"{BRANDING}/petrolyst_logo_standard_white_no_tagline.svg",
|
||||
"brandLogoUrl": f"{BRANDING}/petrolyst_wordmark_white.svg",
|
||||
"colorPrimary": "#35C5B6",
|
||||
"colorLink": "#35C5B6",
|
||||
"colorSuccess": "#35C5B6",
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
# Fuel-monitoring analytics: Superset on atoma.petrolyst.com, reading the
|
||||
# operational store (fuel-db) through Trino. Run only through deploy/deploy.sh
|
||||
# or deploy/rollback.sh: they export SUPERSET_IMAGE_TAG and TRINO_CONFIG_HASH
|
||||
# from the release's ARTIFACTS file, and the :? guards below stop Compose from
|
||||
# running with either unset.
|
||||
name: fuel-monitoring
|
||||
# operational store (fuel-db) directly as the read-only role fuel_ro (Trino was
|
||||
# removed on 29 Oct 2026). Run only through deploy/deploy.sh or
|
||||
# deploy/rollback.sh: they export SUPERSET_IMAGE_TAG from the release's
|
||||
# ARTIFACTS file, and the :? guard below stops Compose from running without it.
|
||||
name: petrolyst
|
||||
|
||||
x-superset-env: &superset-env
|
||||
KEYCLOAK_REALM_URL: ${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/thedigitalbridge}
|
||||
@@ -15,6 +15,8 @@ x-superset: &superset
|
||||
- superset_secret_key
|
||||
- superset_db_password
|
||||
- keycloak_petrolyst_client_secret
|
||||
# fuel_ro's password, read at connect time (superset_config.py)
|
||||
- fuel_db_ro_password
|
||||
|
||||
services:
|
||||
superset:
|
||||
@@ -100,25 +102,6 @@ services:
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
|
||||
trino:
|
||||
image: trinodb/trino:484
|
||||
env_file:
|
||||
- ${PETROLYST_ROOT:-/opt/petrolyst}/env/fuel-monitoring.env
|
||||
volumes:
|
||||
- ./trino/etc:/etc/trino:ro
|
||||
labels:
|
||||
# Compose cannot see content changes in a bind mount; this label changes
|
||||
# with trino/etc, so exactly a config change recreates the container.
|
||||
petrolyst.config-hash: ${TRINO_CONFIG_HASH:?TRINO_CONFIG_HASH unset - run through deploy/deploy.sh}
|
||||
depends_on:
|
||||
fuel-db:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- backend
|
||||
restart: unless-stopped
|
||||
mem_limit: 3g
|
||||
# The image's own healthcheck (/usr/lib/trino/bin/health-check) applies.
|
||||
|
||||
fuel-db:
|
||||
image: timescale/timescaledb:2.31.0-pg17
|
||||
environment:
|
||||
@@ -128,7 +111,7 @@ services:
|
||||
TS_TUNE_NUM_CPUS: "2"
|
||||
secrets:
|
||||
- fuel_db_password
|
||||
- fuel_db_trino_password
|
||||
- fuel_db_ro_password
|
||||
volumes:
|
||||
- fuel_db_data:/var/lib/postgresql/data
|
||||
# A single file, not the folder: the image's own TimescaleDB init scripts
|
||||
@@ -157,9 +140,14 @@ networks:
|
||||
# so fuel-db's loopback port works.
|
||||
fuel-db-tunnel: {}
|
||||
|
||||
# Created under the project's first name, fuel-monitoring; named explicitly so the
|
||||
# data stayed when the project became petrolyst (29 Oct 2026). Compose warns once
|
||||
# that they belong to another project; that is expected.
|
||||
volumes:
|
||||
superset_db_data:
|
||||
name: fuel-monitoring_superset_db_data
|
||||
fuel_db_data:
|
||||
name: fuel-monitoring_fuel_db_data
|
||||
|
||||
secrets:
|
||||
superset_secret_key:
|
||||
@@ -168,7 +156,7 @@ secrets:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/superset_db_password
|
||||
fuel_db_password:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/fuel_db_password
|
||||
fuel_db_trino_password:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/fuel_db_trino_password
|
||||
fuel_db_ro_password:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/fuel_db_ro_password
|
||||
keycloak_petrolyst_client_secret:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/shared/keycloak_petrolyst_client_secret
|
||||
@@ -5,21 +5,24 @@
|
||||
# Loaders log in as their own role but act as fuel_owner (README
|
||||
# "Data loaders"): the default privileges below cover only what
|
||||
# fuel_owner itself creates, not what its members create.
|
||||
# trino_ro what Trino connects as: read-only, also on tables created later.
|
||||
# fuel_ro what Superset and the Petrolyst server read as: SELECT on public,
|
||||
# also on tables created later. Read-only by default as well: a
|
||||
# second lock behind the grants.
|
||||
set -euo pipefail
|
||||
|
||||
trino_password=$(cat /run/secrets/fuel_db_trino_password)
|
||||
ro_password=$(cat /run/secrets/fuel_db_ro_password)
|
||||
|
||||
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" \
|
||||
-v trino_password="$trino_password" <<'SQL'
|
||||
-v ro_password="$ro_password" <<'SQL'
|
||||
CREATE ROLE fuel_owner NOLOGIN;
|
||||
ALTER DATABASE fuel OWNER TO fuel_owner;
|
||||
REVOKE CREATE ON SCHEMA public FROM PUBLIC;
|
||||
|
||||
CREATE ROLE trino_ro LOGIN PASSWORD :'trino_password';
|
||||
GRANT CONNECT ON DATABASE fuel TO trino_ro;
|
||||
GRANT USAGE ON SCHEMA public TO trino_ro;
|
||||
GRANT SELECT ON ALL TABLES IN SCHEMA public TO trino_ro;
|
||||
CREATE ROLE fuel_ro LOGIN PASSWORD :'ro_password';
|
||||
ALTER ROLE fuel_ro SET default_transaction_read_only = on;
|
||||
GRANT CONNECT ON DATABASE fuel TO fuel_ro;
|
||||
GRANT USAGE ON SCHEMA public TO fuel_ro;
|
||||
GRANT SELECT ON ALL TABLES IN SCHEMA public TO fuel_ro;
|
||||
ALTER DEFAULT PRIVILEGES FOR ROLE fuel_owner IN SCHEMA public
|
||||
GRANT SELECT ON TABLES TO trino_ro;
|
||||
GRANT SELECT ON TABLES TO fuel_ro;
|
||||
SQL
|
||||
@@ -1,4 +1,4 @@
|
||||
-- 0002_fm_derivations.sql: internal derivations in schema `fm` (not visible to Trino).
|
||||
-- 0002_fm_derivations.sql: internal derivations in schema `fm` (not readable by fuel_ro).
|
||||
-- DRAFT, not applied. Applied after 0001 by a loader acting as fuel_owner (README in this folder).
|
||||
-- Design: fuel-monitoring .context/plans/2026-10-24-fuel-db-baseline-dashboard.md, §4 and §5.1.
|
||||
BEGIN;
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
-- 0003_bi_views.sql: the views Superset reads through Trino (schema `public`), plus
|
||||
-- fm.v_site_baseline_status. Owned by fuel_owner, so trino_ro can read them through the existing
|
||||
-- default privileges; they run with the owner's rights, so Trino never reads `fm` (it lists fm's
|
||||
-- table names, as it lists every schema's, but any query on them is refused).
|
||||
-- 0003_bi_views.sql: the views Superset reads (schema `public`), plus
|
||||
-- fm.v_site_baseline_status. Owned by fuel_owner, so fuel_ro can read them through the existing
|
||||
-- default privileges; they run with the owner's rights, so fuel_ro never reads `fm` (the catalog
|
||||
-- shows fm's table names, as it shows every schema's, but any query on them is refused).
|
||||
-- DRAFT, not applied. Applied after 0002 by a loader acting as fuel_owner (README in this folder).
|
||||
-- Design: fuel-monitoring .context/plans/2026-10-24-fuel-db-baseline-dashboard.md, §5.2 and §6.
|
||||
BEGIN;
|
||||
@@ -97,7 +97,7 @@ LEFT JOIN pp USING (site_uid) LEFT JOIN fm.v_site_baseline_status bs USING (site
|
||||
LEFT JOIN cal USING (site_uid) LEFT JOIN latest USING (site_uid);
|
||||
|
||||
-- Every view below joins public.v_site for site_uid, site_code, site_name and site_label, so one
|
||||
-- Site filter drives every dataset, and casts to Trino-safe types (varchar(n), integer,
|
||||
-- Site filter drives every dataset, and casts to plain types (varchar(n), integer,
|
||||
-- numeric(p,s), date, timestamp(0), boolean). Live = asserted, reviewed or disputed.
|
||||
|
||||
-- S2, R3: one row per site and measure; the rows stay when a value is not established.
|
||||
|
||||
@@ -10,7 +10,7 @@ The design and its rationale live in the fuel-monitoring corpus:
|
||||
|
||||
## Files
|
||||
|
||||
| File | Creates | Visible to Trino |
|
||||
| File | Creates | Readable by `fuel_ro` (Superset, Petrolyst server) |
|
||||
|---|---|---|
|
||||
| `0001_fm_core.sql` | schema `fm`, `btree_gist` in it, 25 tables, constraints, trigger-backed rules 1–13 | no |
|
||||
| `0002_fm_derivations.sql` | internal views in `fm`: claims → visits → stock intervals → L/h and L/day intervals, dependency hashes, baseline validity | no |
|
||||
@@ -21,7 +21,7 @@ Each file is one transaction. Apply them in order.
|
||||
## Applying (on the server)
|
||||
|
||||
1. Create a loader role as in the repository README, "Data loaders". Its sessions act as
|
||||
`fuel_owner`, so every object belongs to `fuel_owner`. `trino_ro` can then read the `public`
|
||||
`fuel_owner`, so every object belongs to `fuel_owner`. `fuel_ro` can then read the `public`
|
||||
views through the existing default privileges, with no role or grant changes.
|
||||
2. Open the tunnel and apply the three files as the loader:
|
||||
|
||||
@@ -32,13 +32,14 @@ Each file is one transaction. Apply them in order.
|
||||
done
|
||||
```
|
||||
|
||||
3. Check through Trino (`docker exec -it fuel-monitoring-trino-1 trino`):
|
||||
- `SHOW TABLES FROM fuel.public` lists the 18 `v_*` views;
|
||||
- `SELECT count(*) FROM fuel.fm.site` fails with "permission denied for schema fm".
|
||||
3. Check as `fuel_ro`
|
||||
(`docker exec -it petrolyst-fuel-db-1 psql -U postgres -d fuel`, then `SET ROLE fuel_ro;`):
|
||||
- `\dv public.*` lists the 18 `v_*` views;
|
||||
- `SELECT count(*) FROM fm.site;` fails with "permission denied for schema fm".
|
||||
|
||||
The `public` views are owned by `fuel_owner` and run with its rights, so Trino never needs, and
|
||||
never gets, read access to `fm`. Trino's metadata still lists fm's table and column names, as it
|
||||
lists every schema in fuel-db (TimescaleDB's internal ones included); names only, no rows.
|
||||
The `public` views are owned by `fuel_owner` and run with its rights, so `fuel_ro` never needs,
|
||||
and never gets, read access to `fm`. PostgreSQL's catalog still shows every schema's table and
|
||||
column names to any role, so Superset's SQL Lab can list fm's names; names only, no rows.
|
||||
|
||||
## Data
|
||||
|
||||
@@ -91,6 +92,8 @@ it, except baseline versions and their approvals, which live only in fuel-db: ex
|
||||
|
||||
## Verified locally (2026-10-24, scratch only)
|
||||
|
||||
With Trino, before its removal on 29 Oct 2026; `trino_ro` is now `fuel_ro`.
|
||||
|
||||
Setup:
|
||||
- `timescale/timescaledb:2.31.0-pg17`, with this repository's `01-roles.sh`;
|
||||
- a loader created by the README recipe;
|
||||
|
||||
@@ -6,12 +6,12 @@
|
||||
FROM apache/superset:6.1.0@sha256:16b50bbef6648912a79e3293d418fefa743ce555d36c5af6b85d859119ed7f88
|
||||
|
||||
USER root
|
||||
# The 6.1.0 image ships neither an OAuth client nor the Postgres and Trino
|
||||
# drivers.
|
||||
# The 6.1.0 image ships neither an OAuth client nor the Postgres driver.
|
||||
RUN uv pip install --no-cache --python /app/.venv/bin/python \
|
||||
authlib==1.9.1 psycopg2-binary==2.9.13 "trino[sqlalchemy]==0.340.0"
|
||||
authlib==1.9.1 psycopg2-binary==2.9.13
|
||||
|
||||
COPY --from=insights --chown=superset:superset superset_config.py petrolyst_security.py /app/pythonpath/
|
||||
COPY --from=insights --chown=superset:superset roles/ /app/petrolyst-roles/
|
||||
COPY --from=insights --chown=superset:superset assets/branding/ /app/superset/static/assets/branding/
|
||||
COPY --from=insights patches/ /app/patches/petrolyst/
|
||||
# Stop at the first patch or verify that fails, so a later success cannot hide
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
# superset-init: migrate the metadata database, sync roles and permissions, and
|
||||
# register the Trino connection. It is copied into the Superset image, so a
|
||||
# register the connection to fuel-db. It is copied into the Superset image, so a
|
||||
# change here changes the image tag; deploy/deploy.sh runs it on a fresh install
|
||||
# and whenever that tag changes. Every step is idempotent. Unlike the
|
||||
# upstream docker-init.sh it never creates a local admin: users come from
|
||||
@@ -9,6 +9,25 @@ set -euo pipefail
|
||||
|
||||
superset db upgrade
|
||||
superset init
|
||||
|
||||
# Trino was removed on 29 Oct 2026. An instance still holding its connection
|
||||
# "Fuel (Trino)" gets it renamed, so its datasets and dashboards keep their ids;
|
||||
# the next step points it at fuel-db. Drop this once no instance has it.
|
||||
/app/.venv/bin/python - <<'PY'
|
||||
from superset.app import create_app
|
||||
|
||||
with create_app().app_context():
|
||||
from superset import db
|
||||
from superset.models.core import Database
|
||||
|
||||
old = db.session.query(Database).filter_by(database_name="Fuel (Trino)").one_or_none()
|
||||
if old and not db.session.query(Database).filter_by(database_name="Fuel").count():
|
||||
old.database_name = "Fuel"
|
||||
db.session.commit()
|
||||
print('renamed the connection "Fuel (Trino)" to "Fuel"')
|
||||
PY
|
||||
|
||||
# No password here: superset_config.py reads it from /run/secrets at connect time.
|
||||
superset set-database-uri \
|
||||
--database_name "Fuel (Trino)" \
|
||||
--uri "trino://superset@trino:8080/fuel"
|
||||
--database_name "Fuel" \
|
||||
--uri "postgresql+psycopg2://fuel_ro@fuel-db:5432/fuel"
|
||||
@@ -1,3 +0,0 @@
|
||||
# Analytics never writes through Trino. fuel-db's trino_ro role is read-only as
|
||||
# well; this is the second lock.
|
||||
access-control.name=read-only
|
||||
@@ -1,5 +0,0 @@
|
||||
# The fuel-monitoring operational store (service fuel-db), read-only.
|
||||
connector.name=postgresql
|
||||
connection-url=jdbc:postgresql://fuel-db:5432/fuel
|
||||
connection-user=trino_ro
|
||||
connection-password=${ENV:FUEL_DB_TRINO_PASSWORD}
|
||||
@@ -1,14 +0,0 @@
|
||||
# Single node: coordinator and worker in one process. Reachable only on the
|
||||
# backend network (Superset); no port is published and Traefik has no route.
|
||||
coordinator=true
|
||||
node-scheduler.include-coordinator=true
|
||||
http-server.http.port=8080
|
||||
discovery.uri=http://localhost:8080
|
||||
catalog.management=static
|
||||
|
||||
# Memory bounds. The container is capped at 3g (compose.yaml) and jvm.config
|
||||
# gives the heap 75 % of it (~2.25 GB); 1 GB for queries plus 512 MB headroom
|
||||
# leaves room for the rest of the heap.
|
||||
query.max-memory=1GB
|
||||
query.max-memory-per-node=1GB
|
||||
memory.heap-headroom-per-node=512MB
|
||||
@@ -1,14 +0,0 @@
|
||||
-server
|
||||
-agentpath:/usr/lib/trino/bin/libjvmkill.so
|
||||
-XX:InitialRAMPercentage=75
|
||||
-XX:MaxRAMPercentage=75
|
||||
-XX:G1HeapRegionSize=32M
|
||||
-XX:+ExplicitGCInvokesConcurrent
|
||||
-XX:+HeapDumpOnOutOfMemoryError
|
||||
-XX:+ExitOnOutOfMemoryError
|
||||
-XX:-OmitStackTraceInFastThrow
|
||||
-XX:ReservedCodeCacheSize=256M
|
||||
-XX:PerMethodRecompilationCutoff=10000
|
||||
-XX:PerBytecodeRecompilationCutoff=10000
|
||||
-Djdk.attach.allowAttachSelf=true
|
||||
-Djdk.nio.maxCachedBufferSize=2000000
|
||||
@@ -1 +0,0 @@
|
||||
io.trino=INFO
|
||||
@@ -1,3 +0,0 @@
|
||||
node.environment=petrolyst
|
||||
node.data-dir=/data/trino
|
||||
node.id=${ENV:HOSTNAME}
|
||||
Reference in new issue
Block a user