Petrolyst server infrastructure: Traefik, Trino and Superset
Traefik as the server-wide edge (TLS via Let's Encrypt, dashboard on traefik.petrolyst.com behind oauth2-proxy and Keycloak). The fuel-monitoring stack: Superset on atoma.petrolyst.com with Keycloak login and Petrolyst branding, Trino reading the TimescaleDB fuel-db as catalog "fuel". Customization lives in petrolyst-insights/, deployment in the other folders. Deploys run server-side (deploy/deploy.sh) from a Gitea workflow or manually; releases are immutable, Superset metadata is backed up before migrations, and deploy/rollback.sh restores it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Nasim SadatandClaude Opus 5.5 committed
1 parent
24327f9462
commit
d3800ac9dd
34 files changed
+2117
No files matched your search
@@ -0,0 +1,57 @@
|
|||||||
|
# Every push to main deploys everything: Traefik first, then each stack under
|
||||||
|
# stacks/. The runner only uploads; the server validates, deploys and
|
||||||
|
# smoke-tests (deploy/deploy.sh), so this job needs no Docker of its own.
|
||||||
|
name: deploy
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
# Serializes CI runs; deploy.sh's host lock also covers manual rollbacks.
|
||||||
|
concurrency:
|
||||||
|
group: deploy-petrolyst
|
||||||
|
cancel-in-progress: false
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
deploy:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
timeout-minutes: 45
|
||||||
|
env:
|
||||||
|
DEPLOY_HOST: ${{ secrets.DEPLOY_HOST }}
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Lint shell scripts
|
||||||
|
run: |
|
||||||
|
if ! command -v shellcheck >/dev/null; then
|
||||||
|
SUDO=$(command -v sudo || true)
|
||||||
|
$SUDO apt-get update -qq && $SUDO apt-get install -y -qq shellcheck
|
||||||
|
fi
|
||||||
|
shellcheck -x host/*.sh deploy/*.sh stacks/*/superset/init.sh stacks/*/fuel-db/init/*.sh
|
||||||
|
|
||||||
|
- name: Install the deploy key
|
||||||
|
env:
|
||||||
|
DEPLOY_SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
|
||||||
|
DEPLOY_KNOWN_HOSTS: ${{ secrets.DEPLOY_KNOWN_HOSTS }}
|
||||||
|
run: |
|
||||||
|
install -d -m 0700 "$HOME/.ssh"
|
||||||
|
printf '%s\n' "$DEPLOY_SSH_KEY" > "$HOME/.ssh/deploy_key"
|
||||||
|
chmod 0600 "$HOME/.ssh/deploy_key"
|
||||||
|
printf '%s\n' "$DEPLOY_KNOWN_HOSTS" > "$HOME/.ssh/known_hosts"
|
||||||
|
command -v rsync >/dev/null || { SUDO=$(command -v sudo || true); $SUDO apt-get install -y -qq rsync; }
|
||||||
|
|
||||||
|
- name: Upload and deploy
|
||||||
|
run: |
|
||||||
|
ssh_cmd="ssh -i $HOME/.ssh/deploy_key -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -o UserKnownHostsFile=$HOME/.ssh/known_hosts"
|
||||||
|
# A fresh directory per run; only deploy.sh, under the host lock,
|
||||||
|
# turns it into releases/<sha>. --omit-dir-times: the upload's age
|
||||||
|
# must be the server's, because retention prunes uploads by mtime.
|
||||||
|
incoming="/opt/petrolyst/incoming/${GITHUB_SHA}-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT:-1}-$(date -u +%s)"
|
||||||
|
rsync -az --omit-dir-times --exclude .git -e "$ssh_cmd" ./ "deploy@${DEPLOY_HOST}:${incoming}/"
|
||||||
|
$ssh_cmd "deploy@${DEPLOY_HOST}" "${incoming}/deploy/deploy.sh" "$GITHUB_SHA" "$incoming"
|
||||||
|
|
||||||
|
- name: Check from outside
|
||||||
|
run: |
|
||||||
|
test "$(curl -fsS --max-time 20 https://atoma.petrolyst.com/health)" = OK
|
||||||
|
curl -fsS --max-time 20 -o /dev/null -w '%{http_code}\n' https://traefik.petrolyst.com/ | grep -qx 302
|
||||||
+11
@@ -0,0 +1,11 @@
|
|||||||
|
# Secrets and server env files live only on the server (/opt/petrolyst/).
|
||||||
|
*.env
|
||||||
|
.env*
|
||||||
|
secrets/
|
||||||
|
# CI deploy key, generated here during first-time setup (README, Gitea step 3)
|
||||||
|
petrolyst-ci
|
||||||
|
petrolyst-ci.pub
|
||||||
|
# Local test runs
|
||||||
|
.scratch/
|
||||||
|
.DS_Store
|
||||||
|
__pycache__/
|
||||||
@@ -0,0 +1,305 @@
|
|||||||
|
# petrolyst-infra
|
||||||
|
|
||||||
|
Everything that runs on the petrolyst server (109.199.127.84, Ubuntu 24.04), deployed on every
|
||||||
|
push to `main`:
|
||||||
|
|
||||||
|
- **Traefik** at the edge for every project on the server. Its dashboard is at
|
||||||
|
https://traefik.petrolyst.com, behind Keycloak.
|
||||||
|
- **fuel-monitoring** (ATOMA): Superset at https://atoma.petrolyst.com, reading the operational
|
||||||
|
store `fuel-db` (Postgres 17 with TimescaleDB) through Trino. Sign-in goes through Keycloak
|
||||||
|
client `petrolyst`, and the UI carries the Petrolyst brand.
|
||||||
|
|
||||||
|
Customization and deployment are separate, as with media-insights and media-intelligence-docker:
|
||||||
|
|
||||||
|
| Folder | Holds |
|
||||||
|
|---|---|
|
||||||
|
| `petrolyst-insights/` | how Superset looks and behaves for Petrolyst: config, Keycloak login code, branding, patches, dashboards. See its README. |
|
||||||
|
| `traefik/` | edge proxy, TLS, the `sso` middleware (oauth2-proxy) that guards the dashboard |
|
||||||
|
| `stacks/fuel-monitoring/` | Compose stack, Superset Dockerfile, Trino config and catalog, fuel-db roles |
|
||||||
|
| `deploy/` | `deploy.sh` (run by CI), `rollback.sh` (manual), `lib.sh` (shared) |
|
||||||
|
| `host/` | one-time server setup: `bootstrap.sh`, `secrets-init.sh` |
|
||||||
|
| `keycloak/` | the `petrolyst` client for Keycloak's partial import |
|
||||||
|
| `.gitea/workflows/deploy.yml` | the pipeline |
|
||||||
|
|
||||||
|
No credential is in this repository. Secrets exist only on the server, under
|
||||||
|
`/opt/petrolyst/secrets/`; CI holds only the SSH deploy key.
|
||||||
|
|
||||||
|
## How a deploy works
|
||||||
|
|
||||||
|
1. A push to `main` (or a manual run) starts `.gitea/workflows/deploy.yml` on the act_runner. It
|
||||||
|
lints the scripts, rsyncs the checkout to a fresh `/opt/petrolyst/incoming/<sha>-<run>/`, and
|
||||||
|
calls `deploy/deploy.sh <sha> <that dir>` over SSH as user `deploy`.
|
||||||
|
2. `deploy.sh` takes the host lock `/opt/petrolyst/deploy.lock`, which it shares with
|
||||||
|
`rollback.sh`, and keeps it until it exits.
|
||||||
|
3. It finalizes the upload. A new commit is moved to `releases/<sha>`, and the release is made
|
||||||
|
read-only. A re-run of a deployed commit must match it byte for byte; a mismatch aborts.
|
||||||
|
4. It resolves the artifacts and writes them to `releases/<sha>/ARTIFACTS`:
|
||||||
|
- `SUPERSET_IMAGE_TAG` is a hash of exactly the Superset image's build inputs. They include
|
||||||
|
`superset/init.sh`, so changing that file runs `superset-init` again.
|
||||||
|
- `TRINO_CONFIG_HASH` and `TRAEFIK_CONFIG_HASH` are hashes of the bind-mounted config.
|
||||||
|
- Compose recreates a container only when its value changes.
|
||||||
|
5. Preflight, before anything live changes:
|
||||||
|
- every secret and env file exists;
|
||||||
|
- `petrolyst-insights/` holds no deployment files;
|
||||||
|
- `docker compose config` passes for every stack;
|
||||||
|
- the Superset image is built if it is new;
|
||||||
|
- the **downgrade guard** refuses an image older than the metadata database's schema.
|
||||||
|
6. It promotes the release to `/opt/petrolyst/infra`, then starts Traefik, then each stack under
|
||||||
|
`stacks/`.
|
||||||
|
7. For Superset, it reads the metadata database's state:
|
||||||
|
- An empty database is a fresh install: it runs `superset-init`.
|
||||||
|
- If the image changed, or the database is not at the image's migration head: it stops
|
||||||
|
`superset`, writes `backups/superset-meta_<old tag>_<revision>_<time>.dump`, then runs
|
||||||
|
`superset-init` (`db upgrade`, `init`, registers "Fuel (Trino)").
|
||||||
|
- If the image is unchanged and the schema is at its head, it does none of this.
|
||||||
|
8. Smoke tests. A failure turns CI red and prints the stacks' state.
|
||||||
|
- `/health` answers through Traefik with a valid certificate.
|
||||||
|
- `/login/` leads to Keycloak with `client_id=petrolyst` and PKCE.
|
||||||
|
- The dashboard redirects to Keycloak.
|
||||||
|
- Trino reads `fuel`, and Superset reaches Trino.
|
||||||
|
9. Retention:
|
||||||
|
- 5 releases;
|
||||||
|
- backups for 30 days;
|
||||||
|
- the 3 newest Superset images, plus any image that a kept release or backup names;
|
||||||
|
- uploads older than a day.
|
||||||
|
|
||||||
|
The runner then checks both hosts from outside.
|
||||||
|
|
||||||
|
## Server paths
|
||||||
|
|
||||||
|
| Path | Owner | What |
|
||||||
|
|---|---|---|
|
||||||
|
| `/opt/petrolyst/infra` | deploy | live tree (the promoted release) |
|
||||||
|
| `/opt/petrolyst/incoming/<sha>-<run>` | deploy | one CI upload; only `deploy.sh` turns it into a release |
|
||||||
|
| `/opt/petrolyst/releases/<sha>` | deploy, read-only | finalized release plus `ARTIFACTS`; the last 5 are kept |
|
||||||
|
| `/opt/petrolyst/secrets/{shared,traefik,fuel-monitoring}/` | root:deploy 0750, files 0444 | secrets, mounted as `/run/secrets/<name>` |
|
||||||
|
| `/opt/petrolyst/env/fuel-monitoring.env` | root:deploy 0440 | `FUEL_DB_TRINO_PASSWORD` for Trino's `${ENV:…}` |
|
||||||
|
| `/opt/petrolyst/backups/` | deploy 0750 | Superset metadata dumps |
|
||||||
|
| `/opt/petrolyst/state/` | deploy | last deployed SHA and Superset image tag; `rollback-in-progress` during a migration rollback |
|
||||||
|
|
||||||
|
## First-time setup
|
||||||
|
|
||||||
|
### 1. Gitea
|
||||||
|
|
||||||
|
1. Create the private repository `Petrolyst/petrolyst-infra` and push this repository's
|
||||||
|
`main` to it. Enable Actions under Settings → Repository → Units.
|
||||||
|
2. Runner. Check Site/Org Administration → Actions → Runners. If none is registered, run
|
||||||
|
[Gitea Runner](https://docs.gitea.com/runner/installation/docker/) as the container
|
||||||
|
`docker.io/gitea/runner:5.2.2-dind` (privileged, with `/data` and `/var/lib/docker` volumes),
|
||||||
|
for example on the Gitea host (164.68.121.117):
|
||||||
|
- register it as an **organization runner** for `Petrolyst`: take the token from
|
||||||
|
Settings → Actions → Runners → Create new Runner;
|
||||||
|
- set `GITEA_RUNNER_LABELS=ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest`;
|
||||||
|
- note its egress IPv4 (`curl -4 -s https://ifconfig.me`).
|
||||||
|
3. Generate a deploy key on your Mac, used only for this:
|
||||||
|
`ssh-keygen -t ed25519 -N '' -C petrolyst-ci -f petrolyst-ci`
|
||||||
|
4. Add repository secrets (Settings → Actions → Secrets):
|
||||||
|
- `DEPLOY_SSH_KEY`: contents of `petrolyst-ci` (the private key)
|
||||||
|
- `DEPLOY_HOST`: `109.199.127.84`
|
||||||
|
- `DEPLOY_KNOWN_HOSTS`: output of `ssh-keyscan -t ed25519 109.199.127.84`. Compare its
|
||||||
|
fingerprint (`ssh-keygen -lf <file>`) with `ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub`
|
||||||
|
run on the server.
|
||||||
|
|
||||||
|
### 2. Keycloak (realm `thedigitalbridge` on iam.thedigitalbridge.io)
|
||||||
|
|
||||||
|
1. Realm settings → Action → **Partial import**: choose `keycloak/petrolyst-partial-import.json`,
|
||||||
|
and "Skip" if a resource exists. This creates:
|
||||||
|
- the confidential client `petrolyst`: standard flow only, PKCE S256, redirect URIs for both
|
||||||
|
hosts;
|
||||||
|
- its client roles `superset-admin`, `superset-editor`, `superset-viewer` and `traefik-admin`;
|
||||||
|
- a mapper that puts those roles into the claim `petrolyst_roles`.
|
||||||
|
2. Clients → petrolyst → Credentials: copy the client secret (step 3 stores it on the server).
|
||||||
|
3. **MFA**, which Exhibit B requires. A copy of the built-in `browser` flow is not enough, for two
|
||||||
|
reasons: its Cookie step accepts an existing password-only session from another client of the
|
||||||
|
realm, and its conditional OTP subflow skips users who have not set up OTP. Instead:
|
||||||
|
- Authentication → Create flow `petrolyst-mfa` (basic flow).
|
||||||
|
- Add exactly two executions, both **Required**, in this order: `Username Password Form`,
|
||||||
|
then `OTP Form`. Add no Cookie, no Identity Provider Redirector and no conditional subflow.
|
||||||
|
- Clients → petrolyst → Advanced → Authentication flow overrides → Browser flow:
|
||||||
|
`petrolyst-mfa`.
|
||||||
|
|
||||||
|
Result: every sign-in to a Petrolyst app asks for password and OTP, and a user without OTP
|
||||||
|
enrols first. Other clients of the realm are unaffected. There is no silent SSO between
|
||||||
|
Superset and the dashboard.
|
||||||
|
4. Give people roles: Users → (user) → Role mapping → Assign role → filter by clients →
|
||||||
|
`petrolyst`. Group mappings work too.
|
||||||
|
5. Users of the Traefik dashboard need **Email verified** switched on, because oauth2-proxy
|
||||||
|
refuses unverified addresses.
|
||||||
|
|
||||||
|
### 3. Server (as root, `ssh petrolyst`)
|
||||||
|
|
||||||
|
The server needs only the two host scripts; it never gets Gitea credentials.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# on the Mac, in this repository
|
||||||
|
scp host/bootstrap.sh host/secrets-init.sh petrolyst-ci.pub petrolyst:/root/
|
||||||
|
ssh petrolyst
|
||||||
|
# on the server, as root
|
||||||
|
/root/bootstrap.sh --ci-key /root/petrolyst-ci.pub --runner-ip <runner egress IP>
|
||||||
|
/root/secrets-init.sh # generates every secret; reports the missing Keycloak one
|
||||||
|
/root/secrets-init.sh keycloak-secret # paste the secret from Keycloak step 2
|
||||||
|
```
|
||||||
|
|
||||||
|
No runner yet? Pass `--runner-ip 127.0.0.1`. That installs the CI key but leaves it unusable from
|
||||||
|
outside. Deploy manually until a runner exists (see "Manual deploy" below). When the runner is
|
||||||
|
running, run `bootstrap.sh` again with its egress IP.
|
||||||
|
|
||||||
|
`bootstrap.sh` does the following:
|
||||||
|
- installs Docker from Docker's apt repository;
|
||||||
|
- adds 4 GB swap;
|
||||||
|
- turns on ufw for 22, 80 and 443 only;
|
||||||
|
- turns off SSH password login and root password login;
|
||||||
|
- creates the `deploy` user, whose key works only from the runner IP;
|
||||||
|
- creates `/opt/petrolyst` and the Docker network `proxy`.
|
||||||
|
|
||||||
|
**Before closing your root session**, check from a second terminal that `ssh petrolyst` still
|
||||||
|
works.
|
||||||
|
|
||||||
|
### 4. First deploy
|
||||||
|
|
||||||
|
Run the workflow manually (Actions → deploy → Run workflow) or push to `main`. Without a runner,
|
||||||
|
use "Manual deploy" below. The first run builds the Superset image and takes the fresh-install
|
||||||
|
path. Let's Encrypt needs about a minute
|
||||||
|
for the two certificates; the smoke tests wait up to two minutes.
|
||||||
|
|
||||||
|
Then sign in at https://atoma.petrolyst.com with a user who has `superset-admin`. That person
|
||||||
|
becomes the first Superset admin; there is no local admin account.
|
||||||
|
|
||||||
|
## Operating
|
||||||
|
|
||||||
|
| Task | How |
|
||||||
|
|---|---|
|
||||||
|
| Deploy | push to `main` (without a runner: "Manual deploy" below) |
|
||||||
|
| Re-deploy a kept release | `sudo -u deploy /opt/petrolyst/infra/deploy/deploy.sh <sha>` |
|
||||||
|
| Roll back | see below |
|
||||||
|
| Traefik dashboard | https://traefik.petrolyst.com (role `traefik-admin`) |
|
||||||
|
| Trino shell | `docker exec -it fuel-monitoring-trino-1 trino` |
|
||||||
|
| fuel-db from the Mac | `ssh -L 5433:127.0.0.1:5433 petrolyst`, then connect to `localhost:5433`, database `fuel` |
|
||||||
|
| Logs | `docker compose -p fuel-monitoring logs -f superset` (or `-p traefik`) |
|
||||||
|
| Superset break-glass | `docker exec -it fuel-monitoring-superset-1 superset fab ...` |
|
||||||
|
|
||||||
|
**Manual deploy** (no runner): commit first, so that `git status` is clean. Then run this on the
|
||||||
|
Mac, in this repository:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sha=$(git rev-parse HEAD)
|
||||||
|
dir=/opt/petrolyst/incoming/$sha-manual-$(date -u +%s)
|
||||||
|
git archive --format=tar "$sha" |
|
||||||
|
ssh petrolyst "install -d -o deploy -g deploy -m 0755 $dir && tar -x --no-same-owner -C $dir && chown -R deploy:deploy $dir"
|
||||||
|
ssh petrolyst "sudo -u deploy $dir/deploy/deploy.sh $sha $dir"
|
||||||
|
```
|
||||||
|
|
||||||
|
`git archive` sends exactly the commit, never uncommitted or ignored files such as the deploy
|
||||||
|
key. The server then runs the same `deploy.sh` that CI runs, under the same lock. A later push
|
||||||
|
queues a CI run that waits for a runner. Cancel stale waiting runs under Actions before you
|
||||||
|
register one.
|
||||||
|
|
||||||
|
**Rollback** (`deploy/rollback.sh`, manual, under the same lock as deploys):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ls /opt/petrolyst/releases # kept releases
|
||||||
|
sudo -u deploy /opt/petrolyst/infra/deploy/rollback.sh <sha>
|
||||||
|
```
|
||||||
|
|
||||||
|
- If the target image's migration head equals the database's revision, this is a **code-only**
|
||||||
|
rollback: promote and restart.
|
||||||
|
- Otherwise it is a **migration rollback**:
|
||||||
|
1. stop `superset`;
|
||||||
|
2. drop and recreate the `superset` metadata database;
|
||||||
|
3. restore the backup made for the target image, aborting on any error;
|
||||||
|
4. check the restored revision;
|
||||||
|
5. start the retained older image.
|
||||||
|
|
||||||
|
It refuses if that backup or image is gone. `fuel-db` is never touched.
|
||||||
|
|
||||||
|
Before step 1 it writes `state/rollback-in-progress`, and it removes that file only after the
|
||||||
|
smoke tests pass. While the file exists, `deploy.sh` refuses to run. If a rollback is
|
||||||
|
interrupted, run the same command again. It also restores when the failed attempt left the
|
||||||
|
database empty or had already dropped it.
|
||||||
|
|
||||||
|
Afterwards, `git revert` the bad commit and push, so `main` matches. A plain revert without the
|
||||||
|
rollback script is stopped by the downgrade guard.
|
||||||
|
|
||||||
|
**Data loaders** (fuel-db). Every table in `public` must belong to `fuel_owner`. `trino_ro` can
|
||||||
|
read new tables only through `fuel_owner`'s default privileges, and PostgreSQL applies those
|
||||||
|
only to objects `fuel_owner` creates itself. A table that a loader creates as its own role stays
|
||||||
|
unreadable for Trino. Create each loader role so that its sessions act as `fuel_owner`, and it
|
||||||
|
cannot create anything as itself. Run this in
|
||||||
|
`docker exec -it fuel-monitoring-fuel-db-1 psql -U postgres -d fuel`:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE ROLE <loader> LOGIN PASSWORD '<password>';
|
||||||
|
GRANT fuel_owner TO <loader> WITH INHERIT FALSE, SET TRUE;
|
||||||
|
ALTER ROLE <loader> SET role TO 'fuel_owner';
|
||||||
|
```
|
||||||
|
|
||||||
|
Tables it creates, hypertables included, then belong to `fuel_owner`, and Trino can read them.
|
||||||
|
After `SET ROLE NONE`, it cannot create anything in `public`. To repair a table that another
|
||||||
|
role created, run `ALTER TABLE <t> OWNER TO fuel_owner`, then `GRANT SELECT ON <t> TO trino_ro`.
|
||||||
|
|
||||||
|
**Adding a project**: create `stacks/<name>/compose.yaml`, attach the public service to the
|
||||||
|
external network `proxy`, and set `traefik.*` labels with a unique router name and
|
||||||
|
`tls.certresolver: le`. Add the DNS record first. `deploy.sh` picks the new folder up
|
||||||
|
automatically.
|
||||||
|
|
||||||
|
`sso@docker` works only for `traefik.petrolyst.com`. Its cookie, its allowed return hosts and its
|
||||||
|
role (`traefik-admin`) are all tied to the dashboard. An app without a login of its own needs
|
||||||
|
its own gate in its stack:
|
||||||
|
- an oauth2-proxy service with its own cookie secret;
|
||||||
|
- a router for the app host's `/oauth2/` path and a forwardAuth middleware pointing at that
|
||||||
|
service, as in `traefik/compose.yaml`;
|
||||||
|
- `--redirect-url`, `--cookie-domain` and `--whitelist-domain` set to the app's host;
|
||||||
|
- an `--allowed-role` with a new client role in Keycloak;
|
||||||
|
- the app's `/oauth2/callback` added to the `petrolyst` client's redirect URIs.
|
||||||
|
|
||||||
|
## Verified locally (2026-10-23)
|
||||||
|
|
||||||
|
An end-to-end run ran the real `deploy.sh` and `rollback.sh` in a Linux container against
|
||||||
|
Docker Desktop, with a scratch `/opt/petrolyst`. Production differences in that run:
|
||||||
|
- a test CA certificate stood in for Let's Encrypt;
|
||||||
|
- the ACME server in the test copy was unreachable on purpose;
|
||||||
|
- a dummy Keycloak client secret was used.
|
||||||
|
|
||||||
|
Results:
|
||||||
|
- The fresh install took the init path.
|
||||||
|
- An identical re-run recreated nothing and left `ARTIFACTS` byte-identical.
|
||||||
|
- A Trino-only change recreated only `trino`.
|
||||||
|
- A Superset change stopped, backed up, migrated and recreated only `superset`.
|
||||||
|
- A change to `superset/init.sh` alone did the same, and the changed script ran.
|
||||||
|
- A same-SHA upload with different content was refused.
|
||||||
|
- An older image against a newer schema was refused before promotion.
|
||||||
|
- A migration rollback dropped a table added by a "newer migration" and restored the revision.
|
||||||
|
- A code-only rollback brought back the previous image.
|
||||||
|
- A re-deploy of an unchanged image, against a schema one migration behind its head, migrated
|
||||||
|
the schema back to the head.
|
||||||
|
- A migration rollback whose restore failed after the drop left `rollback-in-progress` and an
|
||||||
|
empty database. `deploy.sh` then refused to run. Running the rollback again restored the
|
||||||
|
database and removed the file. It did the same after a simulated stop between dropping and
|
||||||
|
recreating the database. Without the file, a missing database stops the rollback with an error.
|
||||||
|
- The host lock blocked a concurrent run.
|
||||||
|
- Retention pruned read-only releases.
|
||||||
|
- Retention removed a queued upload that a plain `rsync -a` had given an old checkout's
|
||||||
|
directory times. It kept one uploaded with `--omit-dir-times`.
|
||||||
|
- In `fuel-db`, a loader that created a table as its own role left it unreadable for `trino_ro`.
|
||||||
|
With the "Data loaders" recipe, tables (hypertables too) belonged to `fuel_owner` and Trino
|
||||||
|
could read them. After `SET ROLE NONE`, the loader could not create anything.
|
||||||
|
- Smoke tests passed, including the redirects to the real Keycloak with `client_id=petrolyst`
|
||||||
|
and PKCE S256.
|
||||||
|
- TLS 1.1 and unknown SNI were refused, and HTTP redirected to HTTPS.
|
||||||
|
- Trino and `trino_ro` both refused writes.
|
||||||
|
|
||||||
|
Not verified locally:
|
||||||
|
- Let's Encrypt issuance;
|
||||||
|
- a real Keycloak sign-in, OTP and logout (they need the client from setup step 2);
|
||||||
|
- the theme as rendered in a browser;
|
||||||
|
- the Gitea runner.
|
||||||
|
|
||||||
|
## Open items
|
||||||
|
|
||||||
|
- Exhibit B: encryption at rest; confirm the VPS is in a German data centre.
|
||||||
|
- Nightly `pg_dump` of both databases with an offsite copy. Today only the pre-migration
|
||||||
|
Superset dumps exist.
|
||||||
|
- Certificate expiry monitoring. There is no ACME e-mail, so nothing warns if renewal fails
|
||||||
|
between deploys.
|
||||||
|
- Celery worker and beat, if Superset alerts or reports are needed.
|
||||||
|
- A Petrolyst login theme for Keycloak itself, which lives on the Keycloak host.
|
||||||
Executable
+163
@@ -0,0 +1,163 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Server-side deploy, run as the non-root deploy user:
|
||||||
|
# deploy.sh <sha> <incoming-dir> CI: finalize the upload as releases/<sha>, then deploy it
|
||||||
|
# deploy.sh <sha> re-deploy a release that is already finalized
|
||||||
|
#
|
||||||
|
# Order: lock, finalize, artifacts, preflight, image build and downgrade guard
|
||||||
|
# (nothing live touched yet), then promote, Traefik, each stack, smoke tests,
|
||||||
|
# retention. Rollback is a separate, manual script (rollback.sh) under the same
|
||||||
|
# lock.
|
||||||
|
set -euo pipefail
|
||||||
|
# shellcheck source=deploy/lib.sh
|
||||||
|
. "$(dirname "$(readlink -f "$0")")/lib.sh"
|
||||||
|
|
||||||
|
# finalize_release <sha> <upload>: CI never writes into releases/. A new SHA is
|
||||||
|
# moved in atomically; a re-run of a deployed SHA must match it byte for byte
|
||||||
|
# and leaves the finalized release, ARTIFACTS included, untouched.
|
||||||
|
finalize_release() {
|
||||||
|
local release=$RELEASES/$1 upload
|
||||||
|
upload=$(readlink -f "$2") || die "upload $2 does not exist"
|
||||||
|
[[ $upload == "$(readlink -f "$INCOMING")"/* && -d $upload ]] || die "upload $2 is not a directory under $INCOMING"
|
||||||
|
if [ ! -e "$release" ]; then
|
||||||
|
mv "$upload" "$release"
|
||||||
|
log "finalized $release"
|
||||||
|
elif diff -r --exclude=ARTIFACTS "$upload" "$release" >/dev/null; then
|
||||||
|
rm -rf "$upload"
|
||||||
|
log "release $1 already finalized and identical to the upload; reusing it"
|
||||||
|
else
|
||||||
|
die "upload differs from the finalized release $release; one SHA cannot have two contents"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# resolve_artifacts <release>: compute the artifact identities from the
|
||||||
|
# release's own files, record them once in ARTIFACTS, then freeze the release.
|
||||||
|
resolve_artifacts() {
|
||||||
|
local release=$1 computed
|
||||||
|
computed=$(compute_artifacts "$release")
|
||||||
|
if [ -e "$release/ARTIFACTS" ]; then
|
||||||
|
[ "$(cat "$release/ARTIFACTS")" = "$computed" ] || die "$release/ARTIFACTS does not match the release's files"
|
||||||
|
else
|
||||||
|
# A crash between the move and this step leaves no ARTIFACTS; the files
|
||||||
|
# are unchanged, so the computed values are the same.
|
||||||
|
chmod u+w "$release"
|
||||||
|
printf '%s\n' "$computed" >"$release/ARTIFACTS"
|
||||||
|
fi
|
||||||
|
chmod -R a-w "$release"
|
||||||
|
load_artifacts "$release"
|
||||||
|
log "artifacts: superset $SUPERSET_IMAGE_TAG, trino config $TRINO_CONFIG_HASH, traefik config $TRAEFIK_CONFIG_HASH"
|
||||||
|
}
|
||||||
|
|
||||||
|
# prepare_fuel_monitoring <release>: before anything live changes, build the
|
||||||
|
# Superset image from the release and refuse an image older than the schema of
|
||||||
|
# a running metadata database.
|
||||||
|
prepare_fuel_monitoring() {
|
||||||
|
local image=fuel-superset:$SUPERSET_IMAGE_TAG revision
|
||||||
|
if ! "${DOCKER[@]}" image inspect "$image" >/dev/null 2>&1; then
|
||||||
|
log "building $image"
|
||||||
|
compose "$1" stacks/fuel-monitoring build superset
|
||||||
|
fi
|
||||||
|
[ -e "$LIVE/stacks/fuel-monitoring/compose.yaml" ] || return 0
|
||||||
|
[ -n "$(fm ps --status running -q superset-db 2>/dev/null)" ] || return 0
|
||||||
|
revision=$(metadata_revision)
|
||||||
|
[ "$revision" = fresh ] || downgrade_guard "$image" "$revision"
|
||||||
|
}
|
||||||
|
|
||||||
|
downgrade_guard() {
|
||||||
|
alembic "$1" has "$2" ||
|
||||||
|
die "the metadata database is at revision $2, which $1 does not know: the image is older than the schema. Use deploy/rollback.sh."
|
||||||
|
}
|
||||||
|
|
||||||
|
deploy_fuel_monitoring() {
|
||||||
|
local image=fuel-superset:$SUPERSET_IMAGE_TAG revision running head
|
||||||
|
fm up -d --wait superset-db redis
|
||||||
|
revision=$(metadata_revision)
|
||||||
|
if [ "$revision" = fresh ]; then
|
||||||
|
log "metadata database is empty: fresh install, running superset-init"
|
||||||
|
fm run --rm superset-init
|
||||||
|
else
|
||||||
|
# Again here: the database may not have been running during preparation.
|
||||||
|
downgrade_guard "$image" "$revision"
|
||||||
|
running=$(running_superset_tag)
|
||||||
|
# An unchanged tag alone is not enough: the schema may have been changed
|
||||||
|
# under the container (a restore, a manual downgrade).
|
||||||
|
head=$(alembic "$image" heads)
|
||||||
|
if [ "$running" != "$SUPERSET_IMAGE_TAG" ] || [ "$revision" != "$head" ]; then
|
||||||
|
log "Superset image ${running:-unknown} -> $SUPERSET_IMAGE_TAG, schema $revision -> $head: stop, back up, migrate"
|
||||||
|
fm stop superset
|
||||||
|
backup_metadata "${running:-unknown}" "$revision"
|
||||||
|
fm run --rm superset-init
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fm up -d --wait --remove-orphans
|
||||||
|
}
|
||||||
|
|
||||||
|
# Old releases, stale uploads, old backups and unreferenced Superset images.
|
||||||
|
retention() {
|
||||||
|
local current=$1 dir tag referenced
|
||||||
|
while IFS= read -r dir; do
|
||||||
|
[ "${dir##*/}" != "$current" ] || continue
|
||||||
|
chmod -R u+w "$dir"
|
||||||
|
rm -rf "$dir"
|
||||||
|
log "removed release ${dir##*/}"
|
||||||
|
done < <(find "$RELEASES" -mindepth 1 -maxdepth 1 -type d -printf '%T@ %p\n' | sort -rn | tail -n +6 | cut -d' ' -f2-)
|
||||||
|
# Uploads older than a day were never finalized (interrupted CI runs). Their
|
||||||
|
# mtime is the server's (the workflow's rsync omits directory times), and a
|
||||||
|
# CI job lives at most 45 minutes, so an upload being sent or waiting for the
|
||||||
|
# lock is never this old.
|
||||||
|
while IFS= read -r -d '' dir; do
|
||||||
|
chmod -R u+w "$dir"
|
||||||
|
rm -rf "$dir"
|
||||||
|
log "removed stale upload ${dir##*/}"
|
||||||
|
done < <(find "$INCOMING" -mindepth 1 -maxdepth 1 -mmin +1440 -print0)
|
||||||
|
find "$BACKUPS" -maxdepth 1 -name 'superset-meta_*.dump' -mtime +30 -delete
|
||||||
|
find "$BACKUPS" -maxdepth 1 -name '.partial-*' -mmin +60 -delete
|
||||||
|
# Keep the 3 newest images plus every image a kept release or backup names.
|
||||||
|
referenced=$(
|
||||||
|
sed -n 's/^SUPERSET_IMAGE_TAG=//p' "$RELEASES"/*/ARTIFACTS 2>/dev/null || true
|
||||||
|
find "$BACKUPS" -maxdepth 1 -name 'superset-meta_*.dump' -exec basename {} \; |
|
||||||
|
sed -n 's/^superset-meta_\([0-9a-f]\{16\}\)_.*/\1/p'
|
||||||
|
)
|
||||||
|
"${DOCKER[@]}" image ls fuel-superset --format '{{.Tag}}' | tail -n +4 | while IFS= read -r tag; do
|
||||||
|
grep -qxF "$tag" <<<"$referenced" && continue
|
||||||
|
if "${DOCKER[@]}" image rm "fuel-superset:$tag" >/dev/null; then
|
||||||
|
log "removed image fuel-superset:$tag"
|
||||||
|
else
|
||||||
|
log "kept image fuel-superset:$tag (still in use)"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
main() {
|
||||||
|
local sha=${1:-} upload=${2:-} release dir
|
||||||
|
[[ $sha =~ ^[0-9a-f]{40}$ ]] || die "usage: deploy.sh <40-hex sha> [incoming-dir]"
|
||||||
|
acquire_lock 900
|
||||||
|
[ ! -e "$ROLLBACK_MARK" ] ||
|
||||||
|
die "a rollback to $(cat "$ROLLBACK_MARK") did not finish and may have left the metadata database half-restored; run deploy/rollback.sh again first"
|
||||||
|
release=$RELEASES/$sha
|
||||||
|
[ -z "$upload" ] || finalize_release "$sha" "$upload"
|
||||||
|
[ -d "$release" ] || die "no release $release"
|
||||||
|
resolve_artifacts "$release"
|
||||||
|
preflight "$release"
|
||||||
|
prepare_fuel_monitoring "$release"
|
||||||
|
|
||||||
|
# From here on the live state changes; on failure, print the stacks' state.
|
||||||
|
trap on_exit EXIT
|
||||||
|
promote "$release" "$sha"
|
||||||
|
ensure_proxy_network
|
||||||
|
log "traefik"
|
||||||
|
compose "$LIVE" traefik up -d --wait --remove-orphans
|
||||||
|
while IFS= read -r dir; do
|
||||||
|
log "$dir"
|
||||||
|
case $dir in
|
||||||
|
stacks/fuel-monitoring) deploy_fuel_monitoring ;;
|
||||||
|
*) compose "$LIVE" "$dir" up -d --wait --remove-orphans ;;
|
||||||
|
esac
|
||||||
|
done < <(stack_dirs "$LIVE")
|
||||||
|
smoke_tests
|
||||||
|
record_state "$sha"
|
||||||
|
trap - EXIT
|
||||||
|
retention "$sha"
|
||||||
|
log "deployed $sha"
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
+271
@@ -0,0 +1,271 @@
|
|||||||
|
# shellcheck shell=bash
|
||||||
|
# Shared by deploy.sh and rollback.sh; sourced, not executed.
|
||||||
|
|
||||||
|
# Server layout (README "Server paths"). PETROLYST_ROOT exists only so the
|
||||||
|
# scripts can be exercised against a scratch directory; production never sets it.
|
||||||
|
# shellcheck disable=SC2034 # RELEASES and INCOMING are used by the sourcing scripts
|
||||||
|
ROOT=${PETROLYST_ROOT:-/opt/petrolyst}
|
||||||
|
LIVE=$ROOT/infra
|
||||||
|
RELEASES=$ROOT/releases
|
||||||
|
INCOMING=$ROOT/incoming
|
||||||
|
BACKUPS=$ROOT/backups
|
||||||
|
STATE=$ROOT/state
|
||||||
|
LOCK_FILE=$ROOT/deploy.lock
|
||||||
|
# Exists while rollback.sh replaces the metadata database, and stays behind if
|
||||||
|
# it is interrupted there; deploy.sh refuses to run until a rollback completes.
|
||||||
|
ROLLBACK_MARK=$STATE/rollback-in-progress
|
||||||
|
|
||||||
|
ATOMA_HOST=${ATOMA_HOST:-atoma.petrolyst.com}
|
||||||
|
TRAEFIK_HOST=${TRAEFIK_HOST:-traefik.petrolyst.com}
|
||||||
|
KEYCLOAK_REALM_URL=${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/thedigitalbridge}
|
||||||
|
PETROLYST_ROOT=$ROOT
|
||||||
|
export ATOMA_HOST TRAEFIK_HOST KEYCLOAK_REALM_URL PETROLYST_ROOT
|
||||||
|
|
||||||
|
# --host pins the daemon, as in petrolyst-ui/deploy/deploy.sh: a stale
|
||||||
|
# `docker context use` must not send this deploy to another daemon.
|
||||||
|
DOCKER=(docker --host unix:///var/run/docker.sock)
|
||||||
|
|
||||||
|
# Build inputs of the Superset image, relative to a release. Everything the
|
||||||
|
# Dockerfile copies is listed; nothing else is, so the tag changes exactly when
|
||||||
|
# the image would.
|
||||||
|
SUPERSET_INPUTS=(
|
||||||
|
stacks/fuel-monitoring/superset/Dockerfile
|
||||||
|
stacks/fuel-monitoring/superset/init.sh
|
||||||
|
petrolyst-insights/superset_config.py
|
||||||
|
petrolyst-insights/petrolyst_security.py
|
||||||
|
petrolyst-insights/assets/branding
|
||||||
|
petrolyst-insights/patches
|
||||||
|
)
|
||||||
|
|
||||||
|
# Every file the stacks need from /opt/petrolyst (host/secrets-init.sh).
|
||||||
|
REQUIRED_FILES=(
|
||||||
|
secrets/shared/keycloak_petrolyst_client_secret
|
||||||
|
secrets/traefik/oauth2_proxy_cookie_secret
|
||||||
|
secrets/fuel-monitoring/superset_secret_key
|
||||||
|
secrets/fuel-monitoring/superset_db_password
|
||||||
|
secrets/fuel-monitoring/fuel_db_password
|
||||||
|
secrets/fuel-monitoring/fuel_db_trino_password
|
||||||
|
env/fuel-monitoring.env
|
||||||
|
)
|
||||||
|
|
||||||
|
log() { printf '%s %s\n' "$(date -u +%H:%M:%S)" "$*" >&2; }
|
||||||
|
die() { log "ERROR: $*"; exit 1; }
|
||||||
|
|
||||||
|
[ "$(id -u)" != 0 ] || die "run as the deploy user (sudo -u deploy $0 ...), not as root"
|
||||||
|
|
||||||
|
# acquire_lock <seconds>: deploy.sh and rollback.sh share this lock, so a CI
|
||||||
|
# deploy and a manual rollback never run at the same time. Held until exit.
|
||||||
|
acquire_lock() {
|
||||||
|
exec 9>>"$LOCK_FILE"
|
||||||
|
flock -w "$1" 9 || die "another deploy or rollback holds $LOCK_FILE (waited $1 s)"
|
||||||
|
}
|
||||||
|
|
||||||
|
sha256() {
|
||||||
|
if command -v sha256sum >/dev/null; then sha256sum; else shasum -a 256; fi | cut -c1-64
|
||||||
|
}
|
||||||
|
|
||||||
|
# hash_inputs <dir> <path>...: one short digest over the names and contents of
|
||||||
|
# every file under the paths, independent of mtimes and listing order.
|
||||||
|
hash_inputs() {
|
||||||
|
local dir=$1
|
||||||
|
shift
|
||||||
|
(
|
||||||
|
cd "$dir" || exit 1
|
||||||
|
find "$@" -type f -print0 | LC_ALL=C sort -z |
|
||||||
|
while IFS= read -r -d '' f; do printf '%s %s\n' "$f" "$(sha256 <"$f")"; done
|
||||||
|
) | sha256 | cut -c1-16
|
||||||
|
}
|
||||||
|
|
||||||
|
compute_artifacts() {
|
||||||
|
printf 'SUPERSET_IMAGE_TAG=%s\n' "$(hash_inputs "$1" "${SUPERSET_INPUTS[@]}")"
|
||||||
|
printf 'TRINO_CONFIG_HASH=%s\n' "$(hash_inputs "$1" stacks/fuel-monitoring/trino/etc)"
|
||||||
|
# Only the static config: dynamic/ is watched and applies without a restart.
|
||||||
|
printf 'TRAEFIK_CONFIG_HASH=%s\n' "$(hash_inputs "$1" traefik/static)"
|
||||||
|
}
|
||||||
|
|
||||||
|
# load_artifacts <release>: export the release's recorded artifact identities.
|
||||||
|
load_artifacts() {
|
||||||
|
local file=$1/ARTIFACTS line key value
|
||||||
|
[ -s "$file" ] || die "$file is missing"
|
||||||
|
unset SUPERSET_IMAGE_TAG TRINO_CONFIG_HASH TRAEFIK_CONFIG_HASH
|
||||||
|
while IFS= read -r line; do
|
||||||
|
key=${line%%=*}
|
||||||
|
value=${line#*=}
|
||||||
|
[[ $key =~ ^(SUPERSET_IMAGE_TAG|TRINO_CONFIG_HASH|TRAEFIK_CONFIG_HASH)$ && $value =~ ^[0-9a-f]{16}$ ]] ||
|
||||||
|
die "unexpected line in $file: $line"
|
||||||
|
printf -v "$key" '%s' "$value"
|
||||||
|
export "${key?}"
|
||||||
|
done <"$file"
|
||||||
|
[ -n "${SUPERSET_IMAGE_TAG:-}" ] && [ -n "${TRINO_CONFIG_HASH:-}" ] && [ -n "${TRAEFIK_CONFIG_HASH:-}" ] ||
|
||||||
|
die "$file is incomplete"
|
||||||
|
}
|
||||||
|
|
||||||
|
# compose <tree> <dir> <args...>: run Compose for the stack in <tree>/<dir>.
|
||||||
|
compose() {
|
||||||
|
local tree=$1 dir=$2
|
||||||
|
shift 2
|
||||||
|
"${DOCKER[@]}" compose --project-directory "$tree/$dir" -f "$tree/$dir/compose.yaml" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
fm() { compose "$LIVE" stacks/fuel-monitoring "$@"; }
|
||||||
|
|
||||||
|
stack_dirs() {
|
||||||
|
local f
|
||||||
|
for f in "$1"/stacks/*/compose.yaml; do
|
||||||
|
[ -e "$f" ] || continue
|
||||||
|
f=${f#"$1"/}
|
||||||
|
printf '%s\n' "${f%/compose.yaml}"
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
preflight() {
|
||||||
|
local release=$1 f bad dir
|
||||||
|
for f in "${REQUIRED_FILES[@]}"; do
|
||||||
|
[ -s "$ROOT/$f" ] || die "missing or empty: $ROOT/$f (run host/secrets-init.sh as root)"
|
||||||
|
done
|
||||||
|
# oauth2-proxy uses the client secret file byte for byte.
|
||||||
|
f=$ROOT/secrets/shared/keycloak_petrolyst_client_secret
|
||||||
|
[ "$(tail -c1 "$f" | wc -l)" -eq 0 ] ||
|
||||||
|
die "$f ends with a newline; store it again with: host/secrets-init.sh keycloak-secret"
|
||||||
|
bad=$(find "$release/petrolyst-insights" \( -name 'Dockerfile*' -o -name '*compose*.y*ml' \
|
||||||
|
-o -name '*.env' -o -name '.env' -o -path '*/.gitea/*' -o -path '*/.github/*' \) -print)
|
||||||
|
[ -z "$bad" ] || die "petrolyst-insights/ holds customization only; found: $bad"
|
||||||
|
compose "$release" traefik config -q
|
||||||
|
while IFS= read -r dir; do compose "$release" "$dir" config -q; done < <(stack_dirs "$release")
|
||||||
|
}
|
||||||
|
|
||||||
|
# promote <release> <sha>: make the release the live tree. --chmod applies to
|
||||||
|
# the destination only: the live tree stays writable, the release read-only.
|
||||||
|
promote() {
|
||||||
|
mkdir -p "$LIVE"
|
||||||
|
rsync -a --delete --chmod=Du+w,Fu+w "$1/" "$LIVE/"
|
||||||
|
printf '%s\n' "$2" >"$LIVE/REVISION"
|
||||||
|
}
|
||||||
|
|
||||||
|
ensure_proxy_network() {
|
||||||
|
"${DOCKER[@]}" network inspect proxy >/dev/null 2>&1 || "${DOCKER[@]}" network create proxy >/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
# metadata_revision: the metadata database's Alembic revision, "fresh" if it
|
||||||
|
# has no alembic_version table yet; any other error (connection, auth) aborts.
|
||||||
|
metadata_revision() {
|
||||||
|
local out
|
||||||
|
if out=$(fm exec -T superset-db psql -U superset -d superset -tAc 'SELECT version_num FROM alembic_version' 2>&1); then
|
||||||
|
[[ $out =~ ^[0-9a-z_]+$ ]] || die "unexpected alembic_version content: $out"
|
||||||
|
printf '%s\n' "$out"
|
||||||
|
elif [[ $out == *'relation "alembic_version" does not exist'* ]]; then
|
||||||
|
printf 'fresh\n'
|
||||||
|
else
|
||||||
|
die "cannot read the metadata database's revision: $out"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# alembic <image> heads | has <rev>: read Superset's migration graph from the
|
||||||
|
# image without starting the app (loading the scripts through Alembic would
|
||||||
|
# import Superset's models, which need an initialised app).
|
||||||
|
alembic() {
|
||||||
|
"${DOCKER[@]}" run --rm -i --network none --entrypoint python "$1" - "${@:2}" <<'PY'
|
||||||
|
import ast, pathlib, sys
|
||||||
|
|
||||||
|
revs, downs = set(), set()
|
||||||
|
for path in pathlib.Path("/app/superset/migrations/versions").glob("*.py"):
|
||||||
|
for node in ast.parse(path.read_text()).body:
|
||||||
|
if isinstance(node, ast.Assign) and len(node.targets) == 1:
|
||||||
|
target, value = node.targets[0], node.value
|
||||||
|
elif isinstance(node, ast.AnnAssign) and node.value is not None:
|
||||||
|
target, value = node.target, node.value
|
||||||
|
else:
|
||||||
|
continue
|
||||||
|
if isinstance(target, ast.Name) and target.id in ("revision", "down_revision"):
|
||||||
|
found = ast.literal_eval(value)
|
||||||
|
found = found if isinstance(found, (tuple, list)) else (found,)
|
||||||
|
(revs if target.id == "revision" else downs).update(v for v in found if v)
|
||||||
|
if sys.argv[1] == "heads":
|
||||||
|
print(" ".join(sorted(revs - downs)))
|
||||||
|
else:
|
||||||
|
sys.exit(0 if sys.argv[2] in revs else 3)
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
# The image tag of the existing superset container, or of the last successful
|
||||||
|
# deploy if the container is gone; empty if neither is known.
|
||||||
|
running_superset_tag() {
|
||||||
|
local id ref
|
||||||
|
id=$(fm ps -a -q superset 2>/dev/null | head -n1)
|
||||||
|
if [ -n "$id" ]; then
|
||||||
|
ref=$("${DOCKER[@]}" inspect --format '{{.Config.Image}}' "$id")
|
||||||
|
printf '%s\n' "${ref#fuel-superset:}"
|
||||||
|
elif [ -s "$STATE/superset-image-tag" ]; then
|
||||||
|
cat "$STATE/superset-image-tag"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# backup_metadata <image-tag> <revision>: dump the metadata database under a
|
||||||
|
# name that rollback.sh can match to the image whose schema it holds.
|
||||||
|
backup_metadata() {
|
||||||
|
local name tmp
|
||||||
|
name=superset-meta_${1}_${2}_$(date -u +%Y%m%dT%H%M%SZ).dump
|
||||||
|
tmp=$(mktemp "$BACKUPS/.partial-XXXXXX")
|
||||||
|
fm exec -T superset-db pg_dump -U superset -d superset -Fc >"$tmp"
|
||||||
|
[ -s "$tmp" ] || die "metadata backup came out empty"
|
||||||
|
mv "$tmp" "$BACKUPS/$name"
|
||||||
|
log "metadata backup: $BACKUPS/$name"
|
||||||
|
}
|
||||||
|
|
||||||
|
record_state() {
|
||||||
|
mkdir -p "$STATE"
|
||||||
|
printf '%s\n' "$SUPERSET_IMAGE_TAG" >"$STATE/superset-image-tag"
|
||||||
|
printf '%s\n' "$1" >"$STATE/deployed-sha"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Status code and resolved redirect target of one request through the local
|
||||||
|
# Traefik, with certificate verification against the real hostname.
|
||||||
|
probe() {
|
||||||
|
local host=$1 path=$2
|
||||||
|
curl -q --noproxy '*' -sS -o /dev/null --max-time 20 --resolve "$host:443:127.0.0.1" \
|
||||||
|
-w '%{http_code} %{redirect_url}\n' "https://$host$path"
|
||||||
|
}
|
||||||
|
|
||||||
|
smoke_tests() {
|
||||||
|
local i body code location
|
||||||
|
log "smoke tests"
|
||||||
|
# First deploy: Let's Encrypt may need a minute to issue the certificate.
|
||||||
|
for i in $(seq 1 24); do
|
||||||
|
body=$(curl -q --noproxy '*' -fsS --max-time 10 --resolve "$ATOMA_HOST:443:127.0.0.1" \
|
||||||
|
"https://$ATOMA_HOST/health" 2>/dev/null) && [ "$body" = OK ] && break
|
||||||
|
[ "$i" -lt 24 ] || die "https://$ATOMA_HOST/health did not answer OK with a valid certificate"
|
||||||
|
sleep 5
|
||||||
|
done
|
||||||
|
|
||||||
|
read -r code location < <(probe "$ATOMA_HOST" /login/)
|
||||||
|
[ "$code" = 302 ] && [[ $location == "https://$ATOMA_HOST/login/keycloak"* ]] ||
|
||||||
|
die "/login/ should redirect to /login/keycloak, got $code $location"
|
||||||
|
|
||||||
|
read -r code location < <(probe "$ATOMA_HOST" /login/keycloak)
|
||||||
|
[ "$code" = 302 ] && [[ $location == "$KEYCLOAK_REALM_URL/protocol/openid-connect/auth?"* ]] &&
|
||||||
|
[[ $location == *client_id=petrolyst* && $location == *code_challenge_method=S256* ]] ||
|
||||||
|
die "/login/keycloak should redirect to Keycloak with client_id=petrolyst and PKCE S256, got $code $location"
|
||||||
|
|
||||||
|
read -r code location < <(probe "$TRAEFIK_HOST" /)
|
||||||
|
[ "$code" = 302 ] && [[ $location == "$KEYCLOAK_REALM_URL/"* || $location == *"/oauth2/start"* ]] ||
|
||||||
|
die "https://$TRAEFIK_HOST/ should redirect to Keycloak, got $code $location"
|
||||||
|
|
||||||
|
fm exec -T trino trino --execute 'SELECT count(*) FROM fuel.information_schema.schemata' >/dev/null ||
|
||||||
|
die "Trino cannot read the fuel catalog"
|
||||||
|
fm exec -T superset curl -fsS --max-time 10 http://trino:8080/v1/info >/dev/null ||
|
||||||
|
die "Superset cannot reach Trino"
|
||||||
|
log "smoke tests passed"
|
||||||
|
}
|
||||||
|
|
||||||
|
# EXIT trap for the part of a run that changes live state.
|
||||||
|
on_exit() {
|
||||||
|
local status=$?
|
||||||
|
[ "$status" = 0 ] || diagnostics
|
||||||
|
}
|
||||||
|
|
||||||
|
diagnostics() {
|
||||||
|
log "state of the stacks:"
|
||||||
|
compose "$LIVE" traefik ps -a >&2 || true
|
||||||
|
fm ps -a >&2 || true
|
||||||
|
fm logs --no-color --tail=100 >&2 || true
|
||||||
|
}
|
||||||
Executable
+91
@@ -0,0 +1,91 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Manual rollback to a kept release, run on the server as the deploy user:
|
||||||
|
# sudo -u deploy /opt/petrolyst/infra/deploy/rollback.sh <sha>
|
||||||
|
#
|
||||||
|
# Code-only when the target image's Alembic head equals the metadata
|
||||||
|
# database's revision. Otherwise a migration rollback: the metadata database is
|
||||||
|
# dropped, recreated and restored from the backup taken for the target image,
|
||||||
|
# and the restored revision is checked before anything starts. Until the
|
||||||
|
# rollback completes, state/rollback-in-progress blocks deploy.sh; an
|
||||||
|
# interrupted rollback is finished by running it again. Afterwards,
|
||||||
|
# `git revert` and push so main matches; deploy.sh's downgrade guard stops a
|
||||||
|
# plain revert from running against a newer schema.
|
||||||
|
set -euo pipefail
|
||||||
|
# shellcheck source=deploy/lib.sh
|
||||||
|
. "$(dirname "$(readlink -f "$0")")/lib.sh"
|
||||||
|
|
||||||
|
# Destructive steps below are hard-wired to service superset-db and database
|
||||||
|
# superset; nothing here can be pointed at fuel-db.
|
||||||
|
restore_metadata() {
|
||||||
|
local backup=$1 head=$2 restored
|
||||||
|
log "migration rollback: restoring $backup"
|
||||||
|
fm stop superset
|
||||||
|
fm exec -T superset-db psql -U superset -d postgres -v ON_ERROR_STOP=1 \
|
||||||
|
-c 'DROP DATABASE IF EXISTS superset WITH (FORCE)' \
|
||||||
|
-c 'CREATE DATABASE superset OWNER superset'
|
||||||
|
fm exec -T superset-db pg_restore -U superset -d superset --exit-on-error --single-transaction <"$backup"
|
||||||
|
restored=$(metadata_revision)
|
||||||
|
[ "$restored" = "$head" ] ||
|
||||||
|
die "restored revision $restored is not the target head $head; superset stays stopped"
|
||||||
|
}
|
||||||
|
|
||||||
|
# metadata_db_exists: whether database superset exists, asked through the
|
||||||
|
# maintenance database; a connection or authentication error aborts.
|
||||||
|
metadata_db_exists() {
|
||||||
|
local n
|
||||||
|
n=$(fm exec -T superset-db psql -U superset -d postgres -tAc "SELECT count(*) FROM pg_database WHERE datname = 'superset'") ||
|
||||||
|
die "cannot reach the metadata database server"
|
||||||
|
[[ $n =~ ^[01]$ ]] || die "unexpected answer from pg_database: $n"
|
||||||
|
[ "$n" = 1 ]
|
||||||
|
}
|
||||||
|
|
||||||
|
main() {
|
||||||
|
local sha=${1:-} release image revision head backup dir
|
||||||
|
[[ $sha =~ ^[0-9a-f]{40}$ ]] || die "usage: rollback.sh <40-hex sha of a kept release>"
|
||||||
|
acquire_lock 600
|
||||||
|
release=$RELEASES/$sha
|
||||||
|
[ -s "$release/ARTIFACTS" ] || die "no finalized release with ARTIFACTS at $release"
|
||||||
|
# The release's recorded identities, never recomputed.
|
||||||
|
load_artifacts "$release"
|
||||||
|
image=fuel-superset:$SUPERSET_IMAGE_TAG
|
||||||
|
"${DOCKER[@]}" image inspect "$image" >/dev/null 2>&1 || die "image $image is no longer on this host"
|
||||||
|
preflight "$release"
|
||||||
|
|
||||||
|
trap on_exit EXIT
|
||||||
|
fm up -d --wait superset-db redis
|
||||||
|
if [ -e "$ROLLBACK_MARK" ] && ! metadata_db_exists; then
|
||||||
|
# An earlier attempt stopped between DROP and CREATE.
|
||||||
|
revision=fresh
|
||||||
|
else
|
||||||
|
revision=$(metadata_revision)
|
||||||
|
fi
|
||||||
|
# Empty or missing is expected only after an interrupted restore, which left the mark.
|
||||||
|
[ "$revision" != fresh ] || [ -e "$ROLLBACK_MARK" ] || die "the metadata database is empty; there is nothing to roll back"
|
||||||
|
head=$(alembic "$image" heads)
|
||||||
|
[[ $head =~ ^[0-9a-z_]+$ ]] || die "$image has no single migration head: $head"
|
||||||
|
|
||||||
|
if [ "$revision" = "$head" ]; then
|
||||||
|
log "code-only rollback: the schema is already at $head"
|
||||||
|
else
|
||||||
|
backup=$(find "$BACKUPS" -maxdepth 1 -name "superset-meta_${SUPERSET_IMAGE_TAG}_${head}_*.dump" | sort | tail -n1)
|
||||||
|
[ -n "$backup" ] || die "no backup for image $SUPERSET_IMAGE_TAG at revision $head in $BACKUPS"
|
||||||
|
mkdir -p "$STATE"
|
||||||
|
printf '%s\n' "$sha" >"$ROLLBACK_MARK"
|
||||||
|
restore_metadata "$backup" "$head"
|
||||||
|
fi
|
||||||
|
|
||||||
|
promote "$release" "$sha"
|
||||||
|
ensure_proxy_network
|
||||||
|
compose "$LIVE" traefik up -d --wait --remove-orphans
|
||||||
|
# No superset-init: the schema already matches the target image.
|
||||||
|
while IFS= read -r dir; do
|
||||||
|
compose "$LIVE" "$dir" up -d --wait --remove-orphans
|
||||||
|
done < <(stack_dirs "$LIVE")
|
||||||
|
smoke_tests
|
||||||
|
record_state "$sha"
|
||||||
|
rm -f "$ROLLBACK_MARK"
|
||||||
|
trap - EXIT
|
||||||
|
log "rolled back to $sha; now git revert and push so main matches"
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
Executable
+105
@@ -0,0 +1,105 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# One-time host preparation for the petrolyst server (Ubuntu 24.04), run as root.
|
||||||
|
# Idempotent: re-running converges to the same state.
|
||||||
|
#
|
||||||
|
# bootstrap.sh --ci-key <public-key-file> --runner-ip <ip|any>
|
||||||
|
#
|
||||||
|
# --ci-key public half of the dedicated CI deploy key (DEPLOY_SSH_KEY in Gitea)
|
||||||
|
# --runner-ip egress IP of the Gitea act_runner; the deploy key works only from
|
||||||
|
# there ("any" drops that restriction, e.g. before the runner exists)
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
die() { echo "ERROR: $*" >&2; exit 1; }
|
||||||
|
[ "$(id -u)" = 0 ] || die "run as root"
|
||||||
|
|
||||||
|
ci_key='' runner_ip=''
|
||||||
|
while [ $# -gt 0 ]; do
|
||||||
|
case $1 in
|
||||||
|
--ci-key) ci_key=$2; shift 2 ;;
|
||||||
|
--runner-ip) runner_ip=$2; shift 2 ;;
|
||||||
|
*) die "unknown argument $1" ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
[ -s "$ci_key" ] || die "--ci-key <public-key-file> is required"
|
||||||
|
[ -n "$runner_ip" ] || die "--runner-ip <ip|any> is required"
|
||||||
|
grep -qE '^ssh-(ed25519|rsa) ' "$ci_key" || die "$ci_key does not look like an SSH public key"
|
||||||
|
|
||||||
|
ROOT=/opt/petrolyst
|
||||||
|
|
||||||
|
echo "== packages and Docker (official apt repository)"
|
||||||
|
apt-get update -q
|
||||||
|
apt-get install -y -q ca-certificates curl rsync ufw
|
||||||
|
install -m 0755 -d /etc/apt/keyrings
|
||||||
|
curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
|
||||||
|
chmod a+r /etc/apt/keyrings/docker.asc
|
||||||
|
# shellcheck source=/dev/null
|
||||||
|
codename=$(. /etc/os-release && echo "$VERSION_CODENAME")
|
||||||
|
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $codename stable" \
|
||||||
|
>/etc/apt/sources.list.d/docker.list
|
||||||
|
apt-get update -q
|
||||||
|
apt-get install -y -q docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
|
||||||
|
|
||||||
|
daemon_json='{
|
||||||
|
"log-driver": "json-file",
|
||||||
|
"log-opts": { "max-size": "10m", "max-file": "3" },
|
||||||
|
"live-restore": true
|
||||||
|
}'
|
||||||
|
if [ "$(cat /etc/docker/daemon.json 2>/dev/null)" != "$daemon_json" ]; then
|
||||||
|
printf '%s\n' "$daemon_json" >/etc/docker/daemon.json
|
||||||
|
systemctl restart docker
|
||||||
|
fi
|
||||||
|
systemctl enable --now docker
|
||||||
|
|
||||||
|
echo "== swap (4 GB) for Trino, Superset and two Postgres in 7.8 GB"
|
||||||
|
if ! swapon --show=NAME --noheadings | grep -qx /swapfile; then
|
||||||
|
[ -e /swapfile ] || fallocate -l 4G /swapfile
|
||||||
|
chmod 600 /swapfile
|
||||||
|
mkswap /swapfile >/dev/null
|
||||||
|
swapon /swapfile
|
||||||
|
fi
|
||||||
|
grep -q '^/swapfile ' /etc/fstab || echo '/swapfile none swap sw 0 0' >>/etc/fstab
|
||||||
|
echo 'vm.swappiness=10' >/etc/sysctl.d/99-petrolyst.conf
|
||||||
|
sysctl -q -p /etc/sysctl.d/99-petrolyst.conf
|
||||||
|
|
||||||
|
echo "== firewall: 22, 80, 443 only"
|
||||||
|
# Docker-published ports bypass ufw. Only Traefik publishes publicly; every
|
||||||
|
# other published port in the stacks binds to 127.0.0.1.
|
||||||
|
ufw default deny incoming >/dev/null
|
||||||
|
ufw default allow outgoing >/dev/null
|
||||||
|
for port in 22 80 443; do ufw allow "$port/tcp" >/dev/null; done
|
||||||
|
ufw --force enable >/dev/null
|
||||||
|
|
||||||
|
echo "== deploy user (CI)"
|
||||||
|
id deploy >/dev/null 2>&1 || useradd --create-home --shell /bin/bash deploy
|
||||||
|
usermod -aG docker deploy # docker access makes deploy root-equivalent; the key restrictions below matter
|
||||||
|
install -d -o deploy -g deploy -m 0700 /home/deploy/.ssh
|
||||||
|
options=restrict
|
||||||
|
[ "$runner_ip" = any ] || options="restrict,from=\"$runner_ip\""
|
||||||
|
printf '%s %s\n' "$options" "$(cat "$ci_key")" >/home/deploy/.ssh/authorized_keys
|
||||||
|
chown deploy:deploy /home/deploy/.ssh/authorized_keys
|
||||||
|
chmod 0600 /home/deploy/.ssh/authorized_keys
|
||||||
|
|
||||||
|
echo "== SSH: keys only"
|
||||||
|
[ -s /root/.ssh/authorized_keys ] || die "root has no authorized_keys; refusing to turn off password login"
|
||||||
|
# 00- sorts before cloud-init's 50-cloud-init.conf, and sshd keeps the first
|
||||||
|
# value it reads, so these win over its PasswordAuthentication yes.
|
||||||
|
cat >/etc/ssh/sshd_config.d/00-petrolyst.conf <<'EOF'
|
||||||
|
PasswordAuthentication no
|
||||||
|
KbdInteractiveAuthentication no
|
||||||
|
PermitRootLogin prohibit-password
|
||||||
|
EOF
|
||||||
|
sshd -t
|
||||||
|
systemctl reload ssh
|
||||||
|
|
||||||
|
echo "== /opt/petrolyst"
|
||||||
|
install -d -o root -g root -m 0755 "$ROOT"
|
||||||
|
install -d -o deploy -g deploy -m 0755 "$ROOT/infra" "$ROOT/releases" "$ROOT/incoming" "$ROOT/state"
|
||||||
|
install -d -o deploy -g deploy -m 0750 "$ROOT/backups"
|
||||||
|
[ -e "$ROOT/deploy.lock" ] || install -o deploy -g deploy -m 0640 /dev/null "$ROOT/deploy.lock"
|
||||||
|
docker network inspect proxy >/dev/null 2>&1 || docker network create proxy >/dev/null
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "Done. Before closing this session, check from another terminal that"
|
||||||
|
echo " ssh petrolyst still logs you in as root, and"
|
||||||
|
echo " sshd -T | grep -E 'passwordauthentication|permitrootlogin' shows no / prohibit-password."
|
||||||
|
echo "Next: host/secrets-init.sh"
|
||||||
Executable
+86
@@ -0,0 +1,86 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Server secrets and env files, run as root after bootstrap.sh.
|
||||||
|
# secrets-init.sh generate every missing secret; never overwrites one
|
||||||
|
# secrets-init.sh keycloak-secret store the client secret of Keycloak client
|
||||||
|
# "petrolyst" (Clients > petrolyst > Credentials),
|
||||||
|
# read from the terminal or stdin
|
||||||
|
#
|
||||||
|
# Layout: /opt/petrolyst/secrets/{shared,traefik,fuel-monitoring}/ are root:deploy
|
||||||
|
# 0750, so only root and the deploy user can reach them; the files are 0444
|
||||||
|
# because each container reads them under its own uid.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
die() { echo "ERROR: $*" >&2; exit 1; }
|
||||||
|
[ "$(id -u)" = 0 ] || die "run as root"
|
||||||
|
getent group deploy >/dev/null || die "group deploy is missing: run host/bootstrap.sh first"
|
||||||
|
|
||||||
|
ROOT=/opt/petrolyst
|
||||||
|
SECRETS=$ROOT/secrets
|
||||||
|
umask 077
|
||||||
|
install -d -o root -g deploy -m 0750 "$SECRETS" "$SECRETS/shared" "$SECRETS/traefik" "$SECRETS/fuel-monitoring" "$ROOT/env"
|
||||||
|
|
||||||
|
# write_secret <path> <value>: no trailing newline (oauth2-proxy reads the file
|
||||||
|
# byte for byte), replaced atomically.
|
||||||
|
write_secret() {
|
||||||
|
printf '%s' "$2" >"$1.tmp"
|
||||||
|
chown root:deploy "$1.tmp"
|
||||||
|
chmod 0444 "$1.tmp"
|
||||||
|
mv "$1.tmp" "$1"
|
||||||
|
}
|
||||||
|
|
||||||
|
generate() {
|
||||||
|
if [ -s "$1" ]; then
|
||||||
|
echo "kept $1"
|
||||||
|
else
|
||||||
|
write_secret "$1" "$(openssl rand -hex "$2")"
|
||||||
|
echo "generated $1"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
case ${1:-all} in
|
||||||
|
keycloak-secret)
|
||||||
|
if [ -t 0 ]; then
|
||||||
|
printf 'Client secret of Keycloak client "petrolyst": ' >&2
|
||||||
|
IFS= read -rs value
|
||||||
|
echo >&2
|
||||||
|
else
|
||||||
|
IFS= read -r value
|
||||||
|
fi
|
||||||
|
value=$(printf '%s' "$value" | tr -d '[:space:]')
|
||||||
|
[ -n "$value" ] || die "empty secret"
|
||||||
|
write_secret "$SECRETS/shared/keycloak_petrolyst_client_secret" "$value"
|
||||||
|
echo "stored $SECRETS/shared/keycloak_petrolyst_client_secret"
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
all) ;;
|
||||||
|
*) die "usage: secrets-init.sh [keycloak-secret]" ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
# Hex only, so no value ever needs quoting in a URL or a properties file.
|
||||||
|
generate "$SECRETS/fuel-monitoring/superset_secret_key" 32
|
||||||
|
generate "$SECRETS/fuel-monitoring/superset_db_password" 32
|
||||||
|
generate "$SECRETS/fuel-monitoring/fuel_db_password" 32
|
||||||
|
generate "$SECRETS/fuel-monitoring/fuel_db_trino_password" 32
|
||||||
|
# 32 hex characters = a 32-byte cookie key, as oauth2-proxy requires.
|
||||||
|
generate "$SECRETS/traefik/oauth2_proxy_cookie_secret" 16
|
||||||
|
|
||||||
|
# Trino reads the catalog password from its environment (${ENV:...}); keep the
|
||||||
|
# env file in step with the secret file that fuel-db's init script uses.
|
||||||
|
env_file=$ROOT/env/fuel-monitoring.env
|
||||||
|
line="FUEL_DB_TRINO_PASSWORD=$(cat "$SECRETS/fuel-monitoring/fuel_db_trino_password")"
|
||||||
|
if ! grep -qxF "$line" "$env_file" 2>/dev/null; then
|
||||||
|
printf '%s\n' "$line" >"$env_file.tmp"
|
||||||
|
chown root:deploy "$env_file.tmp"
|
||||||
|
chmod 0440 "$env_file.tmp"
|
||||||
|
mv "$env_file.tmp" "$env_file"
|
||||||
|
echo "wrote $env_file"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ ! -s "$SECRETS/shared/keycloak_petrolyst_client_secret" ]; then
|
||||||
|
echo
|
||||||
|
echo "Still missing: the Keycloak client secret. Copy it from Keycloak"
|
||||||
|
echo "(realm thedigitalbridge > Clients > petrolyst > Credentials), then run:"
|
||||||
|
echo " $0 keycloak-secret"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "All secrets present."
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
{
|
||||||
|
"clients": [
|
||||||
|
{
|
||||||
|
"clientId": "petrolyst",
|
||||||
|
"name": "Petrolyst",
|
||||||
|
"description": "Petrolyst apps on the petrolyst server: Superset at atoma.petrolyst.com and the Traefik dashboard at traefik.petrolyst.com (oauth2-proxy). Managed from petrolyst-infra/keycloak/.",
|
||||||
|
"enabled": true,
|
||||||
|
"protocol": "openid-connect",
|
||||||
|
"publicClient": false,
|
||||||
|
"clientAuthenticatorType": "client-secret",
|
||||||
|
"standardFlowEnabled": true,
|
||||||
|
"implicitFlowEnabled": false,
|
||||||
|
"directAccessGrantsEnabled": false,
|
||||||
|
"serviceAccountsEnabled": false,
|
||||||
|
"frontchannelLogout": true,
|
||||||
|
"baseUrl": "https://atoma.petrolyst.com/",
|
||||||
|
"redirectUris": [
|
||||||
|
"https://atoma.petrolyst.com/oauth-authorized/keycloak",
|
||||||
|
"https://traefik.petrolyst.com/oauth2/callback"
|
||||||
|
],
|
||||||
|
"webOrigins": ["+"],
|
||||||
|
"attributes": {
|
||||||
|
"pkce.code.challenge.method": "S256",
|
||||||
|
"post.logout.redirect.uris": "https://atoma.petrolyst.com/*##https://traefik.petrolyst.com/*"
|
||||||
|
},
|
||||||
|
"protocolMappers": [
|
||||||
|
{
|
||||||
|
"name": "petrolyst client roles",
|
||||||
|
"protocol": "openid-connect",
|
||||||
|
"protocolMapper": "oidc-usermodel-client-role-mapper",
|
||||||
|
"consentRequired": false,
|
||||||
|
"config": {
|
||||||
|
"usermodel.clientRoleMapping.clientId": "petrolyst",
|
||||||
|
"multivalued": "true",
|
||||||
|
"claim.name": "petrolyst_roles",
|
||||||
|
"jsonType.label": "String",
|
||||||
|
"id.token.claim": "true",
|
||||||
|
"access.token.claim": "true",
|
||||||
|
"userinfo.token.claim": "true",
|
||||||
|
"introspection.token.claim": "true"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"roles": {
|
||||||
|
"client": {
|
||||||
|
"petrolyst": [
|
||||||
|
{ "name": "superset-admin", "description": "Superset Admin at atoma.petrolyst.com" },
|
||||||
|
{ "name": "superset-editor", "description": "Superset Alpha: build charts and dashboards" },
|
||||||
|
{ "name": "superset-viewer", "description": "Superset Gamma: view dashboards shared with the role" },
|
||||||
|
{ "name": "traefik-admin", "description": "Traefik dashboard at traefik.petrolyst.com" }
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,73 @@
|
|||||||
|
# petrolyst-insights
|
||||||
|
|
||||||
|
The Petrolyst customization of Apache Superset 6.1.0: sign-in, roles, look and feel, patches
|
||||||
|
and dashboards. Consumed by the deployment folders of this repository at image-build time.
|
||||||
|
**Not runnable standalone**: this folder has no Dockerfile, no Compose file, no CI and no
|
||||||
|
secrets. `deploy/deploy.sh` refuses to deploy if one appears here.
|
||||||
|
|
||||||
|
The split follows media-insights (customization) and media-intelligence-docker (deployment):
|
||||||
|
anything that changes how Superset looks or behaves for Petrolyst lives here; anything about how
|
||||||
|
and where it runs lives in `stacks/`, `traefik/`, `deploy/` and `host/`.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
| Path | What it is | In the image at |
|
||||||
|
|---|---|---|
|
||||||
|
| `superset_config.py` | Keycloak sign-in, role mapping, proxy and cookie settings, Redis caches, theme and chart palettes | `/app/pythonpath/` |
|
||||||
|
| `petrolyst_security.py` | `PetrolystSecurityManager`: user info and roles from the Keycloak ID token, `/login/` straight to Keycloak, `/logout/` through Keycloak | `/app/pythonpath/` |
|
||||||
|
| `assets/branding/` | logos (full colour and white), favicons, app icons | `/app/superset/static/assets/branding/`, served at `/static/assets/branding/…` |
|
||||||
|
| `patches/` | runtime patches, `fix_*.py` with `--verify` (see its README); empty at start | `/app/patches/petrolyst/`, applied during the build |
|
||||||
|
| `dashboards/` | exported dashboards and import scripts (see its README) | not in the image |
|
||||||
|
|
||||||
|
How the image consumes this folder: `stacks/fuel-monitoring/compose.yaml` passes it to the build
|
||||||
|
as the named context `insights`, and the Dockerfile copies from it with `COPY --from=insights`.
|
||||||
|
The image tag is a hash of exactly the inputs above that end up in the image, so a change here
|
||||||
|
rebuilds and recreates only the `superset` container.
|
||||||
|
|
||||||
|
## Secrets read
|
||||||
|
|
||||||
|
`superset_config.py` uses `_read_secret(name)` (copied from media-insights): it prefers
|
||||||
|
`/run/secrets/<name>` and falls back to an env var for local runs, and fails at boot on an empty
|
||||||
|
value.
|
||||||
|
|
||||||
|
| Secret | Purpose |
|
||||||
|
|---|---|
|
||||||
|
| `superset_secret_key` | Flask `SECRET_KEY` |
|
||||||
|
| `superset_db_password` | password of user `superset` on the metadata database |
|
||||||
|
| `keycloak_petrolyst_client_secret` | client secret of Keycloak client `petrolyst` |
|
||||||
|
|
||||||
|
On the server they come from `/opt/petrolyst/secrets/` (see the deployment README).
|
||||||
|
|
||||||
|
## Sign-in and roles
|
||||||
|
|
||||||
|
- Provider: Keycloak realm `thedigitalbridge` at `iam.thedigitalbridge.io`, client `petrolyst`,
|
||||||
|
authorization code flow with PKCE S256. The realm URL can be overridden with
|
||||||
|
`KEYCLOAK_REALM_URL`.
|
||||||
|
- Roles come from the client roles of `petrolyst`, which a protocol mapper puts into the claim
|
||||||
|
`petrolyst_roles`, and are re-read at every login:
|
||||||
|
|
||||||
|
| Keycloak client role | Superset role |
|
||||||
|
|---|---|
|
||||||
|
| `superset-admin` | Admin |
|
||||||
|
| `superset-editor` | Alpha |
|
||||||
|
| `superset-viewer` | Gamma |
|
||||||
|
| none | Public (no permissions) |
|
||||||
|
|
||||||
|
- `DASHBOARD_RBAC` is on: a viewer sees a dashboard once an admin adds the Gamma role (or a
|
||||||
|
custom one) to it.
|
||||||
|
|
||||||
|
## Look and feel
|
||||||
|
|
||||||
|
Source: `petrolyst-ui/design/brand/` (tokens `07_UI_Tokens/petrolyst_tokens.css`, logos
|
||||||
|
`public/brand/`, favicons `03_Favicon_App_Icons/`), copied on 2026-10-23 from petrolyst-ui commit
|
||||||
|
`2b2e694`. Copy again when the brand changes.
|
||||||
|
|
||||||
|
- Light theme: action teal `#00695C` for primary and links, navy `#0B1D33` text, `#F8FAFC`
|
||||||
|
layout, full-colour stacked logo.
|
||||||
|
- Dark theme: `#35C5B6` primary, `#071425` / `#12263D` surfaces, white logo.
|
||||||
|
- Font: Inter, Superset's default. Satoshi is licence-restricted and is not used.
|
||||||
|
- Chart palettes: "Petrolyst" categorical (the default, brand teal first) and "Petrolyst Teal" /
|
||||||
|
"Petrolyst Navy" sequential. The categorical palette was validated with the dataviz skill's
|
||||||
|
validator on both chart surfaces; the numbers are in the comment above it in
|
||||||
|
`superset_config.py`. Charts with more than three series in scatter or map form should fold
|
||||||
|
the rest into "Other".
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 441 B |
@@ -0,0 +1,2 @@
|
|||||||
|
<svg xmlns="http://www.w3.org/2000/svg" viewBox="455 175 505 430" role="img" aria-label="Petrolyst micro mark"><path d="M 945.0,214.0 L 878.0,214.0 L 875.0,215.0 L 754.0,341.0 L 749.0,348.0 L 734.0,375.0 L 727.0,398.0 L 726.0,412.0 L 725.0,413.0 L 725.0,437.0 L 724.0,438.0 L 724.0,448.0 L 721.0,464.0 L 714.0,484.0 L 698.0,514.0 L 693.0,521.0 L 685.0,542.0 L 682.0,558.0 L 682.0,596.0 L 701.0,595.0 L 717.0,591.0 L 736.0,580.0 L 746.0,569.0 L 752.0,558.0 L 755.0,548.0 L 755.0,535.0 L 756.0,534.0 L 756.0,453.0 L 757.0,452.0 L 757.0,446.0 L 761.0,430.0 L 770.0,410.0 L 780.0,394.0 L 792.0,380.0 L 946.0,219.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||||
|
<path d="M 503.0,214.0 L 502.0,219.0 L 589.0,312.0 L 596.0,321.0 L 651.0,380.0 L 663.0,395.0 L 671.0,408.0 L 681.0,431.0 L 686.0,456.0 L 686.0,491.0 L 691.0,491.0 L 703.0,472.0 L 709.0,459.0 L 715.0,439.0 L 716.0,423.0 L 717.0,422.0 L 716.0,400.0 L 711.0,379.0 L 701.0,357.0 L 691.0,342.0 L 643.0,289.0 L 572.0,214.0 L 561.0,214.0 L 560.0,215.0 Z" fill="#0B1D33" fill-rule="evenodd"/></svg>
|
||||||
|
After Width: | Height: | Size: 1.0 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 4.6 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 4.9 KiB |
@@ -0,0 +1,35 @@
|
|||||||
|
<svg xmlns="http://www.w3.org/2000/svg" viewBox="55 120 1335 700" role="img" aria-label="Petrolyst">
|
||||||
|
|
||||||
|
<defs>
|
||||||
|
<clipPath id="markClip"><path d="M 943.0,216.0 L 896.0,216.0 L 895.0,217.0 L 892.0,217.0 L 891.0,216.0 L 880.0,216.0 L 879.0,217.0 L 877.0,217.0 L 840.0,255.0 L 840.0,256.0 L 834.0,261.0 L 834.0,262.0 L 814.0,282.0 L 814.0,283.0 L 794.0,303.0 L 794.0,304.0 L 792.0,306.0 L 791.0,306.0 L 791.0,307.0 L 771.0,327.0 L 771.0,328.0 L 756.0,343.0 L 754.0,347.0 L 751.0,350.0 L 751.0,351.0 L 747.0,356.0 L 746.0,359.0 L 744.0,361.0 L 736.0,377.0 L 736.0,379.0 L 735.0,380.0 L 735.0,382.0 L 732.0,389.0 L 731.0,396.0 L 729.0,400.0 L 729.0,405.0 L 728.0,406.0 L 728.0,414.0 L 727.0,415.0 L 727.0,439.0 L 726.0,440.0 L 726.0,450.0 L 725.0,451.0 L 725.0,456.0 L 724.0,457.0 L 724.0,462.0 L 723.0,463.0 L 723.0,466.0 L 722.0,467.0 L 718.0,481.0 L 716.0,484.0 L 716.0,486.0 L 709.0,500.0 L 707.0,502.0 L 705.0,507.0 L 703.0,509.0 L 700.0,516.0 L 695.0,523.0 L 694.0,527.0 L 691.0,532.0 L 688.0,543.0 L 687.0,544.0 L 686.0,552.0 L 685.0,553.0 L 685.0,559.0 L 684.0,560.0 L 684.0,594.0 L 691.0,594.0 L 692.0,593.0 L 705.0,592.0 L 709.0,590.0 L 715.0,589.0 L 725.0,584.0 L 727.0,582.0 L 730.0,581.0 L 733.0,578.0 L 734.0,578.0 L 741.0,571.0 L 741.0,570.0 L 746.0,564.0 L 750.0,556.0 L 750.0,554.0 L 752.0,550.0 L 752.0,547.0 L 753.0,546.0 L 753.0,533.0 L 754.0,532.0 L 754.0,451.0 L 755.0,450.0 L 755.0,444.0 L 756.0,443.0 L 756.0,439.0 L 757.0,438.0 L 757.0,435.0 L 759.0,431.0 L 759.0,428.0 L 761.0,425.0 L 763.0,418.0 L 768.0,408.0 L 770.0,406.0 L 772.0,401.0 L 774.0,399.0 L 778.0,392.0 L 790.0,379.0 L 790.0,378.0 L 822.0,345.0 L 822.0,344.0 L 868.0,297.0 L 868.0,296.0 L 908.0,255.0 L 908.0,254.0 L 912.0,250.0 L 913.0,250.0 L 915.0,248.0 L 915.0,247.0 L 932.0,230.0 L 932.0,229.0 L 944.0,217.0 Z" fill="#000" fill-rule="evenodd"/>
|
||||||
|
<path d="M 505.0,216.0 L 504.0,217.0 L 510.0,223.0 L 510.0,224.0 L 524.0,238.0 L 524.0,239.0 L 538.0,253.0 L 538.0,254.0 L 549.0,265.0 L 549.0,266.0 L 591.0,310.0 L 591.0,311.0 L 598.0,318.0 L 598.0,319.0 L 612.0,333.0 L 612.0,334.0 L 622.0,344.0 L 622.0,345.0 L 630.0,353.0 L 630.0,354.0 L 634.0,357.0 L 634.0,358.0 L 642.0,366.0 L 642.0,367.0 L 653.0,378.0 L 653.0,379.0 L 665.0,393.0 L 668.0,399.0 L 670.0,401.0 L 671.0,404.0 L 673.0,406.0 L 677.0,414.0 L 677.0,416.0 L 680.0,421.0 L 681.0,426.0 L 683.0,429.0 L 683.0,432.0 L 684.0,433.0 L 684.0,436.0 L 685.0,437.0 L 685.0,440.0 L 686.0,441.0 L 687.0,453.0 L 688.0,454.0 L 688.0,489.0 L 689.0,489.0 L 690.0,486.0 L 693.0,483.0 L 696.0,477.0 L 698.0,475.0 L 699.0,472.0 L 701.0,470.0 L 705.0,462.0 L 705.0,460.0 L 707.0,457.0 L 708.0,452.0 L 710.0,449.0 L 712.0,438.0 L 713.0,437.0 L 713.0,433.0 L 714.0,432.0 L 714.0,421.0 L 715.0,420.0 L 715.0,413.0 L 714.0,412.0 L 714.0,402.0 L 713.0,401.0 L 712.0,392.0 L 711.0,391.0 L 711.0,388.0 L 709.0,384.0 L 708.0,378.0 L 699.0,359.0 L 697.0,357.0 L 689.0,344.0 L 677.0,331.0 L 677.0,330.0 L 641.0,292.0 L 641.0,291.0 L 599.0,247.0 L 599.0,246.0 L 587.0,234.0 L 587.0,233.0 L 586.0,233.0 L 584.0,231.0 L 584.0,230.0 L 570.0,216.0 L 563.0,216.0 L 562.0,217.0 L 543.0,217.0 L 542.0,216.0 Z" fill="#000" fill-rule="evenodd"/></clipPath>
|
||||||
|
<linearGradient id="rightGradient" x1="710" y1="180" x2="965" y2="590" gradientUnits="userSpaceOnUse">
|
||||||
|
<stop offset="0%" stop-color="#009688"/>
|
||||||
|
<stop offset="100%" stop-color="#00B6E6"/>
|
||||||
|
</linearGradient>
|
||||||
|
<linearGradient id="wordYGradient" x1="945" y1="660" x2="1090" y2="780" gradientUnits="userSpaceOnUse">
|
||||||
|
<stop offset="0%" stop-color="#009688"/>
|
||||||
|
<stop offset="100%" stop-color="#00B6E6"/>
|
||||||
|
</linearGradient>
|
||||||
|
</defs>
|
||||||
|
|
||||||
|
<g clip-path="url(#markClip)">
|
||||||
|
<rect x="0" y="0" width="735" height="1086" fill="#0B1D33"/>
|
||||||
|
<rect x="735" y="0" width="665" height="1086" fill="url(#rightGradient)"/>
|
||||||
|
</g>
|
||||||
|
<g fill="#0B1D33">
|
||||||
|
<path d="M 250.0,753.0 L 250.0,767.0 L 339.0,767.0 L 339.0,764.0 L 340.0,763.0 L 340.0,757.0 L 339.0,756.0 L 339.0,752.0 L 251.0,752.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||||
|
<path d="M 250.0,715.0 L 250.0,729.0 L 266.0,729.0 L 267.0,730.0 L 301.0,730.0 L 302.0,729.0 L 307.0,729.0 L 308.0,730.0 L 325.0,730.0 L 326.0,729.0 L 340.0,729.0 L 340.0,716.0 L 339.0,715.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||||
|
<path d="M 1260.0,678.0 L 1260.0,692.0 L 1264.0,692.0 L 1265.0,693.0 L 1296.0,693.0 L 1297.0,694.0 L 1297.0,767.0 L 1313.0,767.0 L 1313.0,694.0 L 1314.0,693.0 L 1351.0,693.0 L 1351.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||||
|
<path d="M 867.0,678.0 L 867.0,767.0 L 951.0,767.0 L 951.0,753.0 L 950.0,752.0 L 883.0,752.0 L 882.0,751.0 L 882.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||||
|
<path d="M 551.0,678.0 L 551.0,767.0 L 566.0,767.0 L 566.0,743.0 L 567.0,742.0 L 567.0,739.0 L 566.0,738.0 L 567.0,737.0 L 604.0,737.0 L 614.0,747.0 L 614.0,748.0 L 622.0,756.0 L 622.0,757.0 L 630.0,765.0 L 630.0,766.0 L 631.0,767.0 L 651.0,767.0 L 651.0,766.0 L 649.0,764.0 L 649.0,763.0 L 642.0,756.0 L 642.0,755.0 L 628.0,741.0 L 628.0,740.0 L 626.0,738.0 L 628.0,736.0 L 630.0,736.0 L 631.0,735.0 L 633.0,735.0 L 634.0,734.0 L 636.0,734.0 L 638.0,732.0 L 639.0,732.0 L 645.0,726.0 L 645.0,725.0 L 648.0,720.0 L 648.0,718.0 L 649.0,717.0 L 649.0,713.0 L 650.0,712.0 L 650.0,703.0 L 649.0,702.0 L 649.0,698.0 L 648.0,697.0 L 648.0,695.0 L 647.0,694.0 L 646.0,691.0 L 644.0,689.0 L 644.0,688.0 L 640.0,684.0 L 639.0,684.0 L 637.0,682.0 L 636.0,682.0 L 633.0,680.0 L 631.0,680.0 L 630.0,679.0 L 626.0,679.0 L 625.0,678.0 Z M 567.0,694.0 L 568.0,693.0 L 625.0,693.0 L 626.0,694.0 L 629.0,695.0 L 632.0,698.0 L 632.0,699.0 L 633.0,700.0 L 633.0,713.0 L 632.0,714.0 L 632.0,716.0 L 630.0,718.0 L 630.0,719.0 L 629.0,720.0 L 628.0,720.0 L 626.0,722.0 L 622.0,722.0 L 621.0,723.0 L 568.0,723.0 L 566.0,721.0 L 566.0,703.0 L 567.0,702.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||||
|
<path d="M 398.0,678.0 L 398.0,693.0 L 434.0,693.0 L 435.0,694.0 L 435.0,767.0 L 451.0,767.0 L 451.0,694.0 L 452.0,693.0 L 489.0,693.0 L 489.0,679.0 L 488.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||||
|
<path d="M 250.0,678.0 L 250.0,692.0 L 251.0,692.0 L 252.0,693.0 L 339.0,693.0 L 339.0,691.0 L 340.0,690.0 L 340.0,679.0 L 339.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||||
|
<path d="M 1134.0,683.0 L 1129.0,688.0 L 1129.0,689.0 L 1127.0,691.0 L 1127.0,693.0 L 1126.0,694.0 L 1126.0,696.0 L 1125.0,697.0 L 1125.0,712.0 L 1126.0,713.0 L 1126.0,715.0 L 1127.0,716.0 L 1128.0,719.0 L 1134.0,725.0 L 1135.0,725.0 L 1137.0,727.0 L 1139.0,727.0 L 1140.0,728.0 L 1143.0,728.0 L 1144.0,729.0 L 1185.0,729.0 L 1186.0,730.0 L 1191.0,730.0 L 1192.0,731.0 L 1193.0,731.0 L 1197.0,735.0 L 1197.0,739.0 L 1198.0,740.0 L 1198.0,742.0 L 1197.0,743.0 L 1197.0,746.0 L 1196.0,747.0 L 1196.0,748.0 L 1193.0,751.0 L 1192.0,751.0 L 1189.0,753.0 L 1184.0,753.0 L 1183.0,754.0 L 1151.0,754.0 L 1150.0,753.0 L 1143.0,753.0 L 1142.0,752.0 L 1139.0,752.0 L 1138.0,751.0 L 1135.0,751.0 L 1134.0,750.0 L 1131.0,750.0 L 1130.0,749.0 L 1128.0,749.0 L 1127.0,748.0 L 1125.0,748.0 L 1125.0,760.0 L 1124.0,761.0 L 1126.0,763.0 L 1128.0,763.0 L 1129.0,764.0 L 1131.0,764.0 L 1132.0,765.0 L 1134.0,765.0 L 1135.0,766.0 L 1139.0,766.0 L 1140.0,767.0 L 1146.0,767.0 L 1147.0,768.0 L 1186.0,768.0 L 1187.0,767.0 L 1191.0,767.0 L 1192.0,766.0 L 1195.0,766.0 L 1196.0,765.0 L 1198.0,765.0 L 1199.0,764.0 L 1201.0,764.0 L 1203.0,762.0 L 1204.0,762.0 L 1209.0,757.0 L 1209.0,756.0 L 1211.0,754.0 L 1211.0,753.0 L 1212.0,752.0 L 1212.0,750.0 L 1213.0,749.0 L 1213.0,734.0 L 1212.0,733.0 L 1212.0,730.0 L 1211.0,729.0 L 1211.0,728.0 L 1209.0,726.0 L 1209.0,725.0 L 1204.0,720.0 L 1203.0,720.0 L 1202.0,719.0 L 1201.0,719.0 L 1196.0,716.0 L 1192.0,716.0 L 1191.0,715.0 L 1148.0,715.0 L 1147.0,714.0 L 1145.0,714.0 L 1142.0,711.0 L 1142.0,710.0 L 1141.0,709.0 L 1141.0,699.0 L 1142.0,698.0 L 1142.0,697.0 L 1145.0,694.0 L 1146.0,694.0 L 1147.0,693.0 L 1150.0,693.0 L 1151.0,692.0 L 1190.0,692.0 L 1191.0,693.0 L 1196.0,693.0 L 1197.0,694.0 L 1201.0,694.0 L 1202.0,695.0 L 1204.0,695.0 L 1205.0,696.0 L 1209.0,696.0 L 1209.0,683.0 L 1206.0,681.0 L 1203.0,681.0 L 1202.0,680.0 L 1198.0,680.0 L 1197.0,679.0 L 1193.0,679.0 L 1192.0,678.0 L 1182.0,678.0 L 1181.0,677.0 L 1156.0,677.0 L 1155.0,678.0 L 1146.0,678.0 L 1145.0,679.0 L 1143.0,679.0 L 1142.0,680.0 L 1140.0,680.0 L 1139.0,681.0 L 1138.0,681.0 L 1137.0,682.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||||
|
<path d="M 714.0,691.0 L 714.0,692.0 L 711.0,695.0 L 711.0,696.0 L 709.0,698.0 L 709.0,699.0 L 708.0,700.0 L 708.0,702.0 L 707.0,703.0 L 707.0,706.0 L 706.0,707.0 L 706.0,711.0 L 705.0,712.0 L 705.0,734.0 L 706.0,735.0 L 706.0,739.0 L 707.0,740.0 L 707.0,743.0 L 708.0,744.0 L 708.0,745.0 L 709.0,746.0 L 709.0,747.0 L 710.0,748.0 L 711.0,751.0 L 713.0,753.0 L 713.0,754.0 L 718.0,759.0 L 719.0,759.0 L 721.0,761.0 L 722.0,761.0 L 723.0,762.0 L 724.0,762.0 L 729.0,765.0 L 731.0,765.0 L 732.0,766.0 L 735.0,766.0 L 736.0,767.0 L 741.0,767.0 L 742.0,768.0 L 775.0,768.0 L 776.0,767.0 L 781.0,767.0 L 782.0,766.0 L 785.0,766.0 L 786.0,765.0 L 788.0,765.0 L 789.0,764.0 L 790.0,764.0 L 791.0,763.0 L 792.0,763.0 L 793.0,762.0 L 796.0,761.0 L 798.0,759.0 L 799.0,759.0 L 805.0,753.0 L 805.0,752.0 L 807.0,750.0 L 807.0,749.0 L 809.0,746.0 L 809.0,744.0 L 811.0,741.0 L 811.0,737.0 L 812.0,736.0 L 812.0,711.0 L 811.0,710.0 L 811.0,706.0 L 810.0,705.0 L 810.0,703.0 L 809.0,702.0 L 809.0,700.0 L 808.0,699.0 L 808.0,698.0 L 806.0,696.0 L 806.0,695.0 L 803.0,692.0 L 803.0,691.0 L 799.0,687.0 L 798.0,687.0 L 793.0,683.0 L 792.0,683.0 L 791.0,682.0 L 789.0,682.0 L 786.0,680.0 L 783.0,680.0 L 782.0,679.0 L 779.0,679.0 L 778.0,678.0 L 770.0,678.0 L 769.0,677.0 L 748.0,677.0 L 747.0,678.0 L 739.0,678.0 L 738.0,679.0 L 734.0,679.0 L 733.0,680.0 L 731.0,680.0 L 730.0,681.0 L 728.0,681.0 L 727.0,682.0 L 726.0,682.0 L 725.0,683.0 L 722.0,684.0 L 720.0,686.0 L 719.0,686.0 Z M 725.0,702.0 L 730.0,697.0 L 731.0,697.0 L 733.0,695.0 L 735.0,695.0 L 738.0,693.0 L 742.0,693.0 L 743.0,692.0 L 772.0,692.0 L 773.0,693.0 L 777.0,693.0 L 778.0,694.0 L 781.0,694.0 L 782.0,695.0 L 785.0,696.0 L 787.0,698.0 L 788.0,698.0 L 791.0,701.0 L 791.0,702.0 L 793.0,704.0 L 793.0,705.0 L 794.0,706.0 L 794.0,708.0 L 795.0,709.0 L 795.0,712.0 L 796.0,713.0 L 796.0,733.0 L 795.0,734.0 L 795.0,738.0 L 794.0,739.0 L 793.0,742.0 L 791.0,744.0 L 791.0,745.0 L 788.0,748.0 L 787.0,748.0 L 785.0,750.0 L 784.0,750.0 L 781.0,752.0 L 778.0,752.0 L 777.0,753.0 L 770.0,753.0 L 769.0,754.0 L 747.0,754.0 L 746.0,753.0 L 740.0,753.0 L 739.0,752.0 L 737.0,752.0 L 736.0,751.0 L 734.0,751.0 L 733.0,750.0 L 732.0,750.0 L 730.0,748.0 L 729.0,748.0 L 725.0,744.0 L 725.0,743.0 L 722.0,738.0 L 722.0,734.0 L 721.0,733.0 L 721.0,713.0 L 722.0,712.0 L 722.0,709.0 L 723.0,708.0 L 723.0,706.0 L 724.0,705.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||||
|
<path d="M 95.0,678.0 L 95.0,766.0 L 96.0,767.0 L 111.0,767.0 L 111.0,737.0 L 112.0,736.0 L 171.0,736.0 L 172.0,735.0 L 175.0,735.0 L 176.0,734.0 L 178.0,734.0 L 179.0,733.0 L 180.0,733.0 L 182.0,731.0 L 183.0,731.0 L 186.0,728.0 L 187.0,728.0 L 187.0,727.0 L 190.0,724.0 L 190.0,723.0 L 193.0,718.0 L 193.0,715.0 L 194.0,714.0 L 194.0,702.0 L 193.0,701.0 L 193.0,696.0 L 192.0,695.0 L 192.0,693.0 L 191.0,692.0 L 190.0,689.0 L 188.0,687.0 L 188.0,686.0 L 185.0,683.0 L 184.0,683.0 L 182.0,681.0 L 181.0,681.0 L 180.0,680.0 L 178.0,680.0 L 177.0,679.0 L 174.0,679.0 L 173.0,678.0 L 168.0,678.0 L 167.0,677.0 L 156.0,677.0 L 155.0,678.0 L 135.0,678.0 L 134.0,677.0 L 132.0,677.0 L 131.0,678.0 L 108.0,678.0 L 107.0,677.0 L 105.0,677.0 L 104.0,678.0 Z M 111.0,693.0 L 112.0,692.0 L 169.0,692.0 L 170.0,693.0 L 172.0,693.0 L 177.0,698.0 L 177.0,699.0 L 178.0,700.0 L 178.0,712.0 L 177.0,713.0 L 177.0,715.0 L 176.0,716.0 L 176.0,717.0 L 174.0,719.0 L 173.0,719.0 L 170.0,721.0 L 167.0,721.0 L 166.0,722.0 L 154.0,722.0 L 153.0,721.0 L 152.0,722.0 L 115.0,722.0 L 114.0,721.0 L 113.0,722.0 L 111.0,720.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||||
|
<rect x="250" y="678" width="16" height="90" fill="#0B1D33"/>
|
||||||
|
</g>
|
||||||
|
<path d="M 1084.0,678.0 L 1063.0,678.0 L 1056.0,685.0 L 1056.0,686.0 L 1049.0,693.0 L 1049.0,694.0 L 1042.0,701.0 L 1042.0,702.0 L 1036.0,708.0 L 1036.0,709.0 L 1035.0,710.0 L 1035.0,711.0 L 1033.0,713.0 L 1033.0,714.0 L 1032.0,715.0 L 1032.0,716.0 L 1031.0,717.0 L 1031.0,718.0 L 1030.0,719.0 L 1030.0,720.0 L 1028.0,723.0 L 1028.0,725.0 L 1026.0,728.0 L 1026.0,730.0 L 1025.0,731.0 L 1025.0,733.0 L 1024.0,734.0 L 1024.0,738.0 L 1023.0,739.0 L 1023.0,743.0 L 1022.0,744.0 L 1022.0,752.0 L 1021.0,753.0 L 1021.0,766.0 L 1022.0,767.0 L 1038.0,767.0 L 1038.0,737.0 L 1039.0,736.0 L 1039.0,734.0 L 1040.0,733.0 L 1040.0,732.0 L 1041.0,731.0 L 1041.0,730.0 L 1043.0,728.0 L 1043.0,727.0 L 1046.0,724.0 L 1046.0,723.0 L 1052.0,717.0 L 1052.0,716.0 L 1060.0,708.0 L 1060.0,707.0 L 1065.0,702.0 L 1065.0,701.0 L 1072.0,694.0 L 1072.0,693.0 L 1077.0,688.0 L 1077.0,687.0 L 1082.0,682.0 L 1082.0,681.0 L 1084.0,679.0 L 1085.0,679.0 Z" fill="url(#wordYGradient)" fill-rule="evenodd"/>
|
||||||
|
<path d="M 975.0,678.0 L 975.0,679.0 L 979.0,683.0 L 979.0,684.0 L 983.0,688.0 L 983.0,689.0 L 990.0,696.0 L 990.0,697.0 L 996.0,703.0 L 996.0,704.0 L 1002.0,710.0 L 1002.0,711.0 L 1010.0,719.0 L 1010.0,720.0 L 1014.0,724.0 L 1014.0,725.0 L 1017.0,728.0 L 1017.0,729.0 L 1018.0,730.0 L 1018.0,731.0 L 1019.0,732.0 L 1020.0,732.0 L 1020.0,729.0 L 1021.0,728.0 L 1021.0,726.0 L 1022.0,725.0 L 1022.0,722.0 L 1023.0,721.0 L 1023.0,715.0 L 1024.0,714.0 L 1023.0,713.0 L 1023.0,709.0 L 1022.0,708.0 L 1022.0,707.0 L 1021.0,706.0 L 1021.0,705.0 L 1016.0,700.0 L 1016.0,699.0 L 1010.0,693.0 L 1010.0,692.0 L 1003.0,685.0 L 1003.0,684.0 L 997.0,678.0 Z" fill="url(#wordYGradient)" fill-rule="evenodd"/>
|
||||||
|
</svg>
|
||||||
|
After Width: | Height: | Size: 13 KiB |
@@ -0,0 +1,17 @@
|
|||||||
|
<svg xmlns="http://www.w3.org/2000/svg" viewBox="55 120 1335 700" role="img" aria-label="Petrolyst">
|
||||||
|
<path d="M 943.0,216.0 L 896.0,216.0 L 895.0,217.0 L 892.0,217.0 L 891.0,216.0 L 880.0,216.0 L 879.0,217.0 L 877.0,217.0 L 840.0,255.0 L 840.0,256.0 L 834.0,261.0 L 834.0,262.0 L 814.0,282.0 L 814.0,283.0 L 794.0,303.0 L 794.0,304.0 L 792.0,306.0 L 791.0,306.0 L 791.0,307.0 L 771.0,327.0 L 771.0,328.0 L 756.0,343.0 L 754.0,347.0 L 751.0,350.0 L 751.0,351.0 L 747.0,356.0 L 746.0,359.0 L 744.0,361.0 L 736.0,377.0 L 736.0,379.0 L 735.0,380.0 L 735.0,382.0 L 732.0,389.0 L 731.0,396.0 L 729.0,400.0 L 729.0,405.0 L 728.0,406.0 L 728.0,414.0 L 727.0,415.0 L 727.0,439.0 L 726.0,440.0 L 726.0,450.0 L 725.0,451.0 L 725.0,456.0 L 724.0,457.0 L 724.0,462.0 L 723.0,463.0 L 723.0,466.0 L 722.0,467.0 L 718.0,481.0 L 716.0,484.0 L 716.0,486.0 L 709.0,500.0 L 707.0,502.0 L 705.0,507.0 L 703.0,509.0 L 700.0,516.0 L 695.0,523.0 L 694.0,527.0 L 691.0,532.0 L 688.0,543.0 L 687.0,544.0 L 686.0,552.0 L 685.0,553.0 L 685.0,559.0 L 684.0,560.0 L 684.0,594.0 L 691.0,594.0 L 692.0,593.0 L 705.0,592.0 L 709.0,590.0 L 715.0,589.0 L 725.0,584.0 L 727.0,582.0 L 730.0,581.0 L 733.0,578.0 L 734.0,578.0 L 741.0,571.0 L 741.0,570.0 L 746.0,564.0 L 750.0,556.0 L 750.0,554.0 L 752.0,550.0 L 752.0,547.0 L 753.0,546.0 L 753.0,533.0 L 754.0,532.0 L 754.0,451.0 L 755.0,450.0 L 755.0,444.0 L 756.0,443.0 L 756.0,439.0 L 757.0,438.0 L 757.0,435.0 L 759.0,431.0 L 759.0,428.0 L 761.0,425.0 L 763.0,418.0 L 768.0,408.0 L 770.0,406.0 L 772.0,401.0 L 774.0,399.0 L 778.0,392.0 L 790.0,379.0 L 790.0,378.0 L 822.0,345.0 L 822.0,344.0 L 868.0,297.0 L 868.0,296.0 L 908.0,255.0 L 908.0,254.0 L 912.0,250.0 L 913.0,250.0 L 915.0,248.0 L 915.0,247.0 L 932.0,230.0 L 932.0,229.0 L 944.0,217.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
<path d="M 505.0,216.0 L 504.0,217.0 L 510.0,223.0 L 510.0,224.0 L 524.0,238.0 L 524.0,239.0 L 538.0,253.0 L 538.0,254.0 L 549.0,265.0 L 549.0,266.0 L 591.0,310.0 L 591.0,311.0 L 598.0,318.0 L 598.0,319.0 L 612.0,333.0 L 612.0,334.0 L 622.0,344.0 L 622.0,345.0 L 630.0,353.0 L 630.0,354.0 L 634.0,357.0 L 634.0,358.0 L 642.0,366.0 L 642.0,367.0 L 653.0,378.0 L 653.0,379.0 L 665.0,393.0 L 668.0,399.0 L 670.0,401.0 L 671.0,404.0 L 673.0,406.0 L 677.0,414.0 L 677.0,416.0 L 680.0,421.0 L 681.0,426.0 L 683.0,429.0 L 683.0,432.0 L 684.0,433.0 L 684.0,436.0 L 685.0,437.0 L 685.0,440.0 L 686.0,441.0 L 687.0,453.0 L 688.0,454.0 L 688.0,489.0 L 689.0,489.0 L 690.0,486.0 L 693.0,483.0 L 696.0,477.0 L 698.0,475.0 L 699.0,472.0 L 701.0,470.0 L 705.0,462.0 L 705.0,460.0 L 707.0,457.0 L 708.0,452.0 L 710.0,449.0 L 712.0,438.0 L 713.0,437.0 L 713.0,433.0 L 714.0,432.0 L 714.0,421.0 L 715.0,420.0 L 715.0,413.0 L 714.0,412.0 L 714.0,402.0 L 713.0,401.0 L 712.0,392.0 L 711.0,391.0 L 711.0,388.0 L 709.0,384.0 L 708.0,378.0 L 699.0,359.0 L 697.0,357.0 L 689.0,344.0 L 677.0,331.0 L 677.0,330.0 L 641.0,292.0 L 641.0,291.0 L 599.0,247.0 L 599.0,246.0 L 587.0,234.0 L 587.0,233.0 L 586.0,233.0 L 584.0,231.0 L 584.0,230.0 L 570.0,216.0 L 563.0,216.0 L 562.0,217.0 L 543.0,217.0 L 542.0,216.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
<g fill="#FFFFFF"><path d="M 250.0,753.0 L 250.0,767.0 L 339.0,767.0 L 339.0,764.0 L 340.0,763.0 L 340.0,757.0 L 339.0,756.0 L 339.0,752.0 L 251.0,752.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
<path d="M 250.0,715.0 L 250.0,729.0 L 266.0,729.0 L 267.0,730.0 L 301.0,730.0 L 302.0,729.0 L 307.0,729.0 L 308.0,730.0 L 325.0,730.0 L 326.0,729.0 L 340.0,729.0 L 340.0,716.0 L 339.0,715.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
<path d="M 1260.0,678.0 L 1260.0,692.0 L 1264.0,692.0 L 1265.0,693.0 L 1296.0,693.0 L 1297.0,694.0 L 1297.0,767.0 L 1313.0,767.0 L 1313.0,694.0 L 1314.0,693.0 L 1351.0,693.0 L 1351.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
<path d="M 867.0,678.0 L 867.0,767.0 L 951.0,767.0 L 951.0,753.0 L 950.0,752.0 L 883.0,752.0 L 882.0,751.0 L 882.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
<path d="M 551.0,678.0 L 551.0,767.0 L 566.0,767.0 L 566.0,743.0 L 567.0,742.0 L 567.0,739.0 L 566.0,738.0 L 567.0,737.0 L 604.0,737.0 L 614.0,747.0 L 614.0,748.0 L 622.0,756.0 L 622.0,757.0 L 630.0,765.0 L 630.0,766.0 L 631.0,767.0 L 651.0,767.0 L 651.0,766.0 L 649.0,764.0 L 649.0,763.0 L 642.0,756.0 L 642.0,755.0 L 628.0,741.0 L 628.0,740.0 L 626.0,738.0 L 628.0,736.0 L 630.0,736.0 L 631.0,735.0 L 633.0,735.0 L 634.0,734.0 L 636.0,734.0 L 638.0,732.0 L 639.0,732.0 L 645.0,726.0 L 645.0,725.0 L 648.0,720.0 L 648.0,718.0 L 649.0,717.0 L 649.0,713.0 L 650.0,712.0 L 650.0,703.0 L 649.0,702.0 L 649.0,698.0 L 648.0,697.0 L 648.0,695.0 L 647.0,694.0 L 646.0,691.0 L 644.0,689.0 L 644.0,688.0 L 640.0,684.0 L 639.0,684.0 L 637.0,682.0 L 636.0,682.0 L 633.0,680.0 L 631.0,680.0 L 630.0,679.0 L 626.0,679.0 L 625.0,678.0 Z M 567.0,694.0 L 568.0,693.0 L 625.0,693.0 L 626.0,694.0 L 629.0,695.0 L 632.0,698.0 L 632.0,699.0 L 633.0,700.0 L 633.0,713.0 L 632.0,714.0 L 632.0,716.0 L 630.0,718.0 L 630.0,719.0 L 629.0,720.0 L 628.0,720.0 L 626.0,722.0 L 622.0,722.0 L 621.0,723.0 L 568.0,723.0 L 566.0,721.0 L 566.0,703.0 L 567.0,702.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
<path d="M 398.0,678.0 L 398.0,693.0 L 434.0,693.0 L 435.0,694.0 L 435.0,767.0 L 451.0,767.0 L 451.0,694.0 L 452.0,693.0 L 489.0,693.0 L 489.0,679.0 L 488.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
<path d="M 250.0,678.0 L 250.0,692.0 L 251.0,692.0 L 252.0,693.0 L 339.0,693.0 L 339.0,691.0 L 340.0,690.0 L 340.0,679.0 L 339.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
<path d="M 1134.0,683.0 L 1129.0,688.0 L 1129.0,689.0 L 1127.0,691.0 L 1127.0,693.0 L 1126.0,694.0 L 1126.0,696.0 L 1125.0,697.0 L 1125.0,712.0 L 1126.0,713.0 L 1126.0,715.0 L 1127.0,716.0 L 1128.0,719.0 L 1134.0,725.0 L 1135.0,725.0 L 1137.0,727.0 L 1139.0,727.0 L 1140.0,728.0 L 1143.0,728.0 L 1144.0,729.0 L 1185.0,729.0 L 1186.0,730.0 L 1191.0,730.0 L 1192.0,731.0 L 1193.0,731.0 L 1197.0,735.0 L 1197.0,739.0 L 1198.0,740.0 L 1198.0,742.0 L 1197.0,743.0 L 1197.0,746.0 L 1196.0,747.0 L 1196.0,748.0 L 1193.0,751.0 L 1192.0,751.0 L 1189.0,753.0 L 1184.0,753.0 L 1183.0,754.0 L 1151.0,754.0 L 1150.0,753.0 L 1143.0,753.0 L 1142.0,752.0 L 1139.0,752.0 L 1138.0,751.0 L 1135.0,751.0 L 1134.0,750.0 L 1131.0,750.0 L 1130.0,749.0 L 1128.0,749.0 L 1127.0,748.0 L 1125.0,748.0 L 1125.0,760.0 L 1124.0,761.0 L 1126.0,763.0 L 1128.0,763.0 L 1129.0,764.0 L 1131.0,764.0 L 1132.0,765.0 L 1134.0,765.0 L 1135.0,766.0 L 1139.0,766.0 L 1140.0,767.0 L 1146.0,767.0 L 1147.0,768.0 L 1186.0,768.0 L 1187.0,767.0 L 1191.0,767.0 L 1192.0,766.0 L 1195.0,766.0 L 1196.0,765.0 L 1198.0,765.0 L 1199.0,764.0 L 1201.0,764.0 L 1203.0,762.0 L 1204.0,762.0 L 1209.0,757.0 L 1209.0,756.0 L 1211.0,754.0 L 1211.0,753.0 L 1212.0,752.0 L 1212.0,750.0 L 1213.0,749.0 L 1213.0,734.0 L 1212.0,733.0 L 1212.0,730.0 L 1211.0,729.0 L 1211.0,728.0 L 1209.0,726.0 L 1209.0,725.0 L 1204.0,720.0 L 1203.0,720.0 L 1202.0,719.0 L 1201.0,719.0 L 1196.0,716.0 L 1192.0,716.0 L 1191.0,715.0 L 1148.0,715.0 L 1147.0,714.0 L 1145.0,714.0 L 1142.0,711.0 L 1142.0,710.0 L 1141.0,709.0 L 1141.0,699.0 L 1142.0,698.0 L 1142.0,697.0 L 1145.0,694.0 L 1146.0,694.0 L 1147.0,693.0 L 1150.0,693.0 L 1151.0,692.0 L 1190.0,692.0 L 1191.0,693.0 L 1196.0,693.0 L 1197.0,694.0 L 1201.0,694.0 L 1202.0,695.0 L 1204.0,695.0 L 1205.0,696.0 L 1209.0,696.0 L 1209.0,683.0 L 1206.0,681.0 L 1203.0,681.0 L 1202.0,680.0 L 1198.0,680.0 L 1197.0,679.0 L 1193.0,679.0 L 1192.0,678.0 L 1182.0,678.0 L 1181.0,677.0 L 1156.0,677.0 L 1155.0,678.0 L 1146.0,678.0 L 1145.0,679.0 L 1143.0,679.0 L 1142.0,680.0 L 1140.0,680.0 L 1139.0,681.0 L 1138.0,681.0 L 1137.0,682.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
<path d="M 714.0,691.0 L 714.0,692.0 L 711.0,695.0 L 711.0,696.0 L 709.0,698.0 L 709.0,699.0 L 708.0,700.0 L 708.0,702.0 L 707.0,703.0 L 707.0,706.0 L 706.0,707.0 L 706.0,711.0 L 705.0,712.0 L 705.0,734.0 L 706.0,735.0 L 706.0,739.0 L 707.0,740.0 L 707.0,743.0 L 708.0,744.0 L 708.0,745.0 L 709.0,746.0 L 709.0,747.0 L 710.0,748.0 L 711.0,751.0 L 713.0,753.0 L 713.0,754.0 L 718.0,759.0 L 719.0,759.0 L 721.0,761.0 L 722.0,761.0 L 723.0,762.0 L 724.0,762.0 L 729.0,765.0 L 731.0,765.0 L 732.0,766.0 L 735.0,766.0 L 736.0,767.0 L 741.0,767.0 L 742.0,768.0 L 775.0,768.0 L 776.0,767.0 L 781.0,767.0 L 782.0,766.0 L 785.0,766.0 L 786.0,765.0 L 788.0,765.0 L 789.0,764.0 L 790.0,764.0 L 791.0,763.0 L 792.0,763.0 L 793.0,762.0 L 796.0,761.0 L 798.0,759.0 L 799.0,759.0 L 805.0,753.0 L 805.0,752.0 L 807.0,750.0 L 807.0,749.0 L 809.0,746.0 L 809.0,744.0 L 811.0,741.0 L 811.0,737.0 L 812.0,736.0 L 812.0,711.0 L 811.0,710.0 L 811.0,706.0 L 810.0,705.0 L 810.0,703.0 L 809.0,702.0 L 809.0,700.0 L 808.0,699.0 L 808.0,698.0 L 806.0,696.0 L 806.0,695.0 L 803.0,692.0 L 803.0,691.0 L 799.0,687.0 L 798.0,687.0 L 793.0,683.0 L 792.0,683.0 L 791.0,682.0 L 789.0,682.0 L 786.0,680.0 L 783.0,680.0 L 782.0,679.0 L 779.0,679.0 L 778.0,678.0 L 770.0,678.0 L 769.0,677.0 L 748.0,677.0 L 747.0,678.0 L 739.0,678.0 L 738.0,679.0 L 734.0,679.0 L 733.0,680.0 L 731.0,680.0 L 730.0,681.0 L 728.0,681.0 L 727.0,682.0 L 726.0,682.0 L 725.0,683.0 L 722.0,684.0 L 720.0,686.0 L 719.0,686.0 Z M 725.0,702.0 L 730.0,697.0 L 731.0,697.0 L 733.0,695.0 L 735.0,695.0 L 738.0,693.0 L 742.0,693.0 L 743.0,692.0 L 772.0,692.0 L 773.0,693.0 L 777.0,693.0 L 778.0,694.0 L 781.0,694.0 L 782.0,695.0 L 785.0,696.0 L 787.0,698.0 L 788.0,698.0 L 791.0,701.0 L 791.0,702.0 L 793.0,704.0 L 793.0,705.0 L 794.0,706.0 L 794.0,708.0 L 795.0,709.0 L 795.0,712.0 L 796.0,713.0 L 796.0,733.0 L 795.0,734.0 L 795.0,738.0 L 794.0,739.0 L 793.0,742.0 L 791.0,744.0 L 791.0,745.0 L 788.0,748.0 L 787.0,748.0 L 785.0,750.0 L 784.0,750.0 L 781.0,752.0 L 778.0,752.0 L 777.0,753.0 L 770.0,753.0 L 769.0,754.0 L 747.0,754.0 L 746.0,753.0 L 740.0,753.0 L 739.0,752.0 L 737.0,752.0 L 736.0,751.0 L 734.0,751.0 L 733.0,750.0 L 732.0,750.0 L 730.0,748.0 L 729.0,748.0 L 725.0,744.0 L 725.0,743.0 L 722.0,738.0 L 722.0,734.0 L 721.0,733.0 L 721.0,713.0 L 722.0,712.0 L 722.0,709.0 L 723.0,708.0 L 723.0,706.0 L 724.0,705.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
<path d="M 95.0,678.0 L 95.0,766.0 L 96.0,767.0 L 111.0,767.0 L 111.0,737.0 L 112.0,736.0 L 171.0,736.0 L 172.0,735.0 L 175.0,735.0 L 176.0,734.0 L 178.0,734.0 L 179.0,733.0 L 180.0,733.0 L 182.0,731.0 L 183.0,731.0 L 186.0,728.0 L 187.0,728.0 L 187.0,727.0 L 190.0,724.0 L 190.0,723.0 L 193.0,718.0 L 193.0,715.0 L 194.0,714.0 L 194.0,702.0 L 193.0,701.0 L 193.0,696.0 L 192.0,695.0 L 192.0,693.0 L 191.0,692.0 L 190.0,689.0 L 188.0,687.0 L 188.0,686.0 L 185.0,683.0 L 184.0,683.0 L 182.0,681.0 L 181.0,681.0 L 180.0,680.0 L 178.0,680.0 L 177.0,679.0 L 174.0,679.0 L 173.0,678.0 L 168.0,678.0 L 167.0,677.0 L 156.0,677.0 L 155.0,678.0 L 135.0,678.0 L 134.0,677.0 L 132.0,677.0 L 131.0,678.0 L 108.0,678.0 L 107.0,677.0 L 105.0,677.0 L 104.0,678.0 Z M 111.0,693.0 L 112.0,692.0 L 169.0,692.0 L 170.0,693.0 L 172.0,693.0 L 177.0,698.0 L 177.0,699.0 L 178.0,700.0 L 178.0,712.0 L 177.0,713.0 L 177.0,715.0 L 176.0,716.0 L 176.0,717.0 L 174.0,719.0 L 173.0,719.0 L 170.0,721.0 L 167.0,721.0 L 166.0,722.0 L 154.0,722.0 L 153.0,721.0 L 152.0,722.0 L 115.0,722.0 L 114.0,721.0 L 113.0,722.0 L 111.0,720.0 Z" fill="#FFFFFF" fill-rule="evenodd"/><rect x="250" y="678" width="16" height="90" fill="#FFFFFF"/></g>
|
||||||
|
<path d="M 1084.0,678.0 L 1063.0,678.0 L 1056.0,685.0 L 1056.0,686.0 L 1049.0,693.0 L 1049.0,694.0 L 1042.0,701.0 L 1042.0,702.0 L 1036.0,708.0 L 1036.0,709.0 L 1035.0,710.0 L 1035.0,711.0 L 1033.0,713.0 L 1033.0,714.0 L 1032.0,715.0 L 1032.0,716.0 L 1031.0,717.0 L 1031.0,718.0 L 1030.0,719.0 L 1030.0,720.0 L 1028.0,723.0 L 1028.0,725.0 L 1026.0,728.0 L 1026.0,730.0 L 1025.0,731.0 L 1025.0,733.0 L 1024.0,734.0 L 1024.0,738.0 L 1023.0,739.0 L 1023.0,743.0 L 1022.0,744.0 L 1022.0,752.0 L 1021.0,753.0 L 1021.0,766.0 L 1022.0,767.0 L 1038.0,767.0 L 1038.0,737.0 L 1039.0,736.0 L 1039.0,734.0 L 1040.0,733.0 L 1040.0,732.0 L 1041.0,731.0 L 1041.0,730.0 L 1043.0,728.0 L 1043.0,727.0 L 1046.0,724.0 L 1046.0,723.0 L 1052.0,717.0 L 1052.0,716.0 L 1060.0,708.0 L 1060.0,707.0 L 1065.0,702.0 L 1065.0,701.0 L 1072.0,694.0 L 1072.0,693.0 L 1077.0,688.0 L 1077.0,687.0 L 1082.0,682.0 L 1082.0,681.0 L 1084.0,679.0 L 1085.0,679.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
<path d="M 975.0,678.0 L 975.0,679.0 L 979.0,683.0 L 979.0,684.0 L 983.0,688.0 L 983.0,689.0 L 990.0,696.0 L 990.0,697.0 L 996.0,703.0 L 996.0,704.0 L 1002.0,710.0 L 1002.0,711.0 L 1010.0,719.0 L 1010.0,720.0 L 1014.0,724.0 L 1014.0,725.0 L 1017.0,728.0 L 1017.0,729.0 L 1018.0,730.0 L 1018.0,731.0 L 1019.0,732.0 L 1020.0,732.0 L 1020.0,729.0 L 1021.0,728.0 L 1021.0,726.0 L 1022.0,725.0 L 1022.0,722.0 L 1023.0,721.0 L 1023.0,715.0 L 1024.0,714.0 L 1023.0,713.0 L 1023.0,709.0 L 1022.0,708.0 L 1022.0,707.0 L 1021.0,706.0 L 1021.0,705.0 L 1016.0,700.0 L 1016.0,699.0 L 1010.0,693.0 L 1010.0,692.0 L 1003.0,685.0 L 1003.0,684.0 L 997.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||||
|
|
||||||
|
</svg>
|
||||||
|
After Width: | Height: | Size: 13 KiB |
@@ -0,0 +1,14 @@
|
|||||||
|
# dashboards/
|
||||||
|
|
||||||
|
Exported Superset dashboards and the operator scripts that import them. Not baked into the
|
||||||
|
image, so changes here do not rebuild or restart Superset. Empty until the first fuel
|
||||||
|
dashboards exist.
|
||||||
|
|
||||||
|
Export from a running instance (inside the container, as the server's deploy user):
|
||||||
|
|
||||||
|
docker exec fuel-monitoring-superset-1 superset export-dashboards -f /tmp/dashboards.zip
|
||||||
|
docker cp fuel-monitoring-superset-1:/tmp/dashboards.zip .
|
||||||
|
|
||||||
|
Import is the reverse with `superset import-dashboards -p <zip> -u <username>`. The
|
||||||
|
username must be an existing Superset user, which here means someone who has signed in
|
||||||
|
through Keycloak at least once.
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
# patches/
|
||||||
|
|
||||||
|
Runtime patches applied while the Superset image is built
|
||||||
|
(`stacks/fuel-monitoring/superset/Dockerfile`). Empty until Petrolyst needs one.
|
||||||
|
|
||||||
|
Same convention as media-insights:
|
||||||
|
|
||||||
|
- One file per fix, named `fix_<what>.py`. It patches files inside the installed image
|
||||||
|
(`/app/superset/...` or the venv), for changes that cannot be made through
|
||||||
|
`superset_config.py`.
|
||||||
|
- `python3 fix_<what>.py` applies the patch; `python3 fix_<what>.py --verify` exits non-zero
|
||||||
|
unless the patched code is in place. Both must be idempotent.
|
||||||
|
- The Dockerfile runs both for every `fix_*.py` and stops at the first failure, so a patch that
|
||||||
|
no longer applies to a new Superset version fails the build instead of shipping silently.
|
||||||
|
- Everything in this folder is a build input: changing it gives the image a new tag and
|
||||||
|
recreates the `superset` container on the next deploy.
|
||||||
@@ -0,0 +1,92 @@
|
|||||||
|
"""Keycloak sign-in for Superset at atoma.petrolyst.com.
|
||||||
|
|
||||||
|
Set as CUSTOM_SECURITY_MANAGER in superset_config.py. Checked against
|
||||||
|
Flask-AppBuilder 5.0.2 as bundled in apache/superset:6.1.0. Three changes to
|
||||||
|
the stock behaviour:
|
||||||
|
|
||||||
|
- User info and roles come from the ID token that Keycloak issues for client
|
||||||
|
"petrolyst" (claim petrolyst_roles). FAB's built-in Keycloak branch would read
|
||||||
|
a "groups" claim from the userinfo endpoint instead.
|
||||||
|
- /login/ goes straight to Keycloak. Superset 6.1 normally registers its React
|
||||||
|
login page on /login/, which only shows a "Sign in with keycloak" button.
|
||||||
|
- /logout/ also ends the Keycloak session, so the next visit asks for the
|
||||||
|
password and OTP again.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from typing import Any, Optional
|
||||||
|
from urllib.parse import urlencode
|
||||||
|
|
||||||
|
from flask import g, redirect, request, url_for
|
||||||
|
from flask_appbuilder import expose
|
||||||
|
from flask_appbuilder.security.views import AuthOAuthView
|
||||||
|
from flask_appbuilder.utils.base import get_safe_redirect
|
||||||
|
from flask_login import logout_user
|
||||||
|
from werkzeug.wrappers import Response as WerkzeugResponse
|
||||||
|
|
||||||
|
from superset.security import SupersetSecurityManager
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
PROVIDER = "keycloak"
|
||||||
|
ROLE_CLAIM = "petrolyst_roles"
|
||||||
|
|
||||||
|
|
||||||
|
class PetrolystAuthOAuthView(AuthOAuthView):
|
||||||
|
@expose("/login/")
|
||||||
|
@expose("/login/<provider>")
|
||||||
|
def login(self, provider: Optional[str] = None) -> WerkzeugResponse:
|
||||||
|
authenticated = g.user is not None and g.user.is_authenticated
|
||||||
|
if provider is None and not authenticated:
|
||||||
|
# FAB encodes the query arguments into the OAuth state and runs
|
||||||
|
# get_safe_redirect on "next" again after the callback; checking it
|
||||||
|
# here as well keeps an unsafe value out of the state altogether.
|
||||||
|
args = {}
|
||||||
|
if request.args.get("next"):
|
||||||
|
args["next"] = get_safe_redirect(request.args["next"])
|
||||||
|
return redirect(url_for(".login", provider=PROVIDER, **args))
|
||||||
|
return super().login(provider)
|
||||||
|
|
||||||
|
@expose("/logout/")
|
||||||
|
def logout(self) -> WerkzeugResponse:
|
||||||
|
logout_user()
|
||||||
|
remote = self.appbuilder.sm.oauth_remotes[PROVIDER]
|
||||||
|
end_session = remote.load_server_metadata().get("end_session_endpoint")
|
||||||
|
if not end_session:
|
||||||
|
logger.warning("Keycloak metadata has no end_session_endpoint")
|
||||||
|
return redirect(self.appbuilder.get_url_for_index)
|
||||||
|
# client_id instead of id_token_hint: Keycloak then asks the user to
|
||||||
|
# confirm the logout once, and no ID token has to live in the cookie.
|
||||||
|
params = {
|
||||||
|
"client_id": remote.client_id,
|
||||||
|
"post_logout_redirect_uri": request.host_url,
|
||||||
|
}
|
||||||
|
return redirect(f"{end_session}?{urlencode(params)}")
|
||||||
|
|
||||||
|
|
||||||
|
class PetrolystSecurityManager(SupersetSecurityManager):
|
||||||
|
authoauthview = PetrolystAuthOAuthView
|
||||||
|
# Without this, Superset registers its React login page on /login/ before
|
||||||
|
# FAB's OAuth view, and that page wins the route.
|
||||||
|
register_superset_auth_view = False
|
||||||
|
|
||||||
|
def oauth_user_info(
|
||||||
|
self, provider: str, response: Optional[dict[str, Any]] = None
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
if provider != PROVIDER:
|
||||||
|
raise ValueError(f"unsupported OAuth provider {provider!r}")
|
||||||
|
# Authlib verifies the ID token (signature, issuer, audience, nonce)
|
||||||
|
# and puts its claims under "userinfo".
|
||||||
|
claims = (response or {}).get("userinfo")
|
||||||
|
if not claims:
|
||||||
|
claims = self.oauth_remotes[provider].userinfo()
|
||||||
|
username = claims.get("preferred_username")
|
||||||
|
if not username:
|
||||||
|
raise ValueError("Keycloak token has no preferred_username")
|
||||||
|
return {
|
||||||
|
"username": username,
|
||||||
|
"email": claims.get("email", ""),
|
||||||
|
"first_name": claims.get("given_name", ""),
|
||||||
|
"last_name": claims.get("family_name", ""),
|
||||||
|
"role_keys": list(claims.get(ROLE_CLAIM) or []),
|
||||||
|
}
|
||||||
@@ -0,0 +1,252 @@
|
|||||||
|
"""Superset configuration for Petrolyst (ATOMA fuel monitoring).
|
||||||
|
|
||||||
|
Customization only, and no secrets: those are read from /run/secrets/<name>,
|
||||||
|
which stacks/fuel-monitoring/compose.yaml mounts from /opt/petrolyst/secrets/.
|
||||||
|
Baked into the image at /app/pythonpath/, which is on PYTHONPATH in
|
||||||
|
apache/superset:6.1.0, so Superset loads it without SUPERSET_CONFIG_PATH.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import pathlib
|
||||||
|
from copy import deepcopy
|
||||||
|
|
||||||
|
from flask_appbuilder.security.manager import AUTH_OAUTH
|
||||||
|
from sqlalchemy.engine.url import URL
|
||||||
|
|
||||||
|
from petrolyst_security import PetrolystSecurityManager
|
||||||
|
from superset import config as superset_defaults
|
||||||
|
|
||||||
|
|
||||||
|
def _read_secret(name: str, env_var: str | None = None) -> str:
|
||||||
|
"""Prefer /run/secrets/<name>; fall back to an env var for local dev.
|
||||||
|
|
||||||
|
Uses rstrip('\r\n') (not strip()) so passwords or keys that legitimately
|
||||||
|
contain leading/trailing spaces are not silently mutated. Raises on empty
|
||||||
|
values so boot fails loudly instead of accepting a blank secret.
|
||||||
|
"""
|
||||||
|
path = pathlib.Path(f"/run/secrets/{name}")
|
||||||
|
if path.is_file():
|
||||||
|
value = path.read_text().rstrip("\r\n")
|
||||||
|
else:
|
||||||
|
value = os.environ[env_var or name.upper()]
|
||||||
|
if not value:
|
||||||
|
raise RuntimeError(f"secret {name!r} must not be empty")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
SECRET_KEY = _read_secret("superset_secret_key", "SUPERSET_SECRET_KEY")
|
||||||
|
|
||||||
|
SQLALCHEMY_DATABASE_URI = URL.create(
|
||||||
|
"postgresql+psycopg2",
|
||||||
|
username="superset",
|
||||||
|
password=_read_secret("superset_db_password", "SUPERSET_DB_PASSWORD"),
|
||||||
|
host=os.environ.get("SUPERSET_DB_HOST", "superset-db"),
|
||||||
|
port=5432,
|
||||||
|
database="superset",
|
||||||
|
).render_as_string(hide_password=False)
|
||||||
|
|
||||||
|
# --- Sign-in through Keycloak (client "petrolyst") ----------------------------
|
||||||
|
# Users and roles are managed in Keycloak; there is no local admin account.
|
||||||
|
# Break-glass access: `superset fab ...` inside the container on the server.
|
||||||
|
KEYCLOAK_REALM_URL = os.environ.get(
|
||||||
|
"KEYCLOAK_REALM_URL", "https://iam.thedigitalbridge.io/realms/thedigitalbridge"
|
||||||
|
).rstrip("/")
|
||||||
|
|
||||||
|
AUTH_TYPE = AUTH_OAUTH
|
||||||
|
CUSTOM_SECURITY_MANAGER = PetrolystSecurityManager
|
||||||
|
OAUTH_PROVIDERS = [
|
||||||
|
{
|
||||||
|
"name": "keycloak",
|
||||||
|
"icon": "fa-key",
|
||||||
|
"token_key": "access_token",
|
||||||
|
"remote_app": {
|
||||||
|
"client_id": os.environ.get("KEYCLOAK_CLIENT_ID", "petrolyst"),
|
||||||
|
"client_secret": _read_secret(
|
||||||
|
"keycloak_petrolyst_client_secret", "KEYCLOAK_CLIENT_SECRET"
|
||||||
|
),
|
||||||
|
"server_metadata_url": f"{KEYCLOAK_REALM_URL}/.well-known/openid-configuration",
|
||||||
|
"api_base_url": f"{KEYCLOAK_REALM_URL}/protocol/openid-connect/",
|
||||||
|
"client_kwargs": {
|
||||||
|
"scope": "openid email profile",
|
||||||
|
"code_challenge_method": "S256",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
]
|
||||||
|
|
||||||
|
# Keys are Keycloak client roles of "petrolyst" (claim petrolyst_roles).
|
||||||
|
AUTH_ROLES_MAPPING = {
|
||||||
|
"superset-admin": ["Admin"],
|
||||||
|
"superset-editor": ["Alpha"],
|
||||||
|
"superset-viewer": ["Gamma"],
|
||||||
|
}
|
||||||
|
AUTH_ROLES_SYNC_AT_LOGIN = True
|
||||||
|
AUTH_USER_REGISTRATION = True
|
||||||
|
# Public has no permissions: a Keycloak user without a Superset role signs in
|
||||||
|
# and sees nothing.
|
||||||
|
AUTH_USER_REGISTRATION_ROLE = "Public"
|
||||||
|
|
||||||
|
# Viewers (Gamma) see a dashboard once it lists their role.
|
||||||
|
FEATURE_FLAGS = {"DASHBOARD_RBAC": True}
|
||||||
|
|
||||||
|
# --- Behind Traefik -----------------------------------------------------------
|
||||||
|
ENABLE_PROXY_FIX = True
|
||||||
|
PREFERRED_URL_SCHEME = "https"
|
||||||
|
SESSION_COOKIE_SECURE = True
|
||||||
|
SESSION_COOKIE_HTTPONLY = True
|
||||||
|
SESSION_COOKIE_SAMESITE = "Lax"
|
||||||
|
TALISMAN_CONFIG = deepcopy(superset_defaults.TALISMAN_CONFIG)
|
||||||
|
TALISMAN_CONFIG["session_cookie_secure"] = True
|
||||||
|
|
||||||
|
# --- Redis: caches and rate limits (no Celery, so no results backend) ---------
|
||||||
|
REDIS_URL = os.environ.get("REDIS_URL", "redis://redis:6379")
|
||||||
|
|
||||||
|
CACHE_CONFIG = {
|
||||||
|
"CACHE_TYPE": "RedisCache",
|
||||||
|
"CACHE_DEFAULT_TIMEOUT": 300,
|
||||||
|
"CACHE_KEY_PREFIX": "superset_cache_",
|
||||||
|
"CACHE_REDIS_URL": f"{REDIS_URL}/0",
|
||||||
|
}
|
||||||
|
DATA_CACHE_CONFIG = {
|
||||||
|
"CACHE_TYPE": "RedisCache",
|
||||||
|
"CACHE_DEFAULT_TIMEOUT": 3600,
|
||||||
|
"CACHE_KEY_PREFIX": "superset_data_",
|
||||||
|
"CACHE_REDIS_URL": f"{REDIS_URL}/0",
|
||||||
|
}
|
||||||
|
FILTER_STATE_CACHE_CONFIG = {
|
||||||
|
"CACHE_TYPE": "RedisCache",
|
||||||
|
"CACHE_DEFAULT_TIMEOUT": 86400,
|
||||||
|
"CACHE_KEY_PREFIX": "superset_filter_",
|
||||||
|
"CACHE_REDIS_URL": f"{REDIS_URL}/0",
|
||||||
|
}
|
||||||
|
EXPLORE_FORM_DATA_CACHE_CONFIG = {
|
||||||
|
"CACHE_TYPE": "RedisCache",
|
||||||
|
"CACHE_DEFAULT_TIMEOUT": 86400,
|
||||||
|
"CACHE_KEY_PREFIX": "superset_explore_",
|
||||||
|
"CACHE_REDIS_URL": f"{REDIS_URL}/0",
|
||||||
|
}
|
||||||
|
RATELIMIT_STORAGE_URI = f"{REDIS_URL}/1"
|
||||||
|
|
||||||
|
# --- Petrolyst look and feel --------------------------------------------------
|
||||||
|
# Values from petrolyst-ui/design/brand/07_UI_Tokens/petrolyst_tokens.css.
|
||||||
|
# Assets live in assets/branding/ and are served from /static/assets/branding/.
|
||||||
|
BRANDING = "/static/assets/branding"
|
||||||
|
|
||||||
|
APP_NAME = "Petrolyst"
|
||||||
|
APP_ICON = f"{BRANDING}/petrolyst_logo_standard_fullcolor_no_tagline.svg"
|
||||||
|
FAVICONS = [
|
||||||
|
{"href": f"{BRANDING}/favicon.svg", "type": "image/svg+xml"},
|
||||||
|
{"href": f"{BRANDING}/favicon.ico", "type": "image/x-icon"},
|
||||||
|
{"href": f"{BRANDING}/petrolyst_icon_180.png", "rel": "apple-touch-icon"},
|
||||||
|
]
|
||||||
|
|
||||||
|
THEME_DEFAULT = deepcopy(superset_defaults.THEME_DEFAULT)
|
||||||
|
THEME_DEFAULT["token"].update(
|
||||||
|
{
|
||||||
|
"brandAppName": APP_NAME,
|
||||||
|
"brandLogoAlt": "Petrolyst",
|
||||||
|
"brandLogoUrl": APP_ICON,
|
||||||
|
"brandLogoHref": "/",
|
||||||
|
# The standard logo is stacked (mark over wordmark); 40px keeps the Y
|
||||||
|
# legible and, with the margin, the 60px navbar height.
|
||||||
|
"brandLogoHeight": "40px",
|
||||||
|
"brandLogoMargin": "10px 0",
|
||||||
|
# Action teal, not the raw brand teal #009688: the brand guide keeps
|
||||||
|
# raw teal off small text and controls.
|
||||||
|
"colorPrimary": "#00695C",
|
||||||
|
"colorLink": "#00695C",
|
||||||
|
"colorSuccess": "#0F7B6C",
|
||||||
|
"colorWarning": "#B45309",
|
||||||
|
"colorError": "#B3261E",
|
||||||
|
"colorInfo": "#2563EB",
|
||||||
|
"colorTextBase": "#0B1D33",
|
||||||
|
"colorBgLayout": "#F8FAFC",
|
||||||
|
"colorBorder": "#E2E8F0",
|
||||||
|
"borderRadius": 8,
|
||||||
|
"fontFamily": "Inter, system-ui, sans-serif",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
THEME_DARK = deepcopy(THEME_DEFAULT)
|
||||||
|
THEME_DARK["algorithm"] = "dark"
|
||||||
|
THEME_DARK["token"].update(
|
||||||
|
{
|
||||||
|
"brandLogoUrl": f"{BRANDING}/petrolyst_logo_standard_white_no_tagline.svg",
|
||||||
|
"colorPrimary": "#35C5B6",
|
||||||
|
"colorLink": "#35C5B6",
|
||||||
|
"colorSuccess": "#35C5B6",
|
||||||
|
"colorWarning": "#F59E0B",
|
||||||
|
"colorError": "#FF6B6B",
|
||||||
|
"colorInfo": "#60A5FA",
|
||||||
|
"colorTextBase": "#F8FAFC",
|
||||||
|
"colorBgBase": "#071425",
|
||||||
|
"colorBgLayout": "#071425",
|
||||||
|
"colorBgContainer": "#12263D",
|
||||||
|
"colorBorder": "#526D82",
|
||||||
|
"colorEditorSelection": superset_defaults.THEME_DARK["token"][
|
||||||
|
"colorEditorSelection"
|
||||||
|
],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
# Chart colours. Validated with the dataviz skill's validator on both chart
|
||||||
|
# surfaces (#FFFFFF light, #12263D dark): all slots in both lightness bands and
|
||||||
|
# >= 3:1; worst adjacent CVD dE 13.7, normal-vision dE 26.2; the first three
|
||||||
|
# slots also pass all-pairs (scatter, maps). Brand teal leads; brand cyan
|
||||||
|
# #00B6E6 is snapped to #0097C4 because it is too light for the dark surface
|
||||||
|
# and below 3:1 on white. The other hues come from the validated reference set.
|
||||||
|
# Status meaning (warning, critical) never rides on these colours.
|
||||||
|
EXTRA_CATEGORICAL_COLOR_SCHEMES = [
|
||||||
|
{
|
||||||
|
"id": "petrolyst",
|
||||||
|
"description": "Petrolyst categorical palette, brand teal first",
|
||||||
|
"label": "Petrolyst",
|
||||||
|
"isDefault": True,
|
||||||
|
"colors": [
|
||||||
|
"#009688", # brand teal
|
||||||
|
"#d95926", # orange
|
||||||
|
"#9085e9", # violet
|
||||||
|
"#c98500", # amber
|
||||||
|
"#0097C4", # brand cyan, snapped
|
||||||
|
"#e34948", # red
|
||||||
|
"#3987e5", # blue
|
||||||
|
"#d55181", # magenta
|
||||||
|
],
|
||||||
|
}
|
||||||
|
]
|
||||||
|
|
||||||
|
# One hue, light to dark; the lightest step means "near zero" and may recede.
|
||||||
|
EXTRA_SEQUENTIAL_COLOR_SCHEMES = [
|
||||||
|
{
|
||||||
|
"id": "petrolyst_teal",
|
||||||
|
"description": "Petrolyst teal, light to dark",
|
||||||
|
"label": "Petrolyst Teal",
|
||||||
|
"isDiverging": False,
|
||||||
|
"isDefault": True,
|
||||||
|
"colors": [
|
||||||
|
"#E6F4F2",
|
||||||
|
"#B3DED8",
|
||||||
|
"#7FC8BE",
|
||||||
|
"#4CB2A4",
|
||||||
|
"#009688",
|
||||||
|
"#00695C",
|
||||||
|
"#004D44",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "petrolyst_navy",
|
||||||
|
"description": "Petrolyst navy, light to dark",
|
||||||
|
"label": "Petrolyst Navy",
|
||||||
|
"isDiverging": False,
|
||||||
|
"isDefault": False,
|
||||||
|
"colors": [
|
||||||
|
"#E8EEF5",
|
||||||
|
"#C3D1E2",
|
||||||
|
"#8FA7C4",
|
||||||
|
"#5C7DA6",
|
||||||
|
"#2F5585",
|
||||||
|
"#183A63",
|
||||||
|
"#0B1D33",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
]
|
||||||
@@ -0,0 +1,174 @@
|
|||||||
|
# Fuel-monitoring analytics: Superset on atoma.petrolyst.com, reading the
|
||||||
|
# operational store (fuel-db) through Trino. Run only through deploy/deploy.sh
|
||||||
|
# or deploy/rollback.sh: they export SUPERSET_IMAGE_TAG and TRINO_CONFIG_HASH
|
||||||
|
# from the release's ARTIFACTS file, and the :? guards below stop Compose from
|
||||||
|
# running with either unset.
|
||||||
|
name: fuel-monitoring
|
||||||
|
|
||||||
|
x-superset-env: &superset-env
|
||||||
|
KEYCLOAK_REALM_URL: ${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/thedigitalbridge}
|
||||||
|
|
||||||
|
x-superset: &superset
|
||||||
|
image: fuel-superset:${SUPERSET_IMAGE_TAG:?SUPERSET_IMAGE_TAG unset - run through deploy/deploy.sh}
|
||||||
|
pull_policy: never
|
||||||
|
secrets:
|
||||||
|
- superset_secret_key
|
||||||
|
- superset_db_password
|
||||||
|
- keycloak_petrolyst_client_secret
|
||||||
|
|
||||||
|
services:
|
||||||
|
superset:
|
||||||
|
<<: *superset
|
||||||
|
build:
|
||||||
|
context: ./superset
|
||||||
|
additional_contexts:
|
||||||
|
insights: ../../petrolyst-insights
|
||||||
|
environment:
|
||||||
|
<<: *superset-env
|
||||||
|
SERVER_WORKER_AMOUNT: "2"
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
superset-db:
|
||||||
|
condition: service_healthy
|
||||||
|
redis:
|
||||||
|
condition: service_healthy
|
||||||
|
networks:
|
||||||
|
- backend
|
||||||
|
- proxy
|
||||||
|
mem_limit: 1536m
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "curl", "-fsS", "http://127.0.0.1:8088/health"]
|
||||||
|
interval: 15s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 10
|
||||||
|
start_period: 60s
|
||||||
|
labels:
|
||||||
|
traefik.enable: "true"
|
||||||
|
traefik.docker.network: proxy
|
||||||
|
traefik.http.routers.fm-superset.rule: Host(`${ATOMA_HOST:-atoma.petrolyst.com}`)
|
||||||
|
traefik.http.routers.fm-superset.entrypoints: websecure
|
||||||
|
traefik.http.routers.fm-superset.tls.certresolver: le
|
||||||
|
traefik.http.routers.fm-superset.middlewares: secure-headers@file
|
||||||
|
traefik.http.services.fm-superset.loadbalancer.server.port: "8088"
|
||||||
|
|
||||||
|
# One-shot: never started by `up`; deploy.sh runs it with `compose run --rm`.
|
||||||
|
superset-init:
|
||||||
|
<<: *superset
|
||||||
|
profiles: ["init"]
|
||||||
|
environment: *superset-env
|
||||||
|
command: ["/bin/bash", "/app/petrolyst-init.sh"]
|
||||||
|
depends_on:
|
||||||
|
superset-db:
|
||||||
|
condition: service_healthy
|
||||||
|
redis:
|
||||||
|
condition: service_healthy
|
||||||
|
networks:
|
||||||
|
- backend
|
||||||
|
restart: "no"
|
||||||
|
|
||||||
|
superset-db:
|
||||||
|
image: postgres:17.11-alpine
|
||||||
|
environment:
|
||||||
|
POSTGRES_USER: superset
|
||||||
|
POSTGRES_DB: superset
|
||||||
|
POSTGRES_PASSWORD_FILE: /run/secrets/superset_db_password
|
||||||
|
secrets:
|
||||||
|
- superset_db_password
|
||||||
|
volumes:
|
||||||
|
- superset_db_data:/var/lib/postgresql/data
|
||||||
|
networks:
|
||||||
|
- backend
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: 512m
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U superset -d superset"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 10
|
||||||
|
|
||||||
|
# Cache and rate-limit store only; nothing to persist.
|
||||||
|
redis:
|
||||||
|
image: redis:7.4.11-alpine
|
||||||
|
command: ["redis-server", "--maxmemory", "128mb", "--maxmemory-policy", "allkeys-lru", "--save", "", "--appendonly", "no"]
|
||||||
|
networks:
|
||||||
|
- backend
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: 256m
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "redis-cli", "ping"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 10
|
||||||
|
|
||||||
|
trino:
|
||||||
|
image: trinodb/trino:484
|
||||||
|
env_file:
|
||||||
|
- ${PETROLYST_ROOT:-/opt/petrolyst}/env/fuel-monitoring.env
|
||||||
|
volumes:
|
||||||
|
- ./trino/etc:/etc/trino:ro
|
||||||
|
labels:
|
||||||
|
# Compose cannot see content changes in a bind mount; this label changes
|
||||||
|
# with trino/etc, so exactly a config change recreates the container.
|
||||||
|
petrolyst.config-hash: ${TRINO_CONFIG_HASH:?TRINO_CONFIG_HASH unset - run through deploy/deploy.sh}
|
||||||
|
depends_on:
|
||||||
|
fuel-db:
|
||||||
|
condition: service_healthy
|
||||||
|
networks:
|
||||||
|
- backend
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: 3g
|
||||||
|
# The image's own healthcheck (/usr/lib/trino/bin/health-check) applies.
|
||||||
|
|
||||||
|
fuel-db:
|
||||||
|
image: timescale/timescaledb:2.31.0-pg17
|
||||||
|
environment:
|
||||||
|
POSTGRES_USER: postgres
|
||||||
|
POSTGRES_DB: fuel
|
||||||
|
POSTGRES_PASSWORD_FILE: /run/secrets/fuel_db_password
|
||||||
|
TS_TUNE_NUM_CPUS: "2"
|
||||||
|
secrets:
|
||||||
|
- fuel_db_password
|
||||||
|
- fuel_db_trino_password
|
||||||
|
volumes:
|
||||||
|
- fuel_db_data:/var/lib/postgresql/data
|
||||||
|
# A single file, not the folder: the image's own TimescaleDB init scripts
|
||||||
|
# live in the same directory and must stay visible.
|
||||||
|
- ./fuel-db/init/01-roles.sh:/docker-entrypoint-initdb.d/010_petrolyst_roles.sh:ro
|
||||||
|
networks:
|
||||||
|
- backend
|
||||||
|
- fuel-db-tunnel
|
||||||
|
# Loopback only, for `ssh -L 5433:127.0.0.1:5433 petrolyst`.
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:5433:5432"
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: 1536m
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U postgres -d fuel"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 10
|
||||||
|
|
||||||
|
networks:
|
||||||
|
backend:
|
||||||
|
internal: true
|
||||||
|
proxy:
|
||||||
|
external: true
|
||||||
|
# A port cannot be published from an internal network; this one exists only
|
||||||
|
# so fuel-db's loopback port works.
|
||||||
|
fuel-db-tunnel: {}
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
superset_db_data:
|
||||||
|
fuel_db_data:
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
superset_secret_key:
|
||||||
|
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/superset_secret_key
|
||||||
|
superset_db_password:
|
||||||
|
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/superset_db_password
|
||||||
|
fuel_db_password:
|
||||||
|
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/fuel_db_password
|
||||||
|
fuel_db_trino_password:
|
||||||
|
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/fuel_db_trino_password
|
||||||
|
keycloak_petrolyst_client_secret:
|
||||||
|
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/shared/keycloak_petrolyst_client_secret
|
||||||
+25
@@ -0,0 +1,25 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Runs once, when the fuel-db volume is first initialised (mounted after the
|
||||||
|
# image's own TimescaleDB scripts, as 010_petrolyst_roles.sh).
|
||||||
|
# fuel_owner owns database "fuel", its public schema and every table in it.
|
||||||
|
# Loaders log in as their own role but act as fuel_owner (README
|
||||||
|
# "Data loaders"): the default privileges below cover only what
|
||||||
|
# fuel_owner itself creates, not what its members create.
|
||||||
|
# trino_ro what Trino connects as: read-only, also on tables created later.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
trino_password=$(cat /run/secrets/fuel_db_trino_password)
|
||||||
|
|
||||||
|
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" \
|
||||||
|
-v trino_password="$trino_password" <<'SQL'
|
||||||
|
CREATE ROLE fuel_owner NOLOGIN;
|
||||||
|
ALTER DATABASE fuel OWNER TO fuel_owner;
|
||||||
|
REVOKE CREATE ON SCHEMA public FROM PUBLIC;
|
||||||
|
|
||||||
|
CREATE ROLE trino_ro LOGIN PASSWORD :'trino_password';
|
||||||
|
GRANT CONNECT ON DATABASE fuel TO trino_ro;
|
||||||
|
GRANT USAGE ON SCHEMA public TO trino_ro;
|
||||||
|
GRANT SELECT ON ALL TABLES IN SCHEMA public TO trino_ro;
|
||||||
|
ALTER DEFAULT PRIVILEGES FOR ROLE fuel_owner IN SCHEMA public
|
||||||
|
GRANT SELECT ON TABLES TO trino_ro;
|
||||||
|
SQL
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# Petrolyst Superset image. Deployment only: every customization is copied from
|
||||||
|
# petrolyst-insights/, which compose.yaml passes as the named build context
|
||||||
|
# "insights". deploy/deploy.sh tags the result with a hash of exactly these
|
||||||
|
# inputs (this file plus what is copied below), so the tag changes only when
|
||||||
|
# the image would.
|
||||||
|
FROM apache/superset:6.1.0@sha256:16b50bbef6648912a79e3293d418fefa743ce555d36c5af6b85d859119ed7f88
|
||||||
|
|
||||||
|
USER root
|
||||||
|
# The 6.1.0 image ships neither an OAuth client nor the Postgres and Trino
|
||||||
|
# drivers.
|
||||||
|
RUN uv pip install --no-cache --python /app/.venv/bin/python \
|
||||||
|
authlib==1.9.1 psycopg2-binary==2.9.13 "trino[sqlalchemy]==0.340.0"
|
||||||
|
|
||||||
|
COPY --from=insights --chown=superset:superset superset_config.py petrolyst_security.py /app/pythonpath/
|
||||||
|
COPY --from=insights --chown=superset:superset assets/branding/ /app/superset/static/assets/branding/
|
||||||
|
COPY --from=insights patches/ /app/patches/petrolyst/
|
||||||
|
# Stop at the first patch or verify that fails, so a later success cannot hide
|
||||||
|
# an earlier failure.
|
||||||
|
RUN for p in /app/patches/petrolyst/fix_*.py; do \
|
||||||
|
[ -e "$p" ] || continue; \
|
||||||
|
{ python3 "$p" && python3 "$p" --verify; } || exit 1; \
|
||||||
|
done
|
||||||
|
|
||||||
|
# Deployment, not customization: in the image (and so in the tag) rather than
|
||||||
|
# bind-mounted, so a change to it makes deploy.sh run superset-init again.
|
||||||
|
COPY init.sh /app/petrolyst-init.sh
|
||||||
|
|
||||||
|
USER superset
|
||||||
Executable
+14
@@ -0,0 +1,14 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# superset-init: migrate the metadata database, sync roles and permissions, and
|
||||||
|
# register the Trino connection. It is copied into the Superset image, so a
|
||||||
|
# change here changes the image tag; deploy/deploy.sh runs it on a fresh install
|
||||||
|
# and whenever that tag changes. Every step is idempotent. Unlike the
|
||||||
|
# upstream docker-init.sh it never creates a local admin: users come from
|
||||||
|
# Keycloak.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
superset db upgrade
|
||||||
|
superset init
|
||||||
|
superset set-database-uri \
|
||||||
|
--database_name "Fuel (Trino)" \
|
||||||
|
--uri "trino://superset@trino:8080/fuel"
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
# Analytics never writes through Trino. fuel-db's trino_ro role is read-only as
|
||||||
|
# well; this is the second lock.
|
||||||
|
access-control.name=read-only
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
# The fuel-monitoring operational store (service fuel-db), read-only.
|
||||||
|
connector.name=postgresql
|
||||||
|
connection-url=jdbc:postgresql://fuel-db:5432/fuel
|
||||||
|
connection-user=trino_ro
|
||||||
|
connection-password=${ENV:FUEL_DB_TRINO_PASSWORD}
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
# Single node: coordinator and worker in one process. Reachable only on the
|
||||||
|
# backend network (Superset); no port is published and Traefik has no route.
|
||||||
|
coordinator=true
|
||||||
|
node-scheduler.include-coordinator=true
|
||||||
|
http-server.http.port=8080
|
||||||
|
discovery.uri=http://localhost:8080
|
||||||
|
catalog.management=static
|
||||||
|
|
||||||
|
# Memory bounds. The container is capped at 3g (compose.yaml) and jvm.config
|
||||||
|
# gives the heap 75 % of it (~2.25 GB); 1 GB for queries plus 512 MB headroom
|
||||||
|
# leaves room for the rest of the heap.
|
||||||
|
query.max-memory=1GB
|
||||||
|
query.max-memory-per-node=1GB
|
||||||
|
memory.heap-headroom-per-node=512MB
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
-server
|
||||||
|
-agentpath:/usr/lib/trino/bin/libjvmkill.so
|
||||||
|
-XX:InitialRAMPercentage=75
|
||||||
|
-XX:MaxRAMPercentage=75
|
||||||
|
-XX:G1HeapRegionSize=32M
|
||||||
|
-XX:+ExplicitGCInvokesConcurrent
|
||||||
|
-XX:+HeapDumpOnOutOfMemoryError
|
||||||
|
-XX:+ExitOnOutOfMemoryError
|
||||||
|
-XX:-OmitStackTraceInFastThrow
|
||||||
|
-XX:ReservedCodeCacheSize=256M
|
||||||
|
-XX:PerMethodRecompilationCutoff=10000
|
||||||
|
-XX:PerBytecodeRecompilationCutoff=10000
|
||||||
|
-Djdk.attach.allowAttachSelf=true
|
||||||
|
-Djdk.nio.maxCachedBufferSize=2000000
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
io.trino=INFO
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
node.environment=petrolyst
|
||||||
|
node.data-dir=/data/trino
|
||||||
|
node.id=${ENV:HOSTNAME}
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
# Edge proxy for the whole server. Deployed first by deploy/deploy.sh, which
|
||||||
|
# exports TRAEFIK_CONFIG_HASH from the release's ARTIFACTS file.
|
||||||
|
#
|
||||||
|
# Any stack joins by attaching a service to the external network "proxy" and
|
||||||
|
# setting traefik.* labels; apps without their own login can add the "sso"
|
||||||
|
# middleware (Keycloak client petrolyst, role checked by oauth2-proxy).
|
||||||
|
name: traefik
|
||||||
|
|
||||||
|
services:
|
||||||
|
socket-proxy:
|
||||||
|
image: tecnativa/docker-socket-proxy:v0.5.0
|
||||||
|
environment:
|
||||||
|
CONTAINERS: "1"
|
||||||
|
EVENTS: "1"
|
||||||
|
PING: "1"
|
||||||
|
VERSION: "1"
|
||||||
|
POST: "0"
|
||||||
|
LOG_LEVEL: warning
|
||||||
|
volumes:
|
||||||
|
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||||
|
networks:
|
||||||
|
- socket
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: 64m
|
||||||
|
|
||||||
|
traefik:
|
||||||
|
image: traefik:v3.7.16
|
||||||
|
command: ["--configFile=/etc/traefik/static/traefik.yml"]
|
||||||
|
ports:
|
||||||
|
- "80:80"
|
||||||
|
- "443:443"
|
||||||
|
# Folders, not single files: promotion replaces files by renaming, which a
|
||||||
|
# single-file bind mount would not follow.
|
||||||
|
volumes:
|
||||||
|
- ./static:/etc/traefik/static:ro
|
||||||
|
- ./dynamic:/etc/traefik/dynamic:ro
|
||||||
|
- letsencrypt:/letsencrypt
|
||||||
|
depends_on:
|
||||||
|
- socket-proxy
|
||||||
|
networks:
|
||||||
|
- proxy
|
||||||
|
- socket
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: 256m
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "traefik", "healthcheck", "--configFile=/etc/traefik/static/traefik.yml"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 6
|
||||||
|
labels:
|
||||||
|
petrolyst.config-hash: ${TRAEFIK_CONFIG_HASH:?TRAEFIK_CONFIG_HASH unset - run through deploy/deploy.sh}
|
||||||
|
traefik.enable: "true"
|
||||||
|
traefik.http.routers.dashboard.rule: Host(`${TRAEFIK_HOST:-traefik.petrolyst.com}`)
|
||||||
|
traefik.http.routers.dashboard.entrypoints: websecure
|
||||||
|
traefik.http.routers.dashboard.tls.certresolver: le
|
||||||
|
traefik.http.routers.dashboard.service: api@internal
|
||||||
|
traefik.http.routers.dashboard.middlewares: sso,secure-headers@file
|
||||||
|
# Keycloak gate for the dashboard host only (cookie, return hosts and
|
||||||
|
# role are tied to it; README "Adding a project"): unauthenticated
|
||||||
|
# requests are sent to Keycloak, users without petrolyst:traefik-admin
|
||||||
|
# get 403.
|
||||||
|
traefik.http.middlewares.sso.forwardauth.address: http://oauth2-proxy:4180/
|
||||||
|
traefik.http.middlewares.sso.forwardauth.trustForwardHeader: "true"
|
||||||
|
traefik.http.middlewares.sso.forwardauth.authResponseHeaders: X-Auth-Request-User,X-Auth-Request-Email
|
||||||
|
|
||||||
|
oauth2-proxy:
|
||||||
|
image: quay.io/oauth2-proxy/oauth2-proxy:v7.15.5
|
||||||
|
command:
|
||||||
|
- --provider=keycloak-oidc
|
||||||
|
- --oidc-issuer-url=${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/thedigitalbridge}
|
||||||
|
- --client-id=petrolyst
|
||||||
|
- --client-secret-file=/run/secrets/keycloak_petrolyst_client_secret
|
||||||
|
- --cookie-secret-file=/run/secrets/oauth2_proxy_cookie_secret
|
||||||
|
- --code-challenge-method=S256
|
||||||
|
- --allowed-role=petrolyst:traefik-admin
|
||||||
|
# Required by oauth2-proxy at startup; access is decided by the role above.
|
||||||
|
- --email-domain=*
|
||||||
|
- --redirect-url=https://${TRAEFIK_HOST:-traefik.petrolyst.com}/oauth2/callback
|
||||||
|
- --whitelist-domain=${TRAEFIK_HOST:-traefik.petrolyst.com}
|
||||||
|
- --cookie-domain=${TRAEFIK_HOST:-traefik.petrolyst.com}
|
||||||
|
- --cookie-secure=true
|
||||||
|
- --cookie-samesite=lax
|
||||||
|
- --reverse-proxy=true
|
||||||
|
- --upstream=static://202
|
||||||
|
- --skip-provider-button=true
|
||||||
|
- --set-xauthrequest=true
|
||||||
|
- --http-address=0.0.0.0:4180
|
||||||
|
secrets:
|
||||||
|
- keycloak_petrolyst_client_secret
|
||||||
|
- oauth2_proxy_cookie_secret
|
||||||
|
networks:
|
||||||
|
- proxy
|
||||||
|
restart: unless-stopped
|
||||||
|
mem_limit: 128m
|
||||||
|
labels:
|
||||||
|
traefik.enable: "true"
|
||||||
|
traefik.docker.network: proxy
|
||||||
|
traefik.http.routers.oauth2.rule: Host(`${TRAEFIK_HOST:-traefik.petrolyst.com}`) && PathPrefix(`/oauth2/`)
|
||||||
|
traefik.http.routers.oauth2.entrypoints: websecure
|
||||||
|
traefik.http.routers.oauth2.tls.certresolver: le
|
||||||
|
traefik.http.routers.oauth2.middlewares: secure-headers@file
|
||||||
|
traefik.http.services.oauth2.loadbalancer.server.port: "4180"
|
||||||
|
|
||||||
|
networks:
|
||||||
|
proxy:
|
||||||
|
external: true
|
||||||
|
socket:
|
||||||
|
internal: true
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
letsencrypt:
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
keycloak_petrolyst_client_secret:
|
||||||
|
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/shared/keycloak_petrolyst_client_secret
|
||||||
|
oauth2_proxy_cookie_secret:
|
||||||
|
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/traefik/oauth2_proxy_cookie_secret
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
http:
|
||||||
|
middlewares:
|
||||||
|
# Response headers for every public router (reference as secure-headers@file).
|
||||||
|
secure-headers:
|
||||||
|
headers:
|
||||||
|
stsSeconds: 31536000
|
||||||
|
contentTypeNosniff: true
|
||||||
|
referrerPolicy: strict-origin-when-cross-origin
|
||||||
|
customFrameOptionsValue: SAMEORIGIN
|
||||||
|
permissionsPolicy: "camera=(), microphone=(), geolocation=()"
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
# Applies to every router: TLS 1.2 or newer (Exhibit B), and no certificate for
|
||||||
|
# requests that do not name one of our hosts.
|
||||||
|
tls:
|
||||||
|
options:
|
||||||
|
default:
|
||||||
|
minVersion: VersionTLS12
|
||||||
|
sniStrict: true
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
# Traefik static configuration: the server-wide edge for every stack under
|
||||||
|
# stacks/. Changing anything in static/ changes TRAEFIK_CONFIG_HASH, so deploy.sh
|
||||||
|
# recreates the container; ../dynamic/ is watched and applies without a restart.
|
||||||
|
global:
|
||||||
|
checkNewVersion: false
|
||||||
|
sendAnonymousUsage: false
|
||||||
|
|
||||||
|
entryPoints:
|
||||||
|
web:
|
||||||
|
address: ":80"
|
||||||
|
http:
|
||||||
|
redirections:
|
||||||
|
entryPoint:
|
||||||
|
to: websecure
|
||||||
|
scheme: https
|
||||||
|
permanent: true
|
||||||
|
websecure:
|
||||||
|
address: ":443"
|
||||||
|
# Internal only (not published): serves /ping for the healthcheck.
|
||||||
|
traefik:
|
||||||
|
address: ":8080"
|
||||||
|
|
||||||
|
certificatesResolvers:
|
||||||
|
le:
|
||||||
|
acme:
|
||||||
|
# No e-mail: Let's Encrypt no longer sends expiry mail, and renewal is
|
||||||
|
# automatic. deploy.sh checks the certificates on every deploy.
|
||||||
|
storage: /letsencrypt/acme.json
|
||||||
|
httpChallenge:
|
||||||
|
entryPoint: web
|
||||||
|
|
||||||
|
# The dashboard is served at traefik.petrolyst.com through api@internal, behind
|
||||||
|
# Keycloak (labels on the traefik service in compose.yaml). No insecure port.
|
||||||
|
api:
|
||||||
|
dashboard: true
|
||||||
|
|
||||||
|
ping:
|
||||||
|
entryPoint: traefik
|
||||||
|
|
||||||
|
providers:
|
||||||
|
docker:
|
||||||
|
# Through docker-socket-proxy (read-only container listing and events);
|
||||||
|
# Traefik never mounts the Docker socket itself.
|
||||||
|
endpoint: tcp://socket-proxy:2375
|
||||||
|
exposedByDefault: false
|
||||||
|
network: proxy
|
||||||
|
file:
|
||||||
|
directory: /etc/traefik/dynamic
|
||||||
|
watch: true
|
||||||
|
|
||||||
|
log:
|
||||||
|
level: INFO
|
||||||
|
format: json
|
||||||
|
|
||||||
|
accessLog:
|
||||||
|
format: json
|
||||||
Reference in new issue
Block a user