Petrolyst server infrastructure: Traefik, Trino and Superset
Traefik as the server-wide edge (TLS via Let's Encrypt, dashboard on traefik.petrolyst.com behind oauth2-proxy and Keycloak). The fuel-monitoring stack: Superset on atoma.petrolyst.com with Keycloak login and Petrolyst branding, Trino reading the TimescaleDB fuel-db as catalog "fuel". Customization lives in petrolyst-insights/, deployment in the other folders. Deploys run server-side (deploy/deploy.sh) from a Gitea workflow or manually; releases are immutable, Superset metadata is backed up before migrations, and deploy/rollback.sh restores it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
24327f9462
commit
d3800ac9dd
34 files changed
+2117
No files matched your search
@@ -0,0 +1,57 @@
|
||||
# Every push to main deploys everything: Traefik first, then each stack under
|
||||
# stacks/. The runner only uploads; the server validates, deploys and
|
||||
# smoke-tests (deploy/deploy.sh), so this job needs no Docker of its own.
|
||||
name: deploy
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
# Serializes CI runs; deploy.sh's host lock also covers manual rollbacks.
|
||||
concurrency:
|
||||
group: deploy-petrolyst
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
env:
|
||||
DEPLOY_HOST: ${{ secrets.DEPLOY_HOST }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Lint shell scripts
|
||||
run: |
|
||||
if ! command -v shellcheck >/dev/null; then
|
||||
SUDO=$(command -v sudo || true)
|
||||
$SUDO apt-get update -qq && $SUDO apt-get install -y -qq shellcheck
|
||||
fi
|
||||
shellcheck -x host/*.sh deploy/*.sh stacks/*/superset/init.sh stacks/*/fuel-db/init/*.sh
|
||||
|
||||
- name: Install the deploy key
|
||||
env:
|
||||
DEPLOY_SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
|
||||
DEPLOY_KNOWN_HOSTS: ${{ secrets.DEPLOY_KNOWN_HOSTS }}
|
||||
run: |
|
||||
install -d -m 0700 "$HOME/.ssh"
|
||||
printf '%s\n' "$DEPLOY_SSH_KEY" > "$HOME/.ssh/deploy_key"
|
||||
chmod 0600 "$HOME/.ssh/deploy_key"
|
||||
printf '%s\n' "$DEPLOY_KNOWN_HOSTS" > "$HOME/.ssh/known_hosts"
|
||||
command -v rsync >/dev/null || { SUDO=$(command -v sudo || true); $SUDO apt-get install -y -qq rsync; }
|
||||
|
||||
- name: Upload and deploy
|
||||
run: |
|
||||
ssh_cmd="ssh -i $HOME/.ssh/deploy_key -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -o UserKnownHostsFile=$HOME/.ssh/known_hosts"
|
||||
# A fresh directory per run; only deploy.sh, under the host lock,
|
||||
# turns it into releases/<sha>. --omit-dir-times: the upload's age
|
||||
# must be the server's, because retention prunes uploads by mtime.
|
||||
incoming="/opt/petrolyst/incoming/${GITHUB_SHA}-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT:-1}-$(date -u +%s)"
|
||||
rsync -az --omit-dir-times --exclude .git -e "$ssh_cmd" ./ "deploy@${DEPLOY_HOST}:${incoming}/"
|
||||
$ssh_cmd "deploy@${DEPLOY_HOST}" "${incoming}/deploy/deploy.sh" "$GITHUB_SHA" "$incoming"
|
||||
|
||||
- name: Check from outside
|
||||
run: |
|
||||
test "$(curl -fsS --max-time 20 https://atoma.petrolyst.com/health)" = OK
|
||||
curl -fsS --max-time 20 -o /dev/null -w '%{http_code}\n' https://traefik.petrolyst.com/ | grep -qx 302
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
# Secrets and server env files live only on the server (/opt/petrolyst/).
|
||||
*.env
|
||||
.env*
|
||||
secrets/
|
||||
# CI deploy key, generated here during first-time setup (README, Gitea step 3)
|
||||
petrolyst-ci
|
||||
petrolyst-ci.pub
|
||||
# Local test runs
|
||||
.scratch/
|
||||
.DS_Store
|
||||
__pycache__/
|
||||
@@ -0,0 +1,305 @@
|
||||
# petrolyst-infra
|
||||
|
||||
Everything that runs on the petrolyst server (109.199.127.84, Ubuntu 24.04), deployed on every
|
||||
push to `main`:
|
||||
|
||||
- **Traefik** at the edge for every project on the server. Its dashboard is at
|
||||
https://traefik.petrolyst.com, behind Keycloak.
|
||||
- **fuel-monitoring** (ATOMA): Superset at https://atoma.petrolyst.com, reading the operational
|
||||
store `fuel-db` (Postgres 17 with TimescaleDB) through Trino. Sign-in goes through Keycloak
|
||||
client `petrolyst`, and the UI carries the Petrolyst brand.
|
||||
|
||||
Customization and deployment are separate, as with media-insights and media-intelligence-docker:
|
||||
|
||||
| Folder | Holds |
|
||||
|---|---|
|
||||
| `petrolyst-insights/` | how Superset looks and behaves for Petrolyst: config, Keycloak login code, branding, patches, dashboards. See its README. |
|
||||
| `traefik/` | edge proxy, TLS, the `sso` middleware (oauth2-proxy) that guards the dashboard |
|
||||
| `stacks/fuel-monitoring/` | Compose stack, Superset Dockerfile, Trino config and catalog, fuel-db roles |
|
||||
| `deploy/` | `deploy.sh` (run by CI), `rollback.sh` (manual), `lib.sh` (shared) |
|
||||
| `host/` | one-time server setup: `bootstrap.sh`, `secrets-init.sh` |
|
||||
| `keycloak/` | the `petrolyst` client for Keycloak's partial import |
|
||||
| `.gitea/workflows/deploy.yml` | the pipeline |
|
||||
|
||||
No credential is in this repository. Secrets exist only on the server, under
|
||||
`/opt/petrolyst/secrets/`; CI holds only the SSH deploy key.
|
||||
|
||||
## How a deploy works
|
||||
|
||||
1. A push to `main` (or a manual run) starts `.gitea/workflows/deploy.yml` on the act_runner. It
|
||||
lints the scripts, rsyncs the checkout to a fresh `/opt/petrolyst/incoming/<sha>-<run>/`, and
|
||||
calls `deploy/deploy.sh <sha> <that dir>` over SSH as user `deploy`.
|
||||
2. `deploy.sh` takes the host lock `/opt/petrolyst/deploy.lock`, which it shares with
|
||||
`rollback.sh`, and keeps it until it exits.
|
||||
3. It finalizes the upload. A new commit is moved to `releases/<sha>`, and the release is made
|
||||
read-only. A re-run of a deployed commit must match it byte for byte; a mismatch aborts.
|
||||
4. It resolves the artifacts and writes them to `releases/<sha>/ARTIFACTS`:
|
||||
- `SUPERSET_IMAGE_TAG` is a hash of exactly the Superset image's build inputs. They include
|
||||
`superset/init.sh`, so changing that file runs `superset-init` again.
|
||||
- `TRINO_CONFIG_HASH` and `TRAEFIK_CONFIG_HASH` are hashes of the bind-mounted config.
|
||||
- Compose recreates a container only when its value changes.
|
||||
5. Preflight, before anything live changes:
|
||||
- every secret and env file exists;
|
||||
- `petrolyst-insights/` holds no deployment files;
|
||||
- `docker compose config` passes for every stack;
|
||||
- the Superset image is built if it is new;
|
||||
- the **downgrade guard** refuses an image older than the metadata database's schema.
|
||||
6. It promotes the release to `/opt/petrolyst/infra`, then starts Traefik, then each stack under
|
||||
`stacks/`.
|
||||
7. For Superset, it reads the metadata database's state:
|
||||
- An empty database is a fresh install: it runs `superset-init`.
|
||||
- If the image changed, or the database is not at the image's migration head: it stops
|
||||
`superset`, writes `backups/superset-meta_<old tag>_<revision>_<time>.dump`, then runs
|
||||
`superset-init` (`db upgrade`, `init`, registers "Fuel (Trino)").
|
||||
- If the image is unchanged and the schema is at its head, it does none of this.
|
||||
8. Smoke tests. A failure turns CI red and prints the stacks' state.
|
||||
- `/health` answers through Traefik with a valid certificate.
|
||||
- `/login/` leads to Keycloak with `client_id=petrolyst` and PKCE.
|
||||
- The dashboard redirects to Keycloak.
|
||||
- Trino reads `fuel`, and Superset reaches Trino.
|
||||
9. Retention:
|
||||
- 5 releases;
|
||||
- backups for 30 days;
|
||||
- the 3 newest Superset images, plus any image that a kept release or backup names;
|
||||
- uploads older than a day.
|
||||
|
||||
The runner then checks both hosts from outside.
|
||||
|
||||
## Server paths
|
||||
|
||||
| Path | Owner | What |
|
||||
|---|---|---|
|
||||
| `/opt/petrolyst/infra` | deploy | live tree (the promoted release) |
|
||||
| `/opt/petrolyst/incoming/<sha>-<run>` | deploy | one CI upload; only `deploy.sh` turns it into a release |
|
||||
| `/opt/petrolyst/releases/<sha>` | deploy, read-only | finalized release plus `ARTIFACTS`; the last 5 are kept |
|
||||
| `/opt/petrolyst/secrets/{shared,traefik,fuel-monitoring}/` | root:deploy 0750, files 0444 | secrets, mounted as `/run/secrets/<name>` |
|
||||
| `/opt/petrolyst/env/fuel-monitoring.env` | root:deploy 0440 | `FUEL_DB_TRINO_PASSWORD` for Trino's `${ENV:…}` |
|
||||
| `/opt/petrolyst/backups/` | deploy 0750 | Superset metadata dumps |
|
||||
| `/opt/petrolyst/state/` | deploy | last deployed SHA and Superset image tag; `rollback-in-progress` during a migration rollback |
|
||||
|
||||
## First-time setup
|
||||
|
||||
### 1. Gitea
|
||||
|
||||
1. Create the private repository `Petrolyst/petrolyst-infra` and push this repository's
|
||||
`main` to it. Enable Actions under Settings → Repository → Units.
|
||||
2. Runner. Check Site/Org Administration → Actions → Runners. If none is registered, run
|
||||
[Gitea Runner](https://docs.gitea.com/runner/installation/docker/) as the container
|
||||
`docker.io/gitea/runner:5.2.2-dind` (privileged, with `/data` and `/var/lib/docker` volumes),
|
||||
for example on the Gitea host (164.68.121.117):
|
||||
- register it as an **organization runner** for `Petrolyst`: take the token from
|
||||
Settings → Actions → Runners → Create new Runner;
|
||||
- set `GITEA_RUNNER_LABELS=ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest`;
|
||||
- note its egress IPv4 (`curl -4 -s https://ifconfig.me`).
|
||||
3. Generate a deploy key on your Mac, used only for this:
|
||||
`ssh-keygen -t ed25519 -N '' -C petrolyst-ci -f petrolyst-ci`
|
||||
4. Add repository secrets (Settings → Actions → Secrets):
|
||||
- `DEPLOY_SSH_KEY`: contents of `petrolyst-ci` (the private key)
|
||||
- `DEPLOY_HOST`: `109.199.127.84`
|
||||
- `DEPLOY_KNOWN_HOSTS`: output of `ssh-keyscan -t ed25519 109.199.127.84`. Compare its
|
||||
fingerprint (`ssh-keygen -lf <file>`) with `ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub`
|
||||
run on the server.
|
||||
|
||||
### 2. Keycloak (realm `thedigitalbridge` on iam.thedigitalbridge.io)
|
||||
|
||||
1. Realm settings → Action → **Partial import**: choose `keycloak/petrolyst-partial-import.json`,
|
||||
and "Skip" if a resource exists. This creates:
|
||||
- the confidential client `petrolyst`: standard flow only, PKCE S256, redirect URIs for both
|
||||
hosts;
|
||||
- its client roles `superset-admin`, `superset-editor`, `superset-viewer` and `traefik-admin`;
|
||||
- a mapper that puts those roles into the claim `petrolyst_roles`.
|
||||
2. Clients → petrolyst → Credentials: copy the client secret (step 3 stores it on the server).
|
||||
3. **MFA**, which Exhibit B requires. A copy of the built-in `browser` flow is not enough, for two
|
||||
reasons: its Cookie step accepts an existing password-only session from another client of the
|
||||
realm, and its conditional OTP subflow skips users who have not set up OTP. Instead:
|
||||
- Authentication → Create flow `petrolyst-mfa` (basic flow).
|
||||
- Add exactly two executions, both **Required**, in this order: `Username Password Form`,
|
||||
then `OTP Form`. Add no Cookie, no Identity Provider Redirector and no conditional subflow.
|
||||
- Clients → petrolyst → Advanced → Authentication flow overrides → Browser flow:
|
||||
`petrolyst-mfa`.
|
||||
|
||||
Result: every sign-in to a Petrolyst app asks for password and OTP, and a user without OTP
|
||||
enrols first. Other clients of the realm are unaffected. There is no silent SSO between
|
||||
Superset and the dashboard.
|
||||
4. Give people roles: Users → (user) → Role mapping → Assign role → filter by clients →
|
||||
`petrolyst`. Group mappings work too.
|
||||
5. Users of the Traefik dashboard need **Email verified** switched on, because oauth2-proxy
|
||||
refuses unverified addresses.
|
||||
|
||||
### 3. Server (as root, `ssh petrolyst`)
|
||||
|
||||
The server needs only the two host scripts; it never gets Gitea credentials.
|
||||
|
||||
```bash
|
||||
# on the Mac, in this repository
|
||||
scp host/bootstrap.sh host/secrets-init.sh petrolyst-ci.pub petrolyst:/root/
|
||||
ssh petrolyst
|
||||
# on the server, as root
|
||||
/root/bootstrap.sh --ci-key /root/petrolyst-ci.pub --runner-ip <runner egress IP>
|
||||
/root/secrets-init.sh # generates every secret; reports the missing Keycloak one
|
||||
/root/secrets-init.sh keycloak-secret # paste the secret from Keycloak step 2
|
||||
```
|
||||
|
||||
No runner yet? Pass `--runner-ip 127.0.0.1`. That installs the CI key but leaves it unusable from
|
||||
outside. Deploy manually until a runner exists (see "Manual deploy" below). When the runner is
|
||||
running, run `bootstrap.sh` again with its egress IP.
|
||||
|
||||
`bootstrap.sh` does the following:
|
||||
- installs Docker from Docker's apt repository;
|
||||
- adds 4 GB swap;
|
||||
- turns on ufw for 22, 80 and 443 only;
|
||||
- turns off SSH password login and root password login;
|
||||
- creates the `deploy` user, whose key works only from the runner IP;
|
||||
- creates `/opt/petrolyst` and the Docker network `proxy`.
|
||||
|
||||
**Before closing your root session**, check from a second terminal that `ssh petrolyst` still
|
||||
works.
|
||||
|
||||
### 4. First deploy
|
||||
|
||||
Run the workflow manually (Actions → deploy → Run workflow) or push to `main`. Without a runner,
|
||||
use "Manual deploy" below. The first run builds the Superset image and takes the fresh-install
|
||||
path. Let's Encrypt needs about a minute
|
||||
for the two certificates; the smoke tests wait up to two minutes.
|
||||
|
||||
Then sign in at https://atoma.petrolyst.com with a user who has `superset-admin`. That person
|
||||
becomes the first Superset admin; there is no local admin account.
|
||||
|
||||
## Operating
|
||||
|
||||
| Task | How |
|
||||
|---|---|
|
||||
| Deploy | push to `main` (without a runner: "Manual deploy" below) |
|
||||
| Re-deploy a kept release | `sudo -u deploy /opt/petrolyst/infra/deploy/deploy.sh <sha>` |
|
||||
| Roll back | see below |
|
||||
| Traefik dashboard | https://traefik.petrolyst.com (role `traefik-admin`) |
|
||||
| Trino shell | `docker exec -it fuel-monitoring-trino-1 trino` |
|
||||
| fuel-db from the Mac | `ssh -L 5433:127.0.0.1:5433 petrolyst`, then connect to `localhost:5433`, database `fuel` |
|
||||
| Logs | `docker compose -p fuel-monitoring logs -f superset` (or `-p traefik`) |
|
||||
| Superset break-glass | `docker exec -it fuel-monitoring-superset-1 superset fab ...` |
|
||||
|
||||
**Manual deploy** (no runner): commit first, so that `git status` is clean. Then run this on the
|
||||
Mac, in this repository:
|
||||
|
||||
```bash
|
||||
sha=$(git rev-parse HEAD)
|
||||
dir=/opt/petrolyst/incoming/$sha-manual-$(date -u +%s)
|
||||
git archive --format=tar "$sha" |
|
||||
ssh petrolyst "install -d -o deploy -g deploy -m 0755 $dir && tar -x --no-same-owner -C $dir && chown -R deploy:deploy $dir"
|
||||
ssh petrolyst "sudo -u deploy $dir/deploy/deploy.sh $sha $dir"
|
||||
```
|
||||
|
||||
`git archive` sends exactly the commit, never uncommitted or ignored files such as the deploy
|
||||
key. The server then runs the same `deploy.sh` that CI runs, under the same lock. A later push
|
||||
queues a CI run that waits for a runner. Cancel stale waiting runs under Actions before you
|
||||
register one.
|
||||
|
||||
**Rollback** (`deploy/rollback.sh`, manual, under the same lock as deploys):
|
||||
|
||||
```bash
|
||||
ls /opt/petrolyst/releases # kept releases
|
||||
sudo -u deploy /opt/petrolyst/infra/deploy/rollback.sh <sha>
|
||||
```
|
||||
|
||||
- If the target image's migration head equals the database's revision, this is a **code-only**
|
||||
rollback: promote and restart.
|
||||
- Otherwise it is a **migration rollback**:
|
||||
1. stop `superset`;
|
||||
2. drop and recreate the `superset` metadata database;
|
||||
3. restore the backup made for the target image, aborting on any error;
|
||||
4. check the restored revision;
|
||||
5. start the retained older image.
|
||||
|
||||
It refuses if that backup or image is gone. `fuel-db` is never touched.
|
||||
|
||||
Before step 1 it writes `state/rollback-in-progress`, and it removes that file only after the
|
||||
smoke tests pass. While the file exists, `deploy.sh` refuses to run. If a rollback is
|
||||
interrupted, run the same command again. It also restores when the failed attempt left the
|
||||
database empty or had already dropped it.
|
||||
|
||||
Afterwards, `git revert` the bad commit and push, so `main` matches. A plain revert without the
|
||||
rollback script is stopped by the downgrade guard.
|
||||
|
||||
**Data loaders** (fuel-db). Every table in `public` must belong to `fuel_owner`. `trino_ro` can
|
||||
read new tables only through `fuel_owner`'s default privileges, and PostgreSQL applies those
|
||||
only to objects `fuel_owner` creates itself. A table that a loader creates as its own role stays
|
||||
unreadable for Trino. Create each loader role so that its sessions act as `fuel_owner`, and it
|
||||
cannot create anything as itself. Run this in
|
||||
`docker exec -it fuel-monitoring-fuel-db-1 psql -U postgres -d fuel`:
|
||||
|
||||
```sql
|
||||
CREATE ROLE <loader> LOGIN PASSWORD '<password>';
|
||||
GRANT fuel_owner TO <loader> WITH INHERIT FALSE, SET TRUE;
|
||||
ALTER ROLE <loader> SET role TO 'fuel_owner';
|
||||
```
|
||||
|
||||
Tables it creates, hypertables included, then belong to `fuel_owner`, and Trino can read them.
|
||||
After `SET ROLE NONE`, it cannot create anything in `public`. To repair a table that another
|
||||
role created, run `ALTER TABLE <t> OWNER TO fuel_owner`, then `GRANT SELECT ON <t> TO trino_ro`.
|
||||
|
||||
**Adding a project**: create `stacks/<name>/compose.yaml`, attach the public service to the
|
||||
external network `proxy`, and set `traefik.*` labels with a unique router name and
|
||||
`tls.certresolver: le`. Add the DNS record first. `deploy.sh` picks the new folder up
|
||||
automatically.
|
||||
|
||||
`sso@docker` works only for `traefik.petrolyst.com`. Its cookie, its allowed return hosts and its
|
||||
role (`traefik-admin`) are all tied to the dashboard. An app without a login of its own needs
|
||||
its own gate in its stack:
|
||||
- an oauth2-proxy service with its own cookie secret;
|
||||
- a router for the app host's `/oauth2/` path and a forwardAuth middleware pointing at that
|
||||
service, as in `traefik/compose.yaml`;
|
||||
- `--redirect-url`, `--cookie-domain` and `--whitelist-domain` set to the app's host;
|
||||
- an `--allowed-role` with a new client role in Keycloak;
|
||||
- the app's `/oauth2/callback` added to the `petrolyst` client's redirect URIs.
|
||||
|
||||
## Verified locally (2026-10-23)
|
||||
|
||||
An end-to-end run ran the real `deploy.sh` and `rollback.sh` in a Linux container against
|
||||
Docker Desktop, with a scratch `/opt/petrolyst`. Production differences in that run:
|
||||
- a test CA certificate stood in for Let's Encrypt;
|
||||
- the ACME server in the test copy was unreachable on purpose;
|
||||
- a dummy Keycloak client secret was used.
|
||||
|
||||
Results:
|
||||
- The fresh install took the init path.
|
||||
- An identical re-run recreated nothing and left `ARTIFACTS` byte-identical.
|
||||
- A Trino-only change recreated only `trino`.
|
||||
- A Superset change stopped, backed up, migrated and recreated only `superset`.
|
||||
- A change to `superset/init.sh` alone did the same, and the changed script ran.
|
||||
- A same-SHA upload with different content was refused.
|
||||
- An older image against a newer schema was refused before promotion.
|
||||
- A migration rollback dropped a table added by a "newer migration" and restored the revision.
|
||||
- A code-only rollback brought back the previous image.
|
||||
- A re-deploy of an unchanged image, against a schema one migration behind its head, migrated
|
||||
the schema back to the head.
|
||||
- A migration rollback whose restore failed after the drop left `rollback-in-progress` and an
|
||||
empty database. `deploy.sh` then refused to run. Running the rollback again restored the
|
||||
database and removed the file. It did the same after a simulated stop between dropping and
|
||||
recreating the database. Without the file, a missing database stops the rollback with an error.
|
||||
- The host lock blocked a concurrent run.
|
||||
- Retention pruned read-only releases.
|
||||
- Retention removed a queued upload that a plain `rsync -a` had given an old checkout's
|
||||
directory times. It kept one uploaded with `--omit-dir-times`.
|
||||
- In `fuel-db`, a loader that created a table as its own role left it unreadable for `trino_ro`.
|
||||
With the "Data loaders" recipe, tables (hypertables too) belonged to `fuel_owner` and Trino
|
||||
could read them. After `SET ROLE NONE`, the loader could not create anything.
|
||||
- Smoke tests passed, including the redirects to the real Keycloak with `client_id=petrolyst`
|
||||
and PKCE S256.
|
||||
- TLS 1.1 and unknown SNI were refused, and HTTP redirected to HTTPS.
|
||||
- Trino and `trino_ro` both refused writes.
|
||||
|
||||
Not verified locally:
|
||||
- Let's Encrypt issuance;
|
||||
- a real Keycloak sign-in, OTP and logout (they need the client from setup step 2);
|
||||
- the theme as rendered in a browser;
|
||||
- the Gitea runner.
|
||||
|
||||
## Open items
|
||||
|
||||
- Exhibit B: encryption at rest; confirm the VPS is in a German data centre.
|
||||
- Nightly `pg_dump` of both databases with an offsite copy. Today only the pre-migration
|
||||
Superset dumps exist.
|
||||
- Certificate expiry monitoring. There is no ACME e-mail, so nothing warns if renewal fails
|
||||
between deploys.
|
||||
- Celery worker and beat, if Superset alerts or reports are needed.
|
||||
- A Petrolyst login theme for Keycloak itself, which lives on the Keycloak host.
|
||||
Executable
+163
@@ -0,0 +1,163 @@
|
||||
#!/usr/bin/env bash
|
||||
# Server-side deploy, run as the non-root deploy user:
|
||||
# deploy.sh <sha> <incoming-dir> CI: finalize the upload as releases/<sha>, then deploy it
|
||||
# deploy.sh <sha> re-deploy a release that is already finalized
|
||||
#
|
||||
# Order: lock, finalize, artifacts, preflight, image build and downgrade guard
|
||||
# (nothing live touched yet), then promote, Traefik, each stack, smoke tests,
|
||||
# retention. Rollback is a separate, manual script (rollback.sh) under the same
|
||||
# lock.
|
||||
set -euo pipefail
|
||||
# shellcheck source=deploy/lib.sh
|
||||
. "$(dirname "$(readlink -f "$0")")/lib.sh"
|
||||
|
||||
# finalize_release <sha> <upload>: CI never writes into releases/. A new SHA is
|
||||
# moved in atomically; a re-run of a deployed SHA must match it byte for byte
|
||||
# and leaves the finalized release, ARTIFACTS included, untouched.
|
||||
finalize_release() {
|
||||
local release=$RELEASES/$1 upload
|
||||
upload=$(readlink -f "$2") || die "upload $2 does not exist"
|
||||
[[ $upload == "$(readlink -f "$INCOMING")"/* && -d $upload ]] || die "upload $2 is not a directory under $INCOMING"
|
||||
if [ ! -e "$release" ]; then
|
||||
mv "$upload" "$release"
|
||||
log "finalized $release"
|
||||
elif diff -r --exclude=ARTIFACTS "$upload" "$release" >/dev/null; then
|
||||
rm -rf "$upload"
|
||||
log "release $1 already finalized and identical to the upload; reusing it"
|
||||
else
|
||||
die "upload differs from the finalized release $release; one SHA cannot have two contents"
|
||||
fi
|
||||
}
|
||||
|
||||
# resolve_artifacts <release>: compute the artifact identities from the
|
||||
# release's own files, record them once in ARTIFACTS, then freeze the release.
|
||||
resolve_artifacts() {
|
||||
local release=$1 computed
|
||||
computed=$(compute_artifacts "$release")
|
||||
if [ -e "$release/ARTIFACTS" ]; then
|
||||
[ "$(cat "$release/ARTIFACTS")" = "$computed" ] || die "$release/ARTIFACTS does not match the release's files"
|
||||
else
|
||||
# A crash between the move and this step leaves no ARTIFACTS; the files
|
||||
# are unchanged, so the computed values are the same.
|
||||
chmod u+w "$release"
|
||||
printf '%s\n' "$computed" >"$release/ARTIFACTS"
|
||||
fi
|
||||
chmod -R a-w "$release"
|
||||
load_artifacts "$release"
|
||||
log "artifacts: superset $SUPERSET_IMAGE_TAG, trino config $TRINO_CONFIG_HASH, traefik config $TRAEFIK_CONFIG_HASH"
|
||||
}
|
||||
|
||||
# prepare_fuel_monitoring <release>: before anything live changes, build the
|
||||
# Superset image from the release and refuse an image older than the schema of
|
||||
# a running metadata database.
|
||||
prepare_fuel_monitoring() {
|
||||
local image=fuel-superset:$SUPERSET_IMAGE_TAG revision
|
||||
if ! "${DOCKER[@]}" image inspect "$image" >/dev/null 2>&1; then
|
||||
log "building $image"
|
||||
compose "$1" stacks/fuel-monitoring build superset
|
||||
fi
|
||||
[ -e "$LIVE/stacks/fuel-monitoring/compose.yaml" ] || return 0
|
||||
[ -n "$(fm ps --status running -q superset-db 2>/dev/null)" ] || return 0
|
||||
revision=$(metadata_revision)
|
||||
[ "$revision" = fresh ] || downgrade_guard "$image" "$revision"
|
||||
}
|
||||
|
||||
downgrade_guard() {
|
||||
alembic "$1" has "$2" ||
|
||||
die "the metadata database is at revision $2, which $1 does not know: the image is older than the schema. Use deploy/rollback.sh."
|
||||
}
|
||||
|
||||
deploy_fuel_monitoring() {
|
||||
local image=fuel-superset:$SUPERSET_IMAGE_TAG revision running head
|
||||
fm up -d --wait superset-db redis
|
||||
revision=$(metadata_revision)
|
||||
if [ "$revision" = fresh ]; then
|
||||
log "metadata database is empty: fresh install, running superset-init"
|
||||
fm run --rm superset-init
|
||||
else
|
||||
# Again here: the database may not have been running during preparation.
|
||||
downgrade_guard "$image" "$revision"
|
||||
running=$(running_superset_tag)
|
||||
# An unchanged tag alone is not enough: the schema may have been changed
|
||||
# under the container (a restore, a manual downgrade).
|
||||
head=$(alembic "$image" heads)
|
||||
if [ "$running" != "$SUPERSET_IMAGE_TAG" ] || [ "$revision" != "$head" ]; then
|
||||
log "Superset image ${running:-unknown} -> $SUPERSET_IMAGE_TAG, schema $revision -> $head: stop, back up, migrate"
|
||||
fm stop superset
|
||||
backup_metadata "${running:-unknown}" "$revision"
|
||||
fm run --rm superset-init
|
||||
fi
|
||||
fi
|
||||
fm up -d --wait --remove-orphans
|
||||
}
|
||||
|
||||
# Old releases, stale uploads, old backups and unreferenced Superset images.
|
||||
retention() {
|
||||
local current=$1 dir tag referenced
|
||||
while IFS= read -r dir; do
|
||||
[ "${dir##*/}" != "$current" ] || continue
|
||||
chmod -R u+w "$dir"
|
||||
rm -rf "$dir"
|
||||
log "removed release ${dir##*/}"
|
||||
done < <(find "$RELEASES" -mindepth 1 -maxdepth 1 -type d -printf '%T@ %p\n' | sort -rn | tail -n +6 | cut -d' ' -f2-)
|
||||
# Uploads older than a day were never finalized (interrupted CI runs). Their
|
||||
# mtime is the server's (the workflow's rsync omits directory times), and a
|
||||
# CI job lives at most 45 minutes, so an upload being sent or waiting for the
|
||||
# lock is never this old.
|
||||
while IFS= read -r -d '' dir; do
|
||||
chmod -R u+w "$dir"
|
||||
rm -rf "$dir"
|
||||
log "removed stale upload ${dir##*/}"
|
||||
done < <(find "$INCOMING" -mindepth 1 -maxdepth 1 -mmin +1440 -print0)
|
||||
find "$BACKUPS" -maxdepth 1 -name 'superset-meta_*.dump' -mtime +30 -delete
|
||||
find "$BACKUPS" -maxdepth 1 -name '.partial-*' -mmin +60 -delete
|
||||
# Keep the 3 newest images plus every image a kept release or backup names.
|
||||
referenced=$(
|
||||
sed -n 's/^SUPERSET_IMAGE_TAG=//p' "$RELEASES"/*/ARTIFACTS 2>/dev/null || true
|
||||
find "$BACKUPS" -maxdepth 1 -name 'superset-meta_*.dump' -exec basename {} \; |
|
||||
sed -n 's/^superset-meta_\([0-9a-f]\{16\}\)_.*/\1/p'
|
||||
)
|
||||
"${DOCKER[@]}" image ls fuel-superset --format '{{.Tag}}' | tail -n +4 | while IFS= read -r tag; do
|
||||
grep -qxF "$tag" <<<"$referenced" && continue
|
||||
if "${DOCKER[@]}" image rm "fuel-superset:$tag" >/dev/null; then
|
||||
log "removed image fuel-superset:$tag"
|
||||
else
|
||||
log "kept image fuel-superset:$tag (still in use)"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
main() {
|
||||
local sha=${1:-} upload=${2:-} release dir
|
||||
[[ $sha =~ ^[0-9a-f]{40}$ ]] || die "usage: deploy.sh <40-hex sha> [incoming-dir]"
|
||||
acquire_lock 900
|
||||
[ ! -e "$ROLLBACK_MARK" ] ||
|
||||
die "a rollback to $(cat "$ROLLBACK_MARK") did not finish and may have left the metadata database half-restored; run deploy/rollback.sh again first"
|
||||
release=$RELEASES/$sha
|
||||
[ -z "$upload" ] || finalize_release "$sha" "$upload"
|
||||
[ -d "$release" ] || die "no release $release"
|
||||
resolve_artifacts "$release"
|
||||
preflight "$release"
|
||||
prepare_fuel_monitoring "$release"
|
||||
|
||||
# From here on the live state changes; on failure, print the stacks' state.
|
||||
trap on_exit EXIT
|
||||
promote "$release" "$sha"
|
||||
ensure_proxy_network
|
||||
log "traefik"
|
||||
compose "$LIVE" traefik up -d --wait --remove-orphans
|
||||
while IFS= read -r dir; do
|
||||
log "$dir"
|
||||
case $dir in
|
||||
stacks/fuel-monitoring) deploy_fuel_monitoring ;;
|
||||
*) compose "$LIVE" "$dir" up -d --wait --remove-orphans ;;
|
||||
esac
|
||||
done < <(stack_dirs "$LIVE")
|
||||
smoke_tests
|
||||
record_state "$sha"
|
||||
trap - EXIT
|
||||
retention "$sha"
|
||||
log "deployed $sha"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
+271
@@ -0,0 +1,271 @@
|
||||
# shellcheck shell=bash
|
||||
# Shared by deploy.sh and rollback.sh; sourced, not executed.
|
||||
|
||||
# Server layout (README "Server paths"). PETROLYST_ROOT exists only so the
|
||||
# scripts can be exercised against a scratch directory; production never sets it.
|
||||
# shellcheck disable=SC2034 # RELEASES and INCOMING are used by the sourcing scripts
|
||||
ROOT=${PETROLYST_ROOT:-/opt/petrolyst}
|
||||
LIVE=$ROOT/infra
|
||||
RELEASES=$ROOT/releases
|
||||
INCOMING=$ROOT/incoming
|
||||
BACKUPS=$ROOT/backups
|
||||
STATE=$ROOT/state
|
||||
LOCK_FILE=$ROOT/deploy.lock
|
||||
# Exists while rollback.sh replaces the metadata database, and stays behind if
|
||||
# it is interrupted there; deploy.sh refuses to run until a rollback completes.
|
||||
ROLLBACK_MARK=$STATE/rollback-in-progress
|
||||
|
||||
ATOMA_HOST=${ATOMA_HOST:-atoma.petrolyst.com}
|
||||
TRAEFIK_HOST=${TRAEFIK_HOST:-traefik.petrolyst.com}
|
||||
KEYCLOAK_REALM_URL=${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/thedigitalbridge}
|
||||
PETROLYST_ROOT=$ROOT
|
||||
export ATOMA_HOST TRAEFIK_HOST KEYCLOAK_REALM_URL PETROLYST_ROOT
|
||||
|
||||
# --host pins the daemon, as in petrolyst-ui/deploy/deploy.sh: a stale
|
||||
# `docker context use` must not send this deploy to another daemon.
|
||||
DOCKER=(docker --host unix:///var/run/docker.sock)
|
||||
|
||||
# Build inputs of the Superset image, relative to a release. Everything the
|
||||
# Dockerfile copies is listed; nothing else is, so the tag changes exactly when
|
||||
# the image would.
|
||||
SUPERSET_INPUTS=(
|
||||
stacks/fuel-monitoring/superset/Dockerfile
|
||||
stacks/fuel-monitoring/superset/init.sh
|
||||
petrolyst-insights/superset_config.py
|
||||
petrolyst-insights/petrolyst_security.py
|
||||
petrolyst-insights/assets/branding
|
||||
petrolyst-insights/patches
|
||||
)
|
||||
|
||||
# Every file the stacks need from /opt/petrolyst (host/secrets-init.sh).
|
||||
REQUIRED_FILES=(
|
||||
secrets/shared/keycloak_petrolyst_client_secret
|
||||
secrets/traefik/oauth2_proxy_cookie_secret
|
||||
secrets/fuel-monitoring/superset_secret_key
|
||||
secrets/fuel-monitoring/superset_db_password
|
||||
secrets/fuel-monitoring/fuel_db_password
|
||||
secrets/fuel-monitoring/fuel_db_trino_password
|
||||
env/fuel-monitoring.env
|
||||
)
|
||||
|
||||
log() { printf '%s %s\n' "$(date -u +%H:%M:%S)" "$*" >&2; }
|
||||
die() { log "ERROR: $*"; exit 1; }
|
||||
|
||||
[ "$(id -u)" != 0 ] || die "run as the deploy user (sudo -u deploy $0 ...), not as root"
|
||||
|
||||
# acquire_lock <seconds>: deploy.sh and rollback.sh share this lock, so a CI
|
||||
# deploy and a manual rollback never run at the same time. Held until exit.
|
||||
acquire_lock() {
|
||||
exec 9>>"$LOCK_FILE"
|
||||
flock -w "$1" 9 || die "another deploy or rollback holds $LOCK_FILE (waited $1 s)"
|
||||
}
|
||||
|
||||
sha256() {
|
||||
if command -v sha256sum >/dev/null; then sha256sum; else shasum -a 256; fi | cut -c1-64
|
||||
}
|
||||
|
||||
# hash_inputs <dir> <path>...: one short digest over the names and contents of
|
||||
# every file under the paths, independent of mtimes and listing order.
|
||||
hash_inputs() {
|
||||
local dir=$1
|
||||
shift
|
||||
(
|
||||
cd "$dir" || exit 1
|
||||
find "$@" -type f -print0 | LC_ALL=C sort -z |
|
||||
while IFS= read -r -d '' f; do printf '%s %s\n' "$f" "$(sha256 <"$f")"; done
|
||||
) | sha256 | cut -c1-16
|
||||
}
|
||||
|
||||
compute_artifacts() {
|
||||
printf 'SUPERSET_IMAGE_TAG=%s\n' "$(hash_inputs "$1" "${SUPERSET_INPUTS[@]}")"
|
||||
printf 'TRINO_CONFIG_HASH=%s\n' "$(hash_inputs "$1" stacks/fuel-monitoring/trino/etc)"
|
||||
# Only the static config: dynamic/ is watched and applies without a restart.
|
||||
printf 'TRAEFIK_CONFIG_HASH=%s\n' "$(hash_inputs "$1" traefik/static)"
|
||||
}
|
||||
|
||||
# load_artifacts <release>: export the release's recorded artifact identities.
|
||||
load_artifacts() {
|
||||
local file=$1/ARTIFACTS line key value
|
||||
[ -s "$file" ] || die "$file is missing"
|
||||
unset SUPERSET_IMAGE_TAG TRINO_CONFIG_HASH TRAEFIK_CONFIG_HASH
|
||||
while IFS= read -r line; do
|
||||
key=${line%%=*}
|
||||
value=${line#*=}
|
||||
[[ $key =~ ^(SUPERSET_IMAGE_TAG|TRINO_CONFIG_HASH|TRAEFIK_CONFIG_HASH)$ && $value =~ ^[0-9a-f]{16}$ ]] ||
|
||||
die "unexpected line in $file: $line"
|
||||
printf -v "$key" '%s' "$value"
|
||||
export "${key?}"
|
||||
done <"$file"
|
||||
[ -n "${SUPERSET_IMAGE_TAG:-}" ] && [ -n "${TRINO_CONFIG_HASH:-}" ] && [ -n "${TRAEFIK_CONFIG_HASH:-}" ] ||
|
||||
die "$file is incomplete"
|
||||
}
|
||||
|
||||
# compose <tree> <dir> <args...>: run Compose for the stack in <tree>/<dir>.
|
||||
compose() {
|
||||
local tree=$1 dir=$2
|
||||
shift 2
|
||||
"${DOCKER[@]}" compose --project-directory "$tree/$dir" -f "$tree/$dir/compose.yaml" "$@"
|
||||
}
|
||||
|
||||
fm() { compose "$LIVE" stacks/fuel-monitoring "$@"; }
|
||||
|
||||
stack_dirs() {
|
||||
local f
|
||||
for f in "$1"/stacks/*/compose.yaml; do
|
||||
[ -e "$f" ] || continue
|
||||
f=${f#"$1"/}
|
||||
printf '%s\n' "${f%/compose.yaml}"
|
||||
done
|
||||
}
|
||||
|
||||
preflight() {
|
||||
local release=$1 f bad dir
|
||||
for f in "${REQUIRED_FILES[@]}"; do
|
||||
[ -s "$ROOT/$f" ] || die "missing or empty: $ROOT/$f (run host/secrets-init.sh as root)"
|
||||
done
|
||||
# oauth2-proxy uses the client secret file byte for byte.
|
||||
f=$ROOT/secrets/shared/keycloak_petrolyst_client_secret
|
||||
[ "$(tail -c1 "$f" | wc -l)" -eq 0 ] ||
|
||||
die "$f ends with a newline; store it again with: host/secrets-init.sh keycloak-secret"
|
||||
bad=$(find "$release/petrolyst-insights" \( -name 'Dockerfile*' -o -name '*compose*.y*ml' \
|
||||
-o -name '*.env' -o -name '.env' -o -path '*/.gitea/*' -o -path '*/.github/*' \) -print)
|
||||
[ -z "$bad" ] || die "petrolyst-insights/ holds customization only; found: $bad"
|
||||
compose "$release" traefik config -q
|
||||
while IFS= read -r dir; do compose "$release" "$dir" config -q; done < <(stack_dirs "$release")
|
||||
}
|
||||
|
||||
# promote <release> <sha>: make the release the live tree. --chmod applies to
|
||||
# the destination only: the live tree stays writable, the release read-only.
|
||||
promote() {
|
||||
mkdir -p "$LIVE"
|
||||
rsync -a --delete --chmod=Du+w,Fu+w "$1/" "$LIVE/"
|
||||
printf '%s\n' "$2" >"$LIVE/REVISION"
|
||||
}
|
||||
|
||||
ensure_proxy_network() {
|
||||
"${DOCKER[@]}" network inspect proxy >/dev/null 2>&1 || "${DOCKER[@]}" network create proxy >/dev/null
|
||||
}
|
||||
|
||||
# metadata_revision: the metadata database's Alembic revision, "fresh" if it
|
||||
# has no alembic_version table yet; any other error (connection, auth) aborts.
|
||||
metadata_revision() {
|
||||
local out
|
||||
if out=$(fm exec -T superset-db psql -U superset -d superset -tAc 'SELECT version_num FROM alembic_version' 2>&1); then
|
||||
[[ $out =~ ^[0-9a-z_]+$ ]] || die "unexpected alembic_version content: $out"
|
||||
printf '%s\n' "$out"
|
||||
elif [[ $out == *'relation "alembic_version" does not exist'* ]]; then
|
||||
printf 'fresh\n'
|
||||
else
|
||||
die "cannot read the metadata database's revision: $out"
|
||||
fi
|
||||
}
|
||||
|
||||
# alembic <image> heads | has <rev>: read Superset's migration graph from the
|
||||
# image without starting the app (loading the scripts through Alembic would
|
||||
# import Superset's models, which need an initialised app).
|
||||
alembic() {
|
||||
"${DOCKER[@]}" run --rm -i --network none --entrypoint python "$1" - "${@:2}" <<'PY'
|
||||
import ast, pathlib, sys
|
||||
|
||||
revs, downs = set(), set()
|
||||
for path in pathlib.Path("/app/superset/migrations/versions").glob("*.py"):
|
||||
for node in ast.parse(path.read_text()).body:
|
||||
if isinstance(node, ast.Assign) and len(node.targets) == 1:
|
||||
target, value = node.targets[0], node.value
|
||||
elif isinstance(node, ast.AnnAssign) and node.value is not None:
|
||||
target, value = node.target, node.value
|
||||
else:
|
||||
continue
|
||||
if isinstance(target, ast.Name) and target.id in ("revision", "down_revision"):
|
||||
found = ast.literal_eval(value)
|
||||
found = found if isinstance(found, (tuple, list)) else (found,)
|
||||
(revs if target.id == "revision" else downs).update(v for v in found if v)
|
||||
if sys.argv[1] == "heads":
|
||||
print(" ".join(sorted(revs - downs)))
|
||||
else:
|
||||
sys.exit(0 if sys.argv[2] in revs else 3)
|
||||
PY
|
||||
}
|
||||
|
||||
# The image tag of the existing superset container, or of the last successful
|
||||
# deploy if the container is gone; empty if neither is known.
|
||||
running_superset_tag() {
|
||||
local id ref
|
||||
id=$(fm ps -a -q superset 2>/dev/null | head -n1)
|
||||
if [ -n "$id" ]; then
|
||||
ref=$("${DOCKER[@]}" inspect --format '{{.Config.Image}}' "$id")
|
||||
printf '%s\n' "${ref#fuel-superset:}"
|
||||
elif [ -s "$STATE/superset-image-tag" ]; then
|
||||
cat "$STATE/superset-image-tag"
|
||||
fi
|
||||
}
|
||||
|
||||
# backup_metadata <image-tag> <revision>: dump the metadata database under a
|
||||
# name that rollback.sh can match to the image whose schema it holds.
|
||||
backup_metadata() {
|
||||
local name tmp
|
||||
name=superset-meta_${1}_${2}_$(date -u +%Y%m%dT%H%M%SZ).dump
|
||||
tmp=$(mktemp "$BACKUPS/.partial-XXXXXX")
|
||||
fm exec -T superset-db pg_dump -U superset -d superset -Fc >"$tmp"
|
||||
[ -s "$tmp" ] || die "metadata backup came out empty"
|
||||
mv "$tmp" "$BACKUPS/$name"
|
||||
log "metadata backup: $BACKUPS/$name"
|
||||
}
|
||||
|
||||
record_state() {
|
||||
mkdir -p "$STATE"
|
||||
printf '%s\n' "$SUPERSET_IMAGE_TAG" >"$STATE/superset-image-tag"
|
||||
printf '%s\n' "$1" >"$STATE/deployed-sha"
|
||||
}
|
||||
|
||||
# Status code and resolved redirect target of one request through the local
|
||||
# Traefik, with certificate verification against the real hostname.
|
||||
probe() {
|
||||
local host=$1 path=$2
|
||||
curl -q --noproxy '*' -sS -o /dev/null --max-time 20 --resolve "$host:443:127.0.0.1" \
|
||||
-w '%{http_code} %{redirect_url}\n' "https://$host$path"
|
||||
}
|
||||
|
||||
smoke_tests() {
|
||||
local i body code location
|
||||
log "smoke tests"
|
||||
# First deploy: Let's Encrypt may need a minute to issue the certificate.
|
||||
for i in $(seq 1 24); do
|
||||
body=$(curl -q --noproxy '*' -fsS --max-time 10 --resolve "$ATOMA_HOST:443:127.0.0.1" \
|
||||
"https://$ATOMA_HOST/health" 2>/dev/null) && [ "$body" = OK ] && break
|
||||
[ "$i" -lt 24 ] || die "https://$ATOMA_HOST/health did not answer OK with a valid certificate"
|
||||
sleep 5
|
||||
done
|
||||
|
||||
read -r code location < <(probe "$ATOMA_HOST" /login/)
|
||||
[ "$code" = 302 ] && [[ $location == "https://$ATOMA_HOST/login/keycloak"* ]] ||
|
||||
die "/login/ should redirect to /login/keycloak, got $code $location"
|
||||
|
||||
read -r code location < <(probe "$ATOMA_HOST" /login/keycloak)
|
||||
[ "$code" = 302 ] && [[ $location == "$KEYCLOAK_REALM_URL/protocol/openid-connect/auth?"* ]] &&
|
||||
[[ $location == *client_id=petrolyst* && $location == *code_challenge_method=S256* ]] ||
|
||||
die "/login/keycloak should redirect to Keycloak with client_id=petrolyst and PKCE S256, got $code $location"
|
||||
|
||||
read -r code location < <(probe "$TRAEFIK_HOST" /)
|
||||
[ "$code" = 302 ] && [[ $location == "$KEYCLOAK_REALM_URL/"* || $location == *"/oauth2/start"* ]] ||
|
||||
die "https://$TRAEFIK_HOST/ should redirect to Keycloak, got $code $location"
|
||||
|
||||
fm exec -T trino trino --execute 'SELECT count(*) FROM fuel.information_schema.schemata' >/dev/null ||
|
||||
die "Trino cannot read the fuel catalog"
|
||||
fm exec -T superset curl -fsS --max-time 10 http://trino:8080/v1/info >/dev/null ||
|
||||
die "Superset cannot reach Trino"
|
||||
log "smoke tests passed"
|
||||
}
|
||||
|
||||
# EXIT trap for the part of a run that changes live state.
|
||||
on_exit() {
|
||||
local status=$?
|
||||
[ "$status" = 0 ] || diagnostics
|
||||
}
|
||||
|
||||
diagnostics() {
|
||||
log "state of the stacks:"
|
||||
compose "$LIVE" traefik ps -a >&2 || true
|
||||
fm ps -a >&2 || true
|
||||
fm logs --no-color --tail=100 >&2 || true
|
||||
}
|
||||
Executable
+91
@@ -0,0 +1,91 @@
|
||||
#!/usr/bin/env bash
|
||||
# Manual rollback to a kept release, run on the server as the deploy user:
|
||||
# sudo -u deploy /opt/petrolyst/infra/deploy/rollback.sh <sha>
|
||||
#
|
||||
# Code-only when the target image's Alembic head equals the metadata
|
||||
# database's revision. Otherwise a migration rollback: the metadata database is
|
||||
# dropped, recreated and restored from the backup taken for the target image,
|
||||
# and the restored revision is checked before anything starts. Until the
|
||||
# rollback completes, state/rollback-in-progress blocks deploy.sh; an
|
||||
# interrupted rollback is finished by running it again. Afterwards,
|
||||
# `git revert` and push so main matches; deploy.sh's downgrade guard stops a
|
||||
# plain revert from running against a newer schema.
|
||||
set -euo pipefail
|
||||
# shellcheck source=deploy/lib.sh
|
||||
. "$(dirname "$(readlink -f "$0")")/lib.sh"
|
||||
|
||||
# Destructive steps below are hard-wired to service superset-db and database
|
||||
# superset; nothing here can be pointed at fuel-db.
|
||||
restore_metadata() {
|
||||
local backup=$1 head=$2 restored
|
||||
log "migration rollback: restoring $backup"
|
||||
fm stop superset
|
||||
fm exec -T superset-db psql -U superset -d postgres -v ON_ERROR_STOP=1 \
|
||||
-c 'DROP DATABASE IF EXISTS superset WITH (FORCE)' \
|
||||
-c 'CREATE DATABASE superset OWNER superset'
|
||||
fm exec -T superset-db pg_restore -U superset -d superset --exit-on-error --single-transaction <"$backup"
|
||||
restored=$(metadata_revision)
|
||||
[ "$restored" = "$head" ] ||
|
||||
die "restored revision $restored is not the target head $head; superset stays stopped"
|
||||
}
|
||||
|
||||
# metadata_db_exists: whether database superset exists, asked through the
|
||||
# maintenance database; a connection or authentication error aborts.
|
||||
metadata_db_exists() {
|
||||
local n
|
||||
n=$(fm exec -T superset-db psql -U superset -d postgres -tAc "SELECT count(*) FROM pg_database WHERE datname = 'superset'") ||
|
||||
die "cannot reach the metadata database server"
|
||||
[[ $n =~ ^[01]$ ]] || die "unexpected answer from pg_database: $n"
|
||||
[ "$n" = 1 ]
|
||||
}
|
||||
|
||||
main() {
|
||||
local sha=${1:-} release image revision head backup dir
|
||||
[[ $sha =~ ^[0-9a-f]{40}$ ]] || die "usage: rollback.sh <40-hex sha of a kept release>"
|
||||
acquire_lock 600
|
||||
release=$RELEASES/$sha
|
||||
[ -s "$release/ARTIFACTS" ] || die "no finalized release with ARTIFACTS at $release"
|
||||
# The release's recorded identities, never recomputed.
|
||||
load_artifacts "$release"
|
||||
image=fuel-superset:$SUPERSET_IMAGE_TAG
|
||||
"${DOCKER[@]}" image inspect "$image" >/dev/null 2>&1 || die "image $image is no longer on this host"
|
||||
preflight "$release"
|
||||
|
||||
trap on_exit EXIT
|
||||
fm up -d --wait superset-db redis
|
||||
if [ -e "$ROLLBACK_MARK" ] && ! metadata_db_exists; then
|
||||
# An earlier attempt stopped between DROP and CREATE.
|
||||
revision=fresh
|
||||
else
|
||||
revision=$(metadata_revision)
|
||||
fi
|
||||
# Empty or missing is expected only after an interrupted restore, which left the mark.
|
||||
[ "$revision" != fresh ] || [ -e "$ROLLBACK_MARK" ] || die "the metadata database is empty; there is nothing to roll back"
|
||||
head=$(alembic "$image" heads)
|
||||
[[ $head =~ ^[0-9a-z_]+$ ]] || die "$image has no single migration head: $head"
|
||||
|
||||
if [ "$revision" = "$head" ]; then
|
||||
log "code-only rollback: the schema is already at $head"
|
||||
else
|
||||
backup=$(find "$BACKUPS" -maxdepth 1 -name "superset-meta_${SUPERSET_IMAGE_TAG}_${head}_*.dump" | sort | tail -n1)
|
||||
[ -n "$backup" ] || die "no backup for image $SUPERSET_IMAGE_TAG at revision $head in $BACKUPS"
|
||||
mkdir -p "$STATE"
|
||||
printf '%s\n' "$sha" >"$ROLLBACK_MARK"
|
||||
restore_metadata "$backup" "$head"
|
||||
fi
|
||||
|
||||
promote "$release" "$sha"
|
||||
ensure_proxy_network
|
||||
compose "$LIVE" traefik up -d --wait --remove-orphans
|
||||
# No superset-init: the schema already matches the target image.
|
||||
while IFS= read -r dir; do
|
||||
compose "$LIVE" "$dir" up -d --wait --remove-orphans
|
||||
done < <(stack_dirs "$LIVE")
|
||||
smoke_tests
|
||||
record_state "$sha"
|
||||
rm -f "$ROLLBACK_MARK"
|
||||
trap - EXIT
|
||||
log "rolled back to $sha; now git revert and push so main matches"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Executable
+105
@@ -0,0 +1,105 @@
|
||||
#!/usr/bin/env bash
|
||||
# One-time host preparation for the petrolyst server (Ubuntu 24.04), run as root.
|
||||
# Idempotent: re-running converges to the same state.
|
||||
#
|
||||
# bootstrap.sh --ci-key <public-key-file> --runner-ip <ip|any>
|
||||
#
|
||||
# --ci-key public half of the dedicated CI deploy key (DEPLOY_SSH_KEY in Gitea)
|
||||
# --runner-ip egress IP of the Gitea act_runner; the deploy key works only from
|
||||
# there ("any" drops that restriction, e.g. before the runner exists)
|
||||
set -euo pipefail
|
||||
|
||||
die() { echo "ERROR: $*" >&2; exit 1; }
|
||||
[ "$(id -u)" = 0 ] || die "run as root"
|
||||
|
||||
ci_key='' runner_ip=''
|
||||
while [ $# -gt 0 ]; do
|
||||
case $1 in
|
||||
--ci-key) ci_key=$2; shift 2 ;;
|
||||
--runner-ip) runner_ip=$2; shift 2 ;;
|
||||
*) die "unknown argument $1" ;;
|
||||
esac
|
||||
done
|
||||
[ -s "$ci_key" ] || die "--ci-key <public-key-file> is required"
|
||||
[ -n "$runner_ip" ] || die "--runner-ip <ip|any> is required"
|
||||
grep -qE '^ssh-(ed25519|rsa) ' "$ci_key" || die "$ci_key does not look like an SSH public key"
|
||||
|
||||
ROOT=/opt/petrolyst
|
||||
|
||||
echo "== packages and Docker (official apt repository)"
|
||||
apt-get update -q
|
||||
apt-get install -y -q ca-certificates curl rsync ufw
|
||||
install -m 0755 -d /etc/apt/keyrings
|
||||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
|
||||
chmod a+r /etc/apt/keyrings/docker.asc
|
||||
# shellcheck source=/dev/null
|
||||
codename=$(. /etc/os-release && echo "$VERSION_CODENAME")
|
||||
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $codename stable" \
|
||||
>/etc/apt/sources.list.d/docker.list
|
||||
apt-get update -q
|
||||
apt-get install -y -q docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
|
||||
|
||||
daemon_json='{
|
||||
"log-driver": "json-file",
|
||||
"log-opts": { "max-size": "10m", "max-file": "3" },
|
||||
"live-restore": true
|
||||
}'
|
||||
if [ "$(cat /etc/docker/daemon.json 2>/dev/null)" != "$daemon_json" ]; then
|
||||
printf '%s\n' "$daemon_json" >/etc/docker/daemon.json
|
||||
systemctl restart docker
|
||||
fi
|
||||
systemctl enable --now docker
|
||||
|
||||
echo "== swap (4 GB) for Trino, Superset and two Postgres in 7.8 GB"
|
||||
if ! swapon --show=NAME --noheadings | grep -qx /swapfile; then
|
||||
[ -e /swapfile ] || fallocate -l 4G /swapfile
|
||||
chmod 600 /swapfile
|
||||
mkswap /swapfile >/dev/null
|
||||
swapon /swapfile
|
||||
fi
|
||||
grep -q '^/swapfile ' /etc/fstab || echo '/swapfile none swap sw 0 0' >>/etc/fstab
|
||||
echo 'vm.swappiness=10' >/etc/sysctl.d/99-petrolyst.conf
|
||||
sysctl -q -p /etc/sysctl.d/99-petrolyst.conf
|
||||
|
||||
echo "== firewall: 22, 80, 443 only"
|
||||
# Docker-published ports bypass ufw. Only Traefik publishes publicly; every
|
||||
# other published port in the stacks binds to 127.0.0.1.
|
||||
ufw default deny incoming >/dev/null
|
||||
ufw default allow outgoing >/dev/null
|
||||
for port in 22 80 443; do ufw allow "$port/tcp" >/dev/null; done
|
||||
ufw --force enable >/dev/null
|
||||
|
||||
echo "== deploy user (CI)"
|
||||
id deploy >/dev/null 2>&1 || useradd --create-home --shell /bin/bash deploy
|
||||
usermod -aG docker deploy # docker access makes deploy root-equivalent; the key restrictions below matter
|
||||
install -d -o deploy -g deploy -m 0700 /home/deploy/.ssh
|
||||
options=restrict
|
||||
[ "$runner_ip" = any ] || options="restrict,from=\"$runner_ip\""
|
||||
printf '%s %s\n' "$options" "$(cat "$ci_key")" >/home/deploy/.ssh/authorized_keys
|
||||
chown deploy:deploy /home/deploy/.ssh/authorized_keys
|
||||
chmod 0600 /home/deploy/.ssh/authorized_keys
|
||||
|
||||
echo "== SSH: keys only"
|
||||
[ -s /root/.ssh/authorized_keys ] || die "root has no authorized_keys; refusing to turn off password login"
|
||||
# 00- sorts before cloud-init's 50-cloud-init.conf, and sshd keeps the first
|
||||
# value it reads, so these win over its PasswordAuthentication yes.
|
||||
cat >/etc/ssh/sshd_config.d/00-petrolyst.conf <<'EOF'
|
||||
PasswordAuthentication no
|
||||
KbdInteractiveAuthentication no
|
||||
PermitRootLogin prohibit-password
|
||||
EOF
|
||||
sshd -t
|
||||
systemctl reload ssh
|
||||
|
||||
echo "== /opt/petrolyst"
|
||||
install -d -o root -g root -m 0755 "$ROOT"
|
||||
install -d -o deploy -g deploy -m 0755 "$ROOT/infra" "$ROOT/releases" "$ROOT/incoming" "$ROOT/state"
|
||||
install -d -o deploy -g deploy -m 0750 "$ROOT/backups"
|
||||
[ -e "$ROOT/deploy.lock" ] || install -o deploy -g deploy -m 0640 /dev/null "$ROOT/deploy.lock"
|
||||
docker network inspect proxy >/dev/null 2>&1 || docker network create proxy >/dev/null
|
||||
|
||||
echo
|
||||
echo "Done. Before closing this session, check from another terminal that"
|
||||
echo " ssh petrolyst still logs you in as root, and"
|
||||
echo " sshd -T | grep -E 'passwordauthentication|permitrootlogin' shows no / prohibit-password."
|
||||
echo "Next: host/secrets-init.sh"
|
||||
Executable
+86
@@ -0,0 +1,86 @@
|
||||
#!/usr/bin/env bash
|
||||
# Server secrets and env files, run as root after bootstrap.sh.
|
||||
# secrets-init.sh generate every missing secret; never overwrites one
|
||||
# secrets-init.sh keycloak-secret store the client secret of Keycloak client
|
||||
# "petrolyst" (Clients > petrolyst > Credentials),
|
||||
# read from the terminal or stdin
|
||||
#
|
||||
# Layout: /opt/petrolyst/secrets/{shared,traefik,fuel-monitoring}/ are root:deploy
|
||||
# 0750, so only root and the deploy user can reach them; the files are 0444
|
||||
# because each container reads them under its own uid.
|
||||
set -euo pipefail
|
||||
|
||||
die() { echo "ERROR: $*" >&2; exit 1; }
|
||||
[ "$(id -u)" = 0 ] || die "run as root"
|
||||
getent group deploy >/dev/null || die "group deploy is missing: run host/bootstrap.sh first"
|
||||
|
||||
ROOT=/opt/petrolyst
|
||||
SECRETS=$ROOT/secrets
|
||||
umask 077
|
||||
install -d -o root -g deploy -m 0750 "$SECRETS" "$SECRETS/shared" "$SECRETS/traefik" "$SECRETS/fuel-monitoring" "$ROOT/env"
|
||||
|
||||
# write_secret <path> <value>: no trailing newline (oauth2-proxy reads the file
|
||||
# byte for byte), replaced atomically.
|
||||
write_secret() {
|
||||
printf '%s' "$2" >"$1.tmp"
|
||||
chown root:deploy "$1.tmp"
|
||||
chmod 0444 "$1.tmp"
|
||||
mv "$1.tmp" "$1"
|
||||
}
|
||||
|
||||
generate() {
|
||||
if [ -s "$1" ]; then
|
||||
echo "kept $1"
|
||||
else
|
||||
write_secret "$1" "$(openssl rand -hex "$2")"
|
||||
echo "generated $1"
|
||||
fi
|
||||
}
|
||||
|
||||
case ${1:-all} in
|
||||
keycloak-secret)
|
||||
if [ -t 0 ]; then
|
||||
printf 'Client secret of Keycloak client "petrolyst": ' >&2
|
||||
IFS= read -rs value
|
||||
echo >&2
|
||||
else
|
||||
IFS= read -r value
|
||||
fi
|
||||
value=$(printf '%s' "$value" | tr -d '[:space:]')
|
||||
[ -n "$value" ] || die "empty secret"
|
||||
write_secret "$SECRETS/shared/keycloak_petrolyst_client_secret" "$value"
|
||||
echo "stored $SECRETS/shared/keycloak_petrolyst_client_secret"
|
||||
exit 0
|
||||
;;
|
||||
all) ;;
|
||||
*) die "usage: secrets-init.sh [keycloak-secret]" ;;
|
||||
esac
|
||||
|
||||
# Hex only, so no value ever needs quoting in a URL or a properties file.
|
||||
generate "$SECRETS/fuel-monitoring/superset_secret_key" 32
|
||||
generate "$SECRETS/fuel-monitoring/superset_db_password" 32
|
||||
generate "$SECRETS/fuel-monitoring/fuel_db_password" 32
|
||||
generate "$SECRETS/fuel-monitoring/fuel_db_trino_password" 32
|
||||
# 32 hex characters = a 32-byte cookie key, as oauth2-proxy requires.
|
||||
generate "$SECRETS/traefik/oauth2_proxy_cookie_secret" 16
|
||||
|
||||
# Trino reads the catalog password from its environment (${ENV:...}); keep the
|
||||
# env file in step with the secret file that fuel-db's init script uses.
|
||||
env_file=$ROOT/env/fuel-monitoring.env
|
||||
line="FUEL_DB_TRINO_PASSWORD=$(cat "$SECRETS/fuel-monitoring/fuel_db_trino_password")"
|
||||
if ! grep -qxF "$line" "$env_file" 2>/dev/null; then
|
||||
printf '%s\n' "$line" >"$env_file.tmp"
|
||||
chown root:deploy "$env_file.tmp"
|
||||
chmod 0440 "$env_file.tmp"
|
||||
mv "$env_file.tmp" "$env_file"
|
||||
echo "wrote $env_file"
|
||||
fi
|
||||
|
||||
if [ ! -s "$SECRETS/shared/keycloak_petrolyst_client_secret" ]; then
|
||||
echo
|
||||
echo "Still missing: the Keycloak client secret. Copy it from Keycloak"
|
||||
echo "(realm thedigitalbridge > Clients > petrolyst > Credentials), then run:"
|
||||
echo " $0 keycloak-secret"
|
||||
exit 1
|
||||
fi
|
||||
echo "All secrets present."
|
||||
@@ -0,0 +1,56 @@
|
||||
{
|
||||
"clients": [
|
||||
{
|
||||
"clientId": "petrolyst",
|
||||
"name": "Petrolyst",
|
||||
"description": "Petrolyst apps on the petrolyst server: Superset at atoma.petrolyst.com and the Traefik dashboard at traefik.petrolyst.com (oauth2-proxy). Managed from petrolyst-infra/keycloak/.",
|
||||
"enabled": true,
|
||||
"protocol": "openid-connect",
|
||||
"publicClient": false,
|
||||
"clientAuthenticatorType": "client-secret",
|
||||
"standardFlowEnabled": true,
|
||||
"implicitFlowEnabled": false,
|
||||
"directAccessGrantsEnabled": false,
|
||||
"serviceAccountsEnabled": false,
|
||||
"frontchannelLogout": true,
|
||||
"baseUrl": "https://atoma.petrolyst.com/",
|
||||
"redirectUris": [
|
||||
"https://atoma.petrolyst.com/oauth-authorized/keycloak",
|
||||
"https://traefik.petrolyst.com/oauth2/callback"
|
||||
],
|
||||
"webOrigins": ["+"],
|
||||
"attributes": {
|
||||
"pkce.code.challenge.method": "S256",
|
||||
"post.logout.redirect.uris": "https://atoma.petrolyst.com/*##https://traefik.petrolyst.com/*"
|
||||
},
|
||||
"protocolMappers": [
|
||||
{
|
||||
"name": "petrolyst client roles",
|
||||
"protocol": "openid-connect",
|
||||
"protocolMapper": "oidc-usermodel-client-role-mapper",
|
||||
"consentRequired": false,
|
||||
"config": {
|
||||
"usermodel.clientRoleMapping.clientId": "petrolyst",
|
||||
"multivalued": "true",
|
||||
"claim.name": "petrolyst_roles",
|
||||
"jsonType.label": "String",
|
||||
"id.token.claim": "true",
|
||||
"access.token.claim": "true",
|
||||
"userinfo.token.claim": "true",
|
||||
"introspection.token.claim": "true"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"roles": {
|
||||
"client": {
|
||||
"petrolyst": [
|
||||
{ "name": "superset-admin", "description": "Superset Admin at atoma.petrolyst.com" },
|
||||
{ "name": "superset-editor", "description": "Superset Alpha: build charts and dashboards" },
|
||||
{ "name": "superset-viewer", "description": "Superset Gamma: view dashboards shared with the role" },
|
||||
{ "name": "traefik-admin", "description": "Traefik dashboard at traefik.petrolyst.com" }
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
# petrolyst-insights
|
||||
|
||||
The Petrolyst customization of Apache Superset 6.1.0: sign-in, roles, look and feel, patches
|
||||
and dashboards. Consumed by the deployment folders of this repository at image-build time.
|
||||
**Not runnable standalone**: this folder has no Dockerfile, no Compose file, no CI and no
|
||||
secrets. `deploy/deploy.sh` refuses to deploy if one appears here.
|
||||
|
||||
The split follows media-insights (customization) and media-intelligence-docker (deployment):
|
||||
anything that changes how Superset looks or behaves for Petrolyst lives here; anything about how
|
||||
and where it runs lives in `stacks/`, `traefik/`, `deploy/` and `host/`.
|
||||
|
||||
## Layout
|
||||
|
||||
| Path | What it is | In the image at |
|
||||
|---|---|---|
|
||||
| `superset_config.py` | Keycloak sign-in, role mapping, proxy and cookie settings, Redis caches, theme and chart palettes | `/app/pythonpath/` |
|
||||
| `petrolyst_security.py` | `PetrolystSecurityManager`: user info and roles from the Keycloak ID token, `/login/` straight to Keycloak, `/logout/` through Keycloak | `/app/pythonpath/` |
|
||||
| `assets/branding/` | logos (full colour and white), favicons, app icons | `/app/superset/static/assets/branding/`, served at `/static/assets/branding/…` |
|
||||
| `patches/` | runtime patches, `fix_*.py` with `--verify` (see its README); empty at start | `/app/patches/petrolyst/`, applied during the build |
|
||||
| `dashboards/` | exported dashboards and import scripts (see its README) | not in the image |
|
||||
|
||||
How the image consumes this folder: `stacks/fuel-monitoring/compose.yaml` passes it to the build
|
||||
as the named context `insights`, and the Dockerfile copies from it with `COPY --from=insights`.
|
||||
The image tag is a hash of exactly the inputs above that end up in the image, so a change here
|
||||
rebuilds and recreates only the `superset` container.
|
||||
|
||||
## Secrets read
|
||||
|
||||
`superset_config.py` uses `_read_secret(name)` (copied from media-insights): it prefers
|
||||
`/run/secrets/<name>` and falls back to an env var for local runs, and fails at boot on an empty
|
||||
value.
|
||||
|
||||
| Secret | Purpose |
|
||||
|---|---|
|
||||
| `superset_secret_key` | Flask `SECRET_KEY` |
|
||||
| `superset_db_password` | password of user `superset` on the metadata database |
|
||||
| `keycloak_petrolyst_client_secret` | client secret of Keycloak client `petrolyst` |
|
||||
|
||||
On the server they come from `/opt/petrolyst/secrets/` (see the deployment README).
|
||||
|
||||
## Sign-in and roles
|
||||
|
||||
- Provider: Keycloak realm `thedigitalbridge` at `iam.thedigitalbridge.io`, client `petrolyst`,
|
||||
authorization code flow with PKCE S256. The realm URL can be overridden with
|
||||
`KEYCLOAK_REALM_URL`.
|
||||
- Roles come from the client roles of `petrolyst`, which a protocol mapper puts into the claim
|
||||
`petrolyst_roles`, and are re-read at every login:
|
||||
|
||||
| Keycloak client role | Superset role |
|
||||
|---|---|
|
||||
| `superset-admin` | Admin |
|
||||
| `superset-editor` | Alpha |
|
||||
| `superset-viewer` | Gamma |
|
||||
| none | Public (no permissions) |
|
||||
|
||||
- `DASHBOARD_RBAC` is on: a viewer sees a dashboard once an admin adds the Gamma role (or a
|
||||
custom one) to it.
|
||||
|
||||
## Look and feel
|
||||
|
||||
Source: `petrolyst-ui/design/brand/` (tokens `07_UI_Tokens/petrolyst_tokens.css`, logos
|
||||
`public/brand/`, favicons `03_Favicon_App_Icons/`), copied on 2026-10-23 from petrolyst-ui commit
|
||||
`2b2e694`. Copy again when the brand changes.
|
||||
|
||||
- Light theme: action teal `#00695C` for primary and links, navy `#0B1D33` text, `#F8FAFC`
|
||||
layout, full-colour stacked logo.
|
||||
- Dark theme: `#35C5B6` primary, `#071425` / `#12263D` surfaces, white logo.
|
||||
- Font: Inter, Superset's default. Satoshi is licence-restricted and is not used.
|
||||
- Chart palettes: "Petrolyst" categorical (the default, brand teal first) and "Petrolyst Teal" /
|
||||
"Petrolyst Navy" sequential. The categorical palette was validated with the dataviz skill's
|
||||
validator on both chart surfaces; the numbers are in the comment above it in
|
||||
`superset_config.py`. Charts with more than three series in scatter or map form should fold
|
||||
the rest into "Other".
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 441 B |
@@ -0,0 +1,2 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="455 175 505 430" role="img" aria-label="Petrolyst micro mark"><path d="M 945.0,214.0 L 878.0,214.0 L 875.0,215.0 L 754.0,341.0 L 749.0,348.0 L 734.0,375.0 L 727.0,398.0 L 726.0,412.0 L 725.0,413.0 L 725.0,437.0 L 724.0,438.0 L 724.0,448.0 L 721.0,464.0 L 714.0,484.0 L 698.0,514.0 L 693.0,521.0 L 685.0,542.0 L 682.0,558.0 L 682.0,596.0 L 701.0,595.0 L 717.0,591.0 L 736.0,580.0 L 746.0,569.0 L 752.0,558.0 L 755.0,548.0 L 755.0,535.0 L 756.0,534.0 L 756.0,453.0 L 757.0,452.0 L 757.0,446.0 L 761.0,430.0 L 770.0,410.0 L 780.0,394.0 L 792.0,380.0 L 946.0,219.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 503.0,214.0 L 502.0,219.0 L 589.0,312.0 L 596.0,321.0 L 651.0,380.0 L 663.0,395.0 L 671.0,408.0 L 681.0,431.0 L 686.0,456.0 L 686.0,491.0 L 691.0,491.0 L 703.0,472.0 L 709.0,459.0 L 715.0,439.0 L 716.0,423.0 L 717.0,422.0 L 716.0,400.0 L 711.0,379.0 L 701.0,357.0 L 691.0,342.0 L 643.0,289.0 L 572.0,214.0 L 561.0,214.0 L 560.0,215.0 Z" fill="#0B1D33" fill-rule="evenodd"/></svg>
|
||||
|
After Width: | Height: | Size: 1.0 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 4.6 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 4.9 KiB |
@@ -0,0 +1,35 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="55 120 1335 700" role="img" aria-label="Petrolyst">
|
||||
|
||||
<defs>
|
||||
<clipPath id="markClip"><path d="M 943.0,216.0 L 896.0,216.0 L 895.0,217.0 L 892.0,217.0 L 891.0,216.0 L 880.0,216.0 L 879.0,217.0 L 877.0,217.0 L 840.0,255.0 L 840.0,256.0 L 834.0,261.0 L 834.0,262.0 L 814.0,282.0 L 814.0,283.0 L 794.0,303.0 L 794.0,304.0 L 792.0,306.0 L 791.0,306.0 L 791.0,307.0 L 771.0,327.0 L 771.0,328.0 L 756.0,343.0 L 754.0,347.0 L 751.0,350.0 L 751.0,351.0 L 747.0,356.0 L 746.0,359.0 L 744.0,361.0 L 736.0,377.0 L 736.0,379.0 L 735.0,380.0 L 735.0,382.0 L 732.0,389.0 L 731.0,396.0 L 729.0,400.0 L 729.0,405.0 L 728.0,406.0 L 728.0,414.0 L 727.0,415.0 L 727.0,439.0 L 726.0,440.0 L 726.0,450.0 L 725.0,451.0 L 725.0,456.0 L 724.0,457.0 L 724.0,462.0 L 723.0,463.0 L 723.0,466.0 L 722.0,467.0 L 718.0,481.0 L 716.0,484.0 L 716.0,486.0 L 709.0,500.0 L 707.0,502.0 L 705.0,507.0 L 703.0,509.0 L 700.0,516.0 L 695.0,523.0 L 694.0,527.0 L 691.0,532.0 L 688.0,543.0 L 687.0,544.0 L 686.0,552.0 L 685.0,553.0 L 685.0,559.0 L 684.0,560.0 L 684.0,594.0 L 691.0,594.0 L 692.0,593.0 L 705.0,592.0 L 709.0,590.0 L 715.0,589.0 L 725.0,584.0 L 727.0,582.0 L 730.0,581.0 L 733.0,578.0 L 734.0,578.0 L 741.0,571.0 L 741.0,570.0 L 746.0,564.0 L 750.0,556.0 L 750.0,554.0 L 752.0,550.0 L 752.0,547.0 L 753.0,546.0 L 753.0,533.0 L 754.0,532.0 L 754.0,451.0 L 755.0,450.0 L 755.0,444.0 L 756.0,443.0 L 756.0,439.0 L 757.0,438.0 L 757.0,435.0 L 759.0,431.0 L 759.0,428.0 L 761.0,425.0 L 763.0,418.0 L 768.0,408.0 L 770.0,406.0 L 772.0,401.0 L 774.0,399.0 L 778.0,392.0 L 790.0,379.0 L 790.0,378.0 L 822.0,345.0 L 822.0,344.0 L 868.0,297.0 L 868.0,296.0 L 908.0,255.0 L 908.0,254.0 L 912.0,250.0 L 913.0,250.0 L 915.0,248.0 L 915.0,247.0 L 932.0,230.0 L 932.0,229.0 L 944.0,217.0 Z" fill="#000" fill-rule="evenodd"/>
|
||||
<path d="M 505.0,216.0 L 504.0,217.0 L 510.0,223.0 L 510.0,224.0 L 524.0,238.0 L 524.0,239.0 L 538.0,253.0 L 538.0,254.0 L 549.0,265.0 L 549.0,266.0 L 591.0,310.0 L 591.0,311.0 L 598.0,318.0 L 598.0,319.0 L 612.0,333.0 L 612.0,334.0 L 622.0,344.0 L 622.0,345.0 L 630.0,353.0 L 630.0,354.0 L 634.0,357.0 L 634.0,358.0 L 642.0,366.0 L 642.0,367.0 L 653.0,378.0 L 653.0,379.0 L 665.0,393.0 L 668.0,399.0 L 670.0,401.0 L 671.0,404.0 L 673.0,406.0 L 677.0,414.0 L 677.0,416.0 L 680.0,421.0 L 681.0,426.0 L 683.0,429.0 L 683.0,432.0 L 684.0,433.0 L 684.0,436.0 L 685.0,437.0 L 685.0,440.0 L 686.0,441.0 L 687.0,453.0 L 688.0,454.0 L 688.0,489.0 L 689.0,489.0 L 690.0,486.0 L 693.0,483.0 L 696.0,477.0 L 698.0,475.0 L 699.0,472.0 L 701.0,470.0 L 705.0,462.0 L 705.0,460.0 L 707.0,457.0 L 708.0,452.0 L 710.0,449.0 L 712.0,438.0 L 713.0,437.0 L 713.0,433.0 L 714.0,432.0 L 714.0,421.0 L 715.0,420.0 L 715.0,413.0 L 714.0,412.0 L 714.0,402.0 L 713.0,401.0 L 712.0,392.0 L 711.0,391.0 L 711.0,388.0 L 709.0,384.0 L 708.0,378.0 L 699.0,359.0 L 697.0,357.0 L 689.0,344.0 L 677.0,331.0 L 677.0,330.0 L 641.0,292.0 L 641.0,291.0 L 599.0,247.0 L 599.0,246.0 L 587.0,234.0 L 587.0,233.0 L 586.0,233.0 L 584.0,231.0 L 584.0,230.0 L 570.0,216.0 L 563.0,216.0 L 562.0,217.0 L 543.0,217.0 L 542.0,216.0 Z" fill="#000" fill-rule="evenodd"/></clipPath>
|
||||
<linearGradient id="rightGradient" x1="710" y1="180" x2="965" y2="590" gradientUnits="userSpaceOnUse">
|
||||
<stop offset="0%" stop-color="#009688"/>
|
||||
<stop offset="100%" stop-color="#00B6E6"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="wordYGradient" x1="945" y1="660" x2="1090" y2="780" gradientUnits="userSpaceOnUse">
|
||||
<stop offset="0%" stop-color="#009688"/>
|
||||
<stop offset="100%" stop-color="#00B6E6"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
|
||||
<g clip-path="url(#markClip)">
|
||||
<rect x="0" y="0" width="735" height="1086" fill="#0B1D33"/>
|
||||
<rect x="735" y="0" width="665" height="1086" fill="url(#rightGradient)"/>
|
||||
</g>
|
||||
<g fill="#0B1D33">
|
||||
<path d="M 250.0,753.0 L 250.0,767.0 L 339.0,767.0 L 339.0,764.0 L 340.0,763.0 L 340.0,757.0 L 339.0,756.0 L 339.0,752.0 L 251.0,752.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 250.0,715.0 L 250.0,729.0 L 266.0,729.0 L 267.0,730.0 L 301.0,730.0 L 302.0,729.0 L 307.0,729.0 L 308.0,730.0 L 325.0,730.0 L 326.0,729.0 L 340.0,729.0 L 340.0,716.0 L 339.0,715.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 1260.0,678.0 L 1260.0,692.0 L 1264.0,692.0 L 1265.0,693.0 L 1296.0,693.0 L 1297.0,694.0 L 1297.0,767.0 L 1313.0,767.0 L 1313.0,694.0 L 1314.0,693.0 L 1351.0,693.0 L 1351.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 867.0,678.0 L 867.0,767.0 L 951.0,767.0 L 951.0,753.0 L 950.0,752.0 L 883.0,752.0 L 882.0,751.0 L 882.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 551.0,678.0 L 551.0,767.0 L 566.0,767.0 L 566.0,743.0 L 567.0,742.0 L 567.0,739.0 L 566.0,738.0 L 567.0,737.0 L 604.0,737.0 L 614.0,747.0 L 614.0,748.0 L 622.0,756.0 L 622.0,757.0 L 630.0,765.0 L 630.0,766.0 L 631.0,767.0 L 651.0,767.0 L 651.0,766.0 L 649.0,764.0 L 649.0,763.0 L 642.0,756.0 L 642.0,755.0 L 628.0,741.0 L 628.0,740.0 L 626.0,738.0 L 628.0,736.0 L 630.0,736.0 L 631.0,735.0 L 633.0,735.0 L 634.0,734.0 L 636.0,734.0 L 638.0,732.0 L 639.0,732.0 L 645.0,726.0 L 645.0,725.0 L 648.0,720.0 L 648.0,718.0 L 649.0,717.0 L 649.0,713.0 L 650.0,712.0 L 650.0,703.0 L 649.0,702.0 L 649.0,698.0 L 648.0,697.0 L 648.0,695.0 L 647.0,694.0 L 646.0,691.0 L 644.0,689.0 L 644.0,688.0 L 640.0,684.0 L 639.0,684.0 L 637.0,682.0 L 636.0,682.0 L 633.0,680.0 L 631.0,680.0 L 630.0,679.0 L 626.0,679.0 L 625.0,678.0 Z M 567.0,694.0 L 568.0,693.0 L 625.0,693.0 L 626.0,694.0 L 629.0,695.0 L 632.0,698.0 L 632.0,699.0 L 633.0,700.0 L 633.0,713.0 L 632.0,714.0 L 632.0,716.0 L 630.0,718.0 L 630.0,719.0 L 629.0,720.0 L 628.0,720.0 L 626.0,722.0 L 622.0,722.0 L 621.0,723.0 L 568.0,723.0 L 566.0,721.0 L 566.0,703.0 L 567.0,702.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 398.0,678.0 L 398.0,693.0 L 434.0,693.0 L 435.0,694.0 L 435.0,767.0 L 451.0,767.0 L 451.0,694.0 L 452.0,693.0 L 489.0,693.0 L 489.0,679.0 L 488.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 250.0,678.0 L 250.0,692.0 L 251.0,692.0 L 252.0,693.0 L 339.0,693.0 L 339.0,691.0 L 340.0,690.0 L 340.0,679.0 L 339.0,678.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 1134.0,683.0 L 1129.0,688.0 L 1129.0,689.0 L 1127.0,691.0 L 1127.0,693.0 L 1126.0,694.0 L 1126.0,696.0 L 1125.0,697.0 L 1125.0,712.0 L 1126.0,713.0 L 1126.0,715.0 L 1127.0,716.0 L 1128.0,719.0 L 1134.0,725.0 L 1135.0,725.0 L 1137.0,727.0 L 1139.0,727.0 L 1140.0,728.0 L 1143.0,728.0 L 1144.0,729.0 L 1185.0,729.0 L 1186.0,730.0 L 1191.0,730.0 L 1192.0,731.0 L 1193.0,731.0 L 1197.0,735.0 L 1197.0,739.0 L 1198.0,740.0 L 1198.0,742.0 L 1197.0,743.0 L 1197.0,746.0 L 1196.0,747.0 L 1196.0,748.0 L 1193.0,751.0 L 1192.0,751.0 L 1189.0,753.0 L 1184.0,753.0 L 1183.0,754.0 L 1151.0,754.0 L 1150.0,753.0 L 1143.0,753.0 L 1142.0,752.0 L 1139.0,752.0 L 1138.0,751.0 L 1135.0,751.0 L 1134.0,750.0 L 1131.0,750.0 L 1130.0,749.0 L 1128.0,749.0 L 1127.0,748.0 L 1125.0,748.0 L 1125.0,760.0 L 1124.0,761.0 L 1126.0,763.0 L 1128.0,763.0 L 1129.0,764.0 L 1131.0,764.0 L 1132.0,765.0 L 1134.0,765.0 L 1135.0,766.0 L 1139.0,766.0 L 1140.0,767.0 L 1146.0,767.0 L 1147.0,768.0 L 1186.0,768.0 L 1187.0,767.0 L 1191.0,767.0 L 1192.0,766.0 L 1195.0,766.0 L 1196.0,765.0 L 1198.0,765.0 L 1199.0,764.0 L 1201.0,764.0 L 1203.0,762.0 L 1204.0,762.0 L 1209.0,757.0 L 1209.0,756.0 L 1211.0,754.0 L 1211.0,753.0 L 1212.0,752.0 L 1212.0,750.0 L 1213.0,749.0 L 1213.0,734.0 L 1212.0,733.0 L 1212.0,730.0 L 1211.0,729.0 L 1211.0,728.0 L 1209.0,726.0 L 1209.0,725.0 L 1204.0,720.0 L 1203.0,720.0 L 1202.0,719.0 L 1201.0,719.0 L 1196.0,716.0 L 1192.0,716.0 L 1191.0,715.0 L 1148.0,715.0 L 1147.0,714.0 L 1145.0,714.0 L 1142.0,711.0 L 1142.0,710.0 L 1141.0,709.0 L 1141.0,699.0 L 1142.0,698.0 L 1142.0,697.0 L 1145.0,694.0 L 1146.0,694.0 L 1147.0,693.0 L 1150.0,693.0 L 1151.0,692.0 L 1190.0,692.0 L 1191.0,693.0 L 1196.0,693.0 L 1197.0,694.0 L 1201.0,694.0 L 1202.0,695.0 L 1204.0,695.0 L 1205.0,696.0 L 1209.0,696.0 L 1209.0,683.0 L 1206.0,681.0 L 1203.0,681.0 L 1202.0,680.0 L 1198.0,680.0 L 1197.0,679.0 L 1193.0,679.0 L 1192.0,678.0 L 1182.0,678.0 L 1181.0,677.0 L 1156.0,677.0 L 1155.0,678.0 L 1146.0,678.0 L 1145.0,679.0 L 1143.0,679.0 L 1142.0,680.0 L 1140.0,680.0 L 1139.0,681.0 L 1138.0,681.0 L 1137.0,682.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 714.0,691.0 L 714.0,692.0 L 711.0,695.0 L 711.0,696.0 L 709.0,698.0 L 709.0,699.0 L 708.0,700.0 L 708.0,702.0 L 707.0,703.0 L 707.0,706.0 L 706.0,707.0 L 706.0,711.0 L 705.0,712.0 L 705.0,734.0 L 706.0,735.0 L 706.0,739.0 L 707.0,740.0 L 707.0,743.0 L 708.0,744.0 L 708.0,745.0 L 709.0,746.0 L 709.0,747.0 L 710.0,748.0 L 711.0,751.0 L 713.0,753.0 L 713.0,754.0 L 718.0,759.0 L 719.0,759.0 L 721.0,761.0 L 722.0,761.0 L 723.0,762.0 L 724.0,762.0 L 729.0,765.0 L 731.0,765.0 L 732.0,766.0 L 735.0,766.0 L 736.0,767.0 L 741.0,767.0 L 742.0,768.0 L 775.0,768.0 L 776.0,767.0 L 781.0,767.0 L 782.0,766.0 L 785.0,766.0 L 786.0,765.0 L 788.0,765.0 L 789.0,764.0 L 790.0,764.0 L 791.0,763.0 L 792.0,763.0 L 793.0,762.0 L 796.0,761.0 L 798.0,759.0 L 799.0,759.0 L 805.0,753.0 L 805.0,752.0 L 807.0,750.0 L 807.0,749.0 L 809.0,746.0 L 809.0,744.0 L 811.0,741.0 L 811.0,737.0 L 812.0,736.0 L 812.0,711.0 L 811.0,710.0 L 811.0,706.0 L 810.0,705.0 L 810.0,703.0 L 809.0,702.0 L 809.0,700.0 L 808.0,699.0 L 808.0,698.0 L 806.0,696.0 L 806.0,695.0 L 803.0,692.0 L 803.0,691.0 L 799.0,687.0 L 798.0,687.0 L 793.0,683.0 L 792.0,683.0 L 791.0,682.0 L 789.0,682.0 L 786.0,680.0 L 783.0,680.0 L 782.0,679.0 L 779.0,679.0 L 778.0,678.0 L 770.0,678.0 L 769.0,677.0 L 748.0,677.0 L 747.0,678.0 L 739.0,678.0 L 738.0,679.0 L 734.0,679.0 L 733.0,680.0 L 731.0,680.0 L 730.0,681.0 L 728.0,681.0 L 727.0,682.0 L 726.0,682.0 L 725.0,683.0 L 722.0,684.0 L 720.0,686.0 L 719.0,686.0 Z M 725.0,702.0 L 730.0,697.0 L 731.0,697.0 L 733.0,695.0 L 735.0,695.0 L 738.0,693.0 L 742.0,693.0 L 743.0,692.0 L 772.0,692.0 L 773.0,693.0 L 777.0,693.0 L 778.0,694.0 L 781.0,694.0 L 782.0,695.0 L 785.0,696.0 L 787.0,698.0 L 788.0,698.0 L 791.0,701.0 L 791.0,702.0 L 793.0,704.0 L 793.0,705.0 L 794.0,706.0 L 794.0,708.0 L 795.0,709.0 L 795.0,712.0 L 796.0,713.0 L 796.0,733.0 L 795.0,734.0 L 795.0,738.0 L 794.0,739.0 L 793.0,742.0 L 791.0,744.0 L 791.0,745.0 L 788.0,748.0 L 787.0,748.0 L 785.0,750.0 L 784.0,750.0 L 781.0,752.0 L 778.0,752.0 L 777.0,753.0 L 770.0,753.0 L 769.0,754.0 L 747.0,754.0 L 746.0,753.0 L 740.0,753.0 L 739.0,752.0 L 737.0,752.0 L 736.0,751.0 L 734.0,751.0 L 733.0,750.0 L 732.0,750.0 L 730.0,748.0 L 729.0,748.0 L 725.0,744.0 L 725.0,743.0 L 722.0,738.0 L 722.0,734.0 L 721.0,733.0 L 721.0,713.0 L 722.0,712.0 L 722.0,709.0 L 723.0,708.0 L 723.0,706.0 L 724.0,705.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<path d="M 95.0,678.0 L 95.0,766.0 L 96.0,767.0 L 111.0,767.0 L 111.0,737.0 L 112.0,736.0 L 171.0,736.0 L 172.0,735.0 L 175.0,735.0 L 176.0,734.0 L 178.0,734.0 L 179.0,733.0 L 180.0,733.0 L 182.0,731.0 L 183.0,731.0 L 186.0,728.0 L 187.0,728.0 L 187.0,727.0 L 190.0,724.0 L 190.0,723.0 L 193.0,718.0 L 193.0,715.0 L 194.0,714.0 L 194.0,702.0 L 193.0,701.0 L 193.0,696.0 L 192.0,695.0 L 192.0,693.0 L 191.0,692.0 L 190.0,689.0 L 188.0,687.0 L 188.0,686.0 L 185.0,683.0 L 184.0,683.0 L 182.0,681.0 L 181.0,681.0 L 180.0,680.0 L 178.0,680.0 L 177.0,679.0 L 174.0,679.0 L 173.0,678.0 L 168.0,678.0 L 167.0,677.0 L 156.0,677.0 L 155.0,678.0 L 135.0,678.0 L 134.0,677.0 L 132.0,677.0 L 131.0,678.0 L 108.0,678.0 L 107.0,677.0 L 105.0,677.0 L 104.0,678.0 Z M 111.0,693.0 L 112.0,692.0 L 169.0,692.0 L 170.0,693.0 L 172.0,693.0 L 177.0,698.0 L 177.0,699.0 L 178.0,700.0 L 178.0,712.0 L 177.0,713.0 L 177.0,715.0 L 176.0,716.0 L 176.0,717.0 L 174.0,719.0 L 173.0,719.0 L 170.0,721.0 L 167.0,721.0 L 166.0,722.0 L 154.0,722.0 L 153.0,721.0 L 152.0,722.0 L 115.0,722.0 L 114.0,721.0 L 113.0,722.0 L 111.0,720.0 Z" fill="#0B1D33" fill-rule="evenodd"/>
|
||||
<rect x="250" y="678" width="16" height="90" fill="#0B1D33"/>
|
||||
</g>
|
||||
<path d="M 1084.0,678.0 L 1063.0,678.0 L 1056.0,685.0 L 1056.0,686.0 L 1049.0,693.0 L 1049.0,694.0 L 1042.0,701.0 L 1042.0,702.0 L 1036.0,708.0 L 1036.0,709.0 L 1035.0,710.0 L 1035.0,711.0 L 1033.0,713.0 L 1033.0,714.0 L 1032.0,715.0 L 1032.0,716.0 L 1031.0,717.0 L 1031.0,718.0 L 1030.0,719.0 L 1030.0,720.0 L 1028.0,723.0 L 1028.0,725.0 L 1026.0,728.0 L 1026.0,730.0 L 1025.0,731.0 L 1025.0,733.0 L 1024.0,734.0 L 1024.0,738.0 L 1023.0,739.0 L 1023.0,743.0 L 1022.0,744.0 L 1022.0,752.0 L 1021.0,753.0 L 1021.0,766.0 L 1022.0,767.0 L 1038.0,767.0 L 1038.0,737.0 L 1039.0,736.0 L 1039.0,734.0 L 1040.0,733.0 L 1040.0,732.0 L 1041.0,731.0 L 1041.0,730.0 L 1043.0,728.0 L 1043.0,727.0 L 1046.0,724.0 L 1046.0,723.0 L 1052.0,717.0 L 1052.0,716.0 L 1060.0,708.0 L 1060.0,707.0 L 1065.0,702.0 L 1065.0,701.0 L 1072.0,694.0 L 1072.0,693.0 L 1077.0,688.0 L 1077.0,687.0 L 1082.0,682.0 L 1082.0,681.0 L 1084.0,679.0 L 1085.0,679.0 Z" fill="url(#wordYGradient)" fill-rule="evenodd"/>
|
||||
<path d="M 975.0,678.0 L 975.0,679.0 L 979.0,683.0 L 979.0,684.0 L 983.0,688.0 L 983.0,689.0 L 990.0,696.0 L 990.0,697.0 L 996.0,703.0 L 996.0,704.0 L 1002.0,710.0 L 1002.0,711.0 L 1010.0,719.0 L 1010.0,720.0 L 1014.0,724.0 L 1014.0,725.0 L 1017.0,728.0 L 1017.0,729.0 L 1018.0,730.0 L 1018.0,731.0 L 1019.0,732.0 L 1020.0,732.0 L 1020.0,729.0 L 1021.0,728.0 L 1021.0,726.0 L 1022.0,725.0 L 1022.0,722.0 L 1023.0,721.0 L 1023.0,715.0 L 1024.0,714.0 L 1023.0,713.0 L 1023.0,709.0 L 1022.0,708.0 L 1022.0,707.0 L 1021.0,706.0 L 1021.0,705.0 L 1016.0,700.0 L 1016.0,699.0 L 1010.0,693.0 L 1010.0,692.0 L 1003.0,685.0 L 1003.0,684.0 L 997.0,678.0 Z" fill="url(#wordYGradient)" fill-rule="evenodd"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 13 KiB |
@@ -0,0 +1,17 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="55 120 1335 700" role="img" aria-label="Petrolyst">
|
||||
<path d="M 943.0,216.0 L 896.0,216.0 L 895.0,217.0 L 892.0,217.0 L 891.0,216.0 L 880.0,216.0 L 879.0,217.0 L 877.0,217.0 L 840.0,255.0 L 840.0,256.0 L 834.0,261.0 L 834.0,262.0 L 814.0,282.0 L 814.0,283.0 L 794.0,303.0 L 794.0,304.0 L 792.0,306.0 L 791.0,306.0 L 791.0,307.0 L 771.0,327.0 L 771.0,328.0 L 756.0,343.0 L 754.0,347.0 L 751.0,350.0 L 751.0,351.0 L 747.0,356.0 L 746.0,359.0 L 744.0,361.0 L 736.0,377.0 L 736.0,379.0 L 735.0,380.0 L 735.0,382.0 L 732.0,389.0 L 731.0,396.0 L 729.0,400.0 L 729.0,405.0 L 728.0,406.0 L 728.0,414.0 L 727.0,415.0 L 727.0,439.0 L 726.0,440.0 L 726.0,450.0 L 725.0,451.0 L 725.0,456.0 L 724.0,457.0 L 724.0,462.0 L 723.0,463.0 L 723.0,466.0 L 722.0,467.0 L 718.0,481.0 L 716.0,484.0 L 716.0,486.0 L 709.0,500.0 L 707.0,502.0 L 705.0,507.0 L 703.0,509.0 L 700.0,516.0 L 695.0,523.0 L 694.0,527.0 L 691.0,532.0 L 688.0,543.0 L 687.0,544.0 L 686.0,552.0 L 685.0,553.0 L 685.0,559.0 L 684.0,560.0 L 684.0,594.0 L 691.0,594.0 L 692.0,593.0 L 705.0,592.0 L 709.0,590.0 L 715.0,589.0 L 725.0,584.0 L 727.0,582.0 L 730.0,581.0 L 733.0,578.0 L 734.0,578.0 L 741.0,571.0 L 741.0,570.0 L 746.0,564.0 L 750.0,556.0 L 750.0,554.0 L 752.0,550.0 L 752.0,547.0 L 753.0,546.0 L 753.0,533.0 L 754.0,532.0 L 754.0,451.0 L 755.0,450.0 L 755.0,444.0 L 756.0,443.0 L 756.0,439.0 L 757.0,438.0 L 757.0,435.0 L 759.0,431.0 L 759.0,428.0 L 761.0,425.0 L 763.0,418.0 L 768.0,408.0 L 770.0,406.0 L 772.0,401.0 L 774.0,399.0 L 778.0,392.0 L 790.0,379.0 L 790.0,378.0 L 822.0,345.0 L 822.0,344.0 L 868.0,297.0 L 868.0,296.0 L 908.0,255.0 L 908.0,254.0 L 912.0,250.0 L 913.0,250.0 L 915.0,248.0 L 915.0,247.0 L 932.0,230.0 L 932.0,229.0 L 944.0,217.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 505.0,216.0 L 504.0,217.0 L 510.0,223.0 L 510.0,224.0 L 524.0,238.0 L 524.0,239.0 L 538.0,253.0 L 538.0,254.0 L 549.0,265.0 L 549.0,266.0 L 591.0,310.0 L 591.0,311.0 L 598.0,318.0 L 598.0,319.0 L 612.0,333.0 L 612.0,334.0 L 622.0,344.0 L 622.0,345.0 L 630.0,353.0 L 630.0,354.0 L 634.0,357.0 L 634.0,358.0 L 642.0,366.0 L 642.0,367.0 L 653.0,378.0 L 653.0,379.0 L 665.0,393.0 L 668.0,399.0 L 670.0,401.0 L 671.0,404.0 L 673.0,406.0 L 677.0,414.0 L 677.0,416.0 L 680.0,421.0 L 681.0,426.0 L 683.0,429.0 L 683.0,432.0 L 684.0,433.0 L 684.0,436.0 L 685.0,437.0 L 685.0,440.0 L 686.0,441.0 L 687.0,453.0 L 688.0,454.0 L 688.0,489.0 L 689.0,489.0 L 690.0,486.0 L 693.0,483.0 L 696.0,477.0 L 698.0,475.0 L 699.0,472.0 L 701.0,470.0 L 705.0,462.0 L 705.0,460.0 L 707.0,457.0 L 708.0,452.0 L 710.0,449.0 L 712.0,438.0 L 713.0,437.0 L 713.0,433.0 L 714.0,432.0 L 714.0,421.0 L 715.0,420.0 L 715.0,413.0 L 714.0,412.0 L 714.0,402.0 L 713.0,401.0 L 712.0,392.0 L 711.0,391.0 L 711.0,388.0 L 709.0,384.0 L 708.0,378.0 L 699.0,359.0 L 697.0,357.0 L 689.0,344.0 L 677.0,331.0 L 677.0,330.0 L 641.0,292.0 L 641.0,291.0 L 599.0,247.0 L 599.0,246.0 L 587.0,234.0 L 587.0,233.0 L 586.0,233.0 L 584.0,231.0 L 584.0,230.0 L 570.0,216.0 L 563.0,216.0 L 562.0,217.0 L 543.0,217.0 L 542.0,216.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<g fill="#FFFFFF"><path d="M 250.0,753.0 L 250.0,767.0 L 339.0,767.0 L 339.0,764.0 L 340.0,763.0 L 340.0,757.0 L 339.0,756.0 L 339.0,752.0 L 251.0,752.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 250.0,715.0 L 250.0,729.0 L 266.0,729.0 L 267.0,730.0 L 301.0,730.0 L 302.0,729.0 L 307.0,729.0 L 308.0,730.0 L 325.0,730.0 L 326.0,729.0 L 340.0,729.0 L 340.0,716.0 L 339.0,715.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 1260.0,678.0 L 1260.0,692.0 L 1264.0,692.0 L 1265.0,693.0 L 1296.0,693.0 L 1297.0,694.0 L 1297.0,767.0 L 1313.0,767.0 L 1313.0,694.0 L 1314.0,693.0 L 1351.0,693.0 L 1351.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 867.0,678.0 L 867.0,767.0 L 951.0,767.0 L 951.0,753.0 L 950.0,752.0 L 883.0,752.0 L 882.0,751.0 L 882.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 551.0,678.0 L 551.0,767.0 L 566.0,767.0 L 566.0,743.0 L 567.0,742.0 L 567.0,739.0 L 566.0,738.0 L 567.0,737.0 L 604.0,737.0 L 614.0,747.0 L 614.0,748.0 L 622.0,756.0 L 622.0,757.0 L 630.0,765.0 L 630.0,766.0 L 631.0,767.0 L 651.0,767.0 L 651.0,766.0 L 649.0,764.0 L 649.0,763.0 L 642.0,756.0 L 642.0,755.0 L 628.0,741.0 L 628.0,740.0 L 626.0,738.0 L 628.0,736.0 L 630.0,736.0 L 631.0,735.0 L 633.0,735.0 L 634.0,734.0 L 636.0,734.0 L 638.0,732.0 L 639.0,732.0 L 645.0,726.0 L 645.0,725.0 L 648.0,720.0 L 648.0,718.0 L 649.0,717.0 L 649.0,713.0 L 650.0,712.0 L 650.0,703.0 L 649.0,702.0 L 649.0,698.0 L 648.0,697.0 L 648.0,695.0 L 647.0,694.0 L 646.0,691.0 L 644.0,689.0 L 644.0,688.0 L 640.0,684.0 L 639.0,684.0 L 637.0,682.0 L 636.0,682.0 L 633.0,680.0 L 631.0,680.0 L 630.0,679.0 L 626.0,679.0 L 625.0,678.0 Z M 567.0,694.0 L 568.0,693.0 L 625.0,693.0 L 626.0,694.0 L 629.0,695.0 L 632.0,698.0 L 632.0,699.0 L 633.0,700.0 L 633.0,713.0 L 632.0,714.0 L 632.0,716.0 L 630.0,718.0 L 630.0,719.0 L 629.0,720.0 L 628.0,720.0 L 626.0,722.0 L 622.0,722.0 L 621.0,723.0 L 568.0,723.0 L 566.0,721.0 L 566.0,703.0 L 567.0,702.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 398.0,678.0 L 398.0,693.0 L 434.0,693.0 L 435.0,694.0 L 435.0,767.0 L 451.0,767.0 L 451.0,694.0 L 452.0,693.0 L 489.0,693.0 L 489.0,679.0 L 488.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 250.0,678.0 L 250.0,692.0 L 251.0,692.0 L 252.0,693.0 L 339.0,693.0 L 339.0,691.0 L 340.0,690.0 L 340.0,679.0 L 339.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 1134.0,683.0 L 1129.0,688.0 L 1129.0,689.0 L 1127.0,691.0 L 1127.0,693.0 L 1126.0,694.0 L 1126.0,696.0 L 1125.0,697.0 L 1125.0,712.0 L 1126.0,713.0 L 1126.0,715.0 L 1127.0,716.0 L 1128.0,719.0 L 1134.0,725.0 L 1135.0,725.0 L 1137.0,727.0 L 1139.0,727.0 L 1140.0,728.0 L 1143.0,728.0 L 1144.0,729.0 L 1185.0,729.0 L 1186.0,730.0 L 1191.0,730.0 L 1192.0,731.0 L 1193.0,731.0 L 1197.0,735.0 L 1197.0,739.0 L 1198.0,740.0 L 1198.0,742.0 L 1197.0,743.0 L 1197.0,746.0 L 1196.0,747.0 L 1196.0,748.0 L 1193.0,751.0 L 1192.0,751.0 L 1189.0,753.0 L 1184.0,753.0 L 1183.0,754.0 L 1151.0,754.0 L 1150.0,753.0 L 1143.0,753.0 L 1142.0,752.0 L 1139.0,752.0 L 1138.0,751.0 L 1135.0,751.0 L 1134.0,750.0 L 1131.0,750.0 L 1130.0,749.0 L 1128.0,749.0 L 1127.0,748.0 L 1125.0,748.0 L 1125.0,760.0 L 1124.0,761.0 L 1126.0,763.0 L 1128.0,763.0 L 1129.0,764.0 L 1131.0,764.0 L 1132.0,765.0 L 1134.0,765.0 L 1135.0,766.0 L 1139.0,766.0 L 1140.0,767.0 L 1146.0,767.0 L 1147.0,768.0 L 1186.0,768.0 L 1187.0,767.0 L 1191.0,767.0 L 1192.0,766.0 L 1195.0,766.0 L 1196.0,765.0 L 1198.0,765.0 L 1199.0,764.0 L 1201.0,764.0 L 1203.0,762.0 L 1204.0,762.0 L 1209.0,757.0 L 1209.0,756.0 L 1211.0,754.0 L 1211.0,753.0 L 1212.0,752.0 L 1212.0,750.0 L 1213.0,749.0 L 1213.0,734.0 L 1212.0,733.0 L 1212.0,730.0 L 1211.0,729.0 L 1211.0,728.0 L 1209.0,726.0 L 1209.0,725.0 L 1204.0,720.0 L 1203.0,720.0 L 1202.0,719.0 L 1201.0,719.0 L 1196.0,716.0 L 1192.0,716.0 L 1191.0,715.0 L 1148.0,715.0 L 1147.0,714.0 L 1145.0,714.0 L 1142.0,711.0 L 1142.0,710.0 L 1141.0,709.0 L 1141.0,699.0 L 1142.0,698.0 L 1142.0,697.0 L 1145.0,694.0 L 1146.0,694.0 L 1147.0,693.0 L 1150.0,693.0 L 1151.0,692.0 L 1190.0,692.0 L 1191.0,693.0 L 1196.0,693.0 L 1197.0,694.0 L 1201.0,694.0 L 1202.0,695.0 L 1204.0,695.0 L 1205.0,696.0 L 1209.0,696.0 L 1209.0,683.0 L 1206.0,681.0 L 1203.0,681.0 L 1202.0,680.0 L 1198.0,680.0 L 1197.0,679.0 L 1193.0,679.0 L 1192.0,678.0 L 1182.0,678.0 L 1181.0,677.0 L 1156.0,677.0 L 1155.0,678.0 L 1146.0,678.0 L 1145.0,679.0 L 1143.0,679.0 L 1142.0,680.0 L 1140.0,680.0 L 1139.0,681.0 L 1138.0,681.0 L 1137.0,682.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 714.0,691.0 L 714.0,692.0 L 711.0,695.0 L 711.0,696.0 L 709.0,698.0 L 709.0,699.0 L 708.0,700.0 L 708.0,702.0 L 707.0,703.0 L 707.0,706.0 L 706.0,707.0 L 706.0,711.0 L 705.0,712.0 L 705.0,734.0 L 706.0,735.0 L 706.0,739.0 L 707.0,740.0 L 707.0,743.0 L 708.0,744.0 L 708.0,745.0 L 709.0,746.0 L 709.0,747.0 L 710.0,748.0 L 711.0,751.0 L 713.0,753.0 L 713.0,754.0 L 718.0,759.0 L 719.0,759.0 L 721.0,761.0 L 722.0,761.0 L 723.0,762.0 L 724.0,762.0 L 729.0,765.0 L 731.0,765.0 L 732.0,766.0 L 735.0,766.0 L 736.0,767.0 L 741.0,767.0 L 742.0,768.0 L 775.0,768.0 L 776.0,767.0 L 781.0,767.0 L 782.0,766.0 L 785.0,766.0 L 786.0,765.0 L 788.0,765.0 L 789.0,764.0 L 790.0,764.0 L 791.0,763.0 L 792.0,763.0 L 793.0,762.0 L 796.0,761.0 L 798.0,759.0 L 799.0,759.0 L 805.0,753.0 L 805.0,752.0 L 807.0,750.0 L 807.0,749.0 L 809.0,746.0 L 809.0,744.0 L 811.0,741.0 L 811.0,737.0 L 812.0,736.0 L 812.0,711.0 L 811.0,710.0 L 811.0,706.0 L 810.0,705.0 L 810.0,703.0 L 809.0,702.0 L 809.0,700.0 L 808.0,699.0 L 808.0,698.0 L 806.0,696.0 L 806.0,695.0 L 803.0,692.0 L 803.0,691.0 L 799.0,687.0 L 798.0,687.0 L 793.0,683.0 L 792.0,683.0 L 791.0,682.0 L 789.0,682.0 L 786.0,680.0 L 783.0,680.0 L 782.0,679.0 L 779.0,679.0 L 778.0,678.0 L 770.0,678.0 L 769.0,677.0 L 748.0,677.0 L 747.0,678.0 L 739.0,678.0 L 738.0,679.0 L 734.0,679.0 L 733.0,680.0 L 731.0,680.0 L 730.0,681.0 L 728.0,681.0 L 727.0,682.0 L 726.0,682.0 L 725.0,683.0 L 722.0,684.0 L 720.0,686.0 L 719.0,686.0 Z M 725.0,702.0 L 730.0,697.0 L 731.0,697.0 L 733.0,695.0 L 735.0,695.0 L 738.0,693.0 L 742.0,693.0 L 743.0,692.0 L 772.0,692.0 L 773.0,693.0 L 777.0,693.0 L 778.0,694.0 L 781.0,694.0 L 782.0,695.0 L 785.0,696.0 L 787.0,698.0 L 788.0,698.0 L 791.0,701.0 L 791.0,702.0 L 793.0,704.0 L 793.0,705.0 L 794.0,706.0 L 794.0,708.0 L 795.0,709.0 L 795.0,712.0 L 796.0,713.0 L 796.0,733.0 L 795.0,734.0 L 795.0,738.0 L 794.0,739.0 L 793.0,742.0 L 791.0,744.0 L 791.0,745.0 L 788.0,748.0 L 787.0,748.0 L 785.0,750.0 L 784.0,750.0 L 781.0,752.0 L 778.0,752.0 L 777.0,753.0 L 770.0,753.0 L 769.0,754.0 L 747.0,754.0 L 746.0,753.0 L 740.0,753.0 L 739.0,752.0 L 737.0,752.0 L 736.0,751.0 L 734.0,751.0 L 733.0,750.0 L 732.0,750.0 L 730.0,748.0 L 729.0,748.0 L 725.0,744.0 L 725.0,743.0 L 722.0,738.0 L 722.0,734.0 L 721.0,733.0 L 721.0,713.0 L 722.0,712.0 L 722.0,709.0 L 723.0,708.0 L 723.0,706.0 L 724.0,705.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 95.0,678.0 L 95.0,766.0 L 96.0,767.0 L 111.0,767.0 L 111.0,737.0 L 112.0,736.0 L 171.0,736.0 L 172.0,735.0 L 175.0,735.0 L 176.0,734.0 L 178.0,734.0 L 179.0,733.0 L 180.0,733.0 L 182.0,731.0 L 183.0,731.0 L 186.0,728.0 L 187.0,728.0 L 187.0,727.0 L 190.0,724.0 L 190.0,723.0 L 193.0,718.0 L 193.0,715.0 L 194.0,714.0 L 194.0,702.0 L 193.0,701.0 L 193.0,696.0 L 192.0,695.0 L 192.0,693.0 L 191.0,692.0 L 190.0,689.0 L 188.0,687.0 L 188.0,686.0 L 185.0,683.0 L 184.0,683.0 L 182.0,681.0 L 181.0,681.0 L 180.0,680.0 L 178.0,680.0 L 177.0,679.0 L 174.0,679.0 L 173.0,678.0 L 168.0,678.0 L 167.0,677.0 L 156.0,677.0 L 155.0,678.0 L 135.0,678.0 L 134.0,677.0 L 132.0,677.0 L 131.0,678.0 L 108.0,678.0 L 107.0,677.0 L 105.0,677.0 L 104.0,678.0 Z M 111.0,693.0 L 112.0,692.0 L 169.0,692.0 L 170.0,693.0 L 172.0,693.0 L 177.0,698.0 L 177.0,699.0 L 178.0,700.0 L 178.0,712.0 L 177.0,713.0 L 177.0,715.0 L 176.0,716.0 L 176.0,717.0 L 174.0,719.0 L 173.0,719.0 L 170.0,721.0 L 167.0,721.0 L 166.0,722.0 L 154.0,722.0 L 153.0,721.0 L 152.0,722.0 L 115.0,722.0 L 114.0,721.0 L 113.0,722.0 L 111.0,720.0 Z" fill="#FFFFFF" fill-rule="evenodd"/><rect x="250" y="678" width="16" height="90" fill="#FFFFFF"/></g>
|
||||
<path d="M 1084.0,678.0 L 1063.0,678.0 L 1056.0,685.0 L 1056.0,686.0 L 1049.0,693.0 L 1049.0,694.0 L 1042.0,701.0 L 1042.0,702.0 L 1036.0,708.0 L 1036.0,709.0 L 1035.0,710.0 L 1035.0,711.0 L 1033.0,713.0 L 1033.0,714.0 L 1032.0,715.0 L 1032.0,716.0 L 1031.0,717.0 L 1031.0,718.0 L 1030.0,719.0 L 1030.0,720.0 L 1028.0,723.0 L 1028.0,725.0 L 1026.0,728.0 L 1026.0,730.0 L 1025.0,731.0 L 1025.0,733.0 L 1024.0,734.0 L 1024.0,738.0 L 1023.0,739.0 L 1023.0,743.0 L 1022.0,744.0 L 1022.0,752.0 L 1021.0,753.0 L 1021.0,766.0 L 1022.0,767.0 L 1038.0,767.0 L 1038.0,737.0 L 1039.0,736.0 L 1039.0,734.0 L 1040.0,733.0 L 1040.0,732.0 L 1041.0,731.0 L 1041.0,730.0 L 1043.0,728.0 L 1043.0,727.0 L 1046.0,724.0 L 1046.0,723.0 L 1052.0,717.0 L 1052.0,716.0 L 1060.0,708.0 L 1060.0,707.0 L 1065.0,702.0 L 1065.0,701.0 L 1072.0,694.0 L 1072.0,693.0 L 1077.0,688.0 L 1077.0,687.0 L 1082.0,682.0 L 1082.0,681.0 L 1084.0,679.0 L 1085.0,679.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
<path d="M 975.0,678.0 L 975.0,679.0 L 979.0,683.0 L 979.0,684.0 L 983.0,688.0 L 983.0,689.0 L 990.0,696.0 L 990.0,697.0 L 996.0,703.0 L 996.0,704.0 L 1002.0,710.0 L 1002.0,711.0 L 1010.0,719.0 L 1010.0,720.0 L 1014.0,724.0 L 1014.0,725.0 L 1017.0,728.0 L 1017.0,729.0 L 1018.0,730.0 L 1018.0,731.0 L 1019.0,732.0 L 1020.0,732.0 L 1020.0,729.0 L 1021.0,728.0 L 1021.0,726.0 L 1022.0,725.0 L 1022.0,722.0 L 1023.0,721.0 L 1023.0,715.0 L 1024.0,714.0 L 1023.0,713.0 L 1023.0,709.0 L 1022.0,708.0 L 1022.0,707.0 L 1021.0,706.0 L 1021.0,705.0 L 1016.0,700.0 L 1016.0,699.0 L 1010.0,693.0 L 1010.0,692.0 L 1003.0,685.0 L 1003.0,684.0 L 997.0,678.0 Z" fill="#FFFFFF" fill-rule="evenodd"/>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 13 KiB |
@@ -0,0 +1,14 @@
|
||||
# dashboards/
|
||||
|
||||
Exported Superset dashboards and the operator scripts that import them. Not baked into the
|
||||
image, so changes here do not rebuild or restart Superset. Empty until the first fuel
|
||||
dashboards exist.
|
||||
|
||||
Export from a running instance (inside the container, as the server's deploy user):
|
||||
|
||||
docker exec fuel-monitoring-superset-1 superset export-dashboards -f /tmp/dashboards.zip
|
||||
docker cp fuel-monitoring-superset-1:/tmp/dashboards.zip .
|
||||
|
||||
Import is the reverse with `superset import-dashboards -p <zip> -u <username>`. The
|
||||
username must be an existing Superset user, which here means someone who has signed in
|
||||
through Keycloak at least once.
|
||||
@@ -0,0 +1,16 @@
|
||||
# patches/
|
||||
|
||||
Runtime patches applied while the Superset image is built
|
||||
(`stacks/fuel-monitoring/superset/Dockerfile`). Empty until Petrolyst needs one.
|
||||
|
||||
Same convention as media-insights:
|
||||
|
||||
- One file per fix, named `fix_<what>.py`. It patches files inside the installed image
|
||||
(`/app/superset/...` or the venv), for changes that cannot be made through
|
||||
`superset_config.py`.
|
||||
- `python3 fix_<what>.py` applies the patch; `python3 fix_<what>.py --verify` exits non-zero
|
||||
unless the patched code is in place. Both must be idempotent.
|
||||
- The Dockerfile runs both for every `fix_*.py` and stops at the first failure, so a patch that
|
||||
no longer applies to a new Superset version fails the build instead of shipping silently.
|
||||
- Everything in this folder is a build input: changing it gives the image a new tag and
|
||||
recreates the `superset` container on the next deploy.
|
||||
@@ -0,0 +1,92 @@
|
||||
"""Keycloak sign-in for Superset at atoma.petrolyst.com.
|
||||
|
||||
Set as CUSTOM_SECURITY_MANAGER in superset_config.py. Checked against
|
||||
Flask-AppBuilder 5.0.2 as bundled in apache/superset:6.1.0. Three changes to
|
||||
the stock behaviour:
|
||||
|
||||
- User info and roles come from the ID token that Keycloak issues for client
|
||||
"petrolyst" (claim petrolyst_roles). FAB's built-in Keycloak branch would read
|
||||
a "groups" claim from the userinfo endpoint instead.
|
||||
- /login/ goes straight to Keycloak. Superset 6.1 normally registers its React
|
||||
login page on /login/, which only shows a "Sign in with keycloak" button.
|
||||
- /logout/ also ends the Keycloak session, so the next visit asks for the
|
||||
password and OTP again.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import Any, Optional
|
||||
from urllib.parse import urlencode
|
||||
|
||||
from flask import g, redirect, request, url_for
|
||||
from flask_appbuilder import expose
|
||||
from flask_appbuilder.security.views import AuthOAuthView
|
||||
from flask_appbuilder.utils.base import get_safe_redirect
|
||||
from flask_login import logout_user
|
||||
from werkzeug.wrappers import Response as WerkzeugResponse
|
||||
|
||||
from superset.security import SupersetSecurityManager
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
PROVIDER = "keycloak"
|
||||
ROLE_CLAIM = "petrolyst_roles"
|
||||
|
||||
|
||||
class PetrolystAuthOAuthView(AuthOAuthView):
|
||||
@expose("/login/")
|
||||
@expose("/login/<provider>")
|
||||
def login(self, provider: Optional[str] = None) -> WerkzeugResponse:
|
||||
authenticated = g.user is not None and g.user.is_authenticated
|
||||
if provider is None and not authenticated:
|
||||
# FAB encodes the query arguments into the OAuth state and runs
|
||||
# get_safe_redirect on "next" again after the callback; checking it
|
||||
# here as well keeps an unsafe value out of the state altogether.
|
||||
args = {}
|
||||
if request.args.get("next"):
|
||||
args["next"] = get_safe_redirect(request.args["next"])
|
||||
return redirect(url_for(".login", provider=PROVIDER, **args))
|
||||
return super().login(provider)
|
||||
|
||||
@expose("/logout/")
|
||||
def logout(self) -> WerkzeugResponse:
|
||||
logout_user()
|
||||
remote = self.appbuilder.sm.oauth_remotes[PROVIDER]
|
||||
end_session = remote.load_server_metadata().get("end_session_endpoint")
|
||||
if not end_session:
|
||||
logger.warning("Keycloak metadata has no end_session_endpoint")
|
||||
return redirect(self.appbuilder.get_url_for_index)
|
||||
# client_id instead of id_token_hint: Keycloak then asks the user to
|
||||
# confirm the logout once, and no ID token has to live in the cookie.
|
||||
params = {
|
||||
"client_id": remote.client_id,
|
||||
"post_logout_redirect_uri": request.host_url,
|
||||
}
|
||||
return redirect(f"{end_session}?{urlencode(params)}")
|
||||
|
||||
|
||||
class PetrolystSecurityManager(SupersetSecurityManager):
|
||||
authoauthview = PetrolystAuthOAuthView
|
||||
# Without this, Superset registers its React login page on /login/ before
|
||||
# FAB's OAuth view, and that page wins the route.
|
||||
register_superset_auth_view = False
|
||||
|
||||
def oauth_user_info(
|
||||
self, provider: str, response: Optional[dict[str, Any]] = None
|
||||
) -> dict[str, Any]:
|
||||
if provider != PROVIDER:
|
||||
raise ValueError(f"unsupported OAuth provider {provider!r}")
|
||||
# Authlib verifies the ID token (signature, issuer, audience, nonce)
|
||||
# and puts its claims under "userinfo".
|
||||
claims = (response or {}).get("userinfo")
|
||||
if not claims:
|
||||
claims = self.oauth_remotes[provider].userinfo()
|
||||
username = claims.get("preferred_username")
|
||||
if not username:
|
||||
raise ValueError("Keycloak token has no preferred_username")
|
||||
return {
|
||||
"username": username,
|
||||
"email": claims.get("email", ""),
|
||||
"first_name": claims.get("given_name", ""),
|
||||
"last_name": claims.get("family_name", ""),
|
||||
"role_keys": list(claims.get(ROLE_CLAIM) or []),
|
||||
}
|
||||
@@ -0,0 +1,252 @@
|
||||
"""Superset configuration for Petrolyst (ATOMA fuel monitoring).
|
||||
|
||||
Customization only, and no secrets: those are read from /run/secrets/<name>,
|
||||
which stacks/fuel-monitoring/compose.yaml mounts from /opt/petrolyst/secrets/.
|
||||
Baked into the image at /app/pythonpath/, which is on PYTHONPATH in
|
||||
apache/superset:6.1.0, so Superset loads it without SUPERSET_CONFIG_PATH.
|
||||
"""
|
||||
|
||||
import os
|
||||
import pathlib
|
||||
from copy import deepcopy
|
||||
|
||||
from flask_appbuilder.security.manager import AUTH_OAUTH
|
||||
from sqlalchemy.engine.url import URL
|
||||
|
||||
from petrolyst_security import PetrolystSecurityManager
|
||||
from superset import config as superset_defaults
|
||||
|
||||
|
||||
def _read_secret(name: str, env_var: str | None = None) -> str:
|
||||
"""Prefer /run/secrets/<name>; fall back to an env var for local dev.
|
||||
|
||||
Uses rstrip('\r\n') (not strip()) so passwords or keys that legitimately
|
||||
contain leading/trailing spaces are not silently mutated. Raises on empty
|
||||
values so boot fails loudly instead of accepting a blank secret.
|
||||
"""
|
||||
path = pathlib.Path(f"/run/secrets/{name}")
|
||||
if path.is_file():
|
||||
value = path.read_text().rstrip("\r\n")
|
||||
else:
|
||||
value = os.environ[env_var or name.upper()]
|
||||
if not value:
|
||||
raise RuntimeError(f"secret {name!r} must not be empty")
|
||||
return value
|
||||
|
||||
|
||||
SECRET_KEY = _read_secret("superset_secret_key", "SUPERSET_SECRET_KEY")
|
||||
|
||||
SQLALCHEMY_DATABASE_URI = URL.create(
|
||||
"postgresql+psycopg2",
|
||||
username="superset",
|
||||
password=_read_secret("superset_db_password", "SUPERSET_DB_PASSWORD"),
|
||||
host=os.environ.get("SUPERSET_DB_HOST", "superset-db"),
|
||||
port=5432,
|
||||
database="superset",
|
||||
).render_as_string(hide_password=False)
|
||||
|
||||
# --- Sign-in through Keycloak (client "petrolyst") ----------------------------
|
||||
# Users and roles are managed in Keycloak; there is no local admin account.
|
||||
# Break-glass access: `superset fab ...` inside the container on the server.
|
||||
KEYCLOAK_REALM_URL = os.environ.get(
|
||||
"KEYCLOAK_REALM_URL", "https://iam.thedigitalbridge.io/realms/thedigitalbridge"
|
||||
).rstrip("/")
|
||||
|
||||
AUTH_TYPE = AUTH_OAUTH
|
||||
CUSTOM_SECURITY_MANAGER = PetrolystSecurityManager
|
||||
OAUTH_PROVIDERS = [
|
||||
{
|
||||
"name": "keycloak",
|
||||
"icon": "fa-key",
|
||||
"token_key": "access_token",
|
||||
"remote_app": {
|
||||
"client_id": os.environ.get("KEYCLOAK_CLIENT_ID", "petrolyst"),
|
||||
"client_secret": _read_secret(
|
||||
"keycloak_petrolyst_client_secret", "KEYCLOAK_CLIENT_SECRET"
|
||||
),
|
||||
"server_metadata_url": f"{KEYCLOAK_REALM_URL}/.well-known/openid-configuration",
|
||||
"api_base_url": f"{KEYCLOAK_REALM_URL}/protocol/openid-connect/",
|
||||
"client_kwargs": {
|
||||
"scope": "openid email profile",
|
||||
"code_challenge_method": "S256",
|
||||
},
|
||||
},
|
||||
}
|
||||
]
|
||||
|
||||
# Keys are Keycloak client roles of "petrolyst" (claim petrolyst_roles).
|
||||
AUTH_ROLES_MAPPING = {
|
||||
"superset-admin": ["Admin"],
|
||||
"superset-editor": ["Alpha"],
|
||||
"superset-viewer": ["Gamma"],
|
||||
}
|
||||
AUTH_ROLES_SYNC_AT_LOGIN = True
|
||||
AUTH_USER_REGISTRATION = True
|
||||
# Public has no permissions: a Keycloak user without a Superset role signs in
|
||||
# and sees nothing.
|
||||
AUTH_USER_REGISTRATION_ROLE = "Public"
|
||||
|
||||
# Viewers (Gamma) see a dashboard once it lists their role.
|
||||
FEATURE_FLAGS = {"DASHBOARD_RBAC": True}
|
||||
|
||||
# --- Behind Traefik -----------------------------------------------------------
|
||||
ENABLE_PROXY_FIX = True
|
||||
PREFERRED_URL_SCHEME = "https"
|
||||
SESSION_COOKIE_SECURE = True
|
||||
SESSION_COOKIE_HTTPONLY = True
|
||||
SESSION_COOKIE_SAMESITE = "Lax"
|
||||
TALISMAN_CONFIG = deepcopy(superset_defaults.TALISMAN_CONFIG)
|
||||
TALISMAN_CONFIG["session_cookie_secure"] = True
|
||||
|
||||
# --- Redis: caches and rate limits (no Celery, so no results backend) ---------
|
||||
REDIS_URL = os.environ.get("REDIS_URL", "redis://redis:6379")
|
||||
|
||||
CACHE_CONFIG = {
|
||||
"CACHE_TYPE": "RedisCache",
|
||||
"CACHE_DEFAULT_TIMEOUT": 300,
|
||||
"CACHE_KEY_PREFIX": "superset_cache_",
|
||||
"CACHE_REDIS_URL": f"{REDIS_URL}/0",
|
||||
}
|
||||
DATA_CACHE_CONFIG = {
|
||||
"CACHE_TYPE": "RedisCache",
|
||||
"CACHE_DEFAULT_TIMEOUT": 3600,
|
||||
"CACHE_KEY_PREFIX": "superset_data_",
|
||||
"CACHE_REDIS_URL": f"{REDIS_URL}/0",
|
||||
}
|
||||
FILTER_STATE_CACHE_CONFIG = {
|
||||
"CACHE_TYPE": "RedisCache",
|
||||
"CACHE_DEFAULT_TIMEOUT": 86400,
|
||||
"CACHE_KEY_PREFIX": "superset_filter_",
|
||||
"CACHE_REDIS_URL": f"{REDIS_URL}/0",
|
||||
}
|
||||
EXPLORE_FORM_DATA_CACHE_CONFIG = {
|
||||
"CACHE_TYPE": "RedisCache",
|
||||
"CACHE_DEFAULT_TIMEOUT": 86400,
|
||||
"CACHE_KEY_PREFIX": "superset_explore_",
|
||||
"CACHE_REDIS_URL": f"{REDIS_URL}/0",
|
||||
}
|
||||
RATELIMIT_STORAGE_URI = f"{REDIS_URL}/1"
|
||||
|
||||
# --- Petrolyst look and feel --------------------------------------------------
|
||||
# Values from petrolyst-ui/design/brand/07_UI_Tokens/petrolyst_tokens.css.
|
||||
# Assets live in assets/branding/ and are served from /static/assets/branding/.
|
||||
BRANDING = "/static/assets/branding"
|
||||
|
||||
APP_NAME = "Petrolyst"
|
||||
APP_ICON = f"{BRANDING}/petrolyst_logo_standard_fullcolor_no_tagline.svg"
|
||||
FAVICONS = [
|
||||
{"href": f"{BRANDING}/favicon.svg", "type": "image/svg+xml"},
|
||||
{"href": f"{BRANDING}/favicon.ico", "type": "image/x-icon"},
|
||||
{"href": f"{BRANDING}/petrolyst_icon_180.png", "rel": "apple-touch-icon"},
|
||||
]
|
||||
|
||||
THEME_DEFAULT = deepcopy(superset_defaults.THEME_DEFAULT)
|
||||
THEME_DEFAULT["token"].update(
|
||||
{
|
||||
"brandAppName": APP_NAME,
|
||||
"brandLogoAlt": "Petrolyst",
|
||||
"brandLogoUrl": APP_ICON,
|
||||
"brandLogoHref": "/",
|
||||
# The standard logo is stacked (mark over wordmark); 40px keeps the Y
|
||||
# legible and, with the margin, the 60px navbar height.
|
||||
"brandLogoHeight": "40px",
|
||||
"brandLogoMargin": "10px 0",
|
||||
# Action teal, not the raw brand teal #009688: the brand guide keeps
|
||||
# raw teal off small text and controls.
|
||||
"colorPrimary": "#00695C",
|
||||
"colorLink": "#00695C",
|
||||
"colorSuccess": "#0F7B6C",
|
||||
"colorWarning": "#B45309",
|
||||
"colorError": "#B3261E",
|
||||
"colorInfo": "#2563EB",
|
||||
"colorTextBase": "#0B1D33",
|
||||
"colorBgLayout": "#F8FAFC",
|
||||
"colorBorder": "#E2E8F0",
|
||||
"borderRadius": 8,
|
||||
"fontFamily": "Inter, system-ui, sans-serif",
|
||||
}
|
||||
)
|
||||
|
||||
THEME_DARK = deepcopy(THEME_DEFAULT)
|
||||
THEME_DARK["algorithm"] = "dark"
|
||||
THEME_DARK["token"].update(
|
||||
{
|
||||
"brandLogoUrl": f"{BRANDING}/petrolyst_logo_standard_white_no_tagline.svg",
|
||||
"colorPrimary": "#35C5B6",
|
||||
"colorLink": "#35C5B6",
|
||||
"colorSuccess": "#35C5B6",
|
||||
"colorWarning": "#F59E0B",
|
||||
"colorError": "#FF6B6B",
|
||||
"colorInfo": "#60A5FA",
|
||||
"colorTextBase": "#F8FAFC",
|
||||
"colorBgBase": "#071425",
|
||||
"colorBgLayout": "#071425",
|
||||
"colorBgContainer": "#12263D",
|
||||
"colorBorder": "#526D82",
|
||||
"colorEditorSelection": superset_defaults.THEME_DARK["token"][
|
||||
"colorEditorSelection"
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
# Chart colours. Validated with the dataviz skill's validator on both chart
|
||||
# surfaces (#FFFFFF light, #12263D dark): all slots in both lightness bands and
|
||||
# >= 3:1; worst adjacent CVD dE 13.7, normal-vision dE 26.2; the first three
|
||||
# slots also pass all-pairs (scatter, maps). Brand teal leads; brand cyan
|
||||
# #00B6E6 is snapped to #0097C4 because it is too light for the dark surface
|
||||
# and below 3:1 on white. The other hues come from the validated reference set.
|
||||
# Status meaning (warning, critical) never rides on these colours.
|
||||
EXTRA_CATEGORICAL_COLOR_SCHEMES = [
|
||||
{
|
||||
"id": "petrolyst",
|
||||
"description": "Petrolyst categorical palette, brand teal first",
|
||||
"label": "Petrolyst",
|
||||
"isDefault": True,
|
||||
"colors": [
|
||||
"#009688", # brand teal
|
||||
"#d95926", # orange
|
||||
"#9085e9", # violet
|
||||
"#c98500", # amber
|
||||
"#0097C4", # brand cyan, snapped
|
||||
"#e34948", # red
|
||||
"#3987e5", # blue
|
||||
"#d55181", # magenta
|
||||
],
|
||||
}
|
||||
]
|
||||
|
||||
# One hue, light to dark; the lightest step means "near zero" and may recede.
|
||||
EXTRA_SEQUENTIAL_COLOR_SCHEMES = [
|
||||
{
|
||||
"id": "petrolyst_teal",
|
||||
"description": "Petrolyst teal, light to dark",
|
||||
"label": "Petrolyst Teal",
|
||||
"isDiverging": False,
|
||||
"isDefault": True,
|
||||
"colors": [
|
||||
"#E6F4F2",
|
||||
"#B3DED8",
|
||||
"#7FC8BE",
|
||||
"#4CB2A4",
|
||||
"#009688",
|
||||
"#00695C",
|
||||
"#004D44",
|
||||
],
|
||||
},
|
||||
{
|
||||
"id": "petrolyst_navy",
|
||||
"description": "Petrolyst navy, light to dark",
|
||||
"label": "Petrolyst Navy",
|
||||
"isDiverging": False,
|
||||
"isDefault": False,
|
||||
"colors": [
|
||||
"#E8EEF5",
|
||||
"#C3D1E2",
|
||||
"#8FA7C4",
|
||||
"#5C7DA6",
|
||||
"#2F5585",
|
||||
"#183A63",
|
||||
"#0B1D33",
|
||||
],
|
||||
},
|
||||
]
|
||||
@@ -0,0 +1,174 @@
|
||||
# Fuel-monitoring analytics: Superset on atoma.petrolyst.com, reading the
|
||||
# operational store (fuel-db) through Trino. Run only through deploy/deploy.sh
|
||||
# or deploy/rollback.sh: they export SUPERSET_IMAGE_TAG and TRINO_CONFIG_HASH
|
||||
# from the release's ARTIFACTS file, and the :? guards below stop Compose from
|
||||
# running with either unset.
|
||||
name: fuel-monitoring
|
||||
|
||||
x-superset-env: &superset-env
|
||||
KEYCLOAK_REALM_URL: ${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/thedigitalbridge}
|
||||
|
||||
x-superset: &superset
|
||||
image: fuel-superset:${SUPERSET_IMAGE_TAG:?SUPERSET_IMAGE_TAG unset - run through deploy/deploy.sh}
|
||||
pull_policy: never
|
||||
secrets:
|
||||
- superset_secret_key
|
||||
- superset_db_password
|
||||
- keycloak_petrolyst_client_secret
|
||||
|
||||
services:
|
||||
superset:
|
||||
<<: *superset
|
||||
build:
|
||||
context: ./superset
|
||||
additional_contexts:
|
||||
insights: ../../petrolyst-insights
|
||||
environment:
|
||||
<<: *superset-env
|
||||
SERVER_WORKER_AMOUNT: "2"
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
superset-db:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- backend
|
||||
- proxy
|
||||
mem_limit: 1536m
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-fsS", "http://127.0.0.1:8088/health"]
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
start_period: 60s
|
||||
labels:
|
||||
traefik.enable: "true"
|
||||
traefik.docker.network: proxy
|
||||
traefik.http.routers.fm-superset.rule: Host(`${ATOMA_HOST:-atoma.petrolyst.com}`)
|
||||
traefik.http.routers.fm-superset.entrypoints: websecure
|
||||
traefik.http.routers.fm-superset.tls.certresolver: le
|
||||
traefik.http.routers.fm-superset.middlewares: secure-headers@file
|
||||
traefik.http.services.fm-superset.loadbalancer.server.port: "8088"
|
||||
|
||||
# One-shot: never started by `up`; deploy.sh runs it with `compose run --rm`.
|
||||
superset-init:
|
||||
<<: *superset
|
||||
profiles: ["init"]
|
||||
environment: *superset-env
|
||||
command: ["/bin/bash", "/app/petrolyst-init.sh"]
|
||||
depends_on:
|
||||
superset-db:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- backend
|
||||
restart: "no"
|
||||
|
||||
superset-db:
|
||||
image: postgres:17.11-alpine
|
||||
environment:
|
||||
POSTGRES_USER: superset
|
||||
POSTGRES_DB: superset
|
||||
POSTGRES_PASSWORD_FILE: /run/secrets/superset_db_password
|
||||
secrets:
|
||||
- superset_db_password
|
||||
volumes:
|
||||
- superset_db_data:/var/lib/postgresql/data
|
||||
networks:
|
||||
- backend
|
||||
restart: unless-stopped
|
||||
mem_limit: 512m
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U superset -d superset"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
|
||||
# Cache and rate-limit store only; nothing to persist.
|
||||
redis:
|
||||
image: redis:7.4.11-alpine
|
||||
command: ["redis-server", "--maxmemory", "128mb", "--maxmemory-policy", "allkeys-lru", "--save", "", "--appendonly", "no"]
|
||||
networks:
|
||||
- backend
|
||||
restart: unless-stopped
|
||||
mem_limit: 256m
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
|
||||
trino:
|
||||
image: trinodb/trino:484
|
||||
env_file:
|
||||
- ${PETROLYST_ROOT:-/opt/petrolyst}/env/fuel-monitoring.env
|
||||
volumes:
|
||||
- ./trino/etc:/etc/trino:ro
|
||||
labels:
|
||||
# Compose cannot see content changes in a bind mount; this label changes
|
||||
# with trino/etc, so exactly a config change recreates the container.
|
||||
petrolyst.config-hash: ${TRINO_CONFIG_HASH:?TRINO_CONFIG_HASH unset - run through deploy/deploy.sh}
|
||||
depends_on:
|
||||
fuel-db:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- backend
|
||||
restart: unless-stopped
|
||||
mem_limit: 3g
|
||||
# The image's own healthcheck (/usr/lib/trino/bin/health-check) applies.
|
||||
|
||||
fuel-db:
|
||||
image: timescale/timescaledb:2.31.0-pg17
|
||||
environment:
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_DB: fuel
|
||||
POSTGRES_PASSWORD_FILE: /run/secrets/fuel_db_password
|
||||
TS_TUNE_NUM_CPUS: "2"
|
||||
secrets:
|
||||
- fuel_db_password
|
||||
- fuel_db_trino_password
|
||||
volumes:
|
||||
- fuel_db_data:/var/lib/postgresql/data
|
||||
# A single file, not the folder: the image's own TimescaleDB init scripts
|
||||
# live in the same directory and must stay visible.
|
||||
- ./fuel-db/init/01-roles.sh:/docker-entrypoint-initdb.d/010_petrolyst_roles.sh:ro
|
||||
networks:
|
||||
- backend
|
||||
- fuel-db-tunnel
|
||||
# Loopback only, for `ssh -L 5433:127.0.0.1:5433 petrolyst`.
|
||||
ports:
|
||||
- "127.0.0.1:5433:5432"
|
||||
restart: unless-stopped
|
||||
mem_limit: 1536m
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U postgres -d fuel"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
|
||||
networks:
|
||||
backend:
|
||||
internal: true
|
||||
proxy:
|
||||
external: true
|
||||
# A port cannot be published from an internal network; this one exists only
|
||||
# so fuel-db's loopback port works.
|
||||
fuel-db-tunnel: {}
|
||||
|
||||
volumes:
|
||||
superset_db_data:
|
||||
fuel_db_data:
|
||||
|
||||
secrets:
|
||||
superset_secret_key:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/superset_secret_key
|
||||
superset_db_password:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/superset_db_password
|
||||
fuel_db_password:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/fuel_db_password
|
||||
fuel_db_trino_password:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/fuel_db_trino_password
|
||||
keycloak_petrolyst_client_secret:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/shared/keycloak_petrolyst_client_secret
|
||||
+25
@@ -0,0 +1,25 @@
|
||||
#!/usr/bin/env bash
|
||||
# Runs once, when the fuel-db volume is first initialised (mounted after the
|
||||
# image's own TimescaleDB scripts, as 010_petrolyst_roles.sh).
|
||||
# fuel_owner owns database "fuel", its public schema and every table in it.
|
||||
# Loaders log in as their own role but act as fuel_owner (README
|
||||
# "Data loaders"): the default privileges below cover only what
|
||||
# fuel_owner itself creates, not what its members create.
|
||||
# trino_ro what Trino connects as: read-only, also on tables created later.
|
||||
set -euo pipefail
|
||||
|
||||
trino_password=$(cat /run/secrets/fuel_db_trino_password)
|
||||
|
||||
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" \
|
||||
-v trino_password="$trino_password" <<'SQL'
|
||||
CREATE ROLE fuel_owner NOLOGIN;
|
||||
ALTER DATABASE fuel OWNER TO fuel_owner;
|
||||
REVOKE CREATE ON SCHEMA public FROM PUBLIC;
|
||||
|
||||
CREATE ROLE trino_ro LOGIN PASSWORD :'trino_password';
|
||||
GRANT CONNECT ON DATABASE fuel TO trino_ro;
|
||||
GRANT USAGE ON SCHEMA public TO trino_ro;
|
||||
GRANT SELECT ON ALL TABLES IN SCHEMA public TO trino_ro;
|
||||
ALTER DEFAULT PRIVILEGES FOR ROLE fuel_owner IN SCHEMA public
|
||||
GRANT SELECT ON TABLES TO trino_ro;
|
||||
SQL
|
||||
@@ -0,0 +1,28 @@
|
||||
# Petrolyst Superset image. Deployment only: every customization is copied from
|
||||
# petrolyst-insights/, which compose.yaml passes as the named build context
|
||||
# "insights". deploy/deploy.sh tags the result with a hash of exactly these
|
||||
# inputs (this file plus what is copied below), so the tag changes only when
|
||||
# the image would.
|
||||
FROM apache/superset:6.1.0@sha256:16b50bbef6648912a79e3293d418fefa743ce555d36c5af6b85d859119ed7f88
|
||||
|
||||
USER root
|
||||
# The 6.1.0 image ships neither an OAuth client nor the Postgres and Trino
|
||||
# drivers.
|
||||
RUN uv pip install --no-cache --python /app/.venv/bin/python \
|
||||
authlib==1.9.1 psycopg2-binary==2.9.13 "trino[sqlalchemy]==0.340.0"
|
||||
|
||||
COPY --from=insights --chown=superset:superset superset_config.py petrolyst_security.py /app/pythonpath/
|
||||
COPY --from=insights --chown=superset:superset assets/branding/ /app/superset/static/assets/branding/
|
||||
COPY --from=insights patches/ /app/patches/petrolyst/
|
||||
# Stop at the first patch or verify that fails, so a later success cannot hide
|
||||
# an earlier failure.
|
||||
RUN for p in /app/patches/petrolyst/fix_*.py; do \
|
||||
[ -e "$p" ] || continue; \
|
||||
{ python3 "$p" && python3 "$p" --verify; } || exit 1; \
|
||||
done
|
||||
|
||||
# Deployment, not customization: in the image (and so in the tag) rather than
|
||||
# bind-mounted, so a change to it makes deploy.sh run superset-init again.
|
||||
COPY init.sh /app/petrolyst-init.sh
|
||||
|
||||
USER superset
|
||||
Executable
+14
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env bash
|
||||
# superset-init: migrate the metadata database, sync roles and permissions, and
|
||||
# register the Trino connection. It is copied into the Superset image, so a
|
||||
# change here changes the image tag; deploy/deploy.sh runs it on a fresh install
|
||||
# and whenever that tag changes. Every step is idempotent. Unlike the
|
||||
# upstream docker-init.sh it never creates a local admin: users come from
|
||||
# Keycloak.
|
||||
set -euo pipefail
|
||||
|
||||
superset db upgrade
|
||||
superset init
|
||||
superset set-database-uri \
|
||||
--database_name "Fuel (Trino)" \
|
||||
--uri "trino://superset@trino:8080/fuel"
|
||||
@@ -0,0 +1,3 @@
|
||||
# Analytics never writes through Trino. fuel-db's trino_ro role is read-only as
|
||||
# well; this is the second lock.
|
||||
access-control.name=read-only
|
||||
@@ -0,0 +1,5 @@
|
||||
# The fuel-monitoring operational store (service fuel-db), read-only.
|
||||
connector.name=postgresql
|
||||
connection-url=jdbc:postgresql://fuel-db:5432/fuel
|
||||
connection-user=trino_ro
|
||||
connection-password=${ENV:FUEL_DB_TRINO_PASSWORD}
|
||||
@@ -0,0 +1,14 @@
|
||||
# Single node: coordinator and worker in one process. Reachable only on the
|
||||
# backend network (Superset); no port is published and Traefik has no route.
|
||||
coordinator=true
|
||||
node-scheduler.include-coordinator=true
|
||||
http-server.http.port=8080
|
||||
discovery.uri=http://localhost:8080
|
||||
catalog.management=static
|
||||
|
||||
# Memory bounds. The container is capped at 3g (compose.yaml) and jvm.config
|
||||
# gives the heap 75 % of it (~2.25 GB); 1 GB for queries plus 512 MB headroom
|
||||
# leaves room for the rest of the heap.
|
||||
query.max-memory=1GB
|
||||
query.max-memory-per-node=1GB
|
||||
memory.heap-headroom-per-node=512MB
|
||||
@@ -0,0 +1,14 @@
|
||||
-server
|
||||
-agentpath:/usr/lib/trino/bin/libjvmkill.so
|
||||
-XX:InitialRAMPercentage=75
|
||||
-XX:MaxRAMPercentage=75
|
||||
-XX:G1HeapRegionSize=32M
|
||||
-XX:+ExplicitGCInvokesConcurrent
|
||||
-XX:+HeapDumpOnOutOfMemoryError
|
||||
-XX:+ExitOnOutOfMemoryError
|
||||
-XX:-OmitStackTraceInFastThrow
|
||||
-XX:ReservedCodeCacheSize=256M
|
||||
-XX:PerMethodRecompilationCutoff=10000
|
||||
-XX:PerBytecodeRecompilationCutoff=10000
|
||||
-Djdk.attach.allowAttachSelf=true
|
||||
-Djdk.nio.maxCachedBufferSize=2000000
|
||||
@@ -0,0 +1 @@
|
||||
io.trino=INFO
|
||||
@@ -0,0 +1,3 @@
|
||||
node.environment=petrolyst
|
||||
node.data-dir=/data/trino
|
||||
node.id=${ENV:HOSTNAME}
|
||||
@@ -0,0 +1,117 @@
|
||||
# Edge proxy for the whole server. Deployed first by deploy/deploy.sh, which
|
||||
# exports TRAEFIK_CONFIG_HASH from the release's ARTIFACTS file.
|
||||
#
|
||||
# Any stack joins by attaching a service to the external network "proxy" and
|
||||
# setting traefik.* labels; apps without their own login can add the "sso"
|
||||
# middleware (Keycloak client petrolyst, role checked by oauth2-proxy).
|
||||
name: traefik
|
||||
|
||||
services:
|
||||
socket-proxy:
|
||||
image: tecnativa/docker-socket-proxy:v0.5.0
|
||||
environment:
|
||||
CONTAINERS: "1"
|
||||
EVENTS: "1"
|
||||
PING: "1"
|
||||
VERSION: "1"
|
||||
POST: "0"
|
||||
LOG_LEVEL: warning
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
networks:
|
||||
- socket
|
||||
restart: unless-stopped
|
||||
mem_limit: 64m
|
||||
|
||||
traefik:
|
||||
image: traefik:v3.7.16
|
||||
command: ["--configFile=/etc/traefik/static/traefik.yml"]
|
||||
ports:
|
||||
- "80:80"
|
||||
- "443:443"
|
||||
# Folders, not single files: promotion replaces files by renaming, which a
|
||||
# single-file bind mount would not follow.
|
||||
volumes:
|
||||
- ./static:/etc/traefik/static:ro
|
||||
- ./dynamic:/etc/traefik/dynamic:ro
|
||||
- letsencrypt:/letsencrypt
|
||||
depends_on:
|
||||
- socket-proxy
|
||||
networks:
|
||||
- proxy
|
||||
- socket
|
||||
restart: unless-stopped
|
||||
mem_limit: 256m
|
||||
healthcheck:
|
||||
test: ["CMD", "traefik", "healthcheck", "--configFile=/etc/traefik/static/traefik.yml"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 6
|
||||
labels:
|
||||
petrolyst.config-hash: ${TRAEFIK_CONFIG_HASH:?TRAEFIK_CONFIG_HASH unset - run through deploy/deploy.sh}
|
||||
traefik.enable: "true"
|
||||
traefik.http.routers.dashboard.rule: Host(`${TRAEFIK_HOST:-traefik.petrolyst.com}`)
|
||||
traefik.http.routers.dashboard.entrypoints: websecure
|
||||
traefik.http.routers.dashboard.tls.certresolver: le
|
||||
traefik.http.routers.dashboard.service: api@internal
|
||||
traefik.http.routers.dashboard.middlewares: sso,secure-headers@file
|
||||
# Keycloak gate for the dashboard host only (cookie, return hosts and
|
||||
# role are tied to it; README "Adding a project"): unauthenticated
|
||||
# requests are sent to Keycloak, users without petrolyst:traefik-admin
|
||||
# get 403.
|
||||
traefik.http.middlewares.sso.forwardauth.address: http://oauth2-proxy:4180/
|
||||
traefik.http.middlewares.sso.forwardauth.trustForwardHeader: "true"
|
||||
traefik.http.middlewares.sso.forwardauth.authResponseHeaders: X-Auth-Request-User,X-Auth-Request-Email
|
||||
|
||||
oauth2-proxy:
|
||||
image: quay.io/oauth2-proxy/oauth2-proxy:v7.15.5
|
||||
command:
|
||||
- --provider=keycloak-oidc
|
||||
- --oidc-issuer-url=${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/thedigitalbridge}
|
||||
- --client-id=petrolyst
|
||||
- --client-secret-file=/run/secrets/keycloak_petrolyst_client_secret
|
||||
- --cookie-secret-file=/run/secrets/oauth2_proxy_cookie_secret
|
||||
- --code-challenge-method=S256
|
||||
- --allowed-role=petrolyst:traefik-admin
|
||||
# Required by oauth2-proxy at startup; access is decided by the role above.
|
||||
- --email-domain=*
|
||||
- --redirect-url=https://${TRAEFIK_HOST:-traefik.petrolyst.com}/oauth2/callback
|
||||
- --whitelist-domain=${TRAEFIK_HOST:-traefik.petrolyst.com}
|
||||
- --cookie-domain=${TRAEFIK_HOST:-traefik.petrolyst.com}
|
||||
- --cookie-secure=true
|
||||
- --cookie-samesite=lax
|
||||
- --reverse-proxy=true
|
||||
- --upstream=static://202
|
||||
- --skip-provider-button=true
|
||||
- --set-xauthrequest=true
|
||||
- --http-address=0.0.0.0:4180
|
||||
secrets:
|
||||
- keycloak_petrolyst_client_secret
|
||||
- oauth2_proxy_cookie_secret
|
||||
networks:
|
||||
- proxy
|
||||
restart: unless-stopped
|
||||
mem_limit: 128m
|
||||
labels:
|
||||
traefik.enable: "true"
|
||||
traefik.docker.network: proxy
|
||||
traefik.http.routers.oauth2.rule: Host(`${TRAEFIK_HOST:-traefik.petrolyst.com}`) && PathPrefix(`/oauth2/`)
|
||||
traefik.http.routers.oauth2.entrypoints: websecure
|
||||
traefik.http.routers.oauth2.tls.certresolver: le
|
||||
traefik.http.routers.oauth2.middlewares: secure-headers@file
|
||||
traefik.http.services.oauth2.loadbalancer.server.port: "4180"
|
||||
|
||||
networks:
|
||||
proxy:
|
||||
external: true
|
||||
socket:
|
||||
internal: true
|
||||
|
||||
volumes:
|
||||
letsencrypt:
|
||||
|
||||
secrets:
|
||||
keycloak_petrolyst_client_secret:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/shared/keycloak_petrolyst_client_secret
|
||||
oauth2_proxy_cookie_secret:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/traefik/oauth2_proxy_cookie_secret
|
||||
@@ -0,0 +1,10 @@
|
||||
http:
|
||||
middlewares:
|
||||
# Response headers for every public router (reference as secure-headers@file).
|
||||
secure-headers:
|
||||
headers:
|
||||
stsSeconds: 31536000
|
||||
contentTypeNosniff: true
|
||||
referrerPolicy: strict-origin-when-cross-origin
|
||||
customFrameOptionsValue: SAMEORIGIN
|
||||
permissionsPolicy: "camera=(), microphone=(), geolocation=()"
|
||||
@@ -0,0 +1,7 @@
|
||||
# Applies to every router: TLS 1.2 or newer (Exhibit B), and no certificate for
|
||||
# requests that do not name one of our hosts.
|
||||
tls:
|
||||
options:
|
||||
default:
|
||||
minVersion: VersionTLS12
|
||||
sniStrict: true
|
||||
@@ -0,0 +1,56 @@
|
||||
# Traefik static configuration: the server-wide edge for every stack under
|
||||
# stacks/. Changing anything in static/ changes TRAEFIK_CONFIG_HASH, so deploy.sh
|
||||
# recreates the container; ../dynamic/ is watched and applies without a restart.
|
||||
global:
|
||||
checkNewVersion: false
|
||||
sendAnonymousUsage: false
|
||||
|
||||
entryPoints:
|
||||
web:
|
||||
address: ":80"
|
||||
http:
|
||||
redirections:
|
||||
entryPoint:
|
||||
to: websecure
|
||||
scheme: https
|
||||
permanent: true
|
||||
websecure:
|
||||
address: ":443"
|
||||
# Internal only (not published): serves /ping for the healthcheck.
|
||||
traefik:
|
||||
address: ":8080"
|
||||
|
||||
certificatesResolvers:
|
||||
le:
|
||||
acme:
|
||||
# No e-mail: Let's Encrypt no longer sends expiry mail, and renewal is
|
||||
# automatic. deploy.sh checks the certificates on every deploy.
|
||||
storage: /letsencrypt/acme.json
|
||||
httpChallenge:
|
||||
entryPoint: web
|
||||
|
||||
# The dashboard is served at traefik.petrolyst.com through api@internal, behind
|
||||
# Keycloak (labels on the traefik service in compose.yaml). No insecure port.
|
||||
api:
|
||||
dashboard: true
|
||||
|
||||
ping:
|
||||
entryPoint: traefik
|
||||
|
||||
providers:
|
||||
docker:
|
||||
# Through docker-socket-proxy (read-only container listing and events);
|
||||
# Traefik never mounts the Docker socket itself.
|
||||
endpoint: tcp://socket-proxy:2375
|
||||
exposedByDefault: false
|
||||
network: proxy
|
||||
file:
|
||||
directory: /etc/traefik/dynamic
|
||||
watch: true
|
||||
|
||||
log:
|
||||
level: INFO
|
||||
format: json
|
||||
|
||||
accessLog:
|
||||
format: json
|
||||
Reference in new issue
Block a user