Sign-in by customer: realm petrolyst, row rules, guest role (PD-21)
Keycloak import for realm petrolyst: two clients, petrolyst with MFA for Superset and Traefik, petrolyst-app password only. It lists only app.petrolyst.com and traefik.petrolyst.com, and no customer names, as this repository is public. public.v_operator shows each customer's group. Dashboards overview and site, role Petrolyst Guest, filtered by row. Superset refuses a user with no mapped role. The realm travels with the release: each release's compose files name their realm and secret file, and a rollback needs only its target's secret. Not deployed. Checks earlier on this content: tests/run.sh 109 of 109 and read-only, dashboards import twice, column drift 0, compose config on old and new trees, the secret check on a scratch folder, a rehearsal on a throwaway Keycloak 26.4. Never a real deploy or rollback; shellcheck not installed today, clean when last run. Codex inspected once; its fixes are not re-inspected, on the owner's word. Bands: persistence, Superset. Rules: 1, 2, 3, 6, 14. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Nasim SadatandClaude Opus 5.5 committed
1 parent
ab553526f6
commit
78a498f1ce
20 files changed
+279
-78
No files matched your search
@@ -12,7 +12,7 @@ keep the rules there; a change against a rule or an open conflict goes to the ow
|
||||
- `stacks/`, `traefik/`, `deploy/`, `host/`: how and where things run.
|
||||
- `petrolyst-insights/`: what Superset looks like and does (config, security manager, the guest
|
||||
role, branding, patches, dashboards as code). Its README maps each file to the image.
|
||||
- `keycloak/`: the `petrolyst` client import.
|
||||
- `keycloak/`: the import of realm `petrolyst`'s two clients (`petrolyst`, `petrolyst-app`).
|
||||
|
||||
## Rules
|
||||
- **Commit messages** end their body with the bands and rules touched, for example
|
||||
@@ -26,7 +26,8 @@ keep the rules there; a change against a rule or an open conflict goes to the ow
|
||||
export. `import_bundle.py` never deletes; old objects go by hand (`dashboards/README.md`).
|
||||
- **Embedding** stays off in production (`superset_config.py`) until PD-12's points are decided;
|
||||
the local stack turns it on in its own override.
|
||||
- **ATOMA's users** have no Superset role (PD-13): `atoma-reader` opens the Petrolyst app only.
|
||||
- **Customers' users** have no Superset account (PD-13, PD-21): role `reader` of client `petrolyst-app`
|
||||
opens the Petrolyst app only, for the customer of their Keycloak group; Superset refuses them.
|
||||
- fuel-db migrations (`stacks/fuel-monitoring/fuel-db/migrations/`) are applied by hand (P3).
|
||||
|
||||
## Checks
|
||||
|
||||
@@ -21,7 +21,7 @@ Customization and deployment are separate, as with media-insights and media-inte
|
||||
| `stacks/fuel-monitoring/` | Compose stack, Superset Dockerfile and init, fuel-db roles and migrations |
|
||||
| `deploy/` | `deploy.sh` (run on the server by a manual deploy), `rollback.sh` (manual), `lib.sh` (shared) |
|
||||
| `host/` | one-time server setup: `bootstrap.sh`, `secrets-init.sh` |
|
||||
| `keycloak/` | the `petrolyst` client for Keycloak's partial import |
|
||||
| `keycloak/` | the clients `petrolyst` and `petrolyst-app` for Keycloak's partial import |
|
||||
|
||||
No credential is in this repository. Secrets exist only on the server, under
|
||||
`/opt/petrolyst/secrets/`. A deploy needs only root SSH access to the server (`ssh petrolyst`).
|
||||
@@ -88,16 +88,23 @@ Create the private repository `Petrolyst/petrolyst-infra` and push this reposito
|
||||
it. Gitea only keeps the history: deploys are manual, so it needs no runner, no Actions and no
|
||||
deploy secrets.
|
||||
|
||||
### 2. Keycloak (realm `thedigitalbridge` on iam.thedigitalbridge.io)
|
||||
### 2. Keycloak (realm `petrolyst` on iam.thedigitalbridge.io)
|
||||
|
||||
1. Realm settings → Action → **Partial import**: choose `keycloak/petrolyst-partial-import.json`,
|
||||
1. Realm settings → Action → **Partial import**: choose `keycloak/petrolyst-realm-import.json`,
|
||||
and "Skip" if a resource exists. This creates:
|
||||
- the confidential client `petrolyst`: standard flow only, PKCE S256, redirect URIs for both
|
||||
hosts;
|
||||
- its client roles `superset-admin`, `superset-editor`, `superset-viewer`, `atoma-reader`
|
||||
and `traefik-admin`;
|
||||
- a mapper that puts those roles into the claim `petrolyst_roles`.
|
||||
- the confidential client `petrolyst`, for Superset and Traefik: standard flow only, PKCE S256,
|
||||
redirect URIs for both hosts;
|
||||
- its client roles `superset-admin`, `superset-editor`, `superset-viewer` and `traefik-admin`;
|
||||
- a mapper that puts those roles into the claim `petrolyst_roles`;
|
||||
- the confidential client `petrolyst-app`, for the Petrolyst app, with its role `reader` and two
|
||||
mappers: `petrolyst_roles`, and the person's groups with their full path in `petrolyst_groups`,
|
||||
both in the ID token. Both clients' addresses are on app.petrolyst.com, where everything moves
|
||||
(PD-16, PD-21); atoma.petrolyst.com is going away and is not in the import.
|
||||
2. Clients → petrolyst → Credentials: copy the client secret (step 3 stores it on the server).
|
||||
**Realm settings:** registration off; brute-force detection on; a password policy of at least 12
|
||||
characters, because a password is the only lock for customers' users; SSO session idle 8 hours,
|
||||
max 10 hours, because the app's session is renewed through Keycloak about hourly and the default
|
||||
idle limit of 30 minutes would ask for a password each time.
|
||||
3. **MFA**, which Exhibit B requires. A copy of the built-in `browser` flow is not enough, for two
|
||||
reasons: its Cookie step accepts an existing password-only session from another client of the
|
||||
realm, and its conditional OTP subflow skips users who have not set up OTP. Instead:
|
||||
@@ -107,14 +114,17 @@ deploy secrets.
|
||||
- Clients → petrolyst → Advanced → Authentication flow overrides → Browser flow:
|
||||
`petrolyst-mfa`.
|
||||
|
||||
Result: every sign-in to a Petrolyst app asks for password and OTP, and a user without OTP
|
||||
enrols first. Other clients of the realm are unaffected. There is no silent SSO between
|
||||
Superset and the dashboard.
|
||||
4. Give people roles: Users → (user) → Role mapping → Assign role → filter by clients →
|
||||
`petrolyst`. Group mappings work too. ATOMA's users get `atoma-reader` and nothing else:
|
||||
it opens the Petrolyst app (petrolyst-platform) and maps to no Superset role, so in
|
||||
Superset they see nothing. A realm that already has the client gets the role by hand
|
||||
(Clients → petrolyst → Roles → Create role) or by importing again with "Skip".
|
||||
Result: every sign-in to Superset or Traefik asks for password and OTP, and a user without OTP
|
||||
enrols first. There is no silent SSO between them. Client `petrolyst-app` keeps the realm's own
|
||||
flow: customers' users sign in with a password only, the owner's choice "for now" (PD-21).
|
||||
4. **A customer is a top-level group.** Groups → Create group, named as the customer's
|
||||
`keycloak_group` in fuel-db (without the slash), then Role mapping → Assign role → filter by
|
||||
clients → `petrolyst-app` `reader`. Adding a person to the group is then the only step, and they
|
||||
see that customer in the app and no other. Exactly one customer per person: two give "No access".
|
||||
They need no role on client `petrolyst`, and Superset refuses them. A person in no customer's
|
||||
group sees "No access"; that includes our own staff, who join a customer's group to see it in
|
||||
the app.
|
||||
**Administrators:** Users → (user) → Role mapping → Assign role → filter by clients → `petrolyst`.
|
||||
5. Users of the Traefik dashboard need **Email verified** switched on, because oauth2-proxy
|
||||
refuses unverified addresses.
|
||||
|
||||
@@ -158,7 +168,7 @@ becomes the first Superset admin; there is no local admin account.
|
||||
| Task | How |
|
||||
|---|---|
|
||||
| Deploy | "Manual deploy" below, from a clean `main` |
|
||||
| Re-deploy a kept release | `sudo -u deploy /opt/petrolyst/infra/deploy/deploy.sh <sha>` |
|
||||
| Re-deploy a kept release | `sudo -u deploy /opt/petrolyst/infra/deploy/deploy.sh <sha>`. Not across a change of Keycloak realm: see "Moving to realm petrolyst" |
|
||||
| Roll back | see below |
|
||||
| Traefik dashboard | https://traefik.petrolyst.com (role `traefik-admin`) |
|
||||
| fuel-db shell | `docker exec -it petrolyst-fuel-db-1 psql -U postgres -d fuel` |
|
||||
@@ -293,6 +303,23 @@ docker rm -f $(docker ps -aq --filter label=com.docker.compose.project=fuel-moni
|
||||
7. Releases from before this deploy can no longer run, as their role is gone, and `rollback.sh`
|
||||
stops on them. Remove them from `/opt/petrolyst/releases/`.
|
||||
|
||||
**Moving to realm `petrolyst`, one time.** A deploy of its own, after the deploy that removes Trino:
|
||||
releases from before that one can no longer run, so a combined deploy leaves nothing to roll back to.
|
||||
1. The realm is ready (Keycloak, above) and an admin has signed in on client `petrolyst` there. While
|
||||
Superset still answers at atoma.petrolyst.com, add that address's redirect URI to the client by
|
||||
hand, or nobody can sign in after step 3; remove it when the domain goes.
|
||||
2. As root: `host/secrets-init.sh keycloak-secret`. The value goes into the new file
|
||||
`shared/keycloak_petrolyst_realm_client_secret`, so nothing changes for anyone yet. Leave the old
|
||||
file `shared/keycloak_petrolyst_client_secret` alone.
|
||||
3. Deploy by "Manual deploy" only. That one step moves Superset and Traefik together.
|
||||
4. **Prove it by hand:** an admin signs in to Superset and to the Traefik dashboard. The smoke tests
|
||||
check redirects only, and pass with a wrong secret.
|
||||
5. **To go back:** `rollback.sh <previous sha>` returns to the old realm with the old secret. Afterwards go
|
||||
forward again by "Manual deploy" only: "Re-deploy a kept release" runs the old `lib.sh`, whose realm
|
||||
address would override the new release's.
|
||||
6. Not before 7 days of working sign-ins, and on the owner's word: retire client `petrolyst` in realm
|
||||
`thedigitalbridge` and delete the old secret file. From then on there is no way back.
|
||||
|
||||
**Adding a project**: create `stacks/<name>/compose.yaml`, attach the public service to the
|
||||
external network `proxy`, and set `traefik.*` labels with a unique router name and
|
||||
`tls.certresolver: le`. Add the DNS record first. `deploy.sh` picks the new folder up
|
||||
|
||||
+28
-10
@@ -17,9 +17,11 @@ ROLLBACK_MARK=$STATE/rollback-in-progress
|
||||
|
||||
ATOMA_HOST=${ATOMA_HOST:-atoma.petrolyst.com}
|
||||
TRAEFIK_HOST=${TRAEFIK_HOST:-traefik.petrolyst.com}
|
||||
KEYCLOAK_REALM_URL=${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/thedigitalbridge}
|
||||
PETROLYST_ROOT=$ROOT
|
||||
export ATOMA_HOST TRAEFIK_HOST KEYCLOAK_REALM_URL PETROLYST_ROOT
|
||||
# The Keycloak realm is not set here: each release's compose files name their own, with the secret
|
||||
# file that belongs to it. A rollback, which runs this file from the release it leaves, therefore
|
||||
# returns to the old release's realm and secret. Exporting a value here would override them.
|
||||
export ATOMA_HOST TRAEFIK_HOST PETROLYST_ROOT
|
||||
|
||||
# --host pins the daemon, as in petrolyst-web/deploy/deploy.sh: a stale
|
||||
# `docker context use` must not send this deploy to another daemon.
|
||||
@@ -39,8 +41,8 @@ SUPERSET_INPUTS=(
|
||||
)
|
||||
|
||||
# Every file the stacks need from /opt/petrolyst (host/secrets-init.sh).
|
||||
# The Keycloak client secret is not listed: it depends on the release (check_keycloak_secrets).
|
||||
REQUIRED_FILES=(
|
||||
secrets/shared/keycloak_petrolyst_client_secret
|
||||
secrets/traefik/oauth2_proxy_cookie_secret
|
||||
secrets/fuel-monitoring/superset_secret_key
|
||||
secrets/fuel-monitoring/superset_db_password
|
||||
@@ -126,10 +128,7 @@ preflight() {
|
||||
for f in "${REQUIRED_FILES[@]}"; do
|
||||
[ -s "$ROOT/$f" ] || die "missing or empty: $ROOT/$f (run host/secrets-init.sh as root)"
|
||||
done
|
||||
# oauth2-proxy uses the client secret file byte for byte.
|
||||
f=$ROOT/secrets/shared/keycloak_petrolyst_client_secret
|
||||
[ "$(tail -c1 "$f" | wc -l)" -eq 0 ] ||
|
||||
die "$f ends with a newline; store it again with: host/secrets-init.sh keycloak-secret"
|
||||
check_keycloak_secrets "$release"
|
||||
bad=$(find "$release/petrolyst-insights" \( -name 'Dockerfile*' -o -name '*compose*.y*ml' \
|
||||
-o -name '*.env' -o -name '.env' -o -path '*/.gitea/*' -o -path '*/.github/*' \) -print)
|
||||
[ -z "$bad" ] || die "petrolyst-insights/ holds customization only; found: $bad"
|
||||
@@ -141,6 +140,21 @@ preflight() {
|
||||
die "containers of the old project fuel-monitoring exist; remove them first (README \"Removing the old project\")"
|
||||
}
|
||||
|
||||
# check_keycloak_secrets <release>: the client secret that release's compose files name, which
|
||||
# differs per realm. So a rollback needs only its target's secret, not the one of the release it leaves.
|
||||
check_keycloak_secrets() {
|
||||
local release=$1 names name f
|
||||
names=$(cat "$release/stacks/fuel-monitoring/compose.yaml" "$release/traefik/compose.yaml" |
|
||||
grep -oE 'secrets/shared/keycloak_[a-z_]+' | sort -u) || die "$release names no Keycloak client secret"
|
||||
for name in $names; do
|
||||
f=$ROOT/$name
|
||||
[ -s "$f" ] || die "missing or empty: $f (run host/secrets-init.sh as root)"
|
||||
# oauth2-proxy uses the client secret file byte for byte.
|
||||
[ "$(tail -c1 "$f" | wc -l)" -eq 0 ] ||
|
||||
die "$f ends with a newline; store it again"
|
||||
done
|
||||
}
|
||||
|
||||
# promote <release> <sha>: make the release the live tree. --chmod applies to
|
||||
# the destination only: the live tree stays writable, the release read-only.
|
||||
promote() {
|
||||
@@ -234,7 +248,7 @@ probe() {
|
||||
}
|
||||
|
||||
smoke_tests() {
|
||||
local i body code location
|
||||
local i body code location realm
|
||||
log "smoke tests"
|
||||
# First deploy: Let's Encrypt may need a minute to issue the certificate.
|
||||
for i in $(seq 1 24); do
|
||||
@@ -244,17 +258,21 @@ smoke_tests() {
|
||||
sleep 5
|
||||
done
|
||||
|
||||
# The realm this release runs on. These tests check redirects only, not a real sign-in: they
|
||||
# pass with a wrong client secret, so a person signs in after a change of realm or secret.
|
||||
realm=$(fm exec -T superset printenv KEYCLOAK_REALM_URL) || die "Superset has no KEYCLOAK_REALM_URL"
|
||||
|
||||
read -r code location < <(probe "$ATOMA_HOST" /login/)
|
||||
[ "$code" = 302 ] && [[ $location == "https://$ATOMA_HOST/login/keycloak"* ]] ||
|
||||
die "/login/ should redirect to /login/keycloak, got $code $location"
|
||||
|
||||
read -r code location < <(probe "$ATOMA_HOST" /login/keycloak)
|
||||
[ "$code" = 302 ] && [[ $location == "$KEYCLOAK_REALM_URL/protocol/openid-connect/auth?"* ]] &&
|
||||
[ "$code" = 302 ] && [[ $location == "$realm/protocol/openid-connect/auth?"* ]] &&
|
||||
[[ $location == *client_id=petrolyst* && $location == *code_challenge_method=S256* ]] ||
|
||||
die "/login/keycloak should redirect to Keycloak with client_id=petrolyst and PKCE S256, got $code $location"
|
||||
|
||||
read -r code location < <(probe "$TRAEFIK_HOST" /)
|
||||
[ "$code" = 302 ] && [[ $location == "$KEYCLOAK_REALM_URL/"* || $location == *"/oauth2/start"* ]] ||
|
||||
[ "$code" = 302 ] && [[ $location == "$realm/"* || $location == *"/oauth2/start"* ]] ||
|
||||
die "https://$TRAEFIK_HOST/ should redirect to Keycloak, got $code $location"
|
||||
|
||||
# Superset's own connection "Fuel" (fuel_ro, password from /run/secrets) reads fuel-db and
|
||||
|
||||
@@ -2,7 +2,8 @@
|
||||
# Server secrets, run as root after bootstrap.sh.
|
||||
# secrets-init.sh generate every missing secret; never overwrites one
|
||||
# secrets-init.sh keycloak-secret store the client secret of Keycloak client
|
||||
# "petrolyst" (Clients > petrolyst > Credentials),
|
||||
# "petrolyst" of realm "petrolyst"
|
||||
# (Clients > petrolyst > Credentials),
|
||||
# read from the terminal or stdin
|
||||
#
|
||||
# Layout: /opt/petrolyst/secrets/{shared,traefik,fuel-monitoring}/ are root:deploy
|
||||
@@ -50,8 +51,8 @@ case ${1:-all} in
|
||||
fi
|
||||
value=$(printf '%s' "$value" | tr -d '[:space:]')
|
||||
[ -n "$value" ] || die "empty secret"
|
||||
write_secret "$SECRETS/shared/keycloak_petrolyst_client_secret" "$value"
|
||||
echo "stored $SECRETS/shared/keycloak_petrolyst_client_secret"
|
||||
write_secret "$SECRETS/shared/keycloak_petrolyst_realm_client_secret" "$value"
|
||||
echo "stored $SECRETS/shared/keycloak_petrolyst_realm_client_secret"
|
||||
exit 0
|
||||
;;
|
||||
all) ;;
|
||||
@@ -66,10 +67,10 @@ generate "$SECRETS/fuel-monitoring/fuel_db_ro_password" 32
|
||||
# 32 hex characters = a 32-byte cookie key, as oauth2-proxy requires.
|
||||
generate "$SECRETS/traefik/oauth2_proxy_cookie_secret" 16
|
||||
|
||||
if [ ! -s "$SECRETS/shared/keycloak_petrolyst_client_secret" ]; then
|
||||
if [ ! -s "$SECRETS/shared/keycloak_petrolyst_realm_client_secret" ]; then
|
||||
echo
|
||||
echo "Still missing: the Keycloak client secret. Copy it from Keycloak"
|
||||
echo "(realm thedigitalbridge > Clients > petrolyst > Credentials), then run:"
|
||||
echo "(realm petrolyst > Clients > petrolyst > Credentials), then run:"
|
||||
echo " $0 keycloak-secret"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
{
|
||||
"clients": [
|
||||
{
|
||||
"clientId": "petrolyst",
|
||||
"name": "Petrolyst administration",
|
||||
"description": "Superset and the Traefik dashboard (oauth2-proxy), for our own people, with MFA (browser flow override petrolyst-mfa, set by hand). Managed from petrolyst-infra/keycloak/.",
|
||||
"enabled": true,
|
||||
"protocol": "openid-connect",
|
||||
"publicClient": false,
|
||||
"clientAuthenticatorType": "client-secret",
|
||||
"standardFlowEnabled": true,
|
||||
"implicitFlowEnabled": false,
|
||||
"directAccessGrantsEnabled": false,
|
||||
"serviceAccountsEnabled": false,
|
||||
"frontchannelLogout": true,
|
||||
"baseUrl": "https://app.petrolyst.com/",
|
||||
"redirectUris": [
|
||||
"https://app.petrolyst.com/oauth-authorized/keycloak",
|
||||
"https://traefik.petrolyst.com/oauth2/callback"
|
||||
],
|
||||
"webOrigins": ["+"],
|
||||
"attributes": {
|
||||
"pkce.code.challenge.method": "S256",
|
||||
"post.logout.redirect.uris": "https://app.petrolyst.com/*##https://traefik.petrolyst.com/*"
|
||||
},
|
||||
"protocolMappers": [
|
||||
{
|
||||
"name": "petrolyst client roles",
|
||||
"protocol": "openid-connect",
|
||||
"protocolMapper": "oidc-usermodel-client-role-mapper",
|
||||
"consentRequired": false,
|
||||
"config": {
|
||||
"usermodel.clientRoleMapping.clientId": "petrolyst",
|
||||
"multivalued": "true",
|
||||
"claim.name": "petrolyst_roles",
|
||||
"jsonType.label": "String",
|
||||
"id.token.claim": "true",
|
||||
"access.token.claim": "true",
|
||||
"userinfo.token.claim": "true",
|
||||
"introspection.token.claim": "true"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"clientId": "petrolyst-app",
|
||||
"name": "Petrolyst",
|
||||
"description": "The Petrolyst app (petrolyst-platform server/), for customers' users. Password only (PD-21). The customer comes from the person's top-level group. Managed from petrolyst-infra/keycloak/.",
|
||||
"enabled": true,
|
||||
"protocol": "openid-connect",
|
||||
"publicClient": false,
|
||||
"clientAuthenticatorType": "client-secret",
|
||||
"standardFlowEnabled": true,
|
||||
"implicitFlowEnabled": false,
|
||||
"directAccessGrantsEnabled": false,
|
||||
"serviceAccountsEnabled": false,
|
||||
"frontchannelLogout": true,
|
||||
"baseUrl": "https://app.petrolyst.com/app/",
|
||||
"redirectUris": [
|
||||
"https://app.petrolyst.com/app/auth/callback"
|
||||
],
|
||||
"webOrigins": [],
|
||||
"attributes": {
|
||||
"pkce.code.challenge.method": "S256",
|
||||
"post.logout.redirect.uris": "https://app.petrolyst.com/app/*"
|
||||
},
|
||||
"protocolMappers": [
|
||||
{
|
||||
"name": "petrolyst client roles",
|
||||
"protocol": "openid-connect",
|
||||
"protocolMapper": "oidc-usermodel-client-role-mapper",
|
||||
"consentRequired": false,
|
||||
"config": {
|
||||
"usermodel.clientRoleMapping.clientId": "petrolyst-app",
|
||||
"multivalued": "true",
|
||||
"claim.name": "petrolyst_roles",
|
||||
"jsonType.label": "String",
|
||||
"id.token.claim": "true",
|
||||
"access.token.claim": "false",
|
||||
"userinfo.token.claim": "false",
|
||||
"introspection.token.claim": "false"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "petrolyst groups",
|
||||
"protocol": "openid-connect",
|
||||
"protocolMapper": "oidc-group-membership-mapper",
|
||||
"consentRequired": false,
|
||||
"config": {
|
||||
"full.path": "true",
|
||||
"claim.name": "petrolyst_groups",
|
||||
"id.token.claim": "true",
|
||||
"access.token.claim": "false",
|
||||
"userinfo.token.claim": "false",
|
||||
"introspection.token.claim": "false"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"roles": {
|
||||
"client": {
|
||||
"petrolyst": [
|
||||
{ "name": "superset-admin", "description": "Superset Admin" },
|
||||
{ "name": "superset-editor", "description": "Superset Alpha: build charts and dashboards" },
|
||||
{ "name": "superset-viewer", "description": "Superset Gamma: view dashboards shared with the role" },
|
||||
{ "name": "traefik-admin", "description": "Traefik dashboard at traefik.petrolyst.com" }
|
||||
],
|
||||
"petrolyst-app": [
|
||||
{ "name": "reader", "description": "Reads their customer's data in the Petrolyst app; no role in Superset. Mapped onto each customer's group." }
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -15,7 +15,7 @@ and where it runs lives in `stacks/`, `traefik/`, `deploy/` and `host/`.
|
||||
|---|---|---|
|
||||
| `superset_config.py` | Keycloak sign-in, role mapping, proxy and cookie settings, Redis caches, theme and chart palettes | `/app/pythonpath/` |
|
||||
| `petrolyst_security.py` | `PetrolystSecurityManager`: user info and roles from the Keycloak ID token, `/login/` straight to Keycloak, `/logout/` through Keycloak | `/app/pythonpath/` |
|
||||
| `roles/` | `atoma_guest.py`: the role "ATOMA Guest" for the Petrolyst app's embedded dashboards, with exactly its permissions; run by hand (see "Embedding") | `/app/petrolyst-roles/` |
|
||||
| `roles/` | `petrolyst_guest.py`: the role "Petrolyst Guest" for the Petrolyst app's embedded dashboards, with exactly its permissions; run by hand (see "Embedding") | `/app/petrolyst-roles/` |
|
||||
| `assets/branding/` | logos (full colour and white): the wordmark alone for the navbar, and the stacked logo; favicons, app icons; ATOMA's logo, which the Petrolyst app shows in its map's corner (the server reads this file itself: `PETROLYST_OPERATOR_LOGO_FILE`) | `/app/superset/static/assets/branding/`, served at `/static/assets/branding/…` |
|
||||
| `patches/` | runtime patches, `fix_*.py` with `--verify` (see its README); none at present | `/app/patches/petrolyst/`, applied during the build |
|
||||
| `dashboards/` | exported dashboards and import scripts (see its README) | not in the image |
|
||||
@@ -35,13 +35,13 @@ value.
|
||||
|---|---|
|
||||
| `superset_secret_key` | Flask `SECRET_KEY` |
|
||||
| `superset_db_password` | password of user `superset` on the metadata database |
|
||||
| `keycloak_petrolyst_client_secret` | client secret of Keycloak client `petrolyst` |
|
||||
| `keycloak_petrolyst_client_secret` | client secret of Keycloak client `petrolyst`, realm `petrolyst`; on the server in `shared/keycloak_petrolyst_realm_client_secret` |
|
||||
|
||||
On the server they come from `/opt/petrolyst/secrets/` (see the deployment README).
|
||||
|
||||
## Sign-in and roles
|
||||
|
||||
- Provider: Keycloak realm `thedigitalbridge` at `iam.thedigitalbridge.io`, client `petrolyst`,
|
||||
- Provider: Keycloak realm `petrolyst` at `iam.thedigitalbridge.io`, client `petrolyst`,
|
||||
authorization code flow with PKCE S256. The realm URL can be overridden with
|
||||
`KEYCLOAK_REALM_URL`.
|
||||
- Roles come from the client roles of `petrolyst`, which a protocol mapper puts into the claim
|
||||
@@ -68,14 +68,14 @@ stack** (`.local/fmdb-scratch`); `superset_config.py` here leaves it off, and lo
|
||||
guest secret.
|
||||
|
||||
- The scratch stack's `superset_config_local.py` turns on `EMBEDDED_SUPERSET`, names the guest
|
||||
role `GUEST_ROLE_NAME = "ATOMA Guest"`, reads the signing secret with
|
||||
role `GUEST_ROLE_NAME = "Petrolyst Guest"`, reads the signing secret with
|
||||
`_read_secret("superset_guest_token_secret")` and expects the audience `petrolyst-app`. The
|
||||
Petrolyst server signs the guest passes itself with the same secret.
|
||||
- `roles/atoma_guest.py` gives "ATOMA Guest" exactly what a dashboard needs to be read. Which
|
||||
- `roles/petrolyst_guest.py` gives "Petrolyst Guest" exactly what a dashboard needs to be read. Which
|
||||
dashboard a guest sees comes from the pass, not the role. It is not in `init.sh`; run it once,
|
||||
and again whenever its permissions change:
|
||||
|
||||
docker exec <superset> /app/.venv/bin/python /app/petrolyst-roles/atoma_guest.py
|
||||
docker exec <superset> /app/.venv/bin/python /app/petrolyst-roles/petrolyst_guest.py
|
||||
|
||||
- The app's dashboards and their embedding ids: `dashboards/README.md`.
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ Superset dashboards as code and the operator scripts that import them. Not baked
|
||||
image, so changes here do not rebuild or restart Superset.
|
||||
|
||||
- `atoma/`: ATOMA's two dashboards for the Petrolyst app (PD-12, PD-13; SPEC.md §10):
|
||||
`atoma-overview` (fleet totals) and `atoma-site` (one site, its four sections as tabs). The
|
||||
`overview` (fleet totals) and `site` (one site, its four sections as tabs), named `atoma-overview` and `atoma-site` until 30 Oct. The
|
||||
app embeds them; admins open them in Superset, with the filter column. `SPEC.md` is the
|
||||
design; `build_bundle.py` generates the import bundle and `import_bundle.py` imports it
|
||||
(overwriting the objects the bundle defines).
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# ATOMA dashboards: Overview and Site, for the Petrolyst app
|
||||
|
||||
**Status (29 Oct 2026, PD-13): two dashboards, `atoma-overview` and `atoma-site` (§10), built and
|
||||
**Status (30 Oct 2026, PD-21: every customer sees these two, limited by row; slugs `overview` and `site`, formerly
|
||||
`atoma-overview` and `atoma-site`, which the rest of this file still uses). Two dashboards (§10), built and
|
||||
tested on a local Superset; not imported live.** The Petrolyst app embeds them, and admins open
|
||||
them in Superset with the filter column.
|
||||
- `build_bundle.py` generates the dashboards, every dataset and every chart from this
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate ATOMA's dashboards for the Petrolyst app as one Superset 6.1 import bundle.
|
||||
"""Generate the dashboards of the Petrolyst app as one Superset 6.1 import bundle.
|
||||
|
||||
Implements SPEC.md in this folder: the two dashboards the Petrolyst app embeds, "ATOMA · Overview"
|
||||
(atoma-overview) and "ATOMA · Site" (atoma-site, the four detail tabs), one dataset per
|
||||
public.v_* view and every chart. Nothing in the bundle names a site.
|
||||
Implements SPEC.md in this folder: the two dashboards the app embeds, "Overview" (overview) and
|
||||
"Site" (site, the four detail tabs), one dataset per public.v_* view and every chart. Every
|
||||
customer sees the same dashboards, limited by row: the app's guest pass names the customer (PD-16).
|
||||
Nothing in the bundle names a site or a customer.
|
||||
|
||||
build_bundle.py --database-uuid <uuid of "Fuel"> [--out atoma-dashboards.zip]
|
||||
|
||||
@@ -15,7 +16,8 @@ dashboards (the stock dashboard import leaves existing datasets and charts
|
||||
unchanged).
|
||||
|
||||
Before writing, the bundle is checked: every column a chart or filter uses
|
||||
exists in its dataset, every chart in a Site filter's scope has site_label, the
|
||||
exists in its dataset, every dataset has operator_uid (the guest pass's row rule needs
|
||||
it), every chart in a Site filter's scope has site_label, the
|
||||
Overview sums recorded values only and money only per currency, and no row is
|
||||
wider than 12. Requires PyYAML.
|
||||
"""
|
||||
@@ -496,8 +498,11 @@ def timeseries_common(x: str, metrics: list[Any], groupby: list[str], grain: str
|
||||
|
||||
# --- Overview (PD-12, embedded in the Petrolyst app) ---------------------------------------
|
||||
|
||||
OVERVIEW_SLUG = "atoma-overview"
|
||||
SITE_SLUG = "atoma-site" # one embedded dashboard with the DETAIL_TABS as its tabs
|
||||
OVERVIEW_SLUG = "overview"
|
||||
SITE_SLUG = "site" # one embedded dashboard with the DETAIL_TABS as its tabs
|
||||
# A dashboard's identity, apart from its slug: the names it was first built under, so a re-import
|
||||
# updates the existing two (and keeps their embedding ids) rather than adding two new ones.
|
||||
DASHBOARD_KEYS = {OVERVIEW_SLUG: "atoma-overview", SITE_SLUG: "atoma-site"}
|
||||
|
||||
# SPEC §10: only sums and counts of recorded values, never an average of per-site rates; money
|
||||
# only grouped by currency. The value says whether the metric is money.
|
||||
@@ -935,7 +940,7 @@ def dashboard_yaml(key: str, title: str, slug: str, layout: Layout, metadata: di
|
||||
"certified_by": None,
|
||||
"certification_details": None,
|
||||
"published": True,
|
||||
"uuid": uid("dashboard", key),
|
||||
"uuid": uid("dashboard", DASHBOARD_KEYS[key]),
|
||||
"position": layout.nodes,
|
||||
"metadata": metadata,
|
||||
"version": VERSION,
|
||||
@@ -1000,7 +1005,7 @@ def build(database_uuid: str, columns_path: Path) -> dict[str, Any]:
|
||||
all_charts = {**{c.key: c for c in detail_charts()}, **overview}
|
||||
dashboards: list[tuple[Layout, dict[str, Chart], dict[str, Any]]] = []
|
||||
for key, title in [(OVERVIEW_SLUG, "Overview"), (SITE_SLUG, "Site")]:
|
||||
page = Layout(f"ATOMA · {title}", all_charts)
|
||||
page = Layout(title, all_charts)
|
||||
page.markdown.update({f"notes-{k}": v for k, v in TAB_NOTES.items()})
|
||||
page.markdown["notes-overview"] = OVERVIEW_NOTES
|
||||
if key == OVERVIEW_SLUG:
|
||||
@@ -1013,7 +1018,7 @@ def build(database_uuid: str, columns_path: Path) -> dict[str, Any]:
|
||||
placed = {k: all_charts[k] for k in page.ids}
|
||||
page_metadata = base_metadata(page, filters, cross_filters=False)
|
||||
page_metadata["label_colors"].update(STOCK_STATUS_COLOURS)
|
||||
files[f"dashboards/{key}.yaml"] = dashboard_yaml(key, f"ATOMA · {title}", key, page,
|
||||
files[f"dashboards/{key}.yaml"] = dashboard_yaml(key, title, key, page,
|
||||
page_metadata)
|
||||
dashboards.append((page, placed, page_metadata))
|
||||
|
||||
@@ -1031,6 +1036,12 @@ def check(files: dict[str, Any], views: dict[str, list[tuple[str, str]]],
|
||||
problems = []
|
||||
columns = {v: {c for c, _ in cols} | {c[0] for c in CALCULATED.get(v, [])}
|
||||
for v, cols in views.items()}
|
||||
for view, cols in columns.items():
|
||||
if "operator_uid" not in cols:
|
||||
problems.append(f"{view}: no operator_uid, so the guest pass's row rule cannot hold")
|
||||
for name, (_, _, sql) in VIRTUAL.items():
|
||||
if "operator_uid" not in sql:
|
||||
problems.append(f"{name}: its SQL does not select operator_uid")
|
||||
for view in list(MAIN_DTTM) + list(CALCULATED) + list(METRICS):
|
||||
if view not in columns:
|
||||
problems.append(f"{view}: configured but not in the column snapshot")
|
||||
|
||||
@@ -17,7 +17,7 @@ live view and any drift is reported (exit status 2).
|
||||
The database is never created or changed: the bundle's database uuid must
|
||||
already exist under the name "Fuel", otherwise nothing is imported.
|
||||
|
||||
The dashboards (atoma-overview, atoma-site) get no role: admins see them in Superset, and
|
||||
The dashboards (overview, site) get no role: admins see them in Superset, and
|
||||
the Petrolyst app shows them with guest passes (PD-12). With --embedded-out FILE
|
||||
--embed-domain URL, each of them is made embeddable from that origin (its embedding id stays
|
||||
the same across imports), and FILE receives {slug: embedding id} for the Petrolyst server:
|
||||
@@ -36,7 +36,7 @@ import zipfile
|
||||
import yaml
|
||||
|
||||
DATABASE_NAME = "Fuel"
|
||||
EMBEDDED_PREFIX = "atoma-"
|
||||
EMBEDDED_SLUGS = {"overview", "site"}
|
||||
STEPS = [("datasets", "SqlaTable"), ("charts", "Slice"), ("dashboards", "Dashboard")]
|
||||
|
||||
|
||||
@@ -104,12 +104,17 @@ def main() -> int:
|
||||
if path.startswith("dashboards/"):
|
||||
dashboard = db.session.query(Dashboard).filter_by(
|
||||
uuid=yaml.safe_load(text)["uuid"]).one()
|
||||
if dashboard.slug.startswith(EMBEDDED_PREFIX) and args.embedded_out:
|
||||
if dashboard.slug in EMBEDDED_SLUGS and args.embedded_out:
|
||||
from superset.daos.dashboard import EmbeddedDashboardDAO
|
||||
|
||||
config = EmbeddedDashboardDAO.upsert(dashboard, [args.embed_domain])
|
||||
db.session.flush()
|
||||
embedded[dashboard.slug] = str(config.uuid)
|
||||
# An empty or short list would stop the app's dashboards from opening.
|
||||
if args.embedded_out and set(embedded) != EMBEDDED_SLUGS:
|
||||
db.session.rollback()
|
||||
print(f"embedded: expected {sorted(EMBEDDED_SLUGS)}, got {sorted(embedded)}", file=sys.stderr)
|
||||
return 3
|
||||
db.session.commit()
|
||||
if args.embedded_out:
|
||||
with open(args.embedded_out, "w") as out:
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""Keycloak sign-in for Superset at atoma.petrolyst.com.
|
||||
|
||||
Set as CUSTOM_SECURITY_MANAGER in superset_config.py. Checked against
|
||||
Flask-AppBuilder 5.0.2 as bundled in apache/superset:6.1.0. Three changes to
|
||||
Flask-AppBuilder 5.0.2 as bundled in apache/superset:6.1.0. Four changes to
|
||||
the stock behaviour:
|
||||
|
||||
- User info and roles come from the ID token that Keycloak issues for client
|
||||
@@ -11,6 +11,8 @@ the stock behaviour:
|
||||
login page on /login/, which only shows a "Sign in with keycloak" button.
|
||||
- /logout/ also ends the Keycloak session, so the next visit asks for the
|
||||
password and OTP again.
|
||||
- A Keycloak user with no mapped Superset role is refused, and no account is
|
||||
made. Customers' users belong in the Petrolyst app, never here (PD-21).
|
||||
"""
|
||||
|
||||
import logging
|
||||
@@ -70,6 +72,14 @@ class PetrolystSecurityManager(SupersetSecurityManager):
|
||||
# FAB's OAuth view, and that page wins the route.
|
||||
register_superset_auth_view = False
|
||||
|
||||
def auth_user_oauth(self, userinfo: dict[str, Any]) -> Any:
|
||||
# Only the mapped roles count: FAB's own calculation always adds the
|
||||
# registration role, so it is never empty.
|
||||
if not self.get_roles_from_keys(userinfo.get("role_keys", [])):
|
||||
logger.warning("Keycloak sign-in refused: no mapped Superset role")
|
||||
return None
|
||||
return super().auth_user_oauth(userinfo)
|
||||
|
||||
def oauth_user_info(
|
||||
self, provider: str, response: Optional[dict[str, Any]] = None
|
||||
) -> dict[str, Any]:
|
||||
|
||||
+5
-4
@@ -1,14 +1,15 @@
|
||||
"""The role "ATOMA Guest": what the Petrolyst app's embedded dashboards may do (PD-12).
|
||||
"""The role "Petrolyst Guest": what the Petrolyst app's embedded dashboards may do (PD-12).
|
||||
|
||||
Superset gives every embedded viewer (a guest pass signed for one dashboard) the role named by
|
||||
GUEST_ROLE_NAME. Which dashboard a guest sees comes from the pass, not from the role. Superset
|
||||
GUEST_ROLE_NAME. Which dashboard and which customer's rows a guest sees come from the pass, not from
|
||||
the role. Superset
|
||||
leaves the SQL out of chart-data responses for guests.
|
||||
|
||||
Embedding is switched on in the local stack only (.local/fmdb-scratch, superset_config_local.py),
|
||||
so this script is not part of init.sh; run it by hand inside the Superset container, once, and
|
||||
again whenever PERMISSIONS change:
|
||||
|
||||
docker exec <superset> /app/.venv/bin/python /app/petrolyst-roles/atoma_guest.py
|
||||
docker exec <superset> /app/.venv/bin/python /app/petrolyst-roles/petrolyst_guest.py
|
||||
|
||||
Creates the role if needed and gives it exactly PERMISSIONS: missing ones are added, any other
|
||||
is removed (also one added by hand in the UI). A second run changes nothing.
|
||||
@@ -22,7 +23,7 @@ from __future__ import annotations
|
||||
|
||||
import sys
|
||||
|
||||
ROLE = "ATOMA Guest"
|
||||
ROLE = "Petrolyst Guest"
|
||||
PERMISSIONS = [
|
||||
("can_read", "Dashboard"), # the dashboard and its charts' definitions
|
||||
("can_dashboard", "Superset"), # the dashboard page
|
||||
@@ -64,7 +64,7 @@ SQLALCHEMY_CUSTOM_PASSWORD_STORE = _database_password
|
||||
# Users and roles are managed in Keycloak; there is no local admin account.
|
||||
# Break-glass access: `superset fab ...` inside the container on the server.
|
||||
KEYCLOAK_REALM_URL = os.environ.get(
|
||||
"KEYCLOAK_REALM_URL", "https://iam.thedigitalbridge.io/realms/thedigitalbridge"
|
||||
"KEYCLOAK_REALM_URL", "https://iam.thedigitalbridge.io/realms/petrolyst"
|
||||
).rstrip("/")
|
||||
|
||||
AUTH_TYPE = AUTH_OAUTH
|
||||
@@ -94,12 +94,12 @@ AUTH_ROLES_MAPPING = {
|
||||
"superset-admin": ["Admin"],
|
||||
"superset-editor": ["Alpha"],
|
||||
"superset-viewer": ["Gamma"],
|
||||
# "atoma-reader" opens the Petrolyst app only; here it maps to nothing (Public).
|
||||
# Customers' users have a role of client "petrolyst-app", not of this one: refused here.
|
||||
}
|
||||
AUTH_ROLES_SYNC_AT_LOGIN = True
|
||||
AUTH_USER_REGISTRATION = True
|
||||
# Public has no permissions: a Keycloak user without a Superset role signs in
|
||||
# and sees nothing.
|
||||
# Public has no permissions. A Keycloak user without a mapped role never gets
|
||||
# this far: PetrolystSecurityManager refuses the sign-in and makes no account.
|
||||
AUTH_USER_REGISTRATION_ROLE = "Public"
|
||||
|
||||
# Viewers (Gamma) see a dashboard once it lists their role.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
name: petrolyst
|
||||
|
||||
x-superset-env: &superset-env
|
||||
KEYCLOAK_REALM_URL: ${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/thedigitalbridge}
|
||||
KEYCLOAK_REALM_URL: ${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/petrolyst}
|
||||
|
||||
x-superset: &superset
|
||||
image: fuel-superset:${SUPERSET_IMAGE_TAG:?SUPERSET_IMAGE_TAG unset - run through deploy/deploy.sh}
|
||||
@@ -158,4 +158,4 @@ secrets:
|
||||
fuel_db_ro_password:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/fuel-monitoring/fuel_db_ro_password
|
||||
keycloak_petrolyst_client_secret:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/shared/keycloak_petrolyst_client_secret
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/shared/keycloak_petrolyst_realm_client_secret
|
||||
@@ -18,7 +18,9 @@ CREATE TABLE fm.operator (
|
||||
name text NOT NULL,
|
||||
label_issuer text NOT NULL, -- whose labels this customer's people see (holder_alias.issuer)
|
||||
programme text NOT NULL, -- the programme v_site reports (participation.programme)
|
||||
timezone text NOT NULL -- for a holder with no place, or a place without its own
|
||||
timezone text NOT NULL, -- for a holder with no place, or a place without its own
|
||||
keycloak_group text NOT NULL UNIQUE -- the top-level Keycloak group of this customer's people (PD-21)
|
||||
CHECK (keycloak_group ~ '^/[^/]+$')
|
||||
);
|
||||
|
||||
-- Provenance ------------------------------------------------------------------------------
|
||||
|
||||
@@ -631,4 +631,9 @@ SELECT s.site_uid, s.site_code, s.site_name, s.site_label, s.operator_uid,
|
||||
FROM fm.imported_calculation c JOIN public.v_site s ON s.site_uid = c.holder_uid
|
||||
WHERE c.claim_status IN ('asserted','reviewed','disputed');
|
||||
|
||||
-- Which Keycloak group is which customer (PD-21). Only server/ reads this, at sign-in; it holds no
|
||||
-- site data, and no dashboard has a dataset on it.
|
||||
CREATE VIEW public.v_operator AS
|
||||
SELECT operator_uid, name, keycloak_group FROM fm.operator;
|
||||
|
||||
COMMIT;
|
||||
@@ -14,7 +14,7 @@ The design and its rationale live in the fuel-monitoring corpus:
|
||||
|---|---|---|
|
||||
| `0001_fm_core.sql` | schema `fm`, `btree_gist` in it, 29 tables (customers, assets, places and holders, claims, reviews, baselines), constraints, trigger-backed rules 1–15 | no |
|
||||
| `0002_fm_derivations.sql` | internal views in `fm`: claims → visits → stock intervals → L/h and L/day intervals, dependency hashes, baseline validity | no |
|
||||
| `0003_bi_views.sql` | `fm.v_site_baseline_status` and the 18 `public.v_*` views Superset reads | yes, read-only |
|
||||
| `0003_bi_views.sql` | `fm.v_site_baseline_status` and the 19 `public.v_*` views Superset reads | yes, read-only |
|
||||
| `0004_licence.sql` | schema `licence`: plans, prices, entitlements, customer licences, overrides and their change log (PD-19, PD-20). Holds no customer's licence | no |
|
||||
|
||||
Each file is one transaction. Apply them in order.
|
||||
@@ -51,7 +51,7 @@ point 8, closed 30 Oct 2026).
|
||||
|
||||
3. Check as `fuel_ro`
|
||||
(`docker exec -it petrolyst-fuel-db-1 psql -U postgres -d fuel`, then `SET ROLE fuel_ro;`):
|
||||
- `\dv public.*` lists the 18 `v_*` views;
|
||||
- `\dv public.*` lists the 19 `v_*` views;
|
||||
- `SELECT count(*) FROM fm.holder;` fails with "permission denied for schema fm";
|
||||
- `SELECT count(*) FROM licence.plan;` fails with "permission denied for schema licence".
|
||||
|
||||
|
||||
@@ -7,15 +7,15 @@ BEGIN
|
||||
FOREACH v IN ARRAY ARRAY['v_site','v_kpi_measures','v_hour_meter_evidence','v_assumption','v_baseline_version',
|
||||
'v_baseline_input','v_evidence','v_delivery','v_stock_reading','v_interval','v_runtime',
|
||||
'v_coverage','v_reconciliation','v_data_quality','v_spec_reference','v_modelled_lph',
|
||||
'v_electricity_cost','v_imported_calculation'] LOOP
|
||||
'v_electricity_cost','v_imported_calculation','v_operator'] LOOP
|
||||
EXECUTE format('SELECT count(*) FROM public.%I', v);
|
||||
SELECT EXISTS (SELECT 1 FROM information_schema.columns c
|
||||
WHERE c.table_schema = 'public' AND c.table_name = v AND c.column_name = 'operator_uid') INTO has;
|
||||
IF NOT has THEN RAISE EXCEPTION '% has no operator_uid', v; END IF;
|
||||
n := n + 1;
|
||||
END LOOP;
|
||||
IF n <> 18 THEN RAISE EXCEPTION 'expected 18 views, checked %', n; END IF;
|
||||
IF (SELECT count(*) FROM information_schema.views WHERE table_schema = 'public' AND table_name LIKE 'v\_%') <> 18 THEN
|
||||
IF n <> 19 THEN RAISE EXCEPTION 'expected 19 views, checked %', n; END IF;
|
||||
IF (SELECT count(*) FROM information_schema.views WHERE table_schema = 'public' AND table_name LIKE 'v\_%') <> 19 THEN
|
||||
RAISE EXCEPTION 'public holds a view this list does not name';
|
||||
END IF;
|
||||
|
||||
@@ -32,5 +32,5 @@ BEGIN
|
||||
RAISE EXCEPTION USING ERRCODE = 'TT001', MESSAGE = 'fuel_ro created a table';
|
||||
EXCEPTION WHEN insufficient_privilege OR read_only_sql_transaction THEN NULL;
|
||||
END;
|
||||
RAISE NOTICE 'read-only role: 18 views readable, each with operator_uid; fm and licence refused; writes refused';
|
||||
RAISE NOTICE 'read-only role: 19 views readable, each with operator_uid; fm and licence refused; writes refused';
|
||||
END $$;
|
||||
@@ -52,9 +52,9 @@ CREATE FUNCTION rules_test.sha(c text) RETURNS text LANGUAGE sql IMMUTABLE AS
|
||||
$$ SELECT repeat(c, 64) $$;
|
||||
|
||||
-- Two customers ------------------------------------------------------------------------------
|
||||
INSERT INTO fm.operator (operator_uid, name, label_issuer, programme, timezone) VALUES
|
||||
('OPR-0001', 'Customer A', 'Issuer A', 'Programme A', 'Asia/Kabul'),
|
||||
('OPR-0002', 'Customer B', 'Issuer B', 'Programme B', 'Europe/Berlin');
|
||||
INSERT INTO fm.operator (operator_uid, name, label_issuer, programme, timezone, keycloak_group) VALUES
|
||||
('OPR-0001', 'Customer A', 'Issuer A', 'Programme A', 'Asia/Kabul', '/customer-a'),
|
||||
('OPR-0002', 'Customer B', 'Issuer B', 'Programme B', 'Europe/Berlin', '/customer-b');
|
||||
INSERT INTO fm.source_document (sha256, operator_uid, source_family, kind, issuer, title, corpus_path, contains_personal_data) VALUES
|
||||
(rules_test.sha('a'), 'OPR-0001', 'family-a', 'operator_workbook', 'Issuer A', 'workbook A', 'a.xlsx', false),
|
||||
(rules_test.sha('b'), 'OPR-0002', 'family-b', 'operator_workbook', 'Issuer B', 'workbook B', 'b.xlsx', false);
|
||||
@@ -102,6 +102,11 @@ SELECT 'stock:' || k, rules_test.sha('d'), 'OPR-0001', rules_test.visit(k), 'SIT
|
||||
FROM (VALUES ('SITE-0001:2026-01-01:1', 100.0), ('SITE-0001:2026-01-11:1', 50.0)) x(k, l);
|
||||
|
||||
SELECT rules_test.ok('fixture holds two customers', 'SELECT 1');
|
||||
SELECT rules_test.no('customers: one Keycloak group is one customer', 'duplicate key',
|
||||
'UPDATE fm.operator SET keycloak_group = ''/customer-a'' WHERE operator_uid = ''OPR-0002''');
|
||||
SELECT rules_test.no('customers: the group is a top-level one', 'keycloak_group_check',
|
||||
'UPDATE fm.operator SET keycloak_group = ''/customer-a/team'' WHERE operator_uid = ''OPR-0001''');
|
||||
SELECT rules_test.eq('customers: v_operator gives the group', 'SELECT operator_uid FROM public.v_operator WHERE keycloak_group = ''/customer-b''', 'OPR-0002');
|
||||
|
||||
-- Rule 1: baseline versions --------------------------------------------------------------------
|
||||
\set bv 'INSERT INTO fm.baseline_version (code, operator_uid, holder_uid, metric, population, population_key, window_start, window_end, policy_version, meets_policy, computed_at, computed_by_role'
|
||||
|
||||
@@ -69,7 +69,7 @@ services:
|
||||
image: quay.io/oauth2-proxy/oauth2-proxy:v7.15.5
|
||||
command:
|
||||
- --provider=keycloak-oidc
|
||||
- --oidc-issuer-url=${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/thedigitalbridge}
|
||||
- --oidc-issuer-url=${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/petrolyst}
|
||||
- --client-id=petrolyst
|
||||
- --client-secret-file=/run/secrets/keycloak_petrolyst_client_secret
|
||||
- --cookie-secret-file=/run/secrets/oauth2_proxy_cookie_secret
|
||||
@@ -114,6 +114,6 @@ volumes:
|
||||
|
||||
secrets:
|
||||
keycloak_petrolyst_client_secret:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/shared/keycloak_petrolyst_client_secret
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/shared/keycloak_petrolyst_realm_client_secret
|
||||
oauth2_proxy_cookie_secret:
|
||||
file: ${PETROLYST_ROOT:-/opt/petrolyst}/secrets/traefik/oauth2_proxy_cookie_secret
|
||||
Reference in new issue
Block a user