diff --git a/README.md b/README.md index 91c350c..298fd5c 100644 --- a/README.md +++ b/README.md @@ -110,11 +110,10 @@ deploy secrets. enrols first. Other clients of the realm are unaffected. There is no silent SSO between Superset and the dashboard. 4. Give people roles: Users → (user) → Role mapping → Assign role → filter by clients → - `petrolyst`. Group mappings work too. ATOMA's users get `atoma-reader` and nothing else. - They then see only the ATOMA dashboard: no menus, nothing to edit, and Settings offers only - Logout (Superset role "ATOMA Reader", `petrolyst-insights/roles/atoma_reader.py`). - A realm that already has the client gets the new role by hand (Clients → petrolyst → - Roles → Create role) or by importing again with "Skip". + `petrolyst`. Group mappings work too. ATOMA's users get `atoma-reader` and nothing else: + it opens the Petrolyst app (petrolyst-platform) and maps to no Superset role, so in + Superset they see nothing. A realm that already has the client gets the role by hand + (Clients → petrolyst → Roles → Create role) or by importing again with "Skip". 5. Users of the Traefik dashboard need **Email verified** switched on, because oauth2-proxy refuses unverified addresses. @@ -165,7 +164,6 @@ becomes the first Superset admin; there is no local admin account. | fuel-db from the Mac | `ssh -L 5433:127.0.0.1:5433 petrolyst`, then connect to `localhost:5433`, database `fuel` | | Logs | `docker compose -p fuel-monitoring logs -f superset` (or `-p traefik`) | | Superset break-glass | `docker exec -it fuel-monitoring-superset-1 superset fab ...` | -| Reset the role "ATOMA Reader" (after an edit in the UI) | `docker exec fuel-monitoring-superset-1 /app/.venv/bin/python /app/petrolyst-roles/atoma_reader.py`; a deploy does it whenever the Superset image changes | **Manual deploy**, the only way to deploy (there is no CI runner). On the Mac, in this repository: diff --git a/deploy/lib.sh b/deploy/lib.sh index c1b44a3..b07d641 100644 --- a/deploy/lib.sh +++ b/deploy/lib.sh @@ -21,7 +21,7 @@ KEYCLOAK_REALM_URL=${KEYCLOAK_REALM_URL:-https://iam.thedigitalbridge.io/realms/ PETROLYST_ROOT=$ROOT export ATOMA_HOST TRAEFIK_HOST KEYCLOAK_REALM_URL PETROLYST_ROOT -# --host pins the daemon, as in petrolyst-ui/deploy/deploy.sh: a stale +# --host pins the daemon, as in petrolyst-web/deploy/deploy.sh: a stale # `docker context use` must not send this deploy to another daemon. DOCKER=(docker --host unix:///var/run/docker.sock) @@ -33,8 +33,6 @@ SUPERSET_INPUTS=( stacks/fuel-monitoring/superset/init.sh petrolyst-insights/superset_config.py petrolyst-insights/petrolyst_security.py - petrolyst-insights/petrolyst_reader.py - petrolyst-insights/templates petrolyst-insights/roles petrolyst-insights/assets/branding petrolyst-insights/patches diff --git a/keycloak/petrolyst-partial-import.json b/keycloak/petrolyst-partial-import.json index ac31361..8c5c7d1 100644 --- a/keycloak/petrolyst-partial-import.json +++ b/keycloak/petrolyst-partial-import.json @@ -49,7 +49,7 @@ { "name": "superset-admin", "description": "Superset Admin at atoma.petrolyst.com" }, { "name": "superset-editor", "description": "Superset Alpha: build charts and dashboards" }, { "name": "superset-viewer", "description": "Superset Gamma: view dashboards shared with the role" }, - { "name": "atoma-reader", "description": "Superset ATOMA Reader: read the ATOMA dashboard only" }, + { "name": "atoma-reader", "description": "ATOMA's users: the Petrolyst app (no role in Superset)" }, { "name": "traefik-admin", "description": "Traefik dashboard at traefik.petrolyst.com" } ] } diff --git a/petrolyst-insights/README.md b/petrolyst-insights/README.md index 478c077..0f364c4 100644 --- a/petrolyst-insights/README.md +++ b/petrolyst-insights/README.md @@ -14,12 +14,10 @@ and where it runs lives in `stacks/`, `traefik/`, `deploy/` and `host/`. | Path | What it is | In the image at | |---|---|---| | `superset_config.py` | Keycloak sign-in, role mapping, proxy and cookie settings, Redis caches, theme and chart palettes | `/app/pythonpath/` | -| `petrolyst_security.py` | `PetrolystSecurityManager`: user info and roles from the Keycloak ID token, `/login/` straight to Keycloak, `/logout/` through Keycloak; takes the reader rules | `/app/pythonpath/` | -| `petrolyst_reader.py` | readers ("ATOMA Reader"): reach only dashboards granted to the role, own nothing, get no SQL; land on the ATOMA dashboard; `register` is `FLASK_APP_MUTATOR` | `/app/pythonpath/` | -| `templates/` | `head_custom_extra.html`: loads `reader.css` for readers on every page | `/app/pythonpath/templates/` | -| `roles/` | `atoma_reader.py`: the role "ATOMA Reader" with exactly its permissions, run by `superset-init`; `atoma_guest.py`: the role "ATOMA Guest" for the Petrolyst app's embedded dashboards, run by hand (see "Embedding") | `/app/petrolyst-roles/` | -| `assets/branding/` | logos (full colour and white): the wordmark alone for the navbar, and the stacked logo; favicons, app icons; `reader.css`; ATOMA's logo for the fleet map | `/app/superset/static/assets/branding/`, served at `/static/assets/branding/…` | -| `patches/` | runtime patches, `fix_*.py` with `--verify` (see its README): `fix_redact_query_for_readers.py` | `/app/patches/petrolyst/`, applied during the build | +| `petrolyst_security.py` | `PetrolystSecurityManager`: user info and roles from the Keycloak ID token, `/login/` straight to Keycloak, `/logout/` through Keycloak | `/app/pythonpath/` | +| `roles/` | `atoma_guest.py`: the role "ATOMA Guest" for the Petrolyst app's embedded dashboards, with exactly its permissions; run by hand (see "Embedding") | `/app/petrolyst-roles/` | +| `assets/branding/` | logos (full colour and white): the wordmark alone for the navbar, and the stacked logo; favicons, app icons; ATOMA's logo, which the Petrolyst app shows in its map's corner (`PETROLYST_OPERATOR_LOGO`) | `/app/superset/static/assets/branding/`, served at `/static/assets/branding/…` | +| `patches/` | runtime patches, `fix_*.py` with `--verify` (see its README); none at present | `/app/patches/petrolyst/`, applied during the build | | `dashboards/` | exported dashboards and import scripts (see its README) | not in the image | How the image consumes this folder: `stacks/fuel-monitoring/compose.yaml` passes it to the build @@ -54,7 +52,7 @@ On the server they come from `/opt/petrolyst/secrets/` (see the deployment READM | `superset-admin` | Admin | | `superset-editor` | Alpha | | `superset-viewer` | Gamma | - | `atoma-reader` | ATOMA Reader: the ATOMA dashboard only, read-only (SPEC.md §6) | + | `atoma-reader` | none (Public): ATOMA's users use the Petrolyst app, not Superset (PD-13) | | none | Public (no permissions) | - `DASHBOARD_RBAC` is on: a viewer sees a dashboard once an admin adds the Gamma role (or a @@ -64,7 +62,7 @@ On the server they come from `/opt/petrolyst/secrets/` (see the deployment READM ## Embedding (local trial only) -The Petrolyst app (PD-12, petrolyst-platform `app/` and `services/app/`) shows dashboards inside +The Petrolyst app (PD-12, petrolyst-platform `app/` and `server/`) shows dashboards inside its Insights lens with Superset guest passes. This is switched on **only in the local scratch stack** (`.local/fmdb-scratch`); `superset_config.py` here leaves it off, and loads without the guest secret. @@ -72,10 +70,10 @@ guest secret. - The scratch stack's `superset_config_local.py` turns on `EMBEDDED_SUPERSET`, names the guest role `GUEST_ROLE_NAME = "ATOMA Guest"`, reads the signing secret with `_read_secret("superset_guest_token_secret")` and expects the audience `petrolyst-app`. The - app service signs the guest passes itself with the same secret. -- `roles/atoma_guest.py` gives "ATOMA Guest" the same permissions as "ATOMA Reader". Which + Petrolyst server signs the guest passes itself with the same secret. +- `roles/atoma_guest.py` gives "ATOMA Guest" exactly what a dashboard needs to be read. Which dashboard a guest sees comes from the pass, not the role. It is not in `init.sh`; run it once, - and again whenever the permissions in `atoma_reader.py` change: + and again whenever its permissions change: docker exec /app/.venv/bin/python /app/petrolyst-roles/atoma_guest.py @@ -85,8 +83,8 @@ Production embedding is an open point of PD-12 (`.context/design-decisions.md`). ## Look and feel -Source: `petrolyst-ui/design/brand/` (tokens `07_UI_Tokens/petrolyst_tokens.css`, logos -`public/brand/`, favicons `03_Favicon_App_Icons/`), copied on 2026-10-23 from petrolyst-ui commit +Source: `petrolyst-web/design/brand/` (tokens `07_UI_Tokens/petrolyst_tokens.css`, logos +`public/brand/`, favicons `03_Favicon_App_Icons/`), copied on 2026-10-23 from petrolyst-web (then petrolyst-ui) commit `2b2e694`. Copy again when the brand changes. - Light theme: action teal `#00695C` for primary and links, navy `#0B1D33` text, `#F8FAFC` diff --git a/petrolyst-insights/assets/branding/petrolyst_wordmark_fullcolor.svg b/petrolyst-insights/assets/branding/petrolyst_wordmark_fullcolor.svg index 46d7ee0..37cea0a 100644 --- a/petrolyst-insights/assets/branding/petrolyst_wordmark_fullcolor.svg +++ b/petrolyst-insights/assets/branding/petrolyst_wordmark_fullcolor.svg @@ -1,5 +1,5 @@ - diff --git a/petrolyst-insights/assets/branding/petrolyst_wordmark_white.svg b/petrolyst-insights/assets/branding/petrolyst_wordmark_white.svg index 8b53690..a1f5782 100644 --- a/petrolyst-insights/assets/branding/petrolyst_wordmark_white.svg +++ b/petrolyst-insights/assets/branding/petrolyst_wordmark_white.svg @@ -1,5 +1,5 @@ - diff --git a/petrolyst-insights/assets/branding/reader.css b/petrolyst-insights/assets/branding/reader.css deleted file mode 100644 index 6ae6b24..0000000 --- a/petrolyst-insights/assets/branding/reader.css +++ /dev/null @@ -1,154 +0,0 @@ -/* Readers only (petrolyst_reader.py, templates/head_custom_extra.html). Hides what Superset 6.1 - shows whatever the permissions. Cosmetic: what protects is the role and the reader rules. */ - -/* The dashboard's title row: name, favourite, refresh, owner, last change, "…". */ -[data-test="dashboard-header-container"] { - display: none !important; -} - -/* ATOMA's Fleet tab: no filter column and its resize handle (no filter applies there); the - column is empty then, so the map takes the full width. Site details keeps it. Also hidden - until the dashboard's grid is there, so it does not flash up while the Fleet tab loads. - dashboards/atoma/build_bundle.py checks that this names the Fleet tab. */ -body:has(.ant-tabs-tab-active[data-node-key="TAB-fleet"]) [data-test="dashboard-filters-panel"], -body:has(.ant-tabs-tab-active[data-node-key="TAB-fleet"]) div:has(+ [data-test="dashboard-filters-panel"]), -body:not(:has([data-test="grid-container"])) [data-test="dashboard-filters-panel"], -body:not(:has([data-test="grid-container"])) div:has(+ [data-test="dashboard-filters-panel"]) { - display: none !important; -} - -/* Superset sizes blocks in pixels and measures again only about 0.3 s after loading, so the - map would start beside a column that is not there and then widen. Its block spans all 12 - columns (build_bundle.py checks it), so the full width is always right. Its column grows to - the row's width; the blocks inside it follow. - Height: down to the window's bottom edge instead of the layout's 800 px. Above the block are - 123 px (navbar, margin, tab row) and below it 16, so the map ends 32 px above the edge, as it - does from the sides. At least 592 px: the map's own minimum height, 560, and the padding. */ -.grid-row > .dragdroppable-column:has(#MARKDOWN-fleet-map) { - flex-grow: 1; -} -#MARKDOWN-fleet-map > .resizable-container { - width: 100% !important; - max-width: 100% !important; - height: max(592px, 100vh - 139px) !important; -} - -/* Navbar: "+" (new) and the theme switcher. */ -li.ant-menu-submenu:has(> .ant-menu-submenu-title[data-menu-id$="-new-dropdown"]), -li.ant-menu-submenu:has(> .ant-menu-submenu-title[data-menu-id$="-theme-sub-menu"]) { - display: none !important; -} - -/* Settings: Info and the About group (Superset's name and version); User → Logout remains. */ -li.ant-menu-item[data-menu-id$="-info"], -li.ant-menu-item-group:has([data-menu-id$="-about-info"]), -li.ant-menu-item-divider:has(+ li.ant-menu-item-group [data-menu-id$="-about-info"]) { - display: none !important; -} - -/* ATOMA's top-level tabs as a lens switch in the navbar, centred: Map and Insights (owner, - 29 Oct). Pinned to the window, a scroll-linked animation moves it up as fast as the navbar - scrolls away. Without that support, or narrower than 900 px, the tab row stays in the page. - With the switch, the map's block grows by the tab row's 46 px: 93 px are left above and - below it. */ -@supports (animation-timeline: scroll()) { - @media (min-width: 900px) { - #TABS-main > .ant-tabs-nav { - position: fixed; - top: 9px; - left: 50%; - transform: translateX(-50%); - z-index: 1000; - margin: 0; - padding: 4px; - border-radius: 12px; - background: rgba(11, 29, 51, 0.06); - animation: petrolyst-lens-scroll linear both; - animation-timeline: scroll(root block); - animation-range: 0 61px; - } - #TABS-main > .ant-tabs-nav::before, - #TABS-main > .ant-tabs-nav .ant-tabs-ink-bar, - #TABS-main > .ant-tabs-nav .anchor-link-container { - display: none; - } - #TABS-main > .ant-tabs-nav .ant-tabs-nav-list { - gap: 4px; - } - #TABS-main > .ant-tabs-nav .ant-tabs-tab { - margin: 0 !important; - padding: 7px 18px 7px 14px !important; - border-radius: 9px !important; - font-size: 15px; - font-weight: 600; - color: #3b4a5c; - } - #TABS-main > .ant-tabs-nav .ant-tabs-tab-active { - background: #0b1d33 !important; - } - #TABS-main > .ant-tabs-nav .ant-tabs-tab-active .ant-tabs-tab-btn { - color: #fff !important; - } - #TABS-main > .ant-tabs-nav [data-test="span-title"]::before { - content: ""; - display: inline-block; - width: 18px; - height: 18px; - margin-right: 8px; - vertical-align: -3px; - background: currentColor; - -webkit-mask: var(--lens-icon) center / contain no-repeat; - mask: var(--lens-icon) center / contain no-repeat; - } - #TABS-main .ant-tabs-tab[data-node-key="TAB-fleet"] { - --lens-icon: url("data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2024%2024'%20fill='none'%20stroke='black'%20stroke-width='2'%20stroke-linecap='round'%20stroke-linejoin='round'%3E%3Cpath%20d='M9%204%203%206v14l6-2%206%202%206-2V4l-6%202-6-2z'/%3E%3Cpath%20d='M9%204v14M15%206v14'/%3E%3C/svg%3E"); - } - #TABS-main .ant-tabs-tab[data-node-key="TAB-site"] { - --lens-icon: url("data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2024%2024'%20fill='none'%20stroke='black'%20stroke-width='2'%20stroke-linecap='round'%20stroke-linejoin='round'%3E%3Cpath%20d='M4%2020V10M10%2020V4M16%2020v-7M22%2020H2'/%3E%3C/svg%3E"); - } - #MARKDOWN-fleet-map > .resizable-container { - height: max(592px, 100vh - 93px) !important; - } - /* A third lens, AI, as a teaser: greyed, tagged PRO, not a tab and not clickable (owner, - 29 Oct; no AI lens is built until Botswerk exists). Drawn by the tab list itself, so the - dashboard's layout stays Map and Insights. From 1100 px only: narrower, the wider switch - would cover the logo. */ - @media (min-width: 1100px) { - #TABS-main > .ant-tabs-nav .ant-tabs-nav-list::before { - content: "AI" / "AI, available with Petrolyst Pro"; - order: 1; - display: flex; - align-items: center; - padding: 7px 58px 7px 40px; - border-radius: 9px; - font-size: 15px; - font-weight: 600; - line-height: 22px; - color: #8a96a3; - cursor: not-allowed; - background: url("data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2024%2024'%20fill='none'%20stroke='%238a96a3'%20stroke-width='2'%20stroke-linecap='round'%20stroke-linejoin='round'%3E%3Cpath%20d='M9.937%2015.5A2%202%200%200%200%208.5%2014.063l-6.135-1.582a.5.5%200%200%201%200-.962L8.5%209.936A2%202%200%200%200%209.937%208.5l1.582-6.135a.5.5%200%200%201%20.963%200L14.063%208.5A2%202%200%200%200%2015.5%209.937l6.135%201.581a.5.5%200%200%201%200%20.964L15.5%2014.063a2%202%200%200%200-1.437%201.437l-1.582%206.135a.5.5%200%200%201-.963%200z'/%3E%3Cpath%20d='M20%203v4M22%205h-4'/%3E%3C/svg%3E") 14px center / 18px 18px no-repeat; - } - #TABS-main > .ant-tabs-nav .ant-tabs-nav-list::after { - content: "PRO" / ""; - position: absolute; - right: 14px; - top: 50%; - transform: translateY(-50%); - padding: 3px 7px; - border-radius: 999px; - background: #0f7b6c; - color: #fff; - font-size: 10px; - font-weight: 700; - letter-spacing: 0.06em; - line-height: 1; - pointer-events: none; - } - } - } -} -@keyframes petrolyst-lens-scroll { - to { - transform: translate(-50%, -61px); - } -} diff --git a/petrolyst-insights/dashboards/README.md b/petrolyst-insights/dashboards/README.md index 9487cdb..17fbe2f 100644 --- a/petrolyst-insights/dashboards/README.md +++ b/petrolyst-insights/dashboards/README.md @@ -3,16 +3,11 @@ Superset dashboards as code and the operator scripts that import them. Not baked into the image, so changes here do not rebuild or restart Superset. -- `atoma/`: the ATOMA dashboard, with a Map tab (the fleet map) and an Insights tab (one site's - details), shown to readers as a lens switch in the navbar. `SPEC.md` is - the design; `build_bundle.py` generates the import bundle and `import_bundle.py` imports it - (overwriting the objects the bundle defines). `dashboard.css` is the dashboard's own CSS, - which the bundle carries. - - The Fleet tab is an iframe of the fleet-map view from the petrolyst-platform repository. It - needs that view and its read service served on Superset's own origin under `/views`, and the - image's `superset_config.py` with the iframe setting (SPEC.md §6). Without them the tab stays - empty, and the Site details tab works as before. +- `atoma/`: ATOMA's two dashboards for the Petrolyst app (PD-12, PD-13; SPEC.md §10): + `atoma-overview` (fleet totals) and `atoma-site` (one site, its four sections as tabs). The + app embeds them; admins open them in Superset, with the filter column. `SPEC.md` is the + design; `build_bundle.py` generates the import bundle and `import_bundle.py` imports it + (overwriting the objects the bundle defines). Build and import: @@ -25,23 +20,16 @@ image, so changes here do not rebuild or restart Superset. Exit status 2 means a dataset's columns differ from its view: regenerate `atoma/columns.tsv` (command in its header), rebuild and import again. - The import attaches the role "ATOMA Reader" to the dashboard (a bundle cannot carry roles), - so the role must exist first: `superset-init` creates it (`petrolyst-insights/roles/`). The - Fleet tab shows ATOMA's logo from `petrolyst-insights/assets/branding/atoma_logo.svg`, in the - image. + An import never deletes. Where they exist, delete these by hand after importing: the + dashboards from before 28 Oct ("ATOMA - Sites", "ATOMA - Site Detail") and their charts M1, + M3, M4 and M5; the ATOMA dashboard (slug `atoma`, retired 29 Oct with PD-13) and its chart D0 + "Site"; the one-tab dashboards `atoma-site-summary`, `-fuel`, `-runtime` and `-baseline` + (replaced by `atoma-site` on 29 Oct); and the role "ATOMA Reader". - An import never deletes. On an instance that still has the two dashboards from before - 28 Oct 2026 ("ATOMA - Sites" and "ATOMA - Site Detail") and their charts M1, M3, M4 and M5, - delete those by hand after importing. - - The bundle also carries the Petrolyst app's dashboards (PD-12, SPEC.md §10): `atoma-overview` - (fleet totals) and `atoma-site` (one site, its four sections as tabs; it replaced - `atoma-site-summary`, `-fuel`, `-runtime` and `-baseline` on 29 Oct: delete those by hand - where they exist). - They get no role; the app shows them with guest passes (see "Embedding" in the parent - README; local stack only). Add `--embedded-out FILE --embed-domain ` to the + The dashboards get no role; the app shows them with guest passes (see "Embedding" in the + parent README; local stack only). Add `--embedded-out FILE --embed-domain ` to the import to make them embeddable from the app's origin; FILE then receives - `{slug: embedding id}` for the app service. The ids stay the same across imports. Local + `{slug: embedding id}` for the Petrolyst server. The ids stay the same across imports. Local stack: docker exec fmdb-scratch-superset /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip \ diff --git a/petrolyst-insights/dashboards/atoma/SPEC.md b/petrolyst-insights/dashboards/atoma/SPEC.md index 464772c..a7e6c43 100644 --- a/petrolyst-insights/dashboards/atoma/SPEC.md +++ b/petrolyst-insights/dashboards/atoma/SPEC.md @@ -1,13 +1,17 @@ -# ATOMA dashboard: Fleet map and Site details +# ATOMA dashboards: Overview and Site, for the Petrolyst app -**Status: built and tested on a local Superset (P2, 24 Oct 2026); merged into one dashboard with -the fleet-map view on 28 Oct 2026 (§9); not imported live.** -- `build_bundle.py` generates the dashboard, every dataset and every chart from this +**Status (29 Oct 2026, PD-13): two dashboards, `atoma-overview` and `atoma-site` (§10), built and +tested on a local Superset; not imported live.** The Petrolyst app embeds them, and admins open +them in Superset with the filter column. +- `build_bundle.py` generates the dashboards, every dataset and every chart from this specification, as one Superset import bundle; `import_bundle.py` imports it (§6). -- The local tests changed parts of this design. Each change is marked *(P2)* and §8 lists them. -- The merge of 28 Oct (owner's decisions PD-5 and PD-6, `Petrolist/.context/design-decisions.md`) - replaced the Sites Map dashboard by a Fleet tab holding the fleet-map view of petrolyst-platform. - Changed parts are marked *(28 Oct)*; §9 lists the results. +- **Retired 29 Oct (PD-13):** the "ATOMA" dashboard (slug `atoma`) with its Map tab (the React + fleet-map view of petrolyst-platform) and Insights tab, its D0 header row, and the Superset + reader setup ("ATOMA Reader", `reader.css`). ATOMA's users use the Petrolyst app. §2, §3, §9 and + the readers' part of §6 describe them and stay as history. §4's data rules and four tabs are + current: they are `atoma-site`'s tabs, without D0 (the app shows the site's header). +- Earlier changes are marked *(P2)* (local tests of 24 Oct, §8) and *(28 Oct)* (one dashboard + with the fleet-map view, §9). - Nothing is imported into the live Superset without the owner's go-ahead (phase P3). **Datasets.** The `public.v_*` views of @@ -20,8 +24,7 @@ the fleet-map view on 28 Oct 2026 (§9); not imported live.** - the site-specific facts left out here (positions, distances, pilot sites) are there too; - Q-numbers refer to its §7, and phases P0–P5 to its §8. -One dashboard (two until 28 Oct) replaces the earlier tab-per-site design; nothing in it is -specific to a site. +Nothing in the dashboards is specific to a site. ## 1. Capabilities of the installed Superset 6.1.0 @@ -53,7 +56,7 @@ still holds: the view builds the same `?native_filters=` link. No custom frontend and no patch are needed. -## 2. Navigation flow +## 2. Navigation flow *(retired, PD-13: history)* *(28 Oct)* One dashboard, two top-level tabs. *(29 Oct)* They are named **Map** (`TAB-fleet`) and **Insights** (`TAB-site`), the dashboard's lenses; earlier text in this document calls them Fleet and @@ -93,7 +96,7 @@ Until 28 Oct there were two dashboards: "ATOMA - Sites" (slug `atoma-sites`: a d filters, a sites list) and "ATOMA - Site Detail" (slug `atoma-site-detail`), linked by "Open site detail" and "← Sites map". -## 3. Fleet tab: the fleet-map view *(28 Oct)* +## 3. Fleet tab: the fleet-map view *(28 Oct; retired, PD-13: history)* The Fleet tab holds one Markdown block with an iframe of the fleet-map view, a page of petrolyst-platform (`views/src/fleet-map/`) served under `/views/` on Superset's own origin. The @@ -135,7 +138,10 @@ thousands of sites stay fast. **Sites without an accepted location** are not plotted. The key counts them, and the Site filter in Site details lists every site, placed or not. -## 4. Site details tab (any site) +## 4. Site details: the four tabs of `atoma-site` (any site) + +*(PD-13)* What follows about D0 (the site header row) and the Fleet tab is history: the app shows +the site's header, and its Map lens replaced the Fleet tab. **Filters** - **Site** (`NATIVE_FILTER-site`): @@ -248,10 +254,12 @@ is grouped or filtered by `metric` and `population_key` (both in `v_interval`), across methods or generator sets. The site's L/h figure itself comes only from a baseline version (`v_site`), never from this metric. There is no AVG metric on ratio columns. -**Site details link** *(28 Oct)*. The fleet-map view builds it (petrolyst-platform -`views/src/shared/deepLink.ts`) from its iframe address: the dashboard path, the Site filter's id -and the tab. The rison shape is the one the sites list used until 28 Oct, confirmed in P2, and the -view's tests compare it with that list's Trino output: +**Site details link** *(28 Oct; history since PD-13)*. The fleet-map view built it +(petrolyst-platform `views/src/shared/deepLink.ts`, retired) from its iframe address: the +dashboard path, the Site filter's id and the tab. The Petrolyst app uses the same rison shape for +the Site filter's preset (`app/src/app/core/site-filter.ts`). The rison shape is the one the sites +list used until 28 Oct, confirmed in P2, and the view's tests compared it with that list's Trino +output: ``` /superset/dashboard/atoma/?native_filters= @@ -312,7 +320,8 @@ appears on every Superset page, including the stock dashboard list: a capability same sign-in *(28 Oct: not built yet; locally it has none)*. No Gamma role until the ATOMA-facing edit. -**ATOMA's readers** *(owner, 29 Oct)*. The Superset role **"ATOMA Reader"** (Keycloak client role +**ATOMA's readers** *(owner, 29 Oct; retired the same day, PD-13: readers use the Petrolyst +app; kept as history)*. The Superset role **"ATOMA Reader"** (Keycloak client role `atoma-reader`) reads the ATOMA dashboard and nothing else. - **The role** (`petrolyst-insights/roles/atoma_reader.py`): exactly 12 permissions, set by `superset-init` whenever the image changes, and reset by hand with the command in the README. @@ -357,6 +366,9 @@ ATOMA-facing edit. ## 7. Acceptance checks +*(PD-13)* Written for the retired ATOMA dashboard. Where they name D0 or the fleet map, the app's +site header and Map lens take their place (petrolyst-platform's browser tests). + Run in P2 against a local Superset, and repeated as a smoke test in P3. 1. **New site, no new objects.** Count dashboards, charts and datasets in the metadata database. @@ -425,7 +437,7 @@ The marker palette is unchanged from §3. The dataviz palette check flags slate grey, which is intended for "insufficient evidence", and amber at 2.99:1 contrast. The label is always also shown as text, in the legend, the tooltip and M3. -## 9. Fleet map results (local, 28 Oct 2026) +## 9. Fleet map results (local, 28 Oct 2026) *(history)* Local Superset 6.1.0 rebuilt with the iframe setting, behind a local Traefik that serves Superset and the petrolyst-platform read service on one origin (as on the server); the fuel-db of P2. @@ -547,11 +559,11 @@ These changes have not had an independent inspection (the owner skipped Codex). unchanged), sign-in through Keycloak, the server's routing of `/views`, sign-in for the read service, thousands of sites, and dark mode. -## 10. The Petrolyst app's dashboards *(PD-12, local trial, 29 Oct 2026)* +## 10. The Petrolyst app's dashboards *(PD-12, PD-13; local, 29 Oct 2026)* The Petrolyst app (petrolyst-platform `app/`, PD-12) shows Superset dashboards inside its -Insights lens, with guest passes from its app service. `build_bundle.py` builds them beside the -unchanged ATOMA dashboard: +Insights lens, with guest passes from the Petrolyst server (petrolyst-platform `server/`). +`build_bundle.py` builds them (since PD-13 the only dashboards in the bundle): | Slug | Content | Filters (set by the app; Superset's filter column is hidden) | |---|---|---| @@ -569,11 +581,9 @@ unchanged ATOMA dashboard: recorded values across sites (`OVERVIEW_METRICS`); per-site rates (L/h, L/day) are never averaged; money is grouped by `currency_text`, a missing currency is its own row, and no total crosses currencies. `build_bundle.py` checks both. -- They carry no CSS; `dashboard.css` and `reader.css` are checked against the ATOMA dashboard - only. +- They carry no CSS. - `import_bundle.py --embedded-out FILE --embed-domain URL` makes them embeddable from the app's - origin and writes `{slug: embedding id}` for the app service; the reader role stays on the - ATOMA dashboard only. + origin and writes `{slug: embedding id}` for the Petrolyst server. - Embedding is switched on in the local stack only (`.local/fmdb-scratch`, `superset_config_local.py`); production keeps it off until PD-12's open points are settled. - **Checked (local):** a self-signed guest pass opens its own dashboard and nothing else (another diff --git a/petrolyst-insights/dashboards/atoma/build_bundle.py b/petrolyst-insights/dashboards/atoma/build_bundle.py index 0ca9891..1857b6c 100644 --- a/petrolyst-insights/dashboards/atoma/build_bundle.py +++ b/petrolyst-insights/dashboards/atoma/build_bundle.py @@ -1,10 +1,9 @@ #!/usr/bin/env python3 -"""Generate the ATOMA dashboard as one Superset 6.1 import bundle. +"""Generate ATOMA's dashboards for the Petrolyst app as one Superset 6.1 import bundle. -Implements SPEC.md in this folder: the "ATOMA" dashboard with a Map tab (the -fleet-map view of petrolyst-platform, embedded as an iframe) and an Insights tab, -one dataset per public.v_* view and every chart. Nothing in the bundle names -a site. +Implements SPEC.md in this folder: the two dashboards the Petrolyst app embeds, "ATOMA · Overview" +(atoma-overview) and "ATOMA · Site" (atoma-site, the four detail tabs), one dataset per +public.v_* view and every chart. Nothing in the bundle names a site. build_bundle.py --database-uuid [--out atoma-dashboards.zip] @@ -17,9 +16,8 @@ unchanged). Before writing, the bundle is checked: every column a chart or filter uses exists in its dataset, every chart in a Site filter's scope has site_label, the -fleet map's address names this dashboard, its Site filter and the Insights tab, -every layout id that dashboard.css or reader.css names exists, and no row is wider -than 12. Requires PyYAML. +Overview sums recorded values only and money only per currency, and no row is +wider than 12. Requires PyYAML. """ from __future__ import annotations @@ -43,36 +41,11 @@ DATABASE_NAME = "Fuel (Trino)" DATABASE_URI = "trino://superset@trino:8080/fuel" # as superset-init registers it CATALOG, SCHEMA = "fuel", "public" -SLUG = "atoma" SITE_FILTER = "NATIVE_FILTER-site" PERIOD_FILTER = "NATIVE_FILTER-period" -FLEET_TAB = "TAB-fleet" -SITE_TAB = "TAB-site" -# The tabs' names: the lenses Map and Insights (owner, 29 Oct); readers see them as a switch -# in the navbar (assets/branding/reader.css). -FLEET_TAB_TITLE = "Map" -SITE_TAB_TITLE = "Insights" - -# SPEC §3: the Map tab is the fleet-map view of petrolyst-platform, served on the same -# origin under /views (superset_config.py lets the iframe through the Markdown sanitiser). -# Its parameters name where "Site details" leads: this dashboard, its Site filter and the -# Insights tab, with that tab's name for the map's key. The view builds the native_filters -# link from them. -# The operator's logo in the map's bottom right corner: a file in assets/branding/, served by -# Superset under /static/assets/branding/ (same origin, so the view's CSP allows it). -OPERATOR_LOGO = "atoma_logo.svg" -# The dashboard's own CSS (the Insights tab's inner tabs as section buttons), and the -# readers' stylesheet, which names this layout's ids too (the lens switch, the map's size). -DASHBOARD_CSS = HERE / "dashboard.css" -READER_CSS = HERE.parent.parent / "assets" / "branding" / "reader.css" -FLEET_MAP_SRC = (f"/views/fleet-map/?detail=/superset/dashboard/{SLUG}/" - f"&filter={SITE_FILTER}&tab={SITE_TAB}&tab_label={SITE_TAB_TITLE}" - f"&logo=/static/assets/branding/{OPERATOR_LOGO}&logo_alt=ATOMA") -FLEET_MAP = (f'') # SPEC §3: neutral hues, no red or green, keyed by v_site.baseline_status_label. The -# fleet-map view uses the same colours (petrolyst-platform views/src/shared/status.ts). +# Petrolyst app's map uses the same colours (petrolyst-platform app/src/app/core/status.ts). STATUS_COLOURS = { "Approved baseline": "#009688", "Eligible - not yet approved": "#3987e5", @@ -96,8 +69,8 @@ MAIN_DTTM = { "v_electricity_cost": "bill_month", } -# Calculated columns: (name, Trino SQL, type, label). The fleet-map view's read service -# (petrolyst-platform services/read, site_status) shows the same two texts. +# Calculated columns: (name, Trino SQL, type, label). The Petrolyst server's site_status +# (petrolyst-platform server/) shows the same two texts. CALCULATED = { "v_site": [ ("lph_text", "COALESCE(CAST(lph_estimate AS varchar), 'not available')", "VARCHAR", "L/h"), @@ -588,9 +561,6 @@ def detail_charts() -> list[Chart]: return [ # Handlebars would render these as cards, but it compiles templates with # eval, which Superset's own CSP blocks. One-row tables need no CSP change. - table("D0", "Site", "v_site", [ - "site_code", "site_name", "province", "pilot_status", "baseline_status_text", - "location_status"]), table("S1", "Fuel per running hour", "v_site", [ "lph_text", "lph_method", "lph_window_start", "lph_window_end", "baseline_status_text", "baseline_validity_note", "baseline_decision_ref", "baseline_conditions", @@ -799,7 +769,6 @@ TAB_NOTES = { "the map or the sites list. The Viewing period does not apply to this tab.", } -DETAIL_HEADER = [("D0", 12, 24)] DETAIL_TABS = [ ("summary", "Summary", [ [("MARKDOWN:notes-summary", 12, 17)], @@ -982,8 +951,8 @@ def base_metadata(layout: Layout, filters: list[dict[str, Any]], cross_filters: "map_label_colors": {}, "color_scheme_domain": [], "cross_filters_enabled": cross_filters, - # Filters in Superset's left column (owner, 29 Oct). Readers don't see it on the Fleet - # tab (assets/branding/reader.css), so their map gets the full width. + # Filters in Superset's left column (owner, 29 Oct); the Petrolyst app hides the column + # when it embeds a dashboard and sets the filters itself. "filter_bar_orientation": "VERTICAL", "native_filter_configuration": filters, "chart_configuration": {}, @@ -1021,30 +990,12 @@ def build(database_uuid: str, columns_path: Path) -> dict[str, Any]: for view in sorted(views): files[f"datasets/Fuel_Trino/{view}.yaml"] = dataset_yaml(view, views[view], database_uuid) - charts = {c.key: c for c in detail_charts()} - - # SPEC §2: [Map] [Insights]; the site header and the four detail tabs live in - # Insights, so the Map tab shows the map only. - layout = Layout("ATOMA", charts) - layout.markdown["fleet-map"] = FLEET_MAP - layout.markdown.update({f"notes-{slug}": text for slug, text in TAB_NOTES.items()}) - main_tabs = layout.add("TABS-main", "TABS", "GRID_ID", {}) - layout.rows([[("MARKDOWN:fleet-map", 12, 100)]], layout.tab(FLEET_TAB, FLEET_TAB_TITLE, main_tabs), "fleet") - site_tab = layout.tab(SITE_TAB, SITE_TAB_TITLE, main_tabs) - layout.rows([DETAIL_HEADER], site_tab, "header") - layout.tabs(DETAIL_TABS, site_tab, "TABS-detail") - metadata = base_metadata(layout, detail_filters(layout.ids), cross_filters=False) - metadata["label_colors"].update(STOCK_STATUS_COLOURS) - - files["dashboards/ATOMA.yaml"] = dashboard_yaml("atoma", "ATOMA", SLUG, layout, metadata, - DASHBOARD_CSS.read_text()) - dashboards = [(layout, charts, metadata, (DASHBOARD_CSS, READER_CSS))] - # PD-12: the Petrolyst app embeds the Overview and one site dashboard, whose four detail # tabs are Superset's own (owner, 29 Oct). The app shows the site header and chooses site - # and period, so these have no D0 and no CSS. + # and period. Admins open the same dashboards in Superset, with the filter column. overview = {c.key: c for c in overview_charts()} - all_charts = {**charts, **overview} + all_charts = {**{c.key: c for c in detail_charts()}, **overview} + dashboards: list[tuple[Layout, dict[str, Chart], dict[str, Any]]] = [] for key, title in [(OVERVIEW_SLUG, "Overview"), (SITE_SLUG, "Site")]: page = Layout(f"ATOMA · {title}", all_charts) page.markdown.update({f"notes-{k}": v for k, v in TAB_NOTES.items()}) @@ -1061,7 +1012,7 @@ def build(database_uuid: str, columns_path: Path) -> dict[str, Any]: page_metadata["label_colors"].update(STOCK_STATUS_COLOURS) files[f"dashboards/{key}.yaml"] = dashboard_yaml(key, f"ATOMA · {title}", key, page, page_metadata) - dashboards.append((page, placed, page_metadata, ())) + dashboards.append((page, placed, page_metadata)) for chart in all_charts.values(): name = re.sub(r"\W+", "_", f"{chart.key}_{chart.title}").strip("_") @@ -1072,7 +1023,7 @@ def build(database_uuid: str, columns_path: Path) -> dict[str, Any]: def check(files: dict[str, Any], views: dict[str, list[tuple[str, str]]], - dashboards: list[tuple[Layout, dict[str, Chart], dict[str, Any], tuple[Path, ...]]], + dashboards: list[tuple[Layout, dict[str, Chart], dict[str, Any]]], overview: dict[str, Chart]) -> None: problems = [] columns = {v: {c for c, _ in cols} | {c[0] for c in CALCULATED.get(v, [])} @@ -1083,11 +1034,7 @@ def check(files: dict[str, Any], views: dict[str, list[tuple[str, str]]], for view, col in MAIN_DTTM.items(): if col not in columns.get(view, set()): problems.append(f"{view}: main time column {col} missing") - for layout, charts, metadata, stylesheets in dashboards: - for css in stylesheets: - for ref in sorted(set(re.findall(r"#([A-Z]+-[\w-]+)", css.read_text()))): - if ref not in layout.nodes: - problems.append(f"{css.name}: #{ref} is not in the layout") + for layout, charts, metadata in dashboards: placed = set(layout.ids) if placed != set(charts): problems.append(f"charts not placed or unknown: {sorted(placed ^ set(charts))}") @@ -1104,26 +1051,6 @@ def check(files: dict[str, Any], views: dict[str, list[tuple[str, str]]], for key in in_scope: if col not in columns.get(charts[key].view, set()): problems.append(f"{f['id']}: {key} ({charts[key].view}) lacks {col}") - # The fleet map's "Site details" link is built from its iframe address. - fleet = layout.nodes.get("MARKDOWN-fleet-map") - if fleet is not None: - filter_ids = {f["id"] for f in metadata["native_filter_configuration"]} - for needle, what in [(f"detail=/superset/dashboard/{SLUG}/", "this dashboard"), - (f"filter={SITE_FILTER}", "the Site filter"), - (f"tab={SITE_TAB}", "the Insights tab"), - (f"tab_label={SITE_TAB_TITLE}", "the Insights tab's name")]: - if needle not in fleet["meta"]["code"]: - problems.append(f"fleet map: its address does not name {what}") - if SITE_FILTER not in filter_ids or SITE_TAB not in layout.nodes: - problems.append("fleet map: the Site filter or the Insights tab is missing") - if not (HERE.parent.parent / "assets" / "branding" / OPERATOR_LOGO).is_file(): - problems.append(f"fleet map: assets/branding/{OPERATOR_LOGO} is missing") - if metadata.get("filter_bar_orientation") != "VERTICAL": - problems.append("fleet map: the filters must sit in the left column") - if f'data-node-key="{FLEET_TAB}"' not in READER_CSS.read_text(): - problems.append(f"fleet map: reader.css does not hide the filters on {FLEET_TAB}") - if fleet["meta"]["width"] != 12: - problems.append("fleet map: reader.css stretches it, so it must span all 12 columns") # SPEC §10: the Overview sums recorded values only, and money only per currency. for chart in overview.values(): metrics = chart.params.get("metrics") or [chart.params["metric"]] diff --git a/petrolyst-insights/dashboards/atoma/dashboard.css b/petrolyst-insights/dashboards/atoma/dashboard.css deleted file mode 100644 index 25cf6a1..0000000 --- a/petrolyst-insights/dashboards/atoma/dashboard.css +++ /dev/null @@ -1,33 +0,0 @@ -/* The ATOMA dashboard's own CSS, for everyone who opens it (build_bundle.py puts it into the - bundle and checks that every #id here is in the layout). - - Site details' inner tabs (TABS-detail) as a row of section buttons, so they read as a level - below Fleet | Site details (owner, 29 Oct). Aligned with the content's left edge. - Translucent colours: right in the light and the dark theme. */ -#TABS-detail > .ant-tabs-nav { - margin: 0 16px 16px; -} -#TABS-detail > .ant-tabs-nav::before { - border-bottom: 0; -} -#TABS-detail > .ant-tabs-nav .ant-tabs-ink-bar { - display: none; -} -#TABS-detail > .ant-tabs-nav .ant-tabs-nav-list { - gap: 4px; - padding: 4px; - border-radius: 10px; - background: rgba(127, 127, 127, 0.1); -} -#TABS-detail > .ant-tabs-nav .ant-tabs-tab { - margin: 0 !important; - padding: 6px 14px !important; - border-radius: 8px !important; - font-size: 13px; -} -#TABS-detail > .ant-tabs-nav .ant-tabs-tab-active { - background: rgba(0, 150, 136, 0.14) !important; -} -#TABS-detail > .ant-tabs-nav .ant-tabs-tab-active .ant-tabs-tab-btn { - font-weight: 600; -} diff --git a/petrolyst-insights/dashboards/atoma/import_bundle.py b/petrolyst-insights/dashboards/atoma/import_bundle.py index 53d19d5..762401a 100644 --- a/petrolyst-insights/dashboards/atoma/import_bundle.py +++ b/petrolyst-insights/dashboards/atoma/import_bundle.py @@ -1,4 +1,4 @@ -"""Import the ATOMA dashboard bundle into Superset, overwriting what it defines. +"""Import the ATOMA dashboards bundle into Superset, overwriting what it defines. Runs inside the Superset container, with the image's Python: @@ -17,15 +17,10 @@ live view and any drift is reported (exit status 2). The database is never created or changed: the bundle's database uuid must already exist under the name "Fuel (Trino)", otherwise nothing is imported. -A bundle cannot carry a dashboard's roles, so afterwards the role "ATOMA Reader" -is attached to the ATOMA dashboard: with DASHBOARD_RBAC that is how ATOMA's -readers see it. The role comes from roles/atoma_reader.py (run by superset-init); -without it nothing is imported. - -The Petrolyst app's dashboards (PD-12: atoma-overview, atoma-site) get no role: -the app shows them with guest passes. With --embedded-out FILE --embed-domain URL, -each of them is made embeddable from that origin (its embedding id stays the same -across imports), and FILE receives {slug: embedding id} for the app service: +The dashboards (atoma-overview, atoma-site) get no role: admins see them in Superset, and +the Petrolyst app shows them with guest passes (PD-12). With --embedded-out FILE +--embed-domain URL, each of them is made embeddable from that origin (its embedding id stays +the same across imports), and FILE receives {slug: embedding id} for the Petrolyst server: docker exec /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip \ --user --embedded-out /tmp/embedded-dashboards.json --embed-domain http://127.0.0.1:8098 @@ -41,14 +36,12 @@ import zipfile import yaml DATABASE_NAME = "Fuel (Trino)" -READER_ROLE = "ATOMA Reader" -READER_SLUG = "atoma" # the readers' dashboard; the others are the app's (PD-12) EMBEDDED_PREFIX = "atoma-" STEPS = [("datasets", "SqlaTable"), ("charts", "Slice"), ("dashboards", "Dashboard")] def main() -> int: - parser = argparse.ArgumentParser(description="Import the ATOMA dashboard bundle.") + parser = argparse.ArgumentParser(description="Import the ATOMA dashboards bundle.") parser.add_argument("bundle", nargs="?") parser.add_argument("--user", help="existing Superset user who will own new objects") parser.add_argument("--show-database", action="store_true", @@ -85,10 +78,6 @@ def main() -> int: if user is None: print(f"no Superset user {args.user!r}", file=sys.stderr) return 1 - reader_role = security_manager.find_role(READER_ROLE) - if reader_role is None: - print(f'no role "{READER_ROLE}": run roles/atoma_reader.py first', file=sys.stderr) - return 1 with zipfile.ZipFile(args.bundle) as bundle: contents = get_contents_from_bundle(bundle) databases = [yaml.safe_load(t) for p, t in contents.items() if p.startswith("databases/")] @@ -115,17 +104,13 @@ def main() -> int: if path.startswith("dashboards/"): dashboard = db.session.query(Dashboard).filter_by( uuid=yaml.safe_load(text)["uuid"]).one() - if dashboard.slug == READER_SLUG: - if reader_role not in dashboard.roles: - dashboard.roles.append(reader_role) - elif dashboard.slug.startswith(EMBEDDED_PREFIX) and args.embedded_out: + if dashboard.slug.startswith(EMBEDDED_PREFIX) and args.embedded_out: from superset.daos.dashboard import EmbeddedDashboardDAO config = EmbeddedDashboardDAO.upsert(dashboard, [args.embed_domain]) db.session.flush() embedded[dashboard.slug] = str(config.uuid) db.session.commit() - print(f'dashboards: "{READER_SLUG}" readable for "{READER_ROLE}"') if args.embedded_out: with open(args.embedded_out, "w") as out: json.dump(dict(sorted(embedded.items())), out, indent=2) diff --git a/petrolyst-insights/patches/README.md b/petrolyst-insights/patches/README.md index 981205c..668752d 100644 --- a/petrolyst-insights/patches/README.md +++ b/petrolyst-insights/patches/README.md @@ -1,10 +1,7 @@ # patches/ Runtime patches applied while the Superset image is built -(`stacks/fuel-monitoring/superset/Dockerfile`): - -- `fix_redact_query_for_readers.py`: chart-data responses carry no SQL for readers, as Superset - already does for embedded guests (`petrolyst_reader.py`). +(`stacks/fuel-monitoring/superset/Dockerfile`). Empty until Petrolyst needs one. Same convention as media-insights: diff --git a/petrolyst-insights/patches/fix_redact_query_for_readers.py b/petrolyst-insights/patches/fix_redact_query_for_readers.py deleted file mode 100644 index e983c60..0000000 --- a/petrolyst-insights/patches/fix_redact_query_for_readers.py +++ /dev/null @@ -1,54 +0,0 @@ -"""Readers' chart-data responses carry no SQL. - -Superset 6.1.0 drops each query's SQL from chart-data JSON for embedded guest users only -(superset/charts/data/api.py, ChartDataRestApi._send_chart_response). This extends that -condition with the security manager's redacts_query(), when it has one: the Petrolyst managers -return True for readers (petrolyst_reader.ReaderAccessMixin). It runs when the response is -written, so cached results and result_type=query are covered too. - - python3 fix_redact_query_for_readers.py apply (idempotent) - python3 fix_redact_query_for_readers.py --verify exit 1 unless applied -""" - -import sys -from pathlib import Path - -TARGET = Path("/app/superset/charts/data/api.py") -BEFORE = """\ - if security_manager.is_guest_user(): - for query in queries: - query.pop("query", None) -""" -AFTER = """\ - # Petrolyst (patches/fix_redact_query_for_readers.py): readers too. - if security_manager.is_guest_user() or getattr( - security_manager, "redacts_query", lambda: False - )(): - for query in queries: - query.pop("query", None) -""" - - -def main(argv: list[str]) -> int: - text = TARGET.read_text() - if "--verify" in argv: - compile(text, str(TARGET), "exec") - if text.count(AFTER) == 1 and BEFORE not in text: - print(f"{TARGET}: readers' SQL redaction in place") - return 0 - print(f"{TARGET}: readers' SQL redaction missing", file=sys.stderr) - return 1 - if text.count(AFTER) == 1: - print(f"{TARGET}: already patched") - return 0 - if text.count(BEFORE) != 1: - print(f"{TARGET}: the guest-only redaction was not found once; Superset changed?", - file=sys.stderr) - return 1 - TARGET.write_text(text.replace(BEFORE, AFTER)) - print(f"{TARGET}: patched") - return 0 - - -if __name__ == "__main__": - sys.exit(main(sys.argv[1:])) diff --git a/petrolyst-insights/petrolyst_reader.py b/petrolyst-insights/petrolyst_reader.py deleted file mode 100644 index 6294684..0000000 --- a/petrolyst-insights/petrolyst_reader.py +++ /dev/null @@ -1,150 +0,0 @@ -"""Readers: ATOMA's users, who only read the ATOMA dashboard. - -A reader is a signed-in user whose roles are "ATOMA Reader" and nothing more ("Public" aside, -which carries no permissions here). A user who also has Admin, Alpha or Gamma is not a reader. -The role itself (roles/atoma_reader.py) grants no menus and nothing to write; this module adds -what a role cannot express. Checked against apache/superset:6.1.0. - -- Superset lets an owner past the role check, and anyone onto a dashboard that has neither - roles nor datasets. ReaderAccessMixin lets readers reach a dashboard only through its roles - (DASHBOARD_RBAC), and the dashboard list shows them only those. -- Chart-data responses carry each query's SQL for everyone but embedded guests. Readers get none: - ReaderAccessMixin.redacts_query, read by patches/fix_redact_query_for_readers.py. -- Readers land on the ATOMA dashboard instead of Superset's home page, and any other page they - open (a chart or dashboard list, SQL Lab, Explore, user info) leads there too. Every page - loads reader.css for them (templates/head_custom_extra.html). It hides what Superset shows - whatever the permissions: the dashboard's title row, the theme switcher, Info and About in - Settings; and the filter column on ATOMA's Fleet tab. Its address names its content, since - browsers keep Superset's static files for a year. - -superset_config.py sets FLASK_APP_MUTATOR = register; both security managers (Keycloak in -petrolyst_security.py, database login on the local stack) take ReaderAccessMixin. -""" - -from __future__ import annotations - -import hashlib -from pathlib import Path -from typing import Any - -from flask import Flask, g, has_request_context, redirect, request -from flask_login import current_user -from jinja2 import ChoiceLoader, FileSystemLoader -from werkzeug.wrappers import Response as WerkzeugResponse - -READER_ROLE = "ATOMA Reader" -# Roles a reader may have besides READER_ROLE without stopping being one. -NEUTRAL_ROLES = {"Public"} -LANDING = "/superset/dashboard/atoma/" -# The only pages a reader opens; any other page leads to LANDING. Requests that are not page -# loads (the API, static files) keep their own permission checks. -READER_PAGES = ("/superset/dashboard/", "/login/", "/logout/") -TEMPLATES = Path(__file__).with_name("templates") - - -def _signed_in_user() -> Any: - user = getattr(g, "user", None) - if user is None and has_request_context(): - user = current_user - return user - - -def is_reader(user: Any = None) -> bool: - """True for a signed-in user whose roles are READER_ROLE and nothing that grants more.""" - user = user if user is not None else _signed_in_user() - if user is None or not getattr(user, "is_authenticated", False): - return False - names = {role.name for role in getattr(user, "roles", None) or []} - return READER_ROLE in names and names <= {READER_ROLE, *NEUTRAL_ROLES} - - -class ReaderAccessMixin: - """Reader rules for a SupersetSecurityManager subclass; list it before that base.""" - - def raise_for_ownership(self, resource: Any) -> None: - # is_owner() goes through here: a reader owns nothing, so no owner path to a - # dashboard, chart or dataset remains (also for a user who owned things before). - if is_reader(): - from superset.errors import ErrorLevel, SupersetError, SupersetErrorType - from superset.exceptions import SupersetSecurityException - - raise SupersetSecurityException( - SupersetError( - error_type=SupersetErrorType.MISSING_OWNERSHIP_ERROR, - message=f"A reader does not own {resource}", - level=ErrorLevel.ERROR, - ) - ) - super().raise_for_ownership(resource) # type: ignore[misc] - - def raise_for_access(self, *args: Any, **kwargs: Any) -> None: - # The dashboard page, its API, permalinks and chart data (via can_access_dashboard) - # all check a dashboard here. - dashboard = kwargs.get("dashboard", args[0] if args else None) - if dashboard is not None and is_reader() and not self._reader_may_open(dashboard): - from superset.exceptions import SupersetSecurityException - - raise SupersetSecurityException( - self.get_dashboard_access_error_object(dashboard) # type: ignore[attr-defined] - ) - super().raise_for_access(*args, **kwargs) # type: ignore[misc] - - def _reader_may_open(self, dashboard: Any) -> bool: - """Only DASHBOARD_RBAC: published, and one of its roles is the reader's.""" - mine = {role.id for role in self.get_user_roles()} # type: ignore[attr-defined] - return bool(dashboard.published) and bool(mine & {role.id for role in dashboard.roles}) - - def redacts_query(self) -> bool: - """Whether chart-data responses leave out the SQL (embedded guests aside).""" - return is_reader() - - -def _land_readers() -> WerkzeugResponse | None: - if request.method != "GET" or request.path.startswith(READER_PAGES) or not is_reader(): - return None - # A page load in a browser, or Superset's home page from anything. - if request.headers.get("Sec-Fetch-Dest") == "document" or request.path in ( - "/", - "/superset/welcome/", - ): - return redirect(LANDING) - return None - - -def _limit_dashboard_list() -> None: - """The dashboard list shows readers only published dashboards that carry their roles.""" - from superset import security_manager - from superset.dashboards.filters import DashboardAccessFilter - from superset.models.dashboard import Dashboard - - original = DashboardAccessFilter.apply - if getattr(original, "petrolyst_readers", False): - return - - def apply(self: DashboardAccessFilter, query: Any, value: Any) -> Any: - query = original(self, query, value) - if is_reader(): - role_ids = [role.id for role in security_manager.get_user_roles()] - query = query.filter( - Dashboard.published.is_(True), - Dashboard.roles.any(security_manager.role_model.id.in_(role_ids)), - ) - return query - - apply.petrolyst_readers = True # type: ignore[attr-defined] - DashboardAccessFilter.apply = apply # type: ignore[method-assign] - - -def register(app: Flask) -> None: - """FLASK_APP_MUTATOR: runs once Superset is initialized.""" - # Our templates first: head_custom_extra.html, which Superset's spa.html includes on every - # page, is the only file there. - app.jinja_env.loader = ChoiceLoader([FileSystemLoader(str(TEMPLATES)), app.jinja_env.loader]) - app.jinja_env.globals["petrolyst_is_reader"] = is_reader - # Static files are cached for a year (Cache-Control max-age), so the address carries the - # file's hash: a changed reader.css reaches readers on their next page load. - css = Path(app.static_folder or "") / "assets" / "branding" / "reader.css" - version = hashlib.sha256(css.read_bytes()).hexdigest()[:12] - app.jinja_env.globals["petrolyst_reader_css_version"] = version - app.before_request(_land_readers) - _limit_dashboard_list() diff --git a/petrolyst-insights/petrolyst_security.py b/petrolyst-insights/petrolyst_security.py index b1d504f..f5e6bb0 100644 --- a/petrolyst-insights/petrolyst_security.py +++ b/petrolyst-insights/petrolyst_security.py @@ -11,9 +11,6 @@ the stock behaviour: login page on /login/, which only shows a "Sign in with keycloak" button. - /logout/ also ends the Keycloak session, so the next visit asks for the password and OTP again. - -It also takes the reader rules (petrolyst_reader.ReaderAccessMixin), as does the -local stack's database-login manager. """ import logging @@ -27,7 +24,6 @@ from flask_appbuilder.utils.base import get_safe_redirect from flask_login import logout_user from werkzeug.wrappers import Response as WerkzeugResponse -from petrolyst_reader import ReaderAccessMixin from superset.security import SupersetSecurityManager logger = logging.getLogger(__name__) @@ -68,7 +64,7 @@ class PetrolystAuthOAuthView(AuthOAuthView): return redirect(f"{end_session}?{urlencode(params)}") -class PetrolystSecurityManager(ReaderAccessMixin, SupersetSecurityManager): +class PetrolystSecurityManager(SupersetSecurityManager): authoauthview = PetrolystAuthOAuthView # Without this, Superset registers its React login page on /login/ before # FAB's OAuth view, and that page wins the route. diff --git a/petrolyst-insights/roles/atoma_guest.py b/petrolyst-insights/roles/atoma_guest.py index 484341e..823509c 100644 --- a/petrolyst-insights/roles/atoma_guest.py +++ b/petrolyst-insights/roles/atoma_guest.py @@ -1,23 +1,80 @@ -"""The role "ATOMA Guest": what the Petrolyst app's embedded dashboards may do (PD-12 trial). +"""The role "ATOMA Guest": what the Petrolyst app's embedded dashboards may do (PD-12). Superset gives every embedded viewer (a guest pass signed for one dashboard) the role named by -GUEST_ROLE_NAME. That role gets the same permissions as "ATOMA Reader": reading a dashboard -and its charts' data, nothing else. Which dashboard a guest sees comes from the pass, not from -the role. Superset leaves the SQL out of chart-data responses for guests. +GUEST_ROLE_NAME. Which dashboard a guest sees comes from the pass, not from the role. Superset +leaves the SQL out of chart-data responses for guests. Embedding is switched on in the local stack only (.local/fmdb-scratch, superset_config_local.py), -so this script is not part of init.sh; run it by hand inside the Superset container: +so this script is not part of init.sh; run it by hand inside the Superset container, once, and +again whenever PERMISSIONS change: docker exec /app/.venv/bin/python /app/petrolyst-roles/atoma_guest.py + +Creates the role if needed and gives it exactly PERMISSIONS: missing ones are added, any other +is removed (also one added by hand in the UI). A second run changes nothing. + +The list was established on the local stack by reading the ATOMA dashboard as a reader (the +former "ATOMA Reader" role, retired with PD-13): everything a dashboard asks the server for, and +nothing more. No menu access, nothing to write, no Explore, CSV, sharing, drill or query view. """ from __future__ import annotations import sys -from atoma_reader import PERMISSIONS, sync - ROLE = "ATOMA Guest" +PERMISSIONS = [ + ("can_read", "Dashboard"), # the dashboard and its charts' definitions + ("can_dashboard", "Superset"), # the dashboard page + ("can_dashboard_permalink", "Superset"), # a dashboard permalink + ("can_read", "Chart"), # chart data + ("can_time_range", "Api"), # the Viewing period filter + ("can_read", "DashboardFilterStateRestApi"), # native filter state + ("can_write", "DashboardFilterStateRestApi"), + ("can_read", "DashboardPermalinkRestApi"), + ("can_read", "CurrentUserRestApi"), # who is signed in + ("can_read", "SecurityRestApi"), # the CSRF token + ("can_get", "MenuApi"), + ("can_log", "Superset"), # Superset's event log +] + + +def sync(name: str, permissions: list[tuple[str, str]]) -> int: + """Give the role `name` exactly `permissions`, creating it if needed.""" + from superset.app import create_app + + app = create_app() + with app.app_context(): + from superset import db, security_manager as sm + + wanted = [] + for permission, view in permissions: + pvm = sm.find_permission_view_menu(permission, view) + if pvm is None: + print(f"no permission {permission} on {view}: run `superset init` first?", + file=sys.stderr) + return 1 + wanted.append(pvm) + + role = sm.find_role(name) or sm.add_role(name) + added = [pvm for pvm in wanted if pvm not in role.permissions] + removed = [pvm for pvm in role.permissions if pvm not in wanted] + for pvm in added: + sm.add_permission_role(role, pvm) + for pvm in removed: + sm.del_permission_role(role, pvm) + db.session.commit() + + print(f'role "{name}": {len(wanted)} permissions, {len(added)} added, ' + f"{len(removed)} removed") + for pvm in removed: + print(f" removed {pvm.permission.name} on {pvm.view_menu.name}") + return 0 + + +def main() -> int: + return sync(ROLE, PERMISSIONS) + if __name__ == "__main__": - sys.exit(sync(ROLE, PERMISSIONS)) + sys.exit(main()) diff --git a/petrolyst-insights/roles/atoma_reader.py b/petrolyst-insights/roles/atoma_reader.py deleted file mode 100644 index f08bc84..0000000 --- a/petrolyst-insights/roles/atoma_reader.py +++ /dev/null @@ -1,76 +0,0 @@ -"""The role "ATOMA Reader": read the dashboards granted to it, nothing else. - -Runs inside the Superset container, after `superset init` (stacks/fuel-monitoring/superset/init.sh): - - docker exec /app/.venv/bin/python /app/petrolyst-roles/atoma_reader.py - -Creates the role if needed and gives it exactly PERMISSIONS: missing ones are added, any other -is removed (also one added by hand in the UI). A second run changes nothing. Readers reach a -dashboard only when it lists this role (DASHBOARD_RBAC; dashboards/atoma/import_bundle.py -attaches it), and petrolyst_reader.py adds the rules a role cannot express. - -The list was established on the local stack by reading the ATOMA dashboard as a reader: -everything the dashboard asks the server for, and nothing more. No menu access, nothing to -write, no Explore, CSV, sharing, drill or query view. -""" - -from __future__ import annotations - -import sys - -ROLE = "ATOMA Reader" -PERMISSIONS = [ - ("can_read", "Dashboard"), # the dashboard and its charts' definitions - ("can_dashboard", "Superset"), # the dashboard page - ("can_dashboard_permalink", "Superset"), # a dashboard permalink - ("can_read", "Chart"), # chart data - ("can_time_range", "Api"), # the Viewing period filter - ("can_read", "DashboardFilterStateRestApi"), # native filter state - ("can_write", "DashboardFilterStateRestApi"), - ("can_read", "DashboardPermalinkRestApi"), - ("can_read", "CurrentUserRestApi"), # who is signed in - ("can_read", "SecurityRestApi"), # the CSRF token - ("can_get", "MenuApi"), - ("can_log", "Superset"), # Superset's event log -] - - -def sync(name: str, permissions: list[tuple[str, str]]) -> int: - """Give the role `name` exactly `permissions`, creating it if needed (atoma_guest.py too).""" - from superset.app import create_app - - app = create_app() - with app.app_context(): - from superset import db, security_manager as sm - - wanted = [] - for permission, view in permissions: - pvm = sm.find_permission_view_menu(permission, view) - if pvm is None: - print(f"no permission {permission} on {view}: run `superset init` first?", - file=sys.stderr) - return 1 - wanted.append(pvm) - - role = sm.find_role(name) or sm.add_role(name) - added = [pvm for pvm in wanted if pvm not in role.permissions] - removed = [pvm for pvm in role.permissions if pvm not in wanted] - for pvm in added: - sm.add_permission_role(role, pvm) - for pvm in removed: - sm.del_permission_role(role, pvm) - db.session.commit() - - print(f'role "{name}": {len(wanted)} permissions, {len(added)} added, ' - f"{len(removed)} removed") - for pvm in removed: - print(f" removed {pvm.permission.name} on {pvm.view_menu.name}") - return 0 - - -def main() -> int: - return sync(ROLE, PERMISSIONS) - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/petrolyst-insights/superset_config.py b/petrolyst-insights/superset_config.py index e7a1119..53271d1 100644 --- a/petrolyst-insights/superset_config.py +++ b/petrolyst-insights/superset_config.py @@ -13,7 +13,6 @@ from copy import deepcopy from flask_appbuilder.security.manager import AUTH_OAUTH from sqlalchemy.engine.url import URL -import petrolyst_reader from petrolyst_security import PetrolystSecurityManager from superset import config as superset_defaults @@ -80,8 +79,7 @@ AUTH_ROLES_MAPPING = { "superset-admin": ["Admin"], "superset-editor": ["Alpha"], "superset-viewer": ["Gamma"], - # ATOMA's users: read the ATOMA dashboard, nothing else (petrolyst_reader.py). - "atoma-reader": ["ATOMA Reader"], + # "atoma-reader" opens the Petrolyst app only; here it maps to nothing (Public). } AUTH_ROLES_SYNC_AT_LOGIN = True AUTH_USER_REGISTRATION = True @@ -95,27 +93,13 @@ AUTH_USER_REGISTRATION_ROLE = "Public" # opens it from a link. FEATURE_FLAGS = {"DASHBOARD_RBAC": True, "FILTERBAR_CLOSED_BY_DEFAULT": True} -# Readers ("ATOMA Reader"): land on the ATOMA dashboard, see it without the -# title row and the menus Superset shows to everyone, and reach no other -# dashboard (petrolyst_reader.py). -FLASK_APP_MUTATOR = petrolyst_reader.register - # deck.gl maps: OpenStreetMap raster tiles only. The default list adds Mapbox # styles, which fail without MAPBOX_API_KEY. The default CSP already allows the -# tile host. The ATOMA map credits OSM in a Markdown block beside it. +# tile host. DECKGL_BASE_MAP = [ ["tile://https://tile.openstreetmap.org/{z}/{x}/{y}.png", "OpenStreetMap"], ] -# Markdown blocks may hold an iframe: the ATOMA Fleet tab embeds the fleet-map view -# from petrolyst-platform (/views/fleet-map/). Only these attributes survive. The CSP -# (default-src 'self', no frame-src) keeps frames to this origin, so a dashboard -# editor cannot embed another site. -HTML_SANITIZATION_SCHEMA_EXTENSIONS = { - "tagNames": ["iframe"], - "attributes": {"iframe": ["src", "title", "loading", "style"]}, -} - # --- Behind Traefik ----------------------------------------------------------- ENABLE_PROXY_FIX = True PREFERRED_URL_SCHEME = "https" @@ -155,7 +139,7 @@ EXPLORE_FORM_DATA_CACHE_CONFIG = { RATELIMIT_STORAGE_URI = f"{REDIS_URL}/1" # --- Petrolyst look and feel -------------------------------------------------- -# Values from petrolyst-ui/design/brand/07_UI_Tokens/petrolyst_tokens.css. +# Values from petrolyst-web/design/brand/07_UI_Tokens/petrolyst_tokens.css. # Assets live in assets/branding/ and are served from /static/assets/branding/. BRANDING = "/static/assets/branding" @@ -178,7 +162,7 @@ THEME_DEFAULT["token"].update( # letters were about 5px high. Its own Y carries the brand accent. 24px # (letters about 20px) and the margin fill the 60px navbar height. On the # left, 15px more than the navbar's 16px: the P (1.4px inside the image) - # lines up with the dashboard's content and a reader's fleet map at 32px. + # lines up with the dashboard's content at 32px. "brandLogoHeight": "24px", "brandLogoMargin": "18px 0 18px 15px", # Action teal, not the raw brand teal #009688: the brand guide keeps diff --git a/petrolyst-insights/templates/head_custom_extra.html b/petrolyst-insights/templates/head_custom_extra.html deleted file mode 100644 index 0c8fdc6..0000000 --- a/petrolyst-insights/templates/head_custom_extra.html +++ /dev/null @@ -1,6 +0,0 @@ -{#- Petrolyst: replaces Superset's empty head_custom_extra.html, which spa.html includes on every - page (petrolyst_reader.register puts this folder first). Readers get reader.css; its address - names its content, as browsers keep static files for a year. -#} -{% if petrolyst_is_reader() %} - -{% endif %} diff --git a/stacks/fuel-monitoring/superset/Dockerfile b/stacks/fuel-monitoring/superset/Dockerfile index dc96845..7d22963 100644 --- a/stacks/fuel-monitoring/superset/Dockerfile +++ b/stacks/fuel-monitoring/superset/Dockerfile @@ -11,8 +11,7 @@ USER root RUN uv pip install --no-cache --python /app/.venv/bin/python \ authlib==1.9.1 psycopg2-binary==2.9.13 "trino[sqlalchemy]==0.340.0" -COPY --from=insights --chown=superset:superset superset_config.py petrolyst_security.py petrolyst_reader.py /app/pythonpath/ -COPY --from=insights --chown=superset:superset templates/ /app/pythonpath/templates/ +COPY --from=insights --chown=superset:superset superset_config.py petrolyst_security.py /app/pythonpath/ COPY --from=insights --chown=superset:superset roles/ /app/petrolyst-roles/ COPY --from=insights --chown=superset:superset assets/branding/ /app/superset/static/assets/branding/ COPY --from=insights patches/ /app/patches/petrolyst/ diff --git a/stacks/fuel-monitoring/superset/init.sh b/stacks/fuel-monitoring/superset/init.sh index feba166..d7bd43e 100755 --- a/stacks/fuel-monitoring/superset/init.sh +++ b/stacks/fuel-monitoring/superset/init.sh @@ -12,5 +12,3 @@ superset init superset set-database-uri \ --database_name "Fuel (Trino)" \ --uri "trino://superset@trino:8080/fuel" -# The role "ATOMA Reader", set to exactly its permissions (petrolyst-insights/roles/). -/app/.venv/bin/python /app/petrolyst-roles/atoma_reader.py