Trial state of 29 Oct 2026: ATOMA reader setup, app dashboards, embedding notes
Baseline before the clean-up (PD-13): the PD-11 reader setup and the PD-12 app dashboards as they ran on the local stack, uncommitted until now. Not deployed, not independently reviewed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Nasim SadatandClaude Opus 5.5 committed
1 parent
97284052c1
commit
1b8e57adb4
24 files changed
+2777
-136
No files matched your search
@@ -1,8 +1,13 @@
|
||||
# ATOMA dashboards: Sites Map and Site Detail
|
||||
# ATOMA dashboard: Fleet map and Site details
|
||||
|
||||
**Status: proposed, not built.**
|
||||
- `build_bundle.py` (phase P2) will generate both dashboards, every dataset and every chart from
|
||||
this specification, as one Superset import bundle.
|
||||
**Status: built and tested on a local Superset (P2, 24 Oct 2026); merged into one dashboard with
|
||||
the fleet-map view on 28 Oct 2026 (§9); not imported live.**
|
||||
- `build_bundle.py` generates the dashboard, every dataset and every chart from this
|
||||
specification, as one Superset import bundle; `import_bundle.py` imports it (§6).
|
||||
- The local tests changed parts of this design. Each change is marked *(P2)* and §8 lists them.
|
||||
- The merge of 28 Oct (owner's decisions PD-5 and PD-6, `Petrolist/.context/design-decisions.md`)
|
||||
replaced the Sites Map dashboard by a Fleet tab holding the fleet-map view of petrolyst-platform.
|
||||
Changed parts are marked *(28 Oct)*; §9 lists the results.
|
||||
- Nothing is imported into the live Superset without the owner's go-ahead (phase P3).
|
||||
|
||||
**Datasets.** The `public.v_*` views of
|
||||
@@ -15,72 +20,104 @@
|
||||
- the site-specific facts left out here (positions, distances, pilot sites) are there too;
|
||||
- Q-numbers refer to its §7, and phases P0–P5 to its §8.
|
||||
|
||||
Two dashboards replace the earlier tab-per-site design; nothing in them is specific to a site.
|
||||
One dashboard (two until 28 Oct) replaces the earlier tab-per-site design; nothing in it is
|
||||
specific to a site.
|
||||
|
||||
## 1. Capabilities of the installed Superset 6.1.0
|
||||
|
||||
Source: read-only inspection of `config.py`, `superset_config.py` and the frontend bundle on
|
||||
24 Oct 2026. Items marked *(local test)* are confirmed in P2 before the design relies on them.
|
||||
24 Oct 2026. The former *(local test)* items were tested in P2 (§8); rows marked *(P2)* changed as a result.
|
||||
|
||||
| Requirement | What 6.1.0 provides | Choice |
|
||||
|---|---|---|
|
||||
| Map chart | `deck_scatter` (deck.gl Scatterplot): a modern plugin with the InteractiveChart behaviour. Also: deck_multi, deck_screengrid / hex / grid, country_map, world_map, cartodiagram, legacy `mapbox` | **deck.gl Scatterplot** |
|
||||
| Basemap | `DECKGL_BASE_MAP` unset → OSM raster (`tile.openstreetmap.org`), OSM Swiss topo, Mapbox styles. `MAPBOX_API_KEY` is empty, so the Mapbox styles fail. CSP `connect-src` already allows the OSM host; Traefik sets no CSP | set `DECKGL_BASE_MAP` to **OSM only**; "© OpenStreetMap contributors" in a Markdown under the map |
|
||||
| OSM terms | no key, no cost; tile usage policy: light use, attribution, no bulk download, a valid Referer (ours sends the origin) | fine for an internal user base; viewers' browsers reveal the viewed area (tile coordinates) and their IP to OSMF, not our data. Self-hosted tiles later if needed |
|
||||
| Tooltip | "Tooltip contents" (drag columns) plus "Customize tooltips template" (Handlebars). `js_tooltip` exists but `ENABLE_JAVASCRIPT_CONTROLS` is False | contents and template; **no JavaScript** |
|
||||
| Marker colour | "Categorical color" by a dimension, from the colour scheme; a fixed colour per label through the dashboard's `label_colors` *(local test)* | colour by `baseline_status_label`. Fallback: `deck_multi` with one fixed-colour scatter layer per status |
|
||||
| Marker size | fixed radius, with minimum = maximum pixel radius | uniform |
|
||||
| Tooltip | "Tooltip contents" (drag columns) plus "Customize tooltips template" (Handlebars). `js_tooltip` exists but `ENABLE_JAVASCRIPT_CONTROLS` is False. The contents show column names, not labels, and leave out the colour column | **contents only** *(P2)*: short tooltip-only calculated columns (`code`, `name`, `l_per_h`, `method`, `status`, `validity`, `latest_record`); no template, no JavaScript |
|
||||
| Marker colour | "Categorical color" by a dimension, from the colour scheme; a fixed colour per label through the dashboard's `label_colors` (confirmed in P2) | colour by `baseline_status_label` |
|
||||
| Marker size | fixed radius, with minimum = maximum pixel radius. No outline: the layer is drawn with `stroked: false` | uniform, no outline *(P2)* |
|
||||
| Text cards | The Handlebars chart compiles its template with `eval`; Superset's own CSP (`script-src 'self' 'strict-dynamic'` + nonce) blocks that, so the chart shows a CSP error instead *(P2)* | **no Handlebars**: one-row tables. The CSP stays as it is |
|
||||
| Chart descriptions | "Show description" (`expanded_slices`) never renders in 6.1: the dashboard loads `description_markeddown`, the chart holder reads `description_markdown` *(P2)* | descriptions stay on the charts; the visible reading notes are a Markdown block at the top of each tab |
|
||||
| Viewport | `autozoom` fits the filtered points | on |
|
||||
| Clustering | none in deck.gl; only the legacy `mapbox` chart (`clustering_radius`, needs a Mapbox key) | **no clustering** (§3) |
|
||||
| Marker click | emits a **cross-filter** on the latitude and longitude columns: the exact point, or a box around the click that shrinks as you zoom in. It applies to other charts on the same dashboard | cross-filters the Sites list |
|
||||
| Navigation | **no chart-to-dashboard navigation.** A dashboard reads `?native_filters=<rison>` into preselected native filter values (also `native_filters_key`). Table cells render sanitized HTML (`allow_render_html`) | an "Open site detail" link column in the Sites list |
|
||||
| Native filters | Select (single or multi, required, searchable, "first value by default"), Time range; scoping per chart and tab; cross-filter scoping per chart | §4, §5 |
|
||||
| Markdown iframe *(28 Oct)* | `HTML_SANITIZATION` is on and strips an iframe from a Markdown block; `HTML_SANITIZATION_SCHEMA_EXTENSIONS` lets named tags and attributes through. The CSP (`default-src 'self'`, no `frame-src`) allows frames from this origin only | allow `iframe` with `src`, `title`, `loading`, `style`; the fleet-map view is served on the same origin under `/views` (§3) |
|
||||
| Deep link into a tab *(28 Oct)* | `#TAB-<id>` after the URL opens that tab; it combines with `?native_filters=` (tested 28 Oct) | the "Site details" button opens the Site details tab with the site preset |
|
||||
|
||||
*(28 Oct)* The map moved out of Superset into the fleet-map view (§3). The rows Map chart,
|
||||
Basemap, Tooltip, Marker colour, Marker size, Viewport, Clustering and Marker click describe the
|
||||
deck.gl option the dashboard used until then; the view's choices are in §3. The Navigation row
|
||||
still holds: the view builds the same `?native_filters=` link.
|
||||
|
||||
No custom frontend and no patch are needed.
|
||||
|
||||
## 2. Navigation flow
|
||||
|
||||
```
|
||||
ATOMA — Sites (landing, slug atoma-sites)
|
||||
filters: Site search · Zone · Province · Pilot status (default: active) · Baseline status
|
||||
M1 map: markers fitted to the filtered sites ──hover──► tooltip
|
||||
└─click marker──► cross-filter: M3 Sites list shows that site (or the few under the click)
|
||||
M3 Sites list ─────────────"Open site detail"──► ATOMA — Site Detail ?native_filters=(site preset)
|
||||
M4/M5 Sites without an accepted location ──────"Open site detail"──► same
|
||||
ATOMA — Site Detail (slug atoma-site-detail)
|
||||
filters: Site (exactly one, searchable) · Viewing period
|
||||
tabs: Summary · Fuel & Deliveries · Runtime & Electricity · Baseline & Evidence
|
||||
header link "← Sites map"
|
||||
```
|
||||
|
||||
## 3. Sites Map (landing dashboard)
|
||||
*(28 Oct)* One dashboard, two top-level tabs. *(29 Oct)* They are named **Map** (`TAB-fleet`) and
|
||||
**Insights** (`TAB-site`), the dashboard's lenses; earlier text in this document calls them Fleet and
|
||||
Site details. Readers see them as a lens switch, centred in the navbar (§6).
|
||||
|
||||
```
|
||||
[Filter bar: Site search · Zone · Province · Pilot status · Baseline status]
|
||||
[M1 Map (deck.gl Scatterplot) ................................ w8][M2 Status key + attribution w4]
|
||||
[M3 Sites list (table with link column) ..................................................... w12]
|
||||
[M4 Sites without accepted location: count w3][M5 Sites without accepted location (table) .. w9]
|
||||
ATOMA (slug atoma)
|
||||
filters, in Superset's left column: Site (exactly one, searchable) · Viewing period
|
||||
— both out of scope on Map; readers have no column there (§6)
|
||||
[Map] the fleet-map view (iframe, /views/fleet-map/), the page's full width
|
||||
towers fitted to the placed sites ──hover or click──► popup
|
||||
"Find a site" (code or name) ──► the map flies to it ──► its popup
|
||||
└─"Site details"──► /superset/dashboard/atoma/?native_filters=(site preset)#TAB-site
|
||||
sites without an accepted location: counted in the map's key ("Choose them in Insights",
|
||||
the view's tab_label parameter); chosen in the Site filter
|
||||
[Insights] D0 site header, then the tabs
|
||||
Summary · Fuel & Deliveries · Runtime & Electricity · Baseline & Evidence
|
||||
(a row of section buttons, not a second row of tabs: dashboard.css)
|
||||
```
|
||||
|
||||
| # | Question | Type | Dataset | Content | Missing or insufficient |
|
||||
|---|---|---|---|---|---|
|
||||
| M1 | Where are the sites, and what is each one's baseline status? | deck.gl Scatterplot; lat/lon from `latitude`, `longitude`; uniform size; colour = `baseline_status_label`; autozoom | v_site | tooltip: `site_code`, `site_name`, `lph_estimate` ("not available" when NULL), `lph_method`, `baseline_status_label`, `baseline_validity_note` (when set), `latest_record_on` | sites without an accepted location have NULL lat/lon and are not plotted; M4/M5 show them |
|
||||
| M2 | What do the colours mean? | Markdown (static) | — | the status definitions below; "Colours show evidence status, not performance"; OSM attribution | — |
|
||||
| M3 | Which sites match, and how do I open one? | Table; search box; server pagination | v_site | code, name, province, pilot status, baseline status, L/h (+ method), L/day, latest record, location status, `detail_link` | marker click narrows it; "not available" shown as text |
|
||||
| M4 | How many sites lack an accepted location? | Big Number Total, COUNT(*) with saved filter `location_status <> 'accepted'` | v_site | count; follows the dashboard filters | 0 is shown as 0 |
|
||||
| M5 | Which sites are they? | Table; search box | v_site, same saved filter | code, name, province, `location_status` (awaiting acceptance · missing), baseline status, `detail_link` | — |
|
||||
The filters sit in Superset's left column (`filter_bar_orientation: VERTICAL`) *(owner,
|
||||
29 Oct; `build_bundle.py` checks it)*. A filter shows in the column on the tabs where it is in
|
||||
scope; the others are listed under "Filters out of scope" (the Viewing period on Summary, for
|
||||
example). No filter applies to the Fleet tab, so readers have no column there and the map takes
|
||||
the full width (`reader.css`, §6); admins keep the column on both tabs. The column opens
|
||||
collapsed, a 32 px strip with an expand button (`FILTERBAR_CLOSED_BY_DEFAULT`,
|
||||
`superset_config.py`; owner, 29 Oct); the selected site shows in the Site table above the
|
||||
sections. So for a reader the tabs move sideways by 32 px between Fleet and Site details, or by
|
||||
the column's width once it is expanded; the owner accepted that.
|
||||
|
||||
**Filters** (all on `v_site`, scoped to M1, M3, M4 and M5):
|
||||
- Site search: Select on `site_label`, multi, searchable. The label holds code, name and uid, so
|
||||
code or name both match.
|
||||
- Zone, Province: Select.
|
||||
- Pilot status: default `active`.
|
||||
- Baseline status: Select on `baseline_status_label`.
|
||||
From 28 to 29 Oct the filters sat in a row above the tabs (`HORIZONTAL`). On the Fleet tab that
|
||||
row held only an empty "More filters", and hiding it there made the tabs jump down on Site
|
||||
details. Superset 6.1 has no per-tab setting for the filter bar; it hides it only per link
|
||||
(`?show_filters=0`).
|
||||
|
||||
The cross-filter from M1 is scoped **to M3 only**. M4 and M5 hold the sites without coordinates and
|
||||
must not be emptied by it.
|
||||
Until 28 Oct there were two dashboards: "ATOMA - Sites" (slug `atoma-sites`: a deck.gl map, five
|
||||
filters, a sites list) and "ATOMA - Site Detail" (slug `atoma-site-detail`), linked by "Open site
|
||||
detail" and "← Sites map".
|
||||
|
||||
**Baseline status and marker colours.** Colours are neutral categorical hues, with no red or green
|
||||
## 3. Fleet tab: the fleet-map view *(28 Oct)*
|
||||
|
||||
The Fleet tab holds one Markdown block with an iframe of the fleet-map view, a page of
|
||||
petrolyst-platform (`views/src/fleet-map/`) served under `/views/` on Superset's own origin. The
|
||||
view reads the read service's `site_status` operation (`/views/api/v1/site-status`, from
|
||||
`public.v_site`). It is not a Superset chart: it has no dataset and no native filter, and both
|
||||
filters are out of scope on this tab.
|
||||
|
||||
```
|
||||
[Fleet tab: MARKDOWN-fleet-map, w12 h100]
|
||||
<iframe src="/views/fleet-map/?detail=/superset/dashboard/atoma/&filter=NATIVE_FILTER-site&tab=TAB-site">
|
||||
```
|
||||
|
||||
| Part | Content | Missing or insufficient |
|
||||
|---|---|---|
|
||||
| Map | MapLibre on OpenFreeMap Positron: OpenStreetMap data, English and Dari place names (chosen by the owner on 28 Oct after a side-by-side with OSM standard tiles, which carry the same data). Fitted to the placed sites, clear of the key | no placed site: Afghanistan, and the key says so |
|
||||
| Towers | one per site with an accepted location: a pin holding a lattice tower, in the site's baseline status colour (below); a site that is not an active pilot site faded; uniform size. The site's identity is `site_uid`, since two sites can share a `site_code` (an alias cut to 20 characters) | sites without an accepted location are not drawn |
|
||||
| Popup | opens 120 ms after hover; stays while the pointer is on the tower or the popup; closes 250 ms after it leaves both. A click, tap or keyboard focus pins it; Esc or a click on the map closes it. It springs out of the tower (a fade only with reduced motion) and takes the side with room; when neither side has room for all of it, the side with more room, where its figures scroll while the header, the status with any validity note, and **Site details** stay in view. Content: code, name; status as `baseline_status_text` ("not approved" for Eligible and Provisional); any `baseline_validity_note` right beside the status and the L/h; L/h with method ("not available" when NULL); L/day, "whole record, not per running hour"; latest record; pilot status by name (active, reference, candidate, withdrawn, or not in the pilot); other sites whose towers cover this one (the same or nearly the same position), each a button that switches the popup to it; **Site details** | an approved estimate whose inputs changed is never shown without its validity note |
|
||||
| Site details | a link (`target="_top"`) to this dashboard with the site preset in the Site filter, on the Site details tab: the `?native_filters=` value of §5 plus `#TAB-site`. The iframe's address names the dashboard, the filter and the tab, and `build_bundle.py` checks them | no link when that address is missing or malformed: the view accepts only a dashboard path and filter and tab ids of a fixed shape |
|
||||
| Search | "Find a site: code or name", top left. Matches ignore case: code prefixes first, then codes containing the text, then names containing it; at most 8, each with its status dot and status. Arrow keys and Enter, or the pointer, choose one; the map flies to it at zoom 14 or closer (past the clusters) and pins its popup. Esc clears the field | sites without an accepted location are not searched (owner's choice, 28 Oct): the key counts them and the Site filter finds them; "No site matches" when nothing does |
|
||||
| Operator's logo | ATOMA's logo in its own colours, bottom right above the map's credits, on frosted glass (*owner, 29 Oct*). The dashboard passes it in the iframe's address (`logo=/static/assets/branding/atoma_logo.svg`, `logo_alt=ATOMA`; `build_bundle.py` checks the file exists); the view takes only an image from that folder on its own origin, so it carries no operator's branding | no `logo` in the address: no logo |
|
||||
| Key | the four statuses; "Faded: not an active pilot site"; "Colours show evidence status, not performance"; the number of sites without an accepted location, with "Choose them in Site details" | — |
|
||||
| Keyboard, screen readers | a hidden list of the sites with the same links; focusing one opens its popup | — |
|
||||
|
||||
**Baseline status and tower colours.** Colours are neutral categorical hues, with no red or green
|
||||
and no reference to L/h:
|
||||
|
||||
| Status | Definition (`fm.v_site_baseline_status`) | Colour |
|
||||
@@ -90,24 +127,23 @@ and no reference to L/h:
|
||||
| Provisional | no approval; a proposed version below BP-1, **or** usable intervals with no version yet (no estimate shown, method "no baseline version proposed") | amber #c98500 |
|
||||
| Insufficient evidence | no usable interval. **Every pilot site while no runtime evidence exists** | slate #64748B |
|
||||
|
||||
**Nearby sites.**
|
||||
- Sites a few kilometres apart separate at zoom ≥ 11 (≈ 60 px per 2–4 km at zoom 12) and coincide
|
||||
at country zoom (≈ 2 km per pixel at zoom 6). The pilot's own distances are in the design doc.
|
||||
- Handling:
|
||||
- autozoom to the filtered sites (the default "active" filter opens on the pilot area);
|
||||
- small uniform markers (min = max radius, about 7 px) with an outline;
|
||||
- a click near overlapping markers selects every site in the click box, and M3 lists them;
|
||||
- no clustering.
|
||||
- With hundreds of sites, a `deck_multi` density layer (screengrid) for low zoom is a later option.
|
||||
**Nearby sites.** Towers that would overlap cluster into a teal circle with their count (radius
|
||||
36 px, up to zoom 13); a click on it zooms in until they separate. The pilot's sites, a few
|
||||
kilometres apart, separate at the zoom the map opens with. The towers are one symbol layer, so
|
||||
thousands of sites stay fast.
|
||||
|
||||
## 4. Site Detail (one dashboard, any site)
|
||||
**Sites without an accepted location** are not plotted. The key counts them, and the Site filter
|
||||
in Site details lists every site, placed or not.
|
||||
|
||||
## 4. Site details tab (any site)
|
||||
|
||||
**Filters**
|
||||
- **Site** (`NATIVE_FILTER-site`):
|
||||
- Select on `v_site.site_label`, single value, **required**, searchable, no default;
|
||||
- Select on `v_site.site_label`, single value, **required**, searchable,
|
||||
**first value by default** with `requiredFirst` (the §4 fallback, adopted in P2);
|
||||
- scope: every chart on every tab;
|
||||
- every detail dataset carries `site_label` from `v_site`, so this one filter drives all of
|
||||
them *(local test)*.
|
||||
them (confirmed in P2: every chart request carries the selected site).
|
||||
- **Viewing period** (`NATIVE_FILTER-period`):
|
||||
- Time range, default no filter;
|
||||
- scope: the time-series charts on Fuel & Deliveries and Runtime & Electricity only;
|
||||
@@ -115,24 +151,31 @@ and no reference to L/h:
|
||||
baseline's fixed window.
|
||||
|
||||
**Exactly one site, and never network totals:**
|
||||
1. A required, single-value filter *(local test: with the value cleared, no chart queries
|
||||
unfiltered data)*.
|
||||
2. The header (D0) and KPI (S1) are Handlebars templates that render a site only when exactly one
|
||||
row arrives. Otherwise they show "Select one site".
|
||||
1. A required, single-value filter. *P2: "required" alone does not stop the charts: with no
|
||||
value they queried every site.*
|
||||
2. Therefore "Select first filter value by default", with `requiredFirst`, as the filter editor
|
||||
writes it: the charts wait for the filter, the first site is applied without "Apply filters",
|
||||
and "Clear all" stays disabled. Without `requiredFirst` the first value waits behind "Apply
|
||||
filters" while the charts have already queried every site.
|
||||
3. No Big Number or aggregate tile for site figures. They are computed per site in `v_site`,
|
||||
never aggregated by Superset.
|
||||
4. Fallback, if (1) fails its test: "Select first filter value by default", so the Site filter
|
||||
is never empty.
|
||||
never aggregated by Superset. *(29 Oct, PD-12: the app's Overview, §10, adds up recorded
|
||||
amounts across sites; per-site rates are still never averaged.)*
|
||||
4. The Handlebars "Select one site" guard is gone with Handlebars (§1); with (2) the filter is
|
||||
never empty.
|
||||
|
||||
```
|
||||
[D0 Site header: code + name (large) · province · pilot status · baseline status · "← Sites map"]
|
||||
Summary [S1 L/h KPI card w6][S2 Supporting measures w6]
|
||||
[D0 Site: one-row table, code · name · province · pilot status · baseline status · location] (28 Oct: inside Site details, no link)
|
||||
every tab [Reading notes (Markdown) w12] (P2)
|
||||
Summary [S1 Fuel per running hour: one-row table w12] (P2)
|
||||
[S2 Supporting measures w12]
|
||||
[S3 Evidence coverage timeline (Gantt) w12]
|
||||
Fuel & Deliveries [F1 Deliveries w6][F2 Recorded stock w6]
|
||||
[F5 Deliveries table w12] (P2)
|
||||
[F3 Fuel per calendar day by interval w12]
|
||||
[F4 Interval table w12]
|
||||
Runtime & Electricity[R1 Runtime per day by meter interval w6][R2 Hour-meter evidence w6]
|
||||
[R3 Grid energy, kWh w6][R4 Electricity cost, AFN w6]
|
||||
[R1T Runtime pairs and their validity w12] (P2)
|
||||
[R3 Grid energy, kWh w6][R4 Electricity cost, AFN: bar over table w6]
|
||||
Baseline & Evidence [B1 Baseline versions w6][B2 Inputs of each version w6]
|
||||
[B3 All intervals: eligibility and exclusions w12]
|
||||
[B4 Interval L/h against baseline w12]
|
||||
@@ -144,17 +187,17 @@ Baseline & Evidence [B1 Baseline versions w6][B2 Inputs of each version w6]
|
||||
|
||||
| # | Question it answers | Type | Dataset | Dims / measures (units) | Grain | When data are missing or insufficient |
|
||||
|---|---|---|---|---|---|---|
|
||||
| D0 | Which site am I looking at? | Handlebars | v_site | code and name (large), province, pilot status, baseline status, location status, link "← Sites map" | site | "Select one site" unless exactly one row |
|
||||
| S1 | What is the site's L/h, by which method, over which window, how strong? | Handlebars | v_site | `lph_estimate`, `lph_method`, `baseline_status_label`, `baseline_validity_note`, window, decision; L/day whole record | site | Keeps its position. **In v1 every site shows "Insufficient runtime evidence: no hour-meter readings"**, plus L/day labelled "per calendar day, not per running hour". "Eligible" and "Provisional" carry "not approved" |
|
||||
| D0 | Which site am I looking at? | Table, one row *(P2)* | v_site | code, name, province, pilot status, baseline status, location status *(28 Oct: the link "← Sites map" is gone; the Fleet tab is one click away)* | site | the Site filter is never empty (§4) |
|
||||
| S1 | What is the site's L/h, by which method, over which window, how strong? | Table, one row *(P2)* | v_site | `lph_estimate`, `lph_method`, `baseline_status_label`, `baseline_validity_note`, window, decision; L/day whole record | site | Keeps its position. **In v1 every site shows "Insufficient runtime evidence: no hour-meter readings"**, plus L/day labelled "per calendar day, not per running hour". "Eligible" and "Provisional" carry "not approved" |
|
||||
| S2 | What supports it? | Table | v_kpi_measures | measure, value, unit, `value_kind` (recorded · calculated · assumed · not established), window. Two L/day rows: the whole record (moves with new records) and the L/day reference version (fixed window, its status and any input drift in the note) | site × measure | Rows stay and show "not established" |
|
||||
| S3 | Which periods are covered by which evidence, and where are the gaps? | Gantt | v_coverage | lane; start/end; colour = status | span | Empty lanes are visible; nothing is interpolated |
|
||||
| F1 | When and how much was delivered? | Bar (time axis, **day** grain) | v_delivery (one row per delivery event) | `delivery_date`; **SUM(`delivered_l`)**, titled "Litres delivered per day (all deliveries that day)"; tooltip COUNT(deliveries) | day | Bars only on delivery dates; subtitle "deliveries are not consumption"; F4 lists the individual deliveries |
|
||||
| F2 | What stock was recorded? | Scatter (no lines) | v_stock_reading | `observed_on`; litres; series = reading type × `value_kind` | observation | Points only; nothing between readings |
|
||||
| F3 | How much fuel was used per calendar day in each observed interval? | Bar (categorical x, ordered by start) | v_interval | `interval_label`; MAX(`fuel_l_per_day`) (L/d); colour = `stock_status` | stock interval | Excluded intervals greyed, with the reason in the tooltip; no monthly spreading |
|
||||
| S3 | Which periods are covered by which evidence, and where are the gaps? | Gantt | coverage_detail *(P2: virtual dataset over v_coverage)* | lane name without the view's sort number; start/end; colour = status; tooltip: litres delivered in a stock interval, or a bill's AFN amount | span | Nothing is interpolated. *(P2)* The view emits a lane only where spans exist, so the tab's notes name the four lanes and say that a missing lane means no span could be formed: no evidence of that kind, or too few readings (F2, R2 show single readings) |
|
||||
| F1 | When and how much was delivered? | Bar (time axis, **day** grain) | v_delivery (one row per delivery event) | `delivery_date`; **SUM(`delivered_l`)**, titled "Litres delivered per day (all deliveries that day)" | day | Bars only on delivery dates; "deliveries are not consumption" in the tab's notes. *(P2)* The count and the individual deliveries are in **F5**, a table on v_delivery (date, visit of the day, event, litres, price, sources, review), which also lists a delivery of unknown quantity; its search box shows the number of records |
|
||||
| F2 | What stock was recorded? | Scatter (no lines) | v_stock_reading | `observed_on`; litres; series = reading type × `value_kind`, split by visit when a day has several *(P2)* | observation | Points only; nothing between readings |
|
||||
| F3 | How much fuel was used per calendar day in each observed interval? | Bar (categorical x, ordered by start) | v_interval | `interval_label`; MAX(`fuel_l_per_day`) (L/d); colour = `stock_status` | stock interval | Intervals that are not `ok` greyed (*P2*: fixed grey per status in Site Detail's `label_colors`); the status is the series name, so legend and tooltip give the reason; no monthly spreading |
|
||||
| F4 | Interval detail | Table | v_interval | dates, opening, deliveries, closing, used, L/d, status | interval | Shows the reasons |
|
||||
| R1 | How many hours did each generator run per day? | Bar (categorical) | v_runtime | interval × meter; MAX(h/day). h/day is NULL for an invalid pair (`pair_valid` false: two epochs, a decreasing timer or over 24 h/day); such pairs show no bar and are listed with their flags in the tooltip | meter interval | **v1: empty**, with "No hour-meter readings" in the description |
|
||||
| R1 | How many hours did each generator run per day? | Bar (categorical) | v_runtime | interval × meter; MAX(h/day). h/day is NULL for an invalid pair (`pair_valid` false: two epochs, a decreasing timer or over 24 h/day); such pairs show no bar and are listed with their flags in **R1T**, a pair table with `pair_valid`, `same_epoch`, `monotonic`, `over_24h_per_day` *(P2: the bar tooltip cannot show them)* | meter interval | **v1: empty**, with "No hour-meter readings" in the tab's notes |
|
||||
| R2 | What runtime evidence exists? | Table | v_hour_meter_evidence | meter, mapped set, date, hours, state, epoch | reading | **v1: no rows**, "No hour-meter readings" |
|
||||
| R3 | How much grid energy did the site receive? | Handlebars → bar once kWh exist | v_kpi_measures (kWh row) | kWh per bill month | month | "No kWh records; bills are cost only and are not converted to fuel or hours" |
|
||||
| R3 | How much grid energy did the site receive? | Table, the kWh row *(P2)*; a bar once kWh exist | v_kpi_measures (kWh row) | kWh per bill month | month | "No kWh records; bills are cost only and are not converted to fuel or hours" |
|
||||
| R4 | What did grid electricity cost? | Bar (month) + table | v_electricity_cost | bill month; amount (AFN, inferred) | month | Own chart, never on a fuel axis. A disputed bill is listed without amount or kWh |
|
||||
| B1 | Which baseline versions exist, and who approved them? | Table | v_baseline_version | code, status, value, window, grade, decision | version | "No baseline defined" |
|
||||
| B2 | What went into each version? | Table | v_baseline_input | interval, included, reason, fuel, runtime, drift | version × interval | empty until a version exists |
|
||||
@@ -164,24 +207,31 @@ Baseline & Evidence [B1 Baseline versions w6][B2 Inputs of each version w6]
|
||||
| B6 | What is missing or inconsistent? | Table | v_data_quality | check, status, detail | check | Lists the gaps: no runtime, placeholder rows, no kWh, no transfer records, location awaiting acceptance |
|
||||
| B7 | Which documents do the figures rest on? | Table | v_evidence | title, kind, issuer, date, used for | document | — |
|
||||
| B8 | What did the operator workbook calculate, and is it usable? | Table | v_imported_calculation | label, formula, value, our reading, use, reason, `claim_status` | cell | Shows "excluded" with the reason for "actual L/h" and "CP L/h". A disputed transcription shows its status, without formula or value |
|
||||
| B9 | Does stock depletion match engine-measured fuel? | Table | v_reconciliation | stock L, engine L, `covered_h` of `interval_h`, difference L and %. The window runs from the opening visit's departure to the closing visit's arrival. The difference is shown only for an `ok` stock interval with both visit times recorded, in which every generator is placed at the site throughout and covered for the whole window | interval × generator | "No flow-meter data yet"; partial coverage, unknown visit times or a mid-interval placement show the hours covered and no difference |
|
||||
| B9 | Does stock depletion match engine-measured fuel? | Table | v_reconciliation | engine L of this generator, `covered_h` of `interval_h`; site-wide: stock L, engine L of all generators, `all_generators_metered`, difference L and % *(P2: both engine totals shown, so the difference reconciles)*. The window runs from the opening visit's departure to the closing visit's arrival. The difference is shown only for an `ok` stock interval with both visit times recorded, in which every generator is placed at the site throughout and covered for the whole window | interval × generator | "No flow-meter data yet"; partial coverage, unknown visit times or a mid-interval placement show the hours covered and no difference |
|
||||
| B10 | What does the equipment specification say? | Table | v_spec_reference | attribute, value, unit, basis, applicability, `claim_status` | spec | Titled "Manufacturer and nameplate data, not site performance". A disputed specification shows its status, without value |
|
||||
| B11 | What would L/h be under the stated runtime? | Table | v_modelled_lph | assumption code, status, basis; h/day with low–high; window, `matched_days`; L/day; modelled L/h with low–high; `value_kind` | site × assumption | Titled "Modelled from an assumed runtime, not measured". "No runtime assumption recorded" |
|
||||
|
||||
Modelled L/h (B11) is in v1 (Q9, decided 24 Oct 2026). It pools only the usable intervals inside
|
||||
the runtime assumption's dates, and shows that window and its days. It never reaches S1, the map or
|
||||
the M3 Sites list.
|
||||
the runtime assumption's dates, and shows that window and its days. It never reaches S1 or the
|
||||
fleet map.
|
||||
|
||||
## 5. Datasets and filter mapping
|
||||
|
||||
Physical datasets, one per `public.v_*` view on Trino `fuel.public`. Every dataset has the same
|
||||
Physical datasets, one per `public.v_*` view on Trino `fuel.public`, plus one virtual dataset
|
||||
*(P2)*. `coverage_detail` adds two columns to `v_coverage`:
|
||||
- `lane_name`, the lane without its sort number, which read as a count;
|
||||
- `amount`, a stock interval's delivered litres (from its opening visit up to its closing visit,
|
||||
so the opening refill is included) or a bill's AFN amount.
|
||||
|
||||
Both use only `v_coverage`, `v_delivery` and `v_electricity_cost`. Every dataset has the same
|
||||
four site columns from `v_site`: `site_uid`, `site_code`, `site_name`, `site_label`.
|
||||
|
||||
| Dataset | Grain | Used by | Native filters that apply |
|
||||
|---|---|---|---|
|
||||
| v_site | site | M1–M5, D0, S1 | landing: all five filters, plus the M1 cross-filter (on M3); detail: Site |
|
||||
| v_site | site | D0, S1; the fleet-map view through the read service *(28 Oct)* | Site |
|
||||
| v_kpi_measures | site × measure | S2, R3 | Site |
|
||||
| v_coverage | site × lane × span | S3 | Site |
|
||||
| v_coverage | site × lane × span | — (S3 reads `coverage_detail`) | Site |
|
||||
| coverage_detail *(P2)* | site × lane × span | S3 | Site |
|
||||
| v_delivery | delivery | F1 | Site, Viewing period |
|
||||
| v_stock_reading | observation | F2 | Site, Viewing period |
|
||||
| v_interval | stock interval | F3, F4, B3, B4 | Site; Viewing period on F3 and F4 only |
|
||||
@@ -198,79 +248,335 @@ is grouped or filtered by `metric` and `population_key` (both in `v_interval`),
|
||||
across methods or generator sets. The site's L/h figure itself comes only from a baseline version
|
||||
(`v_site`), never from this metric. There is no AVG metric on ratio columns.
|
||||
|
||||
**Link column.** `detail_link` is a calculated column on the `v_site` dataset, defined in the
|
||||
bundle in Trino SQL. The native filter id stays inside Superset's own configuration:
|
||||
**Site details link** *(28 Oct)*. The fleet-map view builds it (petrolyst-platform
|
||||
`views/src/shared/deepLink.ts`) from its iframe address: the dashboard path, the Site filter's id
|
||||
and the tab. The rison shape is the one the sites list used until 28 Oct, confirmed in P2, and the
|
||||
view's tests compare it with that list's Trino output:
|
||||
|
||||
```sql
|
||||
'<a href="/superset/dashboard/atoma-site-detail/?native_filters='
|
||||
|| url_encode('(NATIVE_FILTER-site:(id:NATIVE_FILTER-site,'
|
||||
|| 'extraFormData:(filters:!((col:site_label,op:IN,val:!(' || q || ')))),'
|
||||
|| 'filterState:(value:!(' || q || ')),ownState:()))')
|
||||
|| '">Open site detail</a>'
|
||||
-- q = '''' || replace(replace(site_label, '!', '!!'), '''', '!''') || '''' -- rison string quoting
|
||||
```
|
||||
/superset/dashboard/atoma/?native_filters=<url-encoded:>
|
||||
(NATIVE_FILTER-site:(id:NATIVE_FILTER-site,extraFormData:(filters:!((col:site_label,op:IN,val:!(q)))),
|
||||
filterState:(value:!(q)),ownState:()))
|
||||
#TAB-site
|
||||
-- q = the site label in rison quotes: 'label', with ! written !! and ' written !'
|
||||
```
|
||||
|
||||
- The exact rison state shape is a *(local test)*.
|
||||
- Fallback: the link opens Site Detail with the Site filter expanded (`expand_filters=1`) and the
|
||||
site's label shown beside the link, to pick by search.
|
||||
|
||||
## 6. Build as code, scale, access
|
||||
|
||||
**Build**
|
||||
- `petrolyst-insights/dashboards/atoma/build_bundle.py` writes one import bundle containing both
|
||||
dashboards, every dataset and every chart:
|
||||
- stable UUIDs (uuid5), so a re-import updates instead of duplicating;
|
||||
- fixed native-filter ids (`NATIVE_FILTER-site`, `-period`, `-search`, …);
|
||||
- `build_bundle.py --database-uuid <uuid>` writes one import bundle containing the dashboard,
|
||||
every dataset and every chart:
|
||||
- stable UUIDs (uuid5), so a re-import updates instead of duplicating; the zip is
|
||||
byte-identical for the same inputs;
|
||||
- fixed native-filter ids (`NATIVE_FILTER-site`, `NATIVE_FILTER-period`) and tab ids
|
||||
(`TAB-fleet`, `TAB-site`, `TAB-summary`, …);
|
||||
- `label_colors` for the four statuses;
|
||||
- the existing "Fuel (Trino)" database UUID, read from an export first.
|
||||
- Import: `superset import-dashboards`.
|
||||
- every view column, from `columns.tsv` (a snapshot of Trino's `information_schema`);
|
||||
- the existing "Fuel (Trino)" database UUID, which differs per instance:
|
||||
`import_bundle.py --show-database` prints it.
|
||||
- Before writing, it checks that every column a chart or filter uses exists in its dataset,
|
||||
that every chart in a Site filter's scope has `site_label`, that the fleet map's iframe
|
||||
address names this dashboard, its Site filter and the Site details tab *(28 Oct)*, and that
|
||||
no row is wider than 12.
|
||||
- Import: `import_bundle.py <zip> --user <username>`, inside the Superset container (*P2*: not
|
||||
`superset import-dashboards`, which leaves existing datasets and charts unchanged on a
|
||||
re-import). It runs the dataset, chart and dashboard importers with overwrite on, never creates
|
||||
or changes the database, and reports columns that differ between a dataset and its view.
|
||||
- A re-import replaces edits made in the Superset UI to these objects.
|
||||
|
||||
**Adding a site**
|
||||
- Import its `site.yaml` and its records. Nothing in either dashboard names a site.
|
||||
- New sites appear in M1 (once their location is accepted) or in M5, in M3, and in the Site filter.
|
||||
- Import its `site.yaml` and its records. Nothing in the dashboard or the fleet-map view names a
|
||||
site.
|
||||
- A new site appears in the Site filter, and on the map once its location is accepted; until
|
||||
then the map's key counts it.
|
||||
|
||||
**Hundreds of sites**
|
||||
- `v_site` has one row per site, and the scatter layer handles thousands of points.
|
||||
- The Select filters load the distinct labels; M3 and M5 paginate on the server.
|
||||
- `v_site` has one row per site; the fleet map draws its towers as one symbol layer and
|
||||
clusters overlapping ones, so thousands of sites stay fast.
|
||||
- The Site filter loads the distinct labels; with thousands of sites, check its value limit
|
||||
(open point 6 of the design decisions).
|
||||
|
||||
**Superset configuration change** (petrolyst-insights; image rebuild in P3):
|
||||
`DECKGL_BASE_MAP = [["tile://https://tile.openstreetmap.org/{z}/{x}/{y}.png", "OpenStreetMap"]]`.
|
||||
The CSP needs no change *(local test: no CSP violation in the browser console)*.
|
||||
**Superset configuration changes** (in `superset_config.py`; image rebuild in P3):
|
||||
- `DECKGL_BASE_MAP = [["tile://https://tile.openstreetmap.org/{z}/{x}/{y}.png", "OpenStreetMap"]]`
|
||||
(no deck.gl chart since 28 Oct; kept for charts made by hand);
|
||||
- *(28 Oct)* `HTML_SANITIZATION_SCHEMA_EXTENSIONS` lets an `iframe` (`src`, `title`, `loading`,
|
||||
`style`) through a Markdown block. The fleet-map view must be served on the same origin under
|
||||
`/views` (petrolyst-infra routing, not part of this step).
|
||||
|
||||
**Access, internal first.** Both dashboards for Admin and Alpha only. No Gamma role until the
|
||||
The CSP needs no change. P2: the tiles load and the map causes no CSP violation. One violation
|
||||
appears on every Superset page, including the stock dashboard list: a capability probe
|
||||
(`Function("")`) in the 6.1 frontend bundle, caught by the bundle itself.
|
||||
|
||||
**Access, internal first.** The dashboard for Admin and Alpha only; the read service behind the
|
||||
same sign-in *(28 Oct: not built yet; locally it has none)*. No Gamma role until the
|
||||
ATOMA-facing edit.
|
||||
|
||||
**ATOMA's readers** *(owner, 29 Oct)*. The Superset role **"ATOMA Reader"** (Keycloak client role
|
||||
`atoma-reader`) reads the ATOMA dashboard and nothing else.
|
||||
- **The role** (`petrolyst-insights/roles/atoma_reader.py`): exactly 12 permissions, set by
|
||||
`superset-init` whenever the image changes, and reset by hand with the command in the README.
|
||||
- Dashboard and Chart `can_read`;
|
||||
- Superset `can_dashboard`, `can_dashboard_permalink`, `can_log`;
|
||||
- Api `can_time_range`;
|
||||
- the filter state (`can_read` and `can_write`), permalinks (`can_read`), the current user,
|
||||
the CSRF token and the menu.
|
||||
|
||||
It has no menus, no write rights, no Explore, CSV, sharing, drill or query view. The bundle
|
||||
import attaches the role to the dashboard (DASHBOARD_RBAC).
|
||||
- **Rules a role cannot express** (`petrolyst_reader.py`, in both security managers):
|
||||
- readers own nothing, and reach a dashboard only through its roles (published, role match);
|
||||
- the dashboard list shows them only those;
|
||||
- their chart-data responses carry no SQL (`patches/fix_redact_query_for_readers.py`;
|
||||
Superset itself drops it only for embedded guests).
|
||||
- **What they see:**
|
||||
- every page they open leads to the ATOMA dashboard;
|
||||
- a stylesheet loaded only for them (`assets/branding/reader.css`) hides the dashboard's
|
||||
title row, the theme switcher, and Info and About in Settings, leaving Logout;
|
||||
- the same stylesheet hides the filter column on the Fleet tab, and while the dashboard
|
||||
loads. It sizes the map to the window: the full width at once (Superset measures widths
|
||||
again only about 0.3 s after loading), and down to 32 px above the window's bottom edge
|
||||
instead of the layout's 800 px, at least 560 px. `build_bundle.py` checks that it names the
|
||||
Fleet tab and that the map spans all 12 columns;
|
||||
- the same stylesheet turns the top-level tabs into a lens switch, centred in the navbar, with
|
||||
an icon each; the map's block then grows by the tab row's height. A scroll-linked animation
|
||||
moves the switch up with the navbar, which scrolls away. Without support for scroll-linked
|
||||
animations, or narrower than 900 px, the tab row stays in the page. From 1100 px the switch
|
||||
shows a third lens, AI, as a teaser: greyed, tagged PRO in green, drawn by the tab list
|
||||
(not a tab, not clickable; the layout keeps Map and Insights). Narrower, it would cover the
|
||||
logo. `build_bundle.py` checks that every layout id the stylesheet names exists;
|
||||
- the stylesheet's address carries a hash of its content
|
||||
(`petrolyst_reader.register`): Superset lets browsers keep static files for a year, so a
|
||||
changed file would otherwise not reach readers who loaded the old one;
|
||||
- Superset already hides the menus, "+" and editing from a role without them.
|
||||
- **One refused request remains, harmless:** Superset stores a dashboard's label colours when
|
||||
they changed (`PUT /api/v1/dashboard/<id>/colors`). A reader may not, so it is refused until
|
||||
someone with edit rights opens the dashboard; the colours show either way.
|
||||
- A reader is someone whose only roles are "ATOMA Reader" (and Public). Anyone with Admin, Alpha
|
||||
or Gamma as well sees everything as before.
|
||||
|
||||
## 7. Acceptance checks
|
||||
|
||||
Run in P2 against a local Superset, and repeated as a smoke test in P3.
|
||||
|
||||
1. **New site, no new objects.** Count dashboards, charts and datasets in the metadata database.
|
||||
Then import a synthetic fourth site with records and an accepted location, and a fifth with no
|
||||
location. The counts are unchanged; site 4 appears in M1, M3 and the Site filter; site 5
|
||||
appears in M5, and M4 goes up by 1.
|
||||
location. The counts are unchanged; site 4 appears on the fleet map and in the Site filter;
|
||||
site 5 appears in the Site filter, and the map's count of sites without an accepted location
|
||||
goes up by 1 *(28 Oct)*.
|
||||
2. **Site switching.** In Site Detail, select pilot site A, then site B. D0 shows the new code and
|
||||
name; every chart's query (View query, or chart-data API) carries `site_label = …<site B>…`; no
|
||||
chart keeps site A's rows.
|
||||
3. **Clearing the selection.** With the Site filter cleared, no chart shows multi-site data; D0 and
|
||||
S1 show "Select one site". If not, switch to the §4 fallback and re-test.
|
||||
4. **Unavailable L/h.** For each pilot site:
|
||||
- the tooltip and M3 say "not available" with the method "no runtime evidence";
|
||||
- the fleet map's popup says "not available" with the method "no runtime evidence" *(28 Oct)*;
|
||||
- S1 shows the insufficient-evidence state plus L/day;
|
||||
- B3 shows every interval excluded, with its reason;
|
||||
- B11 shows a modelled range labelled "not measured", and that figure appears nowhere else.
|
||||
5. **Missing coordinates.** A site without an accepted location is not plotted, is listed in M5
|
||||
and counted in M4, and its link opens its detail.
|
||||
5. **Missing coordinates.** A site without an accepted location is not plotted, is counted in the
|
||||
map's key, and can be chosen in the Site filter *(28 Oct)*.
|
||||
6. **Filter propagation.**
|
||||
- On the landing page, each filter changes M1, M3, M4 and M5, and the default "active" shows
|
||||
exactly the active pilot sites.
|
||||
- A marker click filters M3 only.
|
||||
- In Site Detail, the Viewing period changes F1–F3 and R1, R2, R4, but not S1 or B1–B4.
|
||||
7. **Detail links.** For each active pilot site, "Open site detail" in M3 opens Site Detail with
|
||||
exactly that site selected, and D0 matches. Same from M5.
|
||||
- *(28 Oct)* Both filters are out of scope on the Fleet tab; the fleet map shows every placed
|
||||
site, sites that are not active pilot sites faded.
|
||||
- In Site details, the Viewing period changes F1–F5, R1, R1T, R2, R4 and R4T, but not S1 or
|
||||
B1–B4.
|
||||
7. **Detail links** *(28 Oct)*. For each placed site, "Site details" in its popup opens the Site
|
||||
details tab with exactly that site selected and Summary open; D0 matches, and every chart's
|
||||
request carries that site.
|
||||
8. **Colours.**
|
||||
- Fixtures in each status show the four colours.
|
||||
- "Eligible" never renders as "Approved" unless the fixture has an approved version with a
|
||||
decision reference.
|
||||
- Marker size is identical across sites.
|
||||
9. **Basemap.** OSM tiles load, there are no CSP violations in the console, and the attribution
|
||||
is visible.
|
||||
- *(28 Oct)* An approved site whose inputs changed shows its validity note beside the status
|
||||
and the L/h in the popup.
|
||||
9. **Basemap.** The fleet map's tiles load, the attribution is visible, and neither the view nor
|
||||
the dashboard adds a CSP violation (Superset's own probe, §6, stays) *(28 Oct)*.
|
||||
|
||||
## 8. P2 results (local, 24 Oct 2026)
|
||||
|
||||
Local Superset 6.1.0 image built from this repository, Trino 484, fuel-db with migrations
|
||||
0001–0003 and the corpus data loaded; browser checks in headless Chrome.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| 1 New site | pass: with a synthetic site with an accepted location and one without, the object counts stay unchanged (2 dashboards, 28 charts before R1T was added, 18 datasets); the first is in M1, M3 and the Site filter, the second in M5, and M4 goes from 0 to 1. Fixtures removed afterwards |
|
||||
| 2 Site switching | pass: for each active pilot site, every chart request on the four tabs carries that site's `site_label`, all HTTP 200 (25 per site; F5, added later, re-checked for one site) |
|
||||
| 3 Clearing | pass with the fallback (§4): "required" alone let the charts query every site |
|
||||
| 4 Unavailable L/h | pass: tooltip and M3 "not available", method "Not available: no runtime evidence"; S1 the same plus L/day; B3 every interval excluded with its reason; B11 the modelled range, labelled "not measured", nowhere else |
|
||||
| 5 Missing coordinates | pass (synthetic site, check 1): not plotted, in M4 and M5, its link opens its detail |
|
||||
| 6 Filter propagation | pass: the default "active" reaches M1, M3, M4, M5; a marker click re-queries M3 only, with a latitude/longitude box; the Viewing period reaches F1–F5, R1, R1T, R2, R4 and the bill table, and no Summary or Baseline & Evidence chart |
|
||||
| 7 Detail links | pass: from M3 for each active pilot site, and from M5 |
|
||||
| 8 Colours | pass for the mapping: a throwaway fixture with one point per status shows the four colours and equal marker sizes. Not tested by fixture: that the database never labels an unapproved version "Approved baseline" |
|
||||
| 9 Basemap | pass: OSM tiles load, attribution visible, no CSP violation from the map |
|
||||
|
||||
Changes forced by the tests and by two Codex inspection rounds of 24 Oct (F2 visits, R1T, B9
|
||||
totals, coverage lanes; then F5, grey stock statuses, lane wording), all marked *(P2)* above.
|
||||
The second round's fixes were not inspected again. The grey colour for a non-ok stock interval
|
||||
is configured but not shown by the current data, which has no such interval.
|
||||
- Handlebars charts (D0, S1, R3) became one-row tables.
|
||||
- The Site filter uses "first value by default" with `requiredFirst`.
|
||||
- Reading notes moved into a Markdown block per tab.
|
||||
- The map tooltip uses short tooltip-only columns.
|
||||
- The markers have no outline.
|
||||
|
||||
The marker palette is unchanged from §3. The dataviz palette check flags slate as reading
|
||||
grey, which is intended for "insufficient evidence", and amber at 2.99:1 contrast. The label is
|
||||
always also shown as text, in the legend, the tooltip and M3.
|
||||
|
||||
## 9. Fleet map results (local, 28 Oct 2026)
|
||||
|
||||
Local Superset 6.1.0 rebuilt with the iframe setting, behind a local Traefik that serves Superset
|
||||
and the petrolyst-platform read service on one origin (as on the server); the fuel-db of P2.
|
||||
Browser checks with Playwright in Chrome (`petrolyst-platform/views/e2e/`).
|
||||
|
||||
**Spike, before building**
|
||||
- A Markdown iframe survives the sanitiser with the setting of §6.
|
||||
- A `target="_top"` link inside it navigates the whole dashboard page.
|
||||
- `?native_filters=…#TAB-…` selects the site and opens that tab.
|
||||
- Basemaps compared at zoom 11, 14 and 16 over the pilot area: OpenFreeMap Positron and OSM
|
||||
standard tiles show the same OpenStreetMap data; the owner chose Positron.
|
||||
|
||||
**Bundle:** 1 dashboard, 26 charts, 19 datasets; two builds are byte-identical; the import reports
|
||||
column drift 0. The two earlier dashboards and the charts M1, M3, M4 and M5 were deleted from the
|
||||
local Superset through its API. The live Superset never had them (P3 not done).
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| 2 Site switching | pass: choosing a second site in the Site filter moves every chart request to it |
|
||||
| 3 Clearing | pass: after "Clear all" every chart request still carries exactly one site |
|
||||
| 4 Unavailable L/h | pass for the popup ("not available" and the method); S1, B3 and B11 are unchanged by this step and were not re-run |
|
||||
| 5 Missing coordinates | pass for the count: the map's key shows the 2 sites without an accepted location |
|
||||
| 7 Detail links | pass: for each of the 3 placed sites, hover, then "Site details", opens the Site details tab with Summary active, D0 shows the site, and every chart request carries exactly its `site_label` |
|
||||
| 8 Colours | the validity note beside an approved status and its L/h, and "(not approved)" for Provisional, pass as component tests on fixtures. Live, every site is "Insufficient evidence", so only slate was seen; the other three colours were checked by reading the code against §3, not on screen |
|
||||
| 9 Basemap | pass: tiles load, attribution visible; no CSP violation from the view. Superset's own `script-src eval` probe appears on every Superset page, as in P2 (§6), and nothing else. The test's watcher was checked with a deliberately disallowed image inside the view |
|
||||
|
||||
Also passed: the popup opens on hover, stays while the pointer moves onto it, closes after the
|
||||
pointer leaves; a click pins it; Esc and a click on the map close it; with reduced motion it only
|
||||
fades and nothing pulses. The view's minimum size is 480 × 560 px (`catalogue.json`): there, and
|
||||
also at 480 × 360, every site's popup stays inside the map, and an approved estimate whose inputs
|
||||
changed is never in view without its warning, checked by scrolling the card. At 480 × 560 every
|
||||
site's L/h can be read. A single placed site sits clear of the key. Two sites at the same
|
||||
position can each be opened with the pointer. The states the scratch data lacks (approved with a
|
||||
changed input, one site, two at one spot) are made in the browser by changing the read service's
|
||||
answer; the database is unchanged.
|
||||
|
||||
**Filters on top and the map search** *(owner, later on 28 Oct)*: the filter row was checked on
|
||||
screen first: the Fleet tab shows only "More filters", and Site details shows Site and Viewing
|
||||
period in the row. Browser checks:
|
||||
- the map takes the page's width;
|
||||
- the search finds a site by part of its name and by its code, and its popup's "Site details"
|
||||
opens exactly that site;
|
||||
- a site inside a cluster is taken out of it;
|
||||
- site switching and clearing (checks 2 and 3) still pass with the filters in the row.
|
||||
|
||||
**ATOMA readers and the operator's logo** *(owner, 29 Oct)*: the plan was reviewed by Codex
|
||||
(approved after three rounds: deploy inputs, ownership and no-dataset paths, SQL in responses).
|
||||
Browser checks as the local user `reader`:
|
||||
- every page leads to the dashboard;
|
||||
- no title row and no menus; Settings shows only Logout;
|
||||
- the whole fleet-map suite passes as the reader;
|
||||
- reading needs nothing the role lacks (only the colours PUT above is refused);
|
||||
- no chart-data response carries SQL, fresh, cached or as `result_type=query` (an admin's
|
||||
does);
|
||||
- CSV and editing answer 403;
|
||||
- two probe dashboards (Markdown only without roles, and one the reader owns) stay out of
|
||||
reach: page, API and permalink.
|
||||
|
||||
Also checked:
|
||||
- the role script removes a permission added by hand;
|
||||
- every new file changes the image tag;
|
||||
- the patch's `--verify` fails on unpatched Superset and on a tampered line.
|
||||
|
||||
Found on the way: under many parallel requests the local proxy returned 502s. gunicorn closes
|
||||
idle connections after 2 s (`--keep-alive 2`) and Traefik reused them. The local proxy now drops
|
||||
idle connections after 1 s. **The server has the same pairing** (Traefik in front of gunicorn
|
||||
with the default keep-alive): open item.
|
||||
|
||||
**Final inspection:** two fresh Codex sessions reviewed the build.
|
||||
- The first found four defects, all fixed and re-checked:
|
||||
- the popup could leave a small map;
|
||||
- two sites sharing a `site_code` could open each other's details;
|
||||
- every non-active pilot status was called "reference";
|
||||
- a failed database connection gave a 500 instead of the generic 503.
|
||||
- The second confirmed those fixes and found three more:
|
||||
- in a short card the validity warning could scroll away from the L/h;
|
||||
- a single site was centred under the key;
|
||||
- sites at one position hid each other.
|
||||
|
||||
All three are fixed and tested. Those last fixes, and the filter row and the map search, have
|
||||
not had an independent inspection.
|
||||
|
||||
**Filters back on the left, the readers' Fleet tab, the navbar logo, section buttons** *(owner,
|
||||
29 Oct)*:
|
||||
checked in the browser as `reader` and `admin` at 1400 × 1000, 1920 × 1080, 1440 × 760,
|
||||
1280 × 600 and 1024 wide:
|
||||
- a reader's Fleet tab has no filter column and no resize handle, and the map is as wide as
|
||||
the page and reaches 32 px above the window's bottom edge (no page scroll), at once rather
|
||||
than after Superset measures again; at 600 px high it keeps 560 px and the page scrolls;
|
||||
- Site details shows the column collapsed (32 px, for readers and admins, also after the
|
||||
map's "Site details" link); expanded, it has the Site filter; switching back hides it again;
|
||||
- an admin has the column on both tabs;
|
||||
- the column no longer flashes up while the Fleet tab loads (one frame, about 20 ms, remains:
|
||||
the grid exists before its tabs do);
|
||||
- after a switch to Site details its charts are too wide for one or two frames before
|
||||
Superset measures again; not changed;
|
||||
- a reader's page links `reader.css?v=<hash>`, an admin's none;
|
||||
- the navbar shows the wordmark at 24 px for readers, admins and the login page, white in
|
||||
dark mode;
|
||||
- Site details' inner tabs are a row of section buttons (`dashboard.css`), lined up with the
|
||||
content, all four visible and switching, for readers and admins, light and dark; a
|
||||
vertical list was tried first and dropped: Superset sizes the charts for the full width, so
|
||||
they were cut off on the right;
|
||||
- `build_bundle.py` stops when the orientation is not VERTICAL, when reader.css does not name
|
||||
the Fleet tab, when the map spans fewer than 12 columns, or when `dashboard.css` names an
|
||||
id that is not in the layout.
|
||||
|
||||
- *(lenses)* a reader at 1400 px: the switch centred in the navbar (Map | Insights), the map
|
||||
735 px high in an 860 px window and 32 px above its bottom edge; scrolled by 30, 61 and
|
||||
400 px the switch moves up by exactly as much and then stays out of view; at 820 px the tab
|
||||
row stays in the page; an admin keeps the tab row, named Map and Insights; the map's key
|
||||
says "Choose them in Insights".
|
||||
- *(AI teaser)* a reader at 1400 px: the switch reads Map | Insights | AI PRO, still centred,
|
||||
44 px high; a click on AI changes nothing; at 900 px the switch keeps Map | Insights.
|
||||
|
||||
These changes have not had an independent inspection (the owner skipped Codex).
|
||||
|
||||
**Not run:** check 1 (a synthetic fourth and fifth site), check 6 (the Viewing period's scope is
|
||||
unchanged), sign-in through Keycloak, the server's routing of `/views`, sign-in for the read
|
||||
service, thousands of sites, and dark mode.
|
||||
|
||||
## 10. The Petrolyst app's dashboards *(PD-12, local trial, 29 Oct 2026)*
|
||||
|
||||
The Petrolyst app (petrolyst-platform `app/`, PD-12) shows Superset dashboards inside its
|
||||
Insights lens, with guest passes from its app service. `build_bundle.py` builds them beside the
|
||||
unchanged ATOMA dashboard:
|
||||
|
||||
| Slug | Content | Filters (set by the app; Superset's filter column is hidden) |
|
||||
|---|---|---|
|
||||
| `atoma-overview` | O1–O6: sites by pilot status and by baseline status, litres delivered, fuel cost by currency, electricity billed by currency and basis, litres delivered per month | Viewing period on O3–O6 |
|
||||
| `atoma-site` | the four §4 detail tabs as Superset's own tabs, the same rows and charts; no D0 (the app shows the site's header) | Site (all charts), Viewing period (as §4: fuel and runtime charts only) |
|
||||
|
||||
- **One site dashboard with tabs** *(owner, 29 Oct: "move it back inside the dashboard and make
|
||||
the tab as they were")*: replaces the four one-tab dashboards `atoma-site-summary`, `-fuel`,
|
||||
`-runtime`, `-baseline` and the app's per-section pages. Switching tabs needs no reload. The
|
||||
app opens a site on its first tab; a new viewing period reloads the dashboard on the tab that
|
||||
was open (the app reads it from Superset and names it in the address, `#TAB-fuel`). Delete
|
||||
the four old dashboards by hand on an instance that has them.
|
||||
|
||||
- **Overview rule** (changes §4 rule 3 for the Overview only): only sums and counts of
|
||||
recorded values across sites (`OVERVIEW_METRICS`); per-site rates (L/h, L/day) are never
|
||||
averaged; money is grouped by `currency_text`, a missing currency is its own row, and no
|
||||
total crosses currencies. `build_bundle.py` checks both.
|
||||
- They carry no CSS; `dashboard.css` and `reader.css` are checked against the ATOMA dashboard
|
||||
only.
|
||||
- `import_bundle.py --embedded-out FILE --embed-domain URL` makes them embeddable from the app's
|
||||
origin and writes `{slug: embedding id}` for the app service; the reader role stays on the
|
||||
ATOMA dashboard only.
|
||||
- Embedding is switched on in the local stack only (`.local/fmdb-scratch`,
|
||||
`superset_config_local.py`); production keeps it off until PD-12's open points are settled.
|
||||
- **Checked (local):** a self-signed guest pass opens its own dashboard and nothing else (another
|
||||
embedded dashboard and ATOMA: 404; no pass or a wrong audience: 401); the embedded page answers
|
||||
only to the app's referrer (403 otherwise); chart-data responses to guests carry no SQL.
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,309 @@
|
||||
# Columns of the public.v_* views (migrations 0001-0003), as Trino reports them.
|
||||
# build_bundle.py lists every one in its dataset; import_bundle.py reports drift.
|
||||
# Regenerate after a migration that changes a view:
|
||||
# trino --output-format TSV --execute "select table_name, column_name, data_type
|
||||
# from fuel.information_schema.columns where table_schema = 'public'
|
||||
# and table_name like 'v\_%' escape '\' order by table_name, ordinal_position"
|
||||
v_assumption site_uid varchar(12)
|
||||
v_assumption site_code varchar(20)
|
||||
v_assumption site_name varchar(120)
|
||||
v_assumption site_label varchar(170)
|
||||
v_assumption code varchar(60)
|
||||
v_assumption topic varchar(30)
|
||||
v_assumption asset_uid varchar(40)
|
||||
v_assumption value decimal(10,3)
|
||||
v_assumption value_low decimal(10,3)
|
||||
v_assumption value_high decimal(10,3)
|
||||
v_assumption unit varchar(20)
|
||||
v_assumption applies_from date
|
||||
v_assumption applies_to date
|
||||
v_assumption basis varchar(20)
|
||||
v_assumption derived_from_fuel boolean
|
||||
v_assumption status varchar(12)
|
||||
v_assumption accepted_on date
|
||||
v_assumption decision_ref varchar(12)
|
||||
v_assumption rationale varchar(400)
|
||||
v_assumption contradicted_by varchar(200)
|
||||
v_assumption members varchar(400)
|
||||
v_baseline_input site_uid varchar(12)
|
||||
v_baseline_input site_code varchar(20)
|
||||
v_baseline_input site_name varchar(120)
|
||||
v_baseline_input site_label varchar(170)
|
||||
v_baseline_input baseline_code varchar(60)
|
||||
v_baseline_input baseline_status varchar(12)
|
||||
v_baseline_input interval_label varchar(30)
|
||||
v_baseline_input start_on date
|
||||
v_baseline_input end_on date
|
||||
v_baseline_input included boolean
|
||||
v_baseline_input exclusion_reason varchar(300)
|
||||
v_baseline_input fuel_l decimal(10,1)
|
||||
v_baseline_input runtime_h decimal(10,1)
|
||||
v_baseline_input days integer
|
||||
v_baseline_input eligibility_at_computation varchar(12)
|
||||
v_baseline_input evidence_grade_at_computation varchar(2)
|
||||
v_baseline_input dependencies_changed boolean
|
||||
v_baseline_version site_uid varchar(12)
|
||||
v_baseline_version site_code varchar(20)
|
||||
v_baseline_version site_name varchar(120)
|
||||
v_baseline_version site_label varchar(170)
|
||||
v_baseline_version code varchar(60)
|
||||
v_baseline_version metric varchar(40)
|
||||
v_baseline_version status varchar(12)
|
||||
v_baseline_version population varchar(200)
|
||||
v_baseline_version value decimal(8,3)
|
||||
v_baseline_version range_low decimal(8,3)
|
||||
v_baseline_version range_high decimal(8,3)
|
||||
v_baseline_version unit varchar(8)
|
||||
v_baseline_version window_start date
|
||||
v_baseline_version window_end date
|
||||
v_baseline_version fuel_l decimal(10,1)
|
||||
v_baseline_version runtime_h decimal(10,1)
|
||||
v_baseline_version matched_days integer
|
||||
v_baseline_version policy_version varchar(10)
|
||||
v_baseline_version meets_policy boolean
|
||||
v_baseline_version policy_check varchar(300)
|
||||
v_baseline_version evidence_grade varchar(2)
|
||||
v_baseline_version conditions varchar(200)
|
||||
v_baseline_version season_note varchar(200)
|
||||
v_baseline_version approved_on date
|
||||
v_baseline_version approved_by_role varchar(40)
|
||||
v_baseline_version decision_ref varchar(12)
|
||||
v_baseline_version inputs_unchanged boolean
|
||||
v_baseline_version computed_at timestamp(0)
|
||||
v_coverage site_uid varchar(12)
|
||||
v_coverage site_code varchar(20)
|
||||
v_coverage site_name varchar(120)
|
||||
v_coverage site_label varchar(170)
|
||||
v_coverage lane varchar(60)
|
||||
v_coverage status varchar(40)
|
||||
v_coverage start_on date
|
||||
v_coverage end_on date
|
||||
v_coverage start_ts timestamp(0)
|
||||
v_coverage end_ts timestamp(0)
|
||||
v_data_quality site_uid varchar(12)
|
||||
v_data_quality site_code varchar(20)
|
||||
v_data_quality site_name varchar(120)
|
||||
v_data_quality site_label varchar(170)
|
||||
v_data_quality check_order integer
|
||||
v_data_quality check_code varchar(40)
|
||||
v_data_quality label varchar(80)
|
||||
v_data_quality status varchar(10)
|
||||
v_data_quality detail varchar(300)
|
||||
v_delivery site_uid varchar(12)
|
||||
v_delivery site_code varchar(20)
|
||||
v_delivery site_name varchar(120)
|
||||
v_delivery site_label varchar(170)
|
||||
v_delivery event_key varchar(80)
|
||||
v_delivery delivery_date date
|
||||
v_delivery seq_in_day integer
|
||||
v_delivery delivered_l decimal(8,1)
|
||||
v_delivery unit_price decimal(8,2)
|
||||
v_delivery currency varchar(8)
|
||||
v_delivery source_ref varchar(60)
|
||||
v_delivery n_sources integer
|
||||
v_delivery sources_disagree boolean
|
||||
v_delivery reviewed boolean
|
||||
v_electricity_cost site_uid varchar(12)
|
||||
v_electricity_cost site_code varchar(20)
|
||||
v_electricity_cost site_name varchar(120)
|
||||
v_electricity_cost site_label varchar(170)
|
||||
v_electricity_cost bill_month date
|
||||
v_electricity_cost period_start date
|
||||
v_electricity_cost period_end date
|
||||
v_electricity_cost period_known boolean
|
||||
v_electricity_cost amount decimal(12,2)
|
||||
v_electricity_cost currency varchar(8)
|
||||
v_electricity_cost currency_basis varchar(10)
|
||||
v_electricity_cost kwh decimal(10,1)
|
||||
v_electricity_cost source_ref varchar(60)
|
||||
v_electricity_cost claim_status varchar(12)
|
||||
v_evidence site_uid varchar(12)
|
||||
v_evidence site_code varchar(20)
|
||||
v_evidence site_name varchar(120)
|
||||
v_evidence site_label varchar(170)
|
||||
v_evidence document_key varchar(16)
|
||||
v_evidence title varchar(200)
|
||||
v_evidence kind varchar(20)
|
||||
v_evidence issuer varchar(80)
|
||||
v_evidence document_date date
|
||||
v_evidence used_for varchar(200)
|
||||
v_evidence contains_personal_data boolean
|
||||
v_hour_meter_evidence site_uid varchar(12)
|
||||
v_hour_meter_evidence site_code varchar(20)
|
||||
v_hour_meter_evidence site_name varchar(120)
|
||||
v_hour_meter_evidence site_label varchar(170)
|
||||
v_hour_meter_evidence meter_label varchar(40)
|
||||
v_hour_meter_evidence mapped_set varchar(60)
|
||||
v_hour_meter_evidence reading_date date
|
||||
v_hour_meter_evidence date_basis varchar(30)
|
||||
v_hour_meter_evidence hours decimal(9,1)
|
||||
v_hour_meter_evidence reading_state varchar(16)
|
||||
v_hour_meter_evidence source_field varchar(16)
|
||||
v_hour_meter_evidence epoch_no integer
|
||||
v_hour_meter_evidence claim_status varchar(12)
|
||||
v_imported_calculation site_uid varchar(12)
|
||||
v_imported_calculation site_code varchar(20)
|
||||
v_imported_calculation site_name varchar(120)
|
||||
v_imported_calculation site_label varchar(170)
|
||||
v_imported_calculation cell_ref varchar(40)
|
||||
v_imported_calculation source_label varchar(80)
|
||||
v_imported_calculation formula varchar(200)
|
||||
v_imported_calculation cached_value decimal(14,6)
|
||||
v_imported_calculation unit_as_labelled varchar(30)
|
||||
v_imported_calculation our_reading varchar(300)
|
||||
v_imported_calculation baseline_use varchar(12)
|
||||
v_imported_calculation exclusion_reason varchar(300)
|
||||
v_imported_calculation document_key varchar(16)
|
||||
v_imported_calculation claim_status varchar(12)
|
||||
v_interval site_uid varchar(12)
|
||||
v_interval site_code varchar(20)
|
||||
v_interval site_name varchar(120)
|
||||
v_interval site_label varchar(170)
|
||||
v_interval interval_label varchar(30)
|
||||
v_interval start_on date
|
||||
v_interval end_on date
|
||||
v_interval days integer
|
||||
v_interval opening_l decimal(9,1)
|
||||
v_interval opening_basis varchar(8)
|
||||
v_interval deliveries_l decimal(9,1)
|
||||
v_interval withdrawals_l decimal(9,1)
|
||||
v_interval closing_l decimal(9,1)
|
||||
v_interval closing_basis varchar(8)
|
||||
v_interval fuel_used_l decimal(9,1)
|
||||
v_interval fuel_used_unc_l decimal(9,1)
|
||||
v_interval fuel_l_per_day decimal(8,1)
|
||||
v_interval stock_status varchar(40)
|
||||
v_interval runtime_h decimal(9,1)
|
||||
v_interval runtime_h_per_day decimal(6,2)
|
||||
v_interval n_meters integer
|
||||
v_interval metric varchar(40)
|
||||
v_interval population_key varchar(200)
|
||||
v_interval l_per_h decimal(8,3)
|
||||
v_interval l_per_h_low decimal(8,3)
|
||||
v_interval l_per_h_high decimal(8,3)
|
||||
v_interval eligibility varchar(12)
|
||||
v_interval exclusion_reasons varchar(600)
|
||||
v_interval evidence_grade varchar(2)
|
||||
v_interval conditions varchar(200)
|
||||
v_interval in_current_baseline boolean
|
||||
v_interval baseline_comparable boolean
|
||||
v_interval current_baseline_l_per_h decimal(8,3)
|
||||
v_kpi_measures site_uid varchar(12)
|
||||
v_kpi_measures site_code varchar(20)
|
||||
v_kpi_measures site_name varchar(120)
|
||||
v_kpi_measures site_label varchar(170)
|
||||
v_kpi_measures measure_order integer
|
||||
v_kpi_measures measure varchar(60)
|
||||
v_kpi_measures value decimal(12,2)
|
||||
v_kpi_measures unit varchar(12)
|
||||
v_kpi_measures value_kind varchar(16)
|
||||
v_kpi_measures window_start date
|
||||
v_kpi_measures window_end date
|
||||
v_kpi_measures note varchar(200)
|
||||
v_modelled_lph site_uid varchar(12)
|
||||
v_modelled_lph site_code varchar(20)
|
||||
v_modelled_lph site_name varchar(120)
|
||||
v_modelled_lph site_label varchar(170)
|
||||
v_modelled_lph assumption_code varchar(60)
|
||||
v_modelled_lph status varchar(12)
|
||||
v_modelled_lph basis varchar(20)
|
||||
v_modelled_lph h_per_day decimal(5,2)
|
||||
v_modelled_lph h_per_day_low decimal(5,2)
|
||||
v_modelled_lph h_per_day_high decimal(5,2)
|
||||
v_modelled_lph window_start date
|
||||
v_modelled_lph window_end date
|
||||
v_modelled_lph matched_days integer
|
||||
v_modelled_lph l_per_day decimal(7,1)
|
||||
v_modelled_lph modelled_l_per_h decimal(6,2)
|
||||
v_modelled_lph modelled_l_per_h_low decimal(6,2)
|
||||
v_modelled_lph modelled_l_per_h_high decimal(6,2)
|
||||
v_modelled_lph value_kind varchar(70)
|
||||
v_reconciliation site_uid varchar(12)
|
||||
v_reconciliation site_code varchar(20)
|
||||
v_reconciliation site_name varchar(120)
|
||||
v_reconciliation site_label varchar(170)
|
||||
v_reconciliation interval_label varchar(30)
|
||||
v_reconciliation start_on date
|
||||
v_reconciliation end_on date
|
||||
v_reconciliation stock_status varchar(40)
|
||||
v_reconciliation endpoint_times_known boolean
|
||||
v_reconciliation asset_uid varchar(40)
|
||||
v_reconciliation placed_throughout boolean
|
||||
v_reconciliation stock_balance_l decimal(9,1)
|
||||
v_reconciliation engine_l decimal(9,1)
|
||||
v_reconciliation covered_h decimal(9,1)
|
||||
v_reconciliation interval_h decimal(9,1)
|
||||
v_reconciliation engine_all_generators_l decimal(9,1)
|
||||
v_reconciliation all_generators_metered boolean
|
||||
v_reconciliation difference_l decimal(9,1)
|
||||
v_reconciliation difference_pct decimal(7,1)
|
||||
v_runtime site_uid varchar(12)
|
||||
v_runtime site_code varchar(20)
|
||||
v_runtime site_name varchar(120)
|
||||
v_runtime site_label varchar(170)
|
||||
v_runtime meter_label varchar(40)
|
||||
v_runtime mapped_set varchar(60)
|
||||
v_runtime interval_label varchar(80)
|
||||
v_runtime start_on date
|
||||
v_runtime end_on date
|
||||
v_runtime days integer
|
||||
v_runtime runtime_h decimal(9,1)
|
||||
v_runtime h_per_day decimal(6,2)
|
||||
v_runtime pair_valid boolean
|
||||
v_runtime same_epoch boolean
|
||||
v_runtime monotonic boolean
|
||||
v_runtime over_24h_per_day boolean
|
||||
v_runtime reviewed boolean
|
||||
v_site site_uid varchar(12)
|
||||
v_site site_code varchar(20)
|
||||
v_site site_name varchar(120)
|
||||
v_site site_label varchar(170)
|
||||
v_site zone varchar(40)
|
||||
v_site province varchar(40)
|
||||
v_site district varchar(60)
|
||||
v_site pilot_status varchar(12)
|
||||
v_site latitude decimal(9,6)
|
||||
v_site longitude decimal(9,6)
|
||||
v_site location_status varchar(20)
|
||||
v_site location_basis varchar(20)
|
||||
v_site location_precision_m integer
|
||||
v_site baseline_status varchar(24)
|
||||
v_site baseline_status_label varchar(40)
|
||||
v_site baseline_validity_note varchar(60)
|
||||
v_site lph_estimate decimal(6,2)
|
||||
v_site lph_method varchar(60)
|
||||
v_site lph_window_start date
|
||||
v_site lph_window_end date
|
||||
v_site baseline_conditions varchar(200)
|
||||
v_site baseline_decision_ref varchar(12)
|
||||
v_site fuel_l_per_calendar_day decimal(7,1)
|
||||
v_site latest_record_on date
|
||||
v_spec_reference site_uid varchar(12)
|
||||
v_spec_reference site_code varchar(20)
|
||||
v_spec_reference site_name varchar(120)
|
||||
v_spec_reference site_label varchar(170)
|
||||
v_spec_reference slot varchar(20)
|
||||
v_spec_reference asset_uid varchar(40)
|
||||
v_spec_reference asset_kind varchar(10)
|
||||
v_spec_reference make_model varchar(120)
|
||||
v_spec_reference attribute varchar(40)
|
||||
v_spec_reference load_pct decimal(5,1)
|
||||
v_spec_reference value decimal(12,3)
|
||||
v_spec_reference value_text varchar(120)
|
||||
v_spec_reference unit varchar(20)
|
||||
v_spec_reference basis varchar(30)
|
||||
v_spec_reference applicability varchar(12)
|
||||
v_spec_reference applicability_note varchar(300)
|
||||
v_spec_reference value_kind varchar(30)
|
||||
v_spec_reference claim_status varchar(12)
|
||||
v_stock_reading site_uid varchar(12)
|
||||
v_stock_reading site_code varchar(20)
|
||||
v_stock_reading site_name varchar(120)
|
||||
v_stock_reading site_label varchar(170)
|
||||
v_stock_reading observed_on date
|
||||
v_stock_reading seq_in_day integer
|
||||
v_stock_reading reading_type varchar(20)
|
||||
v_stock_reading litres decimal(8,1)
|
||||
v_stock_reading value_kind varchar(12)
|
||||
v_stock_reading spread_l decimal(8,1)
|
||||
v_stock_reading reviewed boolean
|
||||
|
Can't render this file because it contains an unexpected character in line 4 and column 41.
|
@@ -0,0 +1,33 @@
|
||||
/* The ATOMA dashboard's own CSS, for everyone who opens it (build_bundle.py puts it into the
|
||||
bundle and checks that every #id here is in the layout).
|
||||
|
||||
Site details' inner tabs (TABS-detail) as a row of section buttons, so they read as a level
|
||||
below Fleet | Site details (owner, 29 Oct). Aligned with the content's left edge.
|
||||
Translucent colours: right in the light and the dark theme. */
|
||||
#TABS-detail > .ant-tabs-nav {
|
||||
margin: 0 16px 16px;
|
||||
}
|
||||
#TABS-detail > .ant-tabs-nav::before {
|
||||
border-bottom: 0;
|
||||
}
|
||||
#TABS-detail > .ant-tabs-nav .ant-tabs-ink-bar {
|
||||
display: none;
|
||||
}
|
||||
#TABS-detail > .ant-tabs-nav .ant-tabs-nav-list {
|
||||
gap: 4px;
|
||||
padding: 4px;
|
||||
border-radius: 10px;
|
||||
background: rgba(127, 127, 127, 0.1);
|
||||
}
|
||||
#TABS-detail > .ant-tabs-nav .ant-tabs-tab {
|
||||
margin: 0 !important;
|
||||
padding: 6px 14px !important;
|
||||
border-radius: 8px !important;
|
||||
font-size: 13px;
|
||||
}
|
||||
#TABS-detail > .ant-tabs-nav .ant-tabs-tab-active {
|
||||
background: rgba(0, 150, 136, 0.14) !important;
|
||||
}
|
||||
#TABS-detail > .ant-tabs-nav .ant-tabs-tab-active .ant-tabs-tab-btn {
|
||||
font-weight: 600;
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
"""Import the ATOMA dashboard bundle into Superset, overwriting what it defines.
|
||||
|
||||
Runs inside the Superset container, with the image's Python:
|
||||
|
||||
docker cp import_bundle.py <container>:/tmp/ && docker cp atoma-dashboards.zip <container>:/tmp/
|
||||
docker exec <container> /app/.venv/bin/python /tmp/import_bundle.py --show-database
|
||||
docker exec <container> /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip --user <username>
|
||||
|
||||
`superset import-dashboards` imports a dashboard's datasets and charts with
|
||||
overwrite off, so an existing chart or dataset keeps its old definition. This
|
||||
script runs the dataset, chart and dashboard importers in turn, each with
|
||||
overwrite on (the bundle's metadata.yaml type is set to what each importer
|
||||
expects). A dataset import replaces the column list, so columns missing from
|
||||
the bundle are dropped; afterwards each dataset's columns are compared with the
|
||||
live view and any drift is reported (exit status 2).
|
||||
|
||||
The database is never created or changed: the bundle's database uuid must
|
||||
already exist under the name "Fuel (Trino)", otherwise nothing is imported.
|
||||
|
||||
A bundle cannot carry a dashboard's roles, so afterwards the role "ATOMA Reader"
|
||||
is attached to the ATOMA dashboard: with DASHBOARD_RBAC that is how ATOMA's
|
||||
readers see it. The role comes from roles/atoma_reader.py (run by superset-init);
|
||||
without it nothing is imported.
|
||||
|
||||
The Petrolyst app's dashboards (PD-12: atoma-overview, atoma-site) get no role:
|
||||
the app shows them with guest passes. With --embedded-out FILE --embed-domain URL,
|
||||
each of them is made embeddable from that origin (its embedding id stays the same
|
||||
across imports), and FILE receives {slug: embedding id} for the app service:
|
||||
|
||||
docker exec <container> /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip \
|
||||
--user <username> --embedded-out /tmp/embedded-dashboards.json --embed-domain http://127.0.0.1:8098
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
import zipfile
|
||||
|
||||
import yaml
|
||||
|
||||
DATABASE_NAME = "Fuel (Trino)"
|
||||
READER_ROLE = "ATOMA Reader"
|
||||
READER_SLUG = "atoma" # the readers' dashboard; the others are the app's (PD-12)
|
||||
EMBEDDED_PREFIX = "atoma-"
|
||||
STEPS = [("datasets", "SqlaTable"), ("charts", "Slice"), ("dashboards", "Dashboard")]
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Import the ATOMA dashboard bundle.")
|
||||
parser.add_argument("bundle", nargs="?")
|
||||
parser.add_argument("--user", help="existing Superset user who will own new objects")
|
||||
parser.add_argument("--show-database", action="store_true",
|
||||
help=f'print the uuid of "{DATABASE_NAME}" and stop')
|
||||
parser.add_argument("--embedded-out", help="write {slug: embedding id} of the app's dashboards")
|
||||
parser.add_argument("--embed-domain", help="the app's origin, e.g. http://127.0.0.1:8098")
|
||||
args = parser.parse_args()
|
||||
if not args.show_database and not (args.bundle and args.user):
|
||||
parser.error("give a bundle and --user, or --show-database")
|
||||
if bool(args.embedded_out) != bool(args.embed_domain):
|
||||
parser.error("--embedded-out and --embed-domain go together")
|
||||
|
||||
from superset.app import create_app
|
||||
|
||||
app = create_app()
|
||||
with app.app_context():
|
||||
from superset import db, security_manager
|
||||
from superset.commands.chart.importers.v1 import ImportChartsCommand
|
||||
from superset.commands.dashboard.importers.v1 import ImportDashboardsCommand
|
||||
from superset.commands.dataset.importers.v1 import ImportDatasetsCommand
|
||||
from superset.commands.importers.v1.utils import get_contents_from_bundle
|
||||
from superset.connectors.sqla.models import SqlaTable
|
||||
from superset.models.core import Database
|
||||
from superset.models.dashboard import Dashboard
|
||||
from superset.utils.core import override_user
|
||||
|
||||
if args.show_database:
|
||||
found = db.session.query(Database).filter_by(database_name=DATABASE_NAME).all()
|
||||
for database in found:
|
||||
print(database.uuid)
|
||||
return 0 if found else 1
|
||||
|
||||
user = security_manager.find_user(username=args.user)
|
||||
if user is None:
|
||||
print(f"no Superset user {args.user!r}", file=sys.stderr)
|
||||
return 1
|
||||
reader_role = security_manager.find_role(READER_ROLE)
|
||||
if reader_role is None:
|
||||
print(f'no role "{READER_ROLE}": run roles/atoma_reader.py first', file=sys.stderr)
|
||||
return 1
|
||||
with zipfile.ZipFile(args.bundle) as bundle:
|
||||
contents = get_contents_from_bundle(bundle)
|
||||
databases = [yaml.safe_load(t) for p, t in contents.items() if p.startswith("databases/")]
|
||||
for config in databases:
|
||||
database = db.session.query(Database).filter_by(uuid=config["uuid"]).one_or_none()
|
||||
if database is None or database.database_name != DATABASE_NAME:
|
||||
print(f'database {config["uuid"]} is not "{DATABASE_NAME}" here; '
|
||||
"rebuild the bundle with the uuid from --show-database", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
metadata = yaml.safe_load(contents["metadata.yaml"])
|
||||
commands = {"datasets": ImportDatasetsCommand, "charts": ImportChartsCommand,
|
||||
"dashboards": ImportDashboardsCommand}
|
||||
with override_user(user):
|
||||
for step, model in STEPS:
|
||||
step_contents = dict(contents)
|
||||
step_contents["metadata.yaml"] = yaml.safe_dump({**metadata, "type": model})
|
||||
commands[step](step_contents, overwrite=True).run()
|
||||
count = sum(p.startswith(step + "/") for p in contents)
|
||||
print(f"{step}: {count} imported (overwrite)")
|
||||
|
||||
embedded: dict[str, str] = {}
|
||||
for path, text in contents.items():
|
||||
if path.startswith("dashboards/"):
|
||||
dashboard = db.session.query(Dashboard).filter_by(
|
||||
uuid=yaml.safe_load(text)["uuid"]).one()
|
||||
if dashboard.slug == READER_SLUG:
|
||||
if reader_role not in dashboard.roles:
|
||||
dashboard.roles.append(reader_role)
|
||||
elif dashboard.slug.startswith(EMBEDDED_PREFIX) and args.embedded_out:
|
||||
from superset.daos.dashboard import EmbeddedDashboardDAO
|
||||
|
||||
config = EmbeddedDashboardDAO.upsert(dashboard, [args.embed_domain])
|
||||
db.session.flush()
|
||||
embedded[dashboard.slug] = str(config.uuid)
|
||||
db.session.commit()
|
||||
print(f'dashboards: "{READER_SLUG}" readable for "{READER_ROLE}"')
|
||||
if args.embedded_out:
|
||||
with open(args.embedded_out, "w") as out:
|
||||
json.dump(dict(sorted(embedded.items())), out, indent=2)
|
||||
print(f"embedded: {len(embedded)} dashboard(s) for {args.embed_domain}, "
|
||||
f"ids in {args.embedded_out}")
|
||||
|
||||
drift = 0
|
||||
dataset_uuids = [yaml.safe_load(t)["uuid"] for p, t in contents.items()
|
||||
if p.startswith("datasets/")]
|
||||
for dataset_uuid in dataset_uuids:
|
||||
dataset = db.session.query(SqlaTable).filter_by(uuid=dataset_uuid).one()
|
||||
physical = {c.column_name for c in dataset.columns if not c.expression}
|
||||
live = {c["column_name"] for c in dataset.external_metadata()}
|
||||
if physical != live:
|
||||
drift += 1
|
||||
print(f"drift {dataset.table_name}: only in bundle {sorted(physical - live)}, "
|
||||
f"only in view {sorted(live - physical)}")
|
||||
print(f"column drift: {drift} dataset(s)")
|
||||
return 2 if drift else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in new issue
Block a user