petrolyst-infra/petrolyst-insights/dashboards/atoma/SPEC.md

ATOMA dashboards: Sites Map and Site Detail

Status: proposed, not built.

• build_bundle.py (phase P2) will generate both dashboards, every dataset and every chart from this specification, as one Superset import bundle.
• Nothing is imported into the live Superset without the owner's go-ahead (phase P3).

Datasets. The public.v_* views of stacks/fuel-monitoring/fuel-db/migrations/0003_bi_views.sql, read through Trino as fuel.public.

Design and rationale live in the fuel-monitoring corpus, in .context/plans/2026-10-24-fuel-db-baseline-dashboard.md:

• its §6 is this specification;
• the site-specific facts left out here (positions, distances, pilot sites) are there too;
• Q-numbers refer to its §7, and phases P0–P5 to its §8.

Two dashboards replace the earlier tab-per-site design; nothing in them is specific to a site.

1. Capabilities of the installed Superset 6.1.0

Source: read-only inspection of config.py, superset_config.py and the frontend bundle on 24 Oct 2026. Items marked (local test) are confirmed in P2 before the design relies on them.

Requirement	What 6.1.0 provides	Choice
Map chart	deck_scatter (deck.gl Scatterplot): a modern plugin with the InteractiveChart behaviour. Also: deck_multi, deck_screengrid / hex / grid, country_map, world_map, cartodiagram, legacy mapbox	deck.gl Scatterplot
Basemap	DECKGL_BASE_MAP unset → OSM raster (tile.openstreetmap.org), OSM Swiss topo, Mapbox styles. MAPBOX_API_KEY is empty, so the Mapbox styles fail. CSP connect-src already allows the OSM host; Traefik sets no CSP	set DECKGL_BASE_MAP to OSM only; "© OpenStreetMap contributors" in a Markdown under the map
OSM terms	no key, no cost; tile usage policy: light use, attribution, no bulk download, a valid Referer (ours sends the origin)	fine for an internal user base; viewers' browsers reveal the viewed area (tile coordinates) and their IP to OSMF, not our data. Self-hosted tiles later if needed
Tooltip	"Tooltip contents" (drag columns) plus "Customize tooltips template" (Handlebars). js_tooltip exists but ENABLE_JAVASCRIPT_CONTROLS is False	contents and template; no JavaScript
Marker colour	"Categorical color" by a dimension, from the colour scheme; a fixed colour per label through the dashboard's label_colors (local test)	colour by baseline_status_label. Fallback: deck_multi with one fixed-colour scatter layer per status
Marker size	fixed radius, with minimum = maximum pixel radius	uniform
Viewport	autozoom fits the filtered points	on
Clustering	none in deck.gl; only the legacy mapbox chart (clustering_radius, needs a Mapbox key)	no clustering (§3)
Marker click	emits a cross-filter on the latitude and longitude columns: the exact point, or a box around the click that shrinks as you zoom in. It applies to other charts on the same dashboard	cross-filters the Sites list
Navigation	no chart-to-dashboard navigation. A dashboard reads ?native_filters=<rison> into preselected native filter values (also native_filters_key). Table cells render sanitized HTML (allow_render_html)	an "Open site detail" link column in the Sites list
Native filters	Select (single or multi, required, searchable, "first value by default"), Time range; scoping per chart and tab; cross-filter scoping per chart	§4, §5

No custom frontend and no patch are needed.

2. Navigation flow

ATOMA — Sites  (landing, slug atoma-sites)
  filters: Site search · Zone · Province · Pilot status (default: active) · Baseline status
  M1 map: markers fitted to the filtered sites ──hover──► tooltip
          └─click marker──► cross-filter: M3 Sites list shows that site (or the few under the click)
  M3 Sites list ─────────────"Open site detail"──► ATOMA — Site Detail ?native_filters=(site preset)
  M4/M5 Sites without an accepted location ──────"Open site detail"──► same
ATOMA — Site Detail  (slug atoma-site-detail)
  filters: Site (exactly one, searchable) · Viewing period
  tabs: Summary · Fuel & Deliveries · Runtime & Electricity · Baseline & Evidence
  header link "← Sites map"

3. Sites Map (landing dashboard)

[Filter bar: Site search · Zone · Province · Pilot status · Baseline status]
[M1 Map (deck.gl Scatterplot) ................................ w8][M2 Status key + attribution w4]
[M3 Sites list (table with link column) ..................................................... w12]
[M4 Sites without accepted location: count w3][M5 Sites without accepted location (table) .. w9]

#	Question	Type	Dataset	Content	Missing or insufficient
M1	Where are the sites, and what is each one's baseline status?	deck.gl Scatterplot; lat/lon from latitude, longitude; uniform size; colour = baseline_status_label; autozoom	v_site	tooltip: site_code, site_name, lph_estimate ("not available" when NULL), lph_method, baseline_status_label, baseline_validity_note (when set), latest_record_on	sites without an accepted location have NULL lat/lon and are not plotted; M4/M5 show them
M2	What do the colours mean?	Markdown (static)	—	the status definitions below; "Colours show evidence status, not performance"; OSM attribution	—
M3	Which sites match, and how do I open one?	Table; search box; server pagination	v_site	code, name, province, pilot status, baseline status, L/h (+ method), L/day, latest record, location status, detail_link	marker click narrows it; "not available" shown as text
M4	How many sites lack an accepted location?	Big Number Total, COUNT(*) with saved filter location_status <> 'accepted'	v_site	count; follows the dashboard filters	0 is shown as 0
M5	Which sites are they?	Table; search box	v_site, same saved filter	code, name, province, location_status (awaiting acceptance · missing), baseline status, detail_link	—

Filters (all on v_site, scoped to M1, M3, M4 and M5):

• Site search: Select on site_label, multi, searchable. The label holds code, name and uid, so code or name both match.
• Zone, Province: Select.
• Pilot status: default active.
• Baseline status: Select on baseline_status_label.

The cross-filter from M1 is scoped to M3 only. M4 and M5 hold the sites without coordinates and must not be emptied by it.

Baseline status and marker colours. Colours are neutral categorical hues, with no red or green and no reference to L/h:

Status	Definition (fm.v_site_baseline_status)	Colour
Approved baseline	an approved baseline_version with a decision reference. The only state that says "approved"	teal #009688
Eligible – not yet approved	no approval; the latest proposed KPI version has meets_policy (BP-1: ≥ 2 eligible G1 inputs, ≥ 60 matched days, no conditions, one population and metric)	blue #3987e5
Provisional	no approval; a proposed version below BP-1, or usable intervals with no version yet (no estimate shown, method "no baseline version proposed")	amber #c98500
Insufficient evidence	no usable interval. Every pilot site while no runtime evidence exists	slate #64748B

Nearby sites.

• Sites a few kilometres apart separate at zoom ≥ 11 (≈ 60 px per 2–4 km at zoom 12) and coincide at country zoom (≈ 2 km per pixel at zoom 6). The pilot's own distances are in the design doc.
• Handling:
  • autozoom to the filtered sites (the default "active" filter opens on the pilot area);
  • small uniform markers (min = max radius, about 7 px) with an outline;
  • a click near overlapping markers selects every site in the click box, and M3 lists them;
  • no clustering.
• With hundreds of sites, a deck_multi density layer (screengrid) for low zoom is a later option.

4. Site Detail (one dashboard, any site)

Filters

• Site (NATIVE_FILTER-site):
  • Select on v_site.site_label, single value, required, searchable, no default;
  • scope: every chart on every tab;
  • every detail dataset carries site_label from v_site, so this one filter drives all of them (local test).
• Viewing period (NATIVE_FILTER-period):
  • Time range, default no filter;
  • scope: the time-series charts on Fuel & Deliveries and Runtime & Electricity only;
  • Summary and Baseline & Evidence are outside its scope, so a viewing range never changes a baseline's fixed window.

Exactly one site, and never network totals:

1. A required, single-value filter (local test: with the value cleared, no chart queries unfiltered data).
2. The header (D0) and KPI (S1) are Handlebars templates that render a site only when exactly one row arrives. Otherwise they show "Select one site".
3. No Big Number or aggregate tile for site figures. They are computed per site in v_site, never aggregated by Superset.
4. Fallback, if (1) fails its test: "Select first filter value by default", so the Site filter is never empty.

[D0 Site header: code + name (large) · province · pilot status · baseline status · "← Sites map"]
Summary              [S1 L/h KPI card w6][S2 Supporting measures w6]
                     [S3 Evidence coverage timeline (Gantt) w12]
Fuel & Deliveries    [F1 Deliveries w6][F2 Recorded stock w6]
                     [F3 Fuel per calendar day by interval w12]
                     [F4 Interval table w12]
Runtime & Electricity[R1 Runtime per day by meter interval w6][R2 Hour-meter evidence w6]
                     [R3 Grid energy, kWh w6][R4 Electricity cost, AFN w6]
Baseline & Evidence  [B1 Baseline versions w6][B2 Inputs of each version w6]
                     [B3 All intervals: eligibility and exclusions w12]
                     [B4 Interval L/h against baseline w12]
                     [B5 Assumptions w6][B6 Missing data and quality checks w6]
                     [B7 Evidence references w6][B8 Workbook calculations, assessed w6]
                     [B9 Stock against engine-measured fuel w6][B10 Equipment specifications w6]

#	Question it answers	Type	Dataset	Dims / measures (units)	Grain	When data are missing or insufficient
D0	Which site am I looking at?	Handlebars	v_site	code and name (large), province, pilot status, baseline status, location status, link "← Sites map"	site	"Select one site" unless exactly one row
S1	What is the site's L/h, by which method, over which window, how strong?	Handlebars	v_site	lph_estimate, lph_method, baseline_status_label, baseline_validity_note, window, decision; L/day whole record	site	Keeps its position. In v1 every site shows "Insufficient runtime evidence: no hour-meter readings", plus L/day labelled "per calendar day, not per running hour". "Eligible" and "Provisional" carry "not approved"
S2	What supports it?	Table	v_kpi_measures	measure, value, unit, value_kind (recorded · calculated · assumed · not established), window. Two L/day rows: the whole record (moves with new records) and the L/day reference version (fixed window, its status and any input drift in the note)	site × measure	Rows stay and show "not established"
S3	Which periods are covered by which evidence, and where are the gaps?	Gantt	v_coverage	lane; start/end; colour = status	span	Empty lanes are visible; nothing is interpolated
F1	When and how much was delivered?	Bar (time axis, day grain)	v_delivery (one row per delivery event)	delivery_date; SUM(delivered_l), titled "Litres delivered per day (all deliveries that day)"; tooltip COUNT(deliveries)	day	Bars only on delivery dates; subtitle "deliveries are not consumption"; F4 lists the individual deliveries
F2	What stock was recorded?	Scatter (no lines)	v_stock_reading	observed_on; litres; series = reading type × value_kind	observation	Points only; nothing between readings
F3	How much fuel was used per calendar day in each observed interval?	Bar (categorical x, ordered by start)	v_interval	interval_label; MAX(fuel_l_per_day) (L/d); colour = stock_status	stock interval	Excluded intervals greyed, with the reason in the tooltip; no monthly spreading
F4	Interval detail	Table	v_interval	dates, opening, deliveries, closing, used, L/d, status	interval	Shows the reasons
R1	How many hours did each generator run per day?	Bar (categorical)	v_runtime	interval × meter; MAX(h/day). h/day is NULL for an invalid pair (pair_valid false: two epochs, a decreasing timer or over 24 h/day); such pairs show no bar and are listed with their flags in the tooltip	meter interval	v1: empty, with "No hour-meter readings" in the description
R2	What runtime evidence exists?	Table	v_hour_meter_evidence	meter, mapped set, date, hours, state, epoch	reading	v1: no rows, "No hour-meter readings"
R3	How much grid energy did the site receive?	Handlebars → bar once kWh exist	v_kpi_measures (kWh row)	kWh per bill month	month	"No kWh records; bills are cost only and are not converted to fuel or hours"
R4	What did grid electricity cost?	Bar (month) + table	v_electricity_cost	bill month; amount (AFN, inferred)	month	Own chart, never on a fuel axis. A disputed bill is listed without amount or kWh
B1	Which baseline versions exist, and who approved them?	Table	v_baseline_version	code, status, value, window, grade, decision	version	"No baseline defined"
B2	What went into each version?	Table	v_baseline_input	interval, included, reason, fuel, runtime, drift	version × interval	empty until a version exists
B3	Which intervals are usable, and why not?	Table	v_interval	eligibility, exclusion reasons, grade, conditions, in current baseline	interval	v1: every interval excluded, "no timer reading at both ends" (and "a generator has neither a counter nor an idle assumption")
B4	What L/h did each interval give, against the approved baseline?	Mixed: bars + line	v_interval	x interval_label; A: MAX(l_per_h) where eligibility ≠ excluded; B: MAX(current_baseline_l_per_h) (approved value only, and only beside intervals of the same metric and population, baseline_comparable)	interval	v1: no bars; description points to S1
B5	Which assumptions apply?	Table	v_assumption	code, topic, value and range, basis, status, rationale	assumption	"None recorded"
B6	What is missing or inconsistent?	Table	v_data_quality	check, status, detail	check	Lists the gaps: no runtime, placeholder rows, no kWh, no transfer records, location awaiting acceptance
B7	Which documents do the figures rest on?	Table	v_evidence	title, kind, issuer, date, used for	document	—
B8	What did the operator workbook calculate, and is it usable?	Table	v_imported_calculation	label, formula, value, our reading, use, reason, claim_status	cell	Shows "excluded" with the reason for "actual L/h" and "CP L/h". A disputed transcription shows its status, without formula or value
B9	Does stock depletion match engine-measured fuel?	Table	v_reconciliation	stock L, engine L, covered_h of interval_h, difference L and %. The window runs from the opening visit's departure to the closing visit's arrival. The difference is shown only for an ok stock interval with both visit times recorded, in which every generator is placed at the site throughout and covered for the whole window	interval × generator	"No flow-meter data yet"; partial coverage, unknown visit times or a mid-interval placement show the hours covered and no difference
B10	What does the equipment specification say?	Table	v_spec_reference	attribute, value, unit, basis, applicability, claim_status	spec	Titled "Manufacturer and nameplate data, not site performance". A disputed specification shows its status, without value

Modelled L/h (M3) gets a table on Baseline & Evidence only if Q9 approves a runtime assumption. It pools only the usable intervals inside that assumption's dates, and shows that window and its days.

5. Datasets and filter mapping

Physical datasets, one per public.v_* view on Trino fuel.public. Every dataset has the same four site columns from v_site: site_uid, site_code, site_name, site_label.

Dataset	Grain	Used by	Native filters that apply
v_site	site	M1–M5, D0, S1	landing: all five filters, plus the M1 cross-filter (on M3); detail: Site
v_kpi_measures	site × measure	S2, R3	Site
v_coverage	site × lane × span	S3	Site
v_delivery	delivery	F1	Site, Viewing period
v_stock_reading	observation	F2	Site, Viewing period
v_interval	stock interval	F3, F4, B3, B4	Site; Viewing period on F3 and F4 only
v_runtime	meter interval	R1	Site, Viewing period
v_hour_meter_evidence	reading	R2	Site, Viewing period
v_electricity_cost	bill month	R4	Site, Viewing period
v_baseline_version, v_baseline_input	version (× interval)	B1, B2	Site
v_assumption, v_data_quality, v_evidence, v_imported_calculation, v_reconciliation, v_spec_reference	per view	B5–B10	Site

Metrics. Saved metrics enforce pooling, with the same eligibility filter on numerator and denominator, e.g. pooled_l_per_h = SUM(CASE WHEN eligibility <> 'excluded' THEN fuel_used_l END) / NULLIF(SUM(CASE WHEN eligibility <> 'excluded' THEN runtime_h END), 0). A chart that uses it is grouped or filtered by metric and population_key (both in v_interval), so it never pools across methods or generator sets. The site's L/h figure itself comes only from a baseline version (v_site), never from this metric. There is no AVG metric on ratio columns.

Link column. detail_link is a calculated column on the v_site dataset, defined in the bundle in Trino SQL. The native filter id stays inside Superset's own configuration:

'<a href="/superset/dashboard/atoma-site-detail/?native_filters='
  || url_encode('(NATIVE_FILTER-site:(id:NATIVE_FILTER-site,'
       || 'extraFormData:(filters:!((col:site_label,op:IN,val:!(' || q || ')))),'
       || 'filterState:(value:!(' || q || ')),ownState:()))')
  || '">Open site detail</a>'
-- q = '''' || replace(replace(site_label, '!', '!!'), '''', '!''') || ''''   -- rison string quoting

• The exact rison state shape is a (local test).
• Fallback: the link opens Site Detail with the Site filter expanded (expand_filters=1) and the site's label shown beside the link, to pick by search.

6. Build as code, scale, access

Build

• petrolyst-insights/dashboards/atoma/build_bundle.py writes one import bundle containing both dashboards, every dataset and every chart:
  • stable UUIDs (uuid5), so a re-import updates instead of duplicating;
  • fixed native-filter ids (NATIVE_FILTER-site, -period, -search, …);
  • label_colors for the four statuses;
  • the existing "Fuel (Trino)" database UUID, read from an export first.
• Import: superset import-dashboards.

Adding a site

• Import its site.yaml and its records. Nothing in either dashboard names a site.
• New sites appear in M1 (once their location is accepted) or in M5, in M3, and in the Site filter.

Hundreds of sites

• v_site has one row per site, and the scatter layer handles thousands of points.
• The Select filters load the distinct labels; M3 and M5 paginate on the server.

Superset configuration change (petrolyst-insights; image rebuild in P3): DECKGL_BASE_MAP = [["tile://https://tile.openstreetmap.org/{z}/{x}/{y}.png", "OpenStreetMap"]]. The CSP needs no change (local test: no CSP violation in the browser console).

Access, internal first. Both dashboards for Admin and Alpha only. No Gamma role until the ATOMA-facing edit.

7. Acceptance checks

Run in P2 against a local Superset, and repeated as a smoke test in P3.

1. New site, no new objects. Count dashboards, charts and datasets in the metadata database. Then import a synthetic fourth site with records and an accepted location, and a fifth with no location. The counts are unchanged; site 4 appears in M1, M3 and the Site filter; site 5 appears in M5, and M4 goes up by 1.
2. Site switching. In Site Detail, select pilot site A, then site B. D0 shows the new code and name; every chart's query (View query, or chart-data API) carries site_label = …<site B>…; no chart keeps site A's rows.
3. Clearing the selection. With the Site filter cleared, no chart shows multi-site data; D0 and S1 show "Select one site". If not, switch to the §4 fallback and re-test.
4. Unavailable L/h. For each pilot site:
   • the tooltip and M3 say "not available" with the method "no runtime evidence";
   • S1 shows the insufficient-evidence state plus L/day;
   • B3 shows every interval excluded, with its reason.
5. Missing coordinates. A site without an accepted location is not plotted, is listed in M5 and counted in M4, and its link opens its detail.
6. Filter propagation.
   • On the landing page, each filter changes M1, M3, M4 and M5, and the default "active" shows exactly the active pilot sites.
   • A marker click filters M3 only.
   • In Site Detail, the Viewing period changes F1–F3 and R1, R2, R4, but not S1 or B1–B4.
7. Detail links. For each active pilot site, "Open site detail" in M3 opens Site Detail with exactly that site selected, and D0 matches. Same from M5.
8. Colours.
   • Fixtures in each status show the four colours.
   • "Eligible" never renders as "Approved" unless the fixture has an approved version with a decision reference.
   • Marker size is identical across sites.
9. Basemap. OSM tiles load, there are no CSP violations in the console, and the attribution is visible.