Remove Trino: Superset reads fuel-db directly as fuel_ro (PD-14)
trino/ deleted; the role trino_ro becomes fuel_ro, its password read from a secret file at connect time; dashboards' SQL and column list for PostgreSQL; the Compose project is renamed petrolyst, with volumes pinned to their old names. AGENTS.md points to the architecture context. Not deployed: the next deploy needs README's one-time steps first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Nasim SadatandClaude Opus 5.5 committed
1 parent
686318dfac
commit
8405706ed8
25 files changed
+453
-410
No files matched your search
@@ -11,11 +11,11 @@ image, so changes here do not rebuild or restart Superset.
|
||||
|
||||
Build and import:
|
||||
|
||||
docker cp atoma/import_bundle.py fuel-monitoring-superset-1:/tmp/
|
||||
docker exec fuel-monitoring-superset-1 /app/.venv/bin/python /tmp/import_bundle.py --show-database
|
||||
docker cp atoma/import_bundle.py petrolyst-superset-1:/tmp/
|
||||
docker exec petrolyst-superset-1 /app/.venv/bin/python /tmp/import_bundle.py --show-database
|
||||
python3 atoma/build_bundle.py --database-uuid <printed uuid> --out atoma-dashboards.zip
|
||||
docker cp atoma-dashboards.zip fuel-monitoring-superset-1:/tmp/
|
||||
docker exec fuel-monitoring-superset-1 /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip --user <username>
|
||||
docker cp atoma-dashboards.zip petrolyst-superset-1:/tmp/
|
||||
docker exec petrolyst-superset-1 /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip --user <username>
|
||||
|
||||
Exit status 2 means a dataset's columns differ from its view: regenerate `atoma/columns.tsv`
|
||||
(command in its header), rebuild and import again.
|
||||
@@ -34,12 +34,12 @@ image, so changes here do not rebuild or restart Superset.
|
||||
|
||||
docker exec fmdb-scratch-superset /app/.venv/bin/python /tmp/import_bundle.py /tmp/atoma-dashboards.zip \
|
||||
--user admin --embedded-out /tmp/embedded-dashboards.json --embed-domain http://127.0.0.1:8098
|
||||
docker cp fmdb-scratch-superset:/tmp/embedded-dashboards.json <Petrolist>/.local/fmdb-scratch/
|
||||
docker cp fmdb-scratch-superset:/tmp/embedded-dashboards.json <Petrolyst>/.local/fmdb-scratch/
|
||||
|
||||
Export from a running instance (inside the container, as the server's deploy user):
|
||||
|
||||
docker exec fuel-monitoring-superset-1 superset export-dashboards -f /tmp/dashboards.zip
|
||||
docker cp fuel-monitoring-superset-1:/tmp/dashboards.zip .
|
||||
docker exec petrolyst-superset-1 superset export-dashboards -f /tmp/dashboards.zip
|
||||
docker cp petrolyst-superset-1:/tmp/dashboards.zip .
|
||||
|
||||
Import is the reverse with `superset import-dashboards -p <zip> -u <username>`. The
|
||||
username must be an existing Superset user, which here means someone who has signed in
|
||||
|
||||
@@ -15,8 +15,9 @@ them in Superset with the filter column.
|
||||
- Nothing is imported into the live Superset without the owner's go-ahead (phase P3).
|
||||
|
||||
**Datasets.** The `public.v_*` views of
|
||||
`stacks/fuel-monitoring/fuel-db/migrations/0003_bi_views.sql`, read through Trino as
|
||||
`fuel.public`.
|
||||
`stacks/fuel-monitoring/fuel-db/migrations/0003_bi_views.sql`, read from fuel-db directly (database
|
||||
`fuel`, schema `public`) as the read-only role `fuel_ro`. *Until 29 Oct 2026 Superset read them
|
||||
through Trino as `fuel.public`; Trino was removed then.*
|
||||
|
||||
**Design and rationale** live in the fuel-monitoring corpus, in
|
||||
`.context/plans/2026-10-24-fuel-db-baseline-dashboard.md`:
|
||||
@@ -223,7 +224,7 @@ fleet map.
|
||||
|
||||
## 5. Datasets and filter mapping
|
||||
|
||||
Physical datasets, one per `public.v_*` view on Trino `fuel.public`, plus one virtual dataset
|
||||
Physical datasets, one per `public.v_*` view in fuel-db's `public` schema, plus one virtual dataset
|
||||
*(P2)*. `coverage_detail` adds two columns to `v_coverage`:
|
||||
- `lane_name`, the lane without its sort number, which read as a count;
|
||||
- `amount`, a stock interval's delivered litres (from its opening visit up to its closing visit,
|
||||
@@ -280,8 +281,8 @@ output:
|
||||
- fixed native-filter ids (`NATIVE_FILTER-site`, `NATIVE_FILTER-period`) and tab ids
|
||||
(`TAB-fleet`, `TAB-site`, `TAB-summary`, …);
|
||||
- `label_colors` for the four statuses;
|
||||
- every view column, from `columns.tsv` (a snapshot of Trino's `information_schema`);
|
||||
- the existing "Fuel (Trino)" database UUID, which differs per instance:
|
||||
- every view column, from `columns.tsv` (a snapshot of PostgreSQL's catalog);
|
||||
- the existing "Fuel" database UUID, which differs per instance:
|
||||
`import_bundle.py --show-database` prints it.
|
||||
- Before writing, it checks that every column a chart or filter uses exists in its dataset,
|
||||
that every chart in a Site filter's scope has `site_label`, that the fleet map's iframe
|
||||
|
||||
@@ -5,7 +5,7 @@ Implements SPEC.md in this folder: the two dashboards the Petrolyst app embeds,
|
||||
(atoma-overview) and "ATOMA · Site" (atoma-site, the four detail tabs), one dataset per
|
||||
public.v_* view and every chart. Nothing in the bundle names a site.
|
||||
|
||||
build_bundle.py --database-uuid <uuid of "Fuel (Trino)"> [--out atoma-dashboards.zip]
|
||||
build_bundle.py --database-uuid <uuid of "Fuel"> [--out atoma-dashboards.zip]
|
||||
|
||||
The database uuid differs per Superset instance; `import_bundle.py --show-database`
|
||||
prints it. Output is deterministic: the same inputs give a byte-identical zip,
|
||||
@@ -37,8 +37,10 @@ import yaml
|
||||
HERE = Path(__file__).resolve().parent
|
||||
VERSION = "1.0.0"
|
||||
NAMESPACE = uuid.uuid5(uuid.NAMESPACE_URL, "https://petrolyst.com/superset/atoma-dashboards")
|
||||
DATABASE_NAME = "Fuel (Trino)"
|
||||
DATABASE_URI = "trino://superset@trino:8080/fuel" # as superset-init registers it
|
||||
DATABASE_NAME = "Fuel"
|
||||
# As superset-init registers it: fuel-db directly, as the read-only role fuel_ro. The password
|
||||
# comes from a secret file at connect time (superset_config.py), never from the bundle.
|
||||
DATABASE_URI = "postgresql+psycopg2://fuel_ro@fuel-db:5432/fuel"
|
||||
CATALOG, SCHEMA = "fuel", "public"
|
||||
|
||||
SITE_FILTER = "NATIVE_FILTER-site"
|
||||
@@ -69,7 +71,7 @@ MAIN_DTTM = {
|
||||
"v_electricity_cost": "bill_month",
|
||||
}
|
||||
|
||||
# Calculated columns: (name, Trino SQL, type, label). The Petrolyst server's site_status
|
||||
# Calculated columns: (name, PostgreSQL SQL, type, label). The Petrolyst server's site_status
|
||||
# (petrolyst-platform server/) shows the same two texts.
|
||||
CALCULATED = {
|
||||
"v_site": [
|
||||
@@ -93,7 +95,7 @@ CALCULATED = {
|
||||
],
|
||||
}
|
||||
|
||||
# Virtual datasets: name -> (view it extends, extra output columns, Trino SQL).
|
||||
# Virtual datasets: name -> (view it extends, extra output columns, PostgreSQL SQL).
|
||||
# coverage_detail feeds S3. The view's lane names start with a sort number ("1 Stock
|
||||
# intervals"), which reads as a count on the chart, so it adds lane_name without the number
|
||||
# (the chart still sorts by lane). Its amount column is the tooltip: the litres delivered in a
|
||||
@@ -106,7 +108,7 @@ CALCULATED = {
|
||||
# day cannot be told apart. Everywhere else the date window is exact: on a single-visit day the
|
||||
# opening visit's delivery is included and the closing visit's excluded.
|
||||
VIRTUAL = {
|
||||
"coverage_detail": ("v_coverage", [("lane_name", "varchar(60)"), ("amount", "varchar(120)")], """\
|
||||
"coverage_detail": ("v_coverage", [("lane_name", "character varying(60)"), ("amount", "character varying(120)")], """\
|
||||
SELECT c.site_uid, c.site_code, c.site_name, c.site_label, c.lane, c.status,
|
||||
c.start_on, c.end_on, c.start_ts, c.end_ts,
|
||||
regexp_replace(c.lane, '^[0-9]+ ', '') AS lane_name,
|
||||
@@ -114,19 +116,19 @@ SELECT c.site_uid, c.site_code, c.site_name, c.site_label, c.lane, c.status,
|
||||
CASE WHEN f.ambiguous THEN 'not attributed: a delivery on a day with several visits (see Deliveries)'
|
||||
WHEN f.filled_l IS NULL AND coalesce(f.n_unknown, 0) = 0 THEN 'no delivery'
|
||||
WHEN f.filled_l IS NULL THEN 'filled, quantity not recorded'
|
||||
ELSE format('%,.0f L filled', CAST(f.filled_l AS double)) END
|
||||
ELSE to_char(round(f.filled_l), 'FM999,999,999,990') || ' L filled' END
|
||||
|| CASE WHEN f.n_unknown > 0 AND f.filled_l IS NOT NULL AND NOT f.ambiguous
|
||||
THEN ', plus ' || CAST(f.n_unknown AS varchar) || ' of unknown quantity' ELSE '' END
|
||||
WHEN c.lane = '4 Electricity bills' THEN
|
||||
CASE WHEN b.n_bills > 1 THEN 'several bills start this day (see Electricity bills)'
|
||||
WHEN b.disputed THEN 'bill disputed, amount withheld'
|
||||
WHEN b.amount IS NULL THEN 'bill amount not recorded'
|
||||
ELSE format('%,.0f AFN billed (inferred)', CAST(b.amount AS double)) END
|
||||
ELSE to_char(round(b.amount), 'FM999,999,999,990') || ' AFN billed (inferred)' END
|
||||
END AS amount
|
||||
FROM public.v_coverage c
|
||||
LEFT JOIN (SELECT s.site_uid, s.start_on, s.end_on,
|
||||
sum(CASE WHEN d.delivery_date < s.end_on THEN d.delivered_l END) AS filled_l,
|
||||
count_if(d.delivery_date < s.end_on AND d.delivered_l IS NULL) AS n_unknown,
|
||||
count(*) FILTER (WHERE d.delivery_date < s.end_on AND d.delivered_l IS NULL) AS n_unknown,
|
||||
bool_or(d.delivery_date IN (s.start_on, s.end_on) AND m.site_uid IS NOT NULL) AS ambiguous
|
||||
FROM (SELECT DISTINCT site_uid, start_on, end_on FROM public.v_coverage
|
||||
WHERE lane = '1 Stock intervals') s
|
||||
@@ -216,8 +218,8 @@ def read_columns(path: Path) -> dict[str, list[tuple[str, str]]]:
|
||||
return views
|
||||
|
||||
|
||||
def superset_type(trino_type: str) -> str:
|
||||
return trino_type.upper().replace(",", ", ")
|
||||
def superset_type(data_type: str) -> str:
|
||||
return data_type.upper().replace(",", ", ")
|
||||
|
||||
|
||||
def dataset_yaml(view: str, columns: list[tuple[str, str]], database_uuid: str) -> dict[str, Any]:
|
||||
@@ -968,7 +970,7 @@ def build(database_uuid: str, columns_path: Path) -> dict[str, Any]:
|
||||
views = read_columns(columns_path)
|
||||
files: dict[str, Any] = {
|
||||
"metadata.yaml": {"version": VERSION, "type": "Dashboard", "timestamp": "2026-10-24T00:00:00+00:00"},
|
||||
"databases/Fuel_Trino.yaml": {
|
||||
"databases/Fuel.yaml": {
|
||||
"database_name": DATABASE_NAME,
|
||||
"sqlalchemy_uri": DATABASE_URI,
|
||||
"cache_timeout": None,
|
||||
@@ -988,7 +990,7 @@ def build(database_uuid: str, columns_path: Path) -> dict[str, Any]:
|
||||
for name, (base, extra, _) in VIRTUAL.items():
|
||||
views[name] = views[base] + extra
|
||||
for view in sorted(views):
|
||||
files[f"datasets/Fuel_Trino/{view}.yaml"] = dataset_yaml(view, views[view], database_uuid)
|
||||
files[f"datasets/Fuel/{view}.yaml"] = dataset_yaml(view, views[view], database_uuid)
|
||||
|
||||
# PD-12: the Petrolyst app embeds the Overview and one site dashboard, whose four detail
|
||||
# tabs are Superset's own (owner, 29 Oct). The app shows the site header and chooses site
|
||||
@@ -1083,7 +1085,7 @@ def write_zip(files: dict[str, Any], out: Path) -> None:
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
|
||||
parser.add_argument("--database-uuid", required=True, type=uuid.UUID,
|
||||
help='uuid of the "Fuel (Trino)" database in the target Superset')
|
||||
help='uuid of the "Fuel" database in the target Superset')
|
||||
parser.add_argument("--columns", type=Path, default=HERE / "columns.tsv")
|
||||
parser.add_argument("--out", type=Path, default=Path("atoma-dashboards.zip"))
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
@@ -1,309 +1,310 @@
|
||||
# Columns of the public.v_* views (migrations 0001-0003), as Trino reports them.
|
||||
# Columns of the public.v_* views (migrations 0001-0003), as PostgreSQL reports them.
|
||||
# build_bundle.py lists every one in its dataset; import_bundle.py reports drift.
|
||||
# Regenerate after a migration that changes a view:
|
||||
# trino --output-format TSV --execute "select table_name, column_name, data_type
|
||||
# from fuel.information_schema.columns where table_schema = 'public'
|
||||
# and table_name like 'v\_%' escape '\' order by table_name, ordinal_position"
|
||||
v_assumption site_uid varchar(12)
|
||||
v_assumption site_code varchar(20)
|
||||
v_assumption site_name varchar(120)
|
||||
v_assumption site_label varchar(170)
|
||||
v_assumption code varchar(60)
|
||||
v_assumption topic varchar(30)
|
||||
v_assumption asset_uid varchar(40)
|
||||
v_assumption value decimal(10,3)
|
||||
v_assumption value_low decimal(10,3)
|
||||
v_assumption value_high decimal(10,3)
|
||||
v_assumption unit varchar(20)
|
||||
# Regenerate after a migration that changes a view (psql -d fuel -AtF $'\t' -c "..."):
|
||||
# select c.relname, a.attname, format_type(a.atttypid, a.atttypmod)
|
||||
# from pg_attribute a join pg_class c on c.oid = a.attrelid join pg_namespace n on n.oid = c.relnamespace
|
||||
# where n.nspname = 'public' and c.relkind = 'v' and c.relname like 'v\_%' and a.attnum > 0
|
||||
# and not a.attisdropped order by c.relname, a.attnum
|
||||
v_assumption site_uid character varying(12)
|
||||
v_assumption site_code character varying(20)
|
||||
v_assumption site_name character varying(120)
|
||||
v_assumption site_label character varying(170)
|
||||
v_assumption code character varying(60)
|
||||
v_assumption topic character varying(30)
|
||||
v_assumption asset_uid character varying(40)
|
||||
v_assumption value numeric(10,3)
|
||||
v_assumption value_low numeric(10,3)
|
||||
v_assumption value_high numeric(10,3)
|
||||
v_assumption unit character varying(20)
|
||||
v_assumption applies_from date
|
||||
v_assumption applies_to date
|
||||
v_assumption basis varchar(20)
|
||||
v_assumption basis character varying(20)
|
||||
v_assumption derived_from_fuel boolean
|
||||
v_assumption status varchar(12)
|
||||
v_assumption status character varying(12)
|
||||
v_assumption accepted_on date
|
||||
v_assumption decision_ref varchar(12)
|
||||
v_assumption rationale varchar(400)
|
||||
v_assumption contradicted_by varchar(200)
|
||||
v_assumption members varchar(400)
|
||||
v_baseline_input site_uid varchar(12)
|
||||
v_baseline_input site_code varchar(20)
|
||||
v_baseline_input site_name varchar(120)
|
||||
v_baseline_input site_label varchar(170)
|
||||
v_baseline_input baseline_code varchar(60)
|
||||
v_baseline_input baseline_status varchar(12)
|
||||
v_baseline_input interval_label varchar(30)
|
||||
v_assumption decision_ref character varying(12)
|
||||
v_assumption rationale character varying(400)
|
||||
v_assumption contradicted_by character varying(200)
|
||||
v_assumption members character varying(400)
|
||||
v_baseline_input site_uid character varying(12)
|
||||
v_baseline_input site_code character varying(20)
|
||||
v_baseline_input site_name character varying(120)
|
||||
v_baseline_input site_label character varying(170)
|
||||
v_baseline_input baseline_code character varying(60)
|
||||
v_baseline_input baseline_status character varying(12)
|
||||
v_baseline_input interval_label character varying(30)
|
||||
v_baseline_input start_on date
|
||||
v_baseline_input end_on date
|
||||
v_baseline_input included boolean
|
||||
v_baseline_input exclusion_reason varchar(300)
|
||||
v_baseline_input fuel_l decimal(10,1)
|
||||
v_baseline_input runtime_h decimal(10,1)
|
||||
v_baseline_input exclusion_reason character varying(300)
|
||||
v_baseline_input fuel_l numeric(10,1)
|
||||
v_baseline_input runtime_h numeric(10,1)
|
||||
v_baseline_input days integer
|
||||
v_baseline_input eligibility_at_computation varchar(12)
|
||||
v_baseline_input evidence_grade_at_computation varchar(2)
|
||||
v_baseline_input eligibility_at_computation character varying(12)
|
||||
v_baseline_input evidence_grade_at_computation character varying(2)
|
||||
v_baseline_input dependencies_changed boolean
|
||||
v_baseline_version site_uid varchar(12)
|
||||
v_baseline_version site_code varchar(20)
|
||||
v_baseline_version site_name varchar(120)
|
||||
v_baseline_version site_label varchar(170)
|
||||
v_baseline_version code varchar(60)
|
||||
v_baseline_version metric varchar(40)
|
||||
v_baseline_version status varchar(12)
|
||||
v_baseline_version population varchar(200)
|
||||
v_baseline_version value decimal(8,3)
|
||||
v_baseline_version range_low decimal(8,3)
|
||||
v_baseline_version range_high decimal(8,3)
|
||||
v_baseline_version unit varchar(8)
|
||||
v_baseline_version site_uid character varying(12)
|
||||
v_baseline_version site_code character varying(20)
|
||||
v_baseline_version site_name character varying(120)
|
||||
v_baseline_version site_label character varying(170)
|
||||
v_baseline_version code character varying(60)
|
||||
v_baseline_version metric character varying(40)
|
||||
v_baseline_version status character varying(12)
|
||||
v_baseline_version population character varying(200)
|
||||
v_baseline_version value numeric(8,3)
|
||||
v_baseline_version range_low numeric(8,3)
|
||||
v_baseline_version range_high numeric(8,3)
|
||||
v_baseline_version unit character varying(8)
|
||||
v_baseline_version window_start date
|
||||
v_baseline_version window_end date
|
||||
v_baseline_version fuel_l decimal(10,1)
|
||||
v_baseline_version runtime_h decimal(10,1)
|
||||
v_baseline_version fuel_l numeric(10,1)
|
||||
v_baseline_version runtime_h numeric(10,1)
|
||||
v_baseline_version matched_days integer
|
||||
v_baseline_version policy_version varchar(10)
|
||||
v_baseline_version policy_version character varying(10)
|
||||
v_baseline_version meets_policy boolean
|
||||
v_baseline_version policy_check varchar(300)
|
||||
v_baseline_version evidence_grade varchar(2)
|
||||
v_baseline_version conditions varchar(200)
|
||||
v_baseline_version season_note varchar(200)
|
||||
v_baseline_version policy_check character varying(300)
|
||||
v_baseline_version evidence_grade character varying(2)
|
||||
v_baseline_version conditions character varying(200)
|
||||
v_baseline_version season_note character varying(200)
|
||||
v_baseline_version approved_on date
|
||||
v_baseline_version approved_by_role varchar(40)
|
||||
v_baseline_version decision_ref varchar(12)
|
||||
v_baseline_version approved_by_role character varying(40)
|
||||
v_baseline_version decision_ref character varying(12)
|
||||
v_baseline_version inputs_unchanged boolean
|
||||
v_baseline_version computed_at timestamp(0)
|
||||
v_coverage site_uid varchar(12)
|
||||
v_coverage site_code varchar(20)
|
||||
v_coverage site_name varchar(120)
|
||||
v_coverage site_label varchar(170)
|
||||
v_coverage lane varchar(60)
|
||||
v_coverage status varchar(40)
|
||||
v_baseline_version computed_at timestamp(0) without time zone
|
||||
v_coverage site_uid character varying(12)
|
||||
v_coverage site_code character varying(20)
|
||||
v_coverage site_name character varying(120)
|
||||
v_coverage site_label character varying(170)
|
||||
v_coverage lane character varying(60)
|
||||
v_coverage status character varying(40)
|
||||
v_coverage start_on date
|
||||
v_coverage end_on date
|
||||
v_coverage start_ts timestamp(0)
|
||||
v_coverage end_ts timestamp(0)
|
||||
v_data_quality site_uid varchar(12)
|
||||
v_data_quality site_code varchar(20)
|
||||
v_data_quality site_name varchar(120)
|
||||
v_data_quality site_label varchar(170)
|
||||
v_coverage start_ts timestamp(0) without time zone
|
||||
v_coverage end_ts timestamp(0) without time zone
|
||||
v_data_quality site_uid character varying(12)
|
||||
v_data_quality site_code character varying(20)
|
||||
v_data_quality site_name character varying(120)
|
||||
v_data_quality site_label character varying(170)
|
||||
v_data_quality check_order integer
|
||||
v_data_quality check_code varchar(40)
|
||||
v_data_quality label varchar(80)
|
||||
v_data_quality status varchar(10)
|
||||
v_data_quality detail varchar(300)
|
||||
v_delivery site_uid varchar(12)
|
||||
v_delivery site_code varchar(20)
|
||||
v_delivery site_name varchar(120)
|
||||
v_delivery site_label varchar(170)
|
||||
v_delivery event_key varchar(80)
|
||||
v_data_quality check_code character varying(40)
|
||||
v_data_quality label character varying(80)
|
||||
v_data_quality status character varying(10)
|
||||
v_data_quality detail character varying(300)
|
||||
v_delivery site_uid character varying(12)
|
||||
v_delivery site_code character varying(20)
|
||||
v_delivery site_name character varying(120)
|
||||
v_delivery site_label character varying(170)
|
||||
v_delivery event_key character varying(80)
|
||||
v_delivery delivery_date date
|
||||
v_delivery seq_in_day integer
|
||||
v_delivery delivered_l decimal(8,1)
|
||||
v_delivery unit_price decimal(8,2)
|
||||
v_delivery currency varchar(8)
|
||||
v_delivery source_ref varchar(60)
|
||||
v_delivery delivered_l numeric(8,1)
|
||||
v_delivery unit_price numeric(8,2)
|
||||
v_delivery currency character varying(8)
|
||||
v_delivery source_ref character varying(60)
|
||||
v_delivery n_sources integer
|
||||
v_delivery sources_disagree boolean
|
||||
v_delivery reviewed boolean
|
||||
v_electricity_cost site_uid varchar(12)
|
||||
v_electricity_cost site_code varchar(20)
|
||||
v_electricity_cost site_name varchar(120)
|
||||
v_electricity_cost site_label varchar(170)
|
||||
v_electricity_cost site_uid character varying(12)
|
||||
v_electricity_cost site_code character varying(20)
|
||||
v_electricity_cost site_name character varying(120)
|
||||
v_electricity_cost site_label character varying(170)
|
||||
v_electricity_cost bill_month date
|
||||
v_electricity_cost period_start date
|
||||
v_electricity_cost period_end date
|
||||
v_electricity_cost period_known boolean
|
||||
v_electricity_cost amount decimal(12,2)
|
||||
v_electricity_cost currency varchar(8)
|
||||
v_electricity_cost currency_basis varchar(10)
|
||||
v_electricity_cost kwh decimal(10,1)
|
||||
v_electricity_cost source_ref varchar(60)
|
||||
v_electricity_cost claim_status varchar(12)
|
||||
v_evidence site_uid varchar(12)
|
||||
v_evidence site_code varchar(20)
|
||||
v_evidence site_name varchar(120)
|
||||
v_evidence site_label varchar(170)
|
||||
v_evidence document_key varchar(16)
|
||||
v_evidence title varchar(200)
|
||||
v_evidence kind varchar(20)
|
||||
v_evidence issuer varchar(80)
|
||||
v_electricity_cost amount numeric(12,2)
|
||||
v_electricity_cost currency character varying(8)
|
||||
v_electricity_cost currency_basis character varying(10)
|
||||
v_electricity_cost kwh numeric(10,1)
|
||||
v_electricity_cost source_ref character varying(60)
|
||||
v_electricity_cost claim_status character varying(12)
|
||||
v_evidence site_uid character varying(12)
|
||||
v_evidence site_code character varying(20)
|
||||
v_evidence site_name character varying(120)
|
||||
v_evidence site_label character varying(170)
|
||||
v_evidence document_key character varying(16)
|
||||
v_evidence title character varying(200)
|
||||
v_evidence kind character varying(20)
|
||||
v_evidence issuer character varying(80)
|
||||
v_evidence document_date date
|
||||
v_evidence used_for varchar(200)
|
||||
v_evidence used_for character varying(200)
|
||||
v_evidence contains_personal_data boolean
|
||||
v_hour_meter_evidence site_uid varchar(12)
|
||||
v_hour_meter_evidence site_code varchar(20)
|
||||
v_hour_meter_evidence site_name varchar(120)
|
||||
v_hour_meter_evidence site_label varchar(170)
|
||||
v_hour_meter_evidence meter_label varchar(40)
|
||||
v_hour_meter_evidence mapped_set varchar(60)
|
||||
v_hour_meter_evidence site_uid character varying(12)
|
||||
v_hour_meter_evidence site_code character varying(20)
|
||||
v_hour_meter_evidence site_name character varying(120)
|
||||
v_hour_meter_evidence site_label character varying(170)
|
||||
v_hour_meter_evidence meter_label character varying(40)
|
||||
v_hour_meter_evidence mapped_set character varying(60)
|
||||
v_hour_meter_evidence reading_date date
|
||||
v_hour_meter_evidence date_basis varchar(30)
|
||||
v_hour_meter_evidence hours decimal(9,1)
|
||||
v_hour_meter_evidence reading_state varchar(16)
|
||||
v_hour_meter_evidence source_field varchar(16)
|
||||
v_hour_meter_evidence date_basis character varying(30)
|
||||
v_hour_meter_evidence hours numeric(9,1)
|
||||
v_hour_meter_evidence reading_state character varying(16)
|
||||
v_hour_meter_evidence source_field character varying(16)
|
||||
v_hour_meter_evidence epoch_no integer
|
||||
v_hour_meter_evidence claim_status varchar(12)
|
||||
v_imported_calculation site_uid varchar(12)
|
||||
v_imported_calculation site_code varchar(20)
|
||||
v_imported_calculation site_name varchar(120)
|
||||
v_imported_calculation site_label varchar(170)
|
||||
v_imported_calculation cell_ref varchar(40)
|
||||
v_imported_calculation source_label varchar(80)
|
||||
v_imported_calculation formula varchar(200)
|
||||
v_imported_calculation cached_value decimal(14,6)
|
||||
v_imported_calculation unit_as_labelled varchar(30)
|
||||
v_imported_calculation our_reading varchar(300)
|
||||
v_imported_calculation baseline_use varchar(12)
|
||||
v_imported_calculation exclusion_reason varchar(300)
|
||||
v_imported_calculation document_key varchar(16)
|
||||
v_imported_calculation claim_status varchar(12)
|
||||
v_interval site_uid varchar(12)
|
||||
v_interval site_code varchar(20)
|
||||
v_interval site_name varchar(120)
|
||||
v_interval site_label varchar(170)
|
||||
v_interval interval_label varchar(30)
|
||||
v_hour_meter_evidence claim_status character varying(12)
|
||||
v_imported_calculation site_uid character varying(12)
|
||||
v_imported_calculation site_code character varying(20)
|
||||
v_imported_calculation site_name character varying(120)
|
||||
v_imported_calculation site_label character varying(170)
|
||||
v_imported_calculation cell_ref character varying(40)
|
||||
v_imported_calculation source_label character varying(80)
|
||||
v_imported_calculation formula character varying(200)
|
||||
v_imported_calculation cached_value numeric(14,6)
|
||||
v_imported_calculation unit_as_labelled character varying(30)
|
||||
v_imported_calculation our_reading character varying(300)
|
||||
v_imported_calculation baseline_use character varying(12)
|
||||
v_imported_calculation exclusion_reason character varying(300)
|
||||
v_imported_calculation document_key character varying(16)
|
||||
v_imported_calculation claim_status character varying(12)
|
||||
v_interval site_uid character varying(12)
|
||||
v_interval site_code character varying(20)
|
||||
v_interval site_name character varying(120)
|
||||
v_interval site_label character varying(170)
|
||||
v_interval interval_label character varying(30)
|
||||
v_interval start_on date
|
||||
v_interval end_on date
|
||||
v_interval days integer
|
||||
v_interval opening_l decimal(9,1)
|
||||
v_interval opening_basis varchar(8)
|
||||
v_interval deliveries_l decimal(9,1)
|
||||
v_interval withdrawals_l decimal(9,1)
|
||||
v_interval closing_l decimal(9,1)
|
||||
v_interval closing_basis varchar(8)
|
||||
v_interval fuel_used_l decimal(9,1)
|
||||
v_interval fuel_used_unc_l decimal(9,1)
|
||||
v_interval fuel_l_per_day decimal(8,1)
|
||||
v_interval stock_status varchar(40)
|
||||
v_interval runtime_h decimal(9,1)
|
||||
v_interval runtime_h_per_day decimal(6,2)
|
||||
v_interval opening_l numeric(9,1)
|
||||
v_interval opening_basis character varying(8)
|
||||
v_interval deliveries_l numeric(9,1)
|
||||
v_interval withdrawals_l numeric(9,1)
|
||||
v_interval closing_l numeric(9,1)
|
||||
v_interval closing_basis character varying(8)
|
||||
v_interval fuel_used_l numeric(9,1)
|
||||
v_interval fuel_used_unc_l numeric(9,1)
|
||||
v_interval fuel_l_per_day numeric(8,1)
|
||||
v_interval stock_status character varying(40)
|
||||
v_interval runtime_h numeric(9,1)
|
||||
v_interval runtime_h_per_day numeric(6,2)
|
||||
v_interval n_meters integer
|
||||
v_interval metric varchar(40)
|
||||
v_interval population_key varchar(200)
|
||||
v_interval l_per_h decimal(8,3)
|
||||
v_interval l_per_h_low decimal(8,3)
|
||||
v_interval l_per_h_high decimal(8,3)
|
||||
v_interval eligibility varchar(12)
|
||||
v_interval exclusion_reasons varchar(600)
|
||||
v_interval evidence_grade varchar(2)
|
||||
v_interval conditions varchar(200)
|
||||
v_interval metric character varying(40)
|
||||
v_interval population_key character varying(200)
|
||||
v_interval l_per_h numeric(8,3)
|
||||
v_interval l_per_h_low numeric(8,3)
|
||||
v_interval l_per_h_high numeric(8,3)
|
||||
v_interval eligibility character varying(12)
|
||||
v_interval exclusion_reasons character varying(600)
|
||||
v_interval evidence_grade character varying(2)
|
||||
v_interval conditions character varying(200)
|
||||
v_interval in_current_baseline boolean
|
||||
v_interval baseline_comparable boolean
|
||||
v_interval current_baseline_l_per_h decimal(8,3)
|
||||
v_kpi_measures site_uid varchar(12)
|
||||
v_kpi_measures site_code varchar(20)
|
||||
v_kpi_measures site_name varchar(120)
|
||||
v_kpi_measures site_label varchar(170)
|
||||
v_interval current_baseline_l_per_h numeric(8,3)
|
||||
v_kpi_measures site_uid character varying(12)
|
||||
v_kpi_measures site_code character varying(20)
|
||||
v_kpi_measures site_name character varying(120)
|
||||
v_kpi_measures site_label character varying(170)
|
||||
v_kpi_measures measure_order integer
|
||||
v_kpi_measures measure varchar(60)
|
||||
v_kpi_measures value decimal(12,2)
|
||||
v_kpi_measures unit varchar(12)
|
||||
v_kpi_measures value_kind varchar(16)
|
||||
v_kpi_measures measure character varying(60)
|
||||
v_kpi_measures value numeric(12,2)
|
||||
v_kpi_measures unit character varying(12)
|
||||
v_kpi_measures value_kind character varying(16)
|
||||
v_kpi_measures window_start date
|
||||
v_kpi_measures window_end date
|
||||
v_kpi_measures note varchar(200)
|
||||
v_modelled_lph site_uid varchar(12)
|
||||
v_modelled_lph site_code varchar(20)
|
||||
v_modelled_lph site_name varchar(120)
|
||||
v_modelled_lph site_label varchar(170)
|
||||
v_modelled_lph assumption_code varchar(60)
|
||||
v_modelled_lph status varchar(12)
|
||||
v_modelled_lph basis varchar(20)
|
||||
v_modelled_lph h_per_day decimal(5,2)
|
||||
v_modelled_lph h_per_day_low decimal(5,2)
|
||||
v_modelled_lph h_per_day_high decimal(5,2)
|
||||
v_kpi_measures note character varying(200)
|
||||
v_modelled_lph site_uid character varying(12)
|
||||
v_modelled_lph site_code character varying(20)
|
||||
v_modelled_lph site_name character varying(120)
|
||||
v_modelled_lph site_label character varying(170)
|
||||
v_modelled_lph assumption_code character varying(60)
|
||||
v_modelled_lph status character varying(12)
|
||||
v_modelled_lph basis character varying(20)
|
||||
v_modelled_lph h_per_day numeric(5,2)
|
||||
v_modelled_lph h_per_day_low numeric(5,2)
|
||||
v_modelled_lph h_per_day_high numeric(5,2)
|
||||
v_modelled_lph window_start date
|
||||
v_modelled_lph window_end date
|
||||
v_modelled_lph matched_days integer
|
||||
v_modelled_lph l_per_day decimal(7,1)
|
||||
v_modelled_lph modelled_l_per_h decimal(6,2)
|
||||
v_modelled_lph modelled_l_per_h_low decimal(6,2)
|
||||
v_modelled_lph modelled_l_per_h_high decimal(6,2)
|
||||
v_modelled_lph value_kind varchar(70)
|
||||
v_reconciliation site_uid varchar(12)
|
||||
v_reconciliation site_code varchar(20)
|
||||
v_reconciliation site_name varchar(120)
|
||||
v_reconciliation site_label varchar(170)
|
||||
v_reconciliation interval_label varchar(30)
|
||||
v_modelled_lph l_per_day numeric(7,1)
|
||||
v_modelled_lph modelled_l_per_h numeric(6,2)
|
||||
v_modelled_lph modelled_l_per_h_low numeric(6,2)
|
||||
v_modelled_lph modelled_l_per_h_high numeric(6,2)
|
||||
v_modelled_lph value_kind character varying(70)
|
||||
v_reconciliation site_uid character varying(12)
|
||||
v_reconciliation site_code character varying(20)
|
||||
v_reconciliation site_name character varying(120)
|
||||
v_reconciliation site_label character varying(170)
|
||||
v_reconciliation interval_label character varying(30)
|
||||
v_reconciliation start_on date
|
||||
v_reconciliation end_on date
|
||||
v_reconciliation stock_status varchar(40)
|
||||
v_reconciliation stock_status character varying(40)
|
||||
v_reconciliation endpoint_times_known boolean
|
||||
v_reconciliation asset_uid varchar(40)
|
||||
v_reconciliation asset_uid character varying(40)
|
||||
v_reconciliation placed_throughout boolean
|
||||
v_reconciliation stock_balance_l decimal(9,1)
|
||||
v_reconciliation engine_l decimal(9,1)
|
||||
v_reconciliation covered_h decimal(9,1)
|
||||
v_reconciliation interval_h decimal(9,1)
|
||||
v_reconciliation engine_all_generators_l decimal(9,1)
|
||||
v_reconciliation stock_balance_l numeric(9,1)
|
||||
v_reconciliation engine_l numeric(9,1)
|
||||
v_reconciliation covered_h numeric(9,1)
|
||||
v_reconciliation interval_h numeric(9,1)
|
||||
v_reconciliation engine_all_generators_l numeric(9,1)
|
||||
v_reconciliation all_generators_metered boolean
|
||||
v_reconciliation difference_l decimal(9,1)
|
||||
v_reconciliation difference_pct decimal(7,1)
|
||||
v_runtime site_uid varchar(12)
|
||||
v_runtime site_code varchar(20)
|
||||
v_runtime site_name varchar(120)
|
||||
v_runtime site_label varchar(170)
|
||||
v_runtime meter_label varchar(40)
|
||||
v_runtime mapped_set varchar(60)
|
||||
v_runtime interval_label varchar(80)
|
||||
v_reconciliation difference_l numeric(9,1)
|
||||
v_reconciliation difference_pct numeric(7,1)
|
||||
v_runtime site_uid character varying(12)
|
||||
v_runtime site_code character varying(20)
|
||||
v_runtime site_name character varying(120)
|
||||
v_runtime site_label character varying(170)
|
||||
v_runtime meter_label character varying(40)
|
||||
v_runtime mapped_set character varying(60)
|
||||
v_runtime interval_label character varying(80)
|
||||
v_runtime start_on date
|
||||
v_runtime end_on date
|
||||
v_runtime days integer
|
||||
v_runtime runtime_h decimal(9,1)
|
||||
v_runtime h_per_day decimal(6,2)
|
||||
v_runtime runtime_h numeric(9,1)
|
||||
v_runtime h_per_day numeric(6,2)
|
||||
v_runtime pair_valid boolean
|
||||
v_runtime same_epoch boolean
|
||||
v_runtime monotonic boolean
|
||||
v_runtime over_24h_per_day boolean
|
||||
v_runtime reviewed boolean
|
||||
v_site site_uid varchar(12)
|
||||
v_site site_code varchar(20)
|
||||
v_site site_name varchar(120)
|
||||
v_site site_label varchar(170)
|
||||
v_site zone varchar(40)
|
||||
v_site province varchar(40)
|
||||
v_site district varchar(60)
|
||||
v_site pilot_status varchar(12)
|
||||
v_site latitude decimal(9,6)
|
||||
v_site longitude decimal(9,6)
|
||||
v_site location_status varchar(20)
|
||||
v_site location_basis varchar(20)
|
||||
v_site site_uid character varying(12)
|
||||
v_site site_code character varying(20)
|
||||
v_site site_name character varying(120)
|
||||
v_site site_label character varying(170)
|
||||
v_site zone character varying(40)
|
||||
v_site province character varying(40)
|
||||
v_site district character varying(60)
|
||||
v_site pilot_status character varying(12)
|
||||
v_site latitude numeric(9,6)
|
||||
v_site longitude numeric(9,6)
|
||||
v_site location_status character varying(20)
|
||||
v_site location_basis character varying(20)
|
||||
v_site location_precision_m integer
|
||||
v_site baseline_status varchar(24)
|
||||
v_site baseline_status_label varchar(40)
|
||||
v_site baseline_validity_note varchar(60)
|
||||
v_site lph_estimate decimal(6,2)
|
||||
v_site lph_method varchar(60)
|
||||
v_site baseline_status character varying(24)
|
||||
v_site baseline_status_label character varying(40)
|
||||
v_site baseline_validity_note character varying(60)
|
||||
v_site lph_estimate numeric(6,2)
|
||||
v_site lph_method character varying(60)
|
||||
v_site lph_window_start date
|
||||
v_site lph_window_end date
|
||||
v_site baseline_conditions varchar(200)
|
||||
v_site baseline_decision_ref varchar(12)
|
||||
v_site fuel_l_per_calendar_day decimal(7,1)
|
||||
v_site baseline_conditions character varying(200)
|
||||
v_site baseline_decision_ref character varying(12)
|
||||
v_site fuel_l_per_calendar_day numeric(7,1)
|
||||
v_site latest_record_on date
|
||||
v_spec_reference site_uid varchar(12)
|
||||
v_spec_reference site_code varchar(20)
|
||||
v_spec_reference site_name varchar(120)
|
||||
v_spec_reference site_label varchar(170)
|
||||
v_spec_reference slot varchar(20)
|
||||
v_spec_reference asset_uid varchar(40)
|
||||
v_spec_reference asset_kind varchar(10)
|
||||
v_spec_reference make_model varchar(120)
|
||||
v_spec_reference attribute varchar(40)
|
||||
v_spec_reference load_pct decimal(5,1)
|
||||
v_spec_reference value decimal(12,3)
|
||||
v_spec_reference value_text varchar(120)
|
||||
v_spec_reference unit varchar(20)
|
||||
v_spec_reference basis varchar(30)
|
||||
v_spec_reference applicability varchar(12)
|
||||
v_spec_reference applicability_note varchar(300)
|
||||
v_spec_reference value_kind varchar(30)
|
||||
v_spec_reference claim_status varchar(12)
|
||||
v_stock_reading site_uid varchar(12)
|
||||
v_stock_reading site_code varchar(20)
|
||||
v_stock_reading site_name varchar(120)
|
||||
v_stock_reading site_label varchar(170)
|
||||
v_spec_reference site_uid character varying(12)
|
||||
v_spec_reference site_code character varying(20)
|
||||
v_spec_reference site_name character varying(120)
|
||||
v_spec_reference site_label character varying(170)
|
||||
v_spec_reference slot character varying(20)
|
||||
v_spec_reference asset_uid character varying(40)
|
||||
v_spec_reference asset_kind character varying(10)
|
||||
v_spec_reference make_model character varying(120)
|
||||
v_spec_reference attribute character varying(40)
|
||||
v_spec_reference load_pct numeric(5,1)
|
||||
v_spec_reference value numeric(12,3)
|
||||
v_spec_reference value_text character varying(120)
|
||||
v_spec_reference unit character varying(20)
|
||||
v_spec_reference basis character varying(30)
|
||||
v_spec_reference applicability character varying(12)
|
||||
v_spec_reference applicability_note character varying(300)
|
||||
v_spec_reference value_kind character varying(30)
|
||||
v_spec_reference claim_status character varying(12)
|
||||
v_stock_reading site_uid character varying(12)
|
||||
v_stock_reading site_code character varying(20)
|
||||
v_stock_reading site_name character varying(120)
|
||||
v_stock_reading site_label character varying(170)
|
||||
v_stock_reading observed_on date
|
||||
v_stock_reading seq_in_day integer
|
||||
v_stock_reading reading_type varchar(20)
|
||||
v_stock_reading litres decimal(8,1)
|
||||
v_stock_reading value_kind varchar(12)
|
||||
v_stock_reading spread_l decimal(8,1)
|
||||
v_stock_reading reading_type character varying(20)
|
||||
v_stock_reading litres numeric(8,1)
|
||||
v_stock_reading value_kind character varying(12)
|
||||
v_stock_reading spread_l numeric(8,1)
|
||||
v_stock_reading reviewed boolean
|
||||
|
Can't render this file because it contains an unexpected character in line 4 and column 41.
|
@@ -15,7 +15,7 @@ the bundle are dropped; afterwards each dataset's columns are compared with the
|
||||
live view and any drift is reported (exit status 2).
|
||||
|
||||
The database is never created or changed: the bundle's database uuid must
|
||||
already exist under the name "Fuel (Trino)", otherwise nothing is imported.
|
||||
already exist under the name "Fuel", otherwise nothing is imported.
|
||||
|
||||
The dashboards (atoma-overview, atoma-site) get no role: admins see them in Superset, and
|
||||
the Petrolyst app shows them with guest passes (PD-12). With --embedded-out FILE
|
||||
@@ -35,7 +35,7 @@ import zipfile
|
||||
|
||||
import yaml
|
||||
|
||||
DATABASE_NAME = "Fuel (Trino)"
|
||||
DATABASE_NAME = "Fuel"
|
||||
EMBEDDED_PREFIX = "atoma-"
|
||||
STEPS = [("datasets", "SqlaTable"), ("charts", "Slice"), ("dashboards", "Dashboard")]
|
||||
|
||||
|
||||
@@ -45,6 +45,21 @@ SQLALCHEMY_DATABASE_URI = URL.create(
|
||||
database="superset",
|
||||
).render_as_string(hide_password=False)
|
||||
|
||||
# --- The data: fuel-db ----------------------------------------------------------
|
||||
# Superset reads fuel-db directly as the read-only role fuel_ro (no Trino since 29 Oct 2026);
|
||||
# superset-init registers that connection, "Fuel", without a password. Passwords of data
|
||||
# connections come from secret files at connect time, so none is kept in Superset's own
|
||||
# database. A connection added in the UI gets no password this way.
|
||||
_DATABASE_PASSWORD_SECRETS = {"fuel_ro": "fuel_db_ro_password"}
|
||||
|
||||
|
||||
def _database_password(url: URL) -> str | None:
|
||||
secret = _DATABASE_PASSWORD_SECRETS.get(url.username or "")
|
||||
return _read_secret(secret) if secret else None
|
||||
|
||||
|
||||
SQLALCHEMY_CUSTOM_PASSWORD_STORE = _database_password
|
||||
|
||||
# --- Sign-in through Keycloak (client "petrolyst") ----------------------------
|
||||
# Users and roles are managed in Keycloak; there is no local admin account.
|
||||
# Break-glass access: `superset fab ...` inside the container on the server.
|
||||
|
||||
Reference in new issue
Block a user