deploy: remove the CI pipeline; deploys are manual

No runner was ever active, so the pipeline never ran. Removes .gitea/workflows/deploy.yml and the CI key and runner IP from host/bootstrap.sh (a re-run removes the key an earlier bootstrap installed). The README makes "Manual deploy" the one documented procedure: commit on main, shellcheck in Docker (-S warning; the pipeline's lint step), git archive over root SSH, deploy.sh as user deploy, the outside checks, then push. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 09:58:22 +02:00
1 parent bf96edc828
commit bc27bf2356
@@ -1,7 +1,6 @@
 # petrolyst-infra

-Everything that runs on the petrolyst server (109.199.127.84, Ubuntu 24.04), deployed on every
-push to `main`:
+Everything that runs on the petrolyst server (109.199.127.84, Ubuntu 24.04):

 - **Traefik** at the edge for every project on the server. Its dashboard is at
  https://traefik.petrolyst.com, behind Keycloak.
@@ -9,6 +8,9 @@ push to `main`:
  store `fuel-db` (Postgres 17 with TimescaleDB) through Trino. Sign-in goes through Keycloak
  client `petrolyst`, and the UI carries the Petrolyst brand.

+**Deploys are manual.** A person deploys a clean commit of `main` from the Mac ("Manual deploy"
+under Operating). There is no CI runner, and a push to Gitea deploys nothing.
+
 Customization and deployment are separate, as with media-insights and media-intelligence-docker:

 | Folder | Holds |
@@ -16,19 +18,19 @@ Customization and deployment are separate, as with media-insights and media-inte
 | `petrolyst-insights/` | how Superset looks and behaves for Petrolyst: config, Keycloak login code, branding, patches, dashboards. See its README. |
 | `traefik/` | edge proxy, TLS, the `sso` middleware (oauth2-proxy) that guards the dashboard |
 | `stacks/fuel-monitoring/` | Compose stack, Superset Dockerfile, Trino config and catalog, fuel-db roles |
-| `deploy/` | `deploy.sh` (run by CI), `rollback.sh` (manual), `lib.sh` (shared) |
+| `deploy/` | `deploy.sh` (run on the server by a manual deploy), `rollback.sh` (manual), `lib.sh` (shared) |
 | `host/` | one-time server setup: `bootstrap.sh`, `secrets-init.sh` |
 | `keycloak/` | the `petrolyst` client for Keycloak's partial import |
-| `.gitea/workflows/deploy.yml` | the pipeline |

 No credential is in this repository. Secrets exist only on the server, under
-`/opt/petrolyst/secrets/`; CI holds only the SSH deploy key.
+`/opt/petrolyst/secrets/`. A deploy needs only root SSH access to the server (`ssh petrolyst`).

 ## How a deploy works

-1. A push to `main` (or a manual run) starts `.gitea/workflows/deploy.yml` on the act_runner. It
-   lints the scripts, rsyncs the checkout to a fresh `/opt/petrolyst/incoming/<sha>-<run>/`, and
-   calls `deploy/deploy.sh <sha> <that dir>` over SSH as user `deploy`.
+1. A person runs "Manual deploy" (Operating, below) from a clean checkout of `main`: the scripts
+   are linted, `git archive` sends exactly that commit to a fresh
+   `/opt/petrolyst/incoming/<sha>-manual-<time>/`, and `deploy/deploy.sh <sha> <that dir>` runs
+   there as user `deploy` (root hands over with `sudo -u deploy`).
 2. `deploy.sh` takes the host lock `/opt/petrolyst/deploy.lock`, which it shares with
   `rollback.sh`, and keeps it until it exits.
 3. It finalizes the upload. A new commit is moved to `releases/<sha>`, and the release is made
@@ -52,7 +54,7 @@ No credential is in this repository. Secrets exist only on the server, under
     `superset`, writes `backups/superset-meta_<old tag>_<revision>_<time>.dump`, then runs
     `superset-init` (`db upgrade`, `init`, registers "Fuel (Trino)").
   - If the image is unchanged and the schema is at its head, it does none of this.
-8. Smoke tests. A failure turns CI red and prints the stacks' state.
+8. Smoke tests. A failure ends the deploy with an error and prints the stacks' state.
   - `/health` answers through Traefik with a valid certificate.
   - `/login/` leads to Keycloak with `client_id=petrolyst` and PKCE.
   - The dashboard redirects to Keycloak.
@@ -63,14 +65,14 @@ No credential is in this repository. Secrets exist only on the server, under
   - the 3 newest Superset images, plus any image that a kept release or backup names;
   - uploads older than a day.

-The runner then checks both hosts from outside.
+The manual deploy then checks both hosts from outside.

 ## Server paths

 | Path | Owner | What |
 |---|---|---|
 | `/opt/petrolyst/infra` | deploy | live tree (the promoted release) |
-| `/opt/petrolyst/incoming/<sha>-<run>` | deploy | one CI upload; only `deploy.sh` turns it into a release |
+| `/opt/petrolyst/incoming/<sha>-manual-<time>` | deploy | one upload; only `deploy.sh` turns it into a release |
 | `/opt/petrolyst/releases/<sha>` | deploy, read-only | finalized release plus `ARTIFACTS`; the last 5 are kept |
 | `/opt/petrolyst/secrets/{shared,traefik,fuel-monitoring}/` | root:deploy 0750, files 0444 | secrets, mounted as `/run/secrets/<name>` |
 | `/opt/petrolyst/env/fuel-monitoring.env` | root:deploy 0440 | `FUEL_DB_TRINO_PASSWORD` for Trino's `${ENV:…}` |
@@ -81,24 +83,9 @@ The runner then checks both hosts from outside.

 ### 1. Gitea

-1. Create the private repository `Petrolyst/petrolyst-infra` and push this repository's
-   `main` to it. Enable Actions under Settings → Repository → Units.
-2. Runner. Check Site/Org Administration → Actions → Runners. If none is registered, run
-   [Gitea Runner](https://docs.gitea.com/runner/installation/docker/) as the container
-   `docker.io/gitea/runner:5.2.2-dind` (privileged, with `/data` and `/var/lib/docker` volumes),
-   for example on the Gitea host (164.68.121.117):
-   - register it as an **organization runner** for `Petrolyst`: take the token from
-     Settings → Actions → Runners → Create new Runner;
-   - set `GITEA_RUNNER_LABELS=ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest`;
-   - note its egress IPv4 (`curl -4 -s https://ifconfig.me`).
-3. Generate a deploy key on your Mac, used only for this:
-   `ssh-keygen -t ed25519 -N '' -C petrolyst-ci -f petrolyst-ci`
-4. Add repository secrets (Settings → Actions → Secrets):
-   - `DEPLOY_SSH_KEY`: contents of `petrolyst-ci` (the private key)
-   - `DEPLOY_HOST`: `109.199.127.84`
-   - `DEPLOY_KNOWN_HOSTS`: output of `ssh-keyscan -t ed25519 109.199.127.84`. Compare its
-     fingerprint (`ssh-keygen -lf <file>`) with `ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub`
-     run on the server.
+Create the private repository `Petrolyst/petrolyst-infra` and push this repository's `main` to
+it. Gitea only keeps the history: deploys are manual, so it needs no runner, no Actions and no
+deploy secrets.

 ### 2. Keycloak (realm `thedigitalbridge` on iam.thedigitalbridge.io)

@@ -132,24 +119,21 @@ The server needs only the two host scripts; it never gets Gitea credentials.

 ```bash
 # on the Mac, in this repository
-scp host/bootstrap.sh host/secrets-init.sh petrolyst-ci.pub petrolyst:/root/
+scp host/bootstrap.sh host/secrets-init.sh petrolyst:/root/
 ssh petrolyst
 # on the server, as root
-/root/bootstrap.sh --ci-key /root/petrolyst-ci.pub --runner-ip <runner egress IP>
+/root/bootstrap.sh
 /root/secrets-init.sh                   # generates every secret; reports the missing Keycloak one
 /root/secrets-init.sh keycloak-secret   # paste the secret from Keycloak step 2
 ```

-No runner yet? Pass `--runner-ip 127.0.0.1`. That installs the CI key but leaves it unusable from
-outside. Deploy manually until a runner exists (see "Manual deploy" below). When the runner is
-running, run `bootstrap.sh` again with its egress IP.
-
 `bootstrap.sh` does the following:
 - installs Docker from Docker's apt repository;
 - adds 4 GB swap;
 - turns on ufw for 22, 80 and 443 only;
 - turns off SSH password login and root password login;
- creates the `deploy` user, whose key works only from the runner IP;
+- creates the `deploy` user, which runs `deploy.sh`. It has no SSH key and is reached only
+  through `sudo -u deploy` from root; a re-run removes the CI key an earlier version installed;
 - creates `/opt/petrolyst` and the Docker network `proxy`.

 **Before closing your root session**, check from a second terminal that `ssh petrolyst` still
@@ -157,8 +141,7 @@ works.

 ### 4. First deploy

-Run the workflow manually (Actions → deploy → Run workflow) or push to `main`. Without a runner,
-use "Manual deploy" below. The first run builds the Superset image and takes the fresh-install
+Use "Manual deploy" below. The first run builds the Superset image and takes the fresh-install
 path. Let's Encrypt needs about a minute
 for the two certificates; the smoke tests wait up to two minutes.

@@ -169,7 +152,7 @@ becomes the first Superset admin; there is no local admin account.

 | Task | How |
 |---|---|
-| Deploy | push to `main` (without a runner: "Manual deploy" below) |
+| Deploy | "Manual deploy" below, from a clean `main` |
 | Re-deploy a kept release | `sudo -u deploy /opt/petrolyst/infra/deploy/deploy.sh <sha>` |
 | Roll back | see below |
 | Traefik dashboard | https://traefik.petrolyst.com (role `traefik-admin`) |
@@ -178,21 +161,41 @@ becomes the first Superset admin; there is no local admin account.
 | Logs | `docker compose -p fuel-monitoring logs -f superset` (or `-p traefik`) |
 | Superset break-glass | `docker exec -it fuel-monitoring-superset-1 superset fab ...` |

-**Manual deploy** (no runner): commit first, so that `git status` is clean. Then run this on the
-Mac, in this repository:
+**Manual deploy**, the only way to deploy (there is no CI runner). On the Mac, in this
+repository:

-```bash
-sha=$(git rev-parse HEAD)
-dir=/opt/petrolyst/incoming/$sha-manual-$(date -u +%s)
-git archive --format=tar "$sha" |
-  ssh petrolyst "install -d -o deploy -g deploy -m 0755 $dir && tar -x --no-same-owner -C $dir && chown -R deploy:deploy $dir"
-ssh petrolyst "sudo -u deploy $dir/deploy/deploy.sh $sha $dir"
-```
+1. Commit on `main`, so that `git status` is clean. The live commit is in
+   `/opt/petrolyst/state/deployed-sha` (`ssh petrolyst cat /opt/petrolyst/state/deployed-sha`).
+2. Lint the shell scripts. Warnings and errors must be zero; the info notes on the deliberate
+   `test && test || die` guards in `deploy/lib.sh` (SC2015) are filtered out:

-`git archive` sends exactly the commit, never uncommitted or ignored files such as the deploy
-key. The server then runs the same `deploy.sh` that CI runs, under the same lock. A later push
-queues a CI run that waits for a runner. Cancel stale waiting runs under Actions before you
-register one.
+   ```bash
+   docker run --rm -v "$PWD:/mnt" -w /mnt koalaman/shellcheck:v0.10.0 -S warning -x \
+     host/*.sh deploy/*.sh stacks/*/superset/init.sh stacks/*/fuel-db/init/*.sh
+   ```
+
+3. Deploy:
+
+   ```bash
+   sha=$(git rev-parse HEAD)
+   dir=/opt/petrolyst/incoming/$sha-manual-$(date -u +%s)
+   git archive --format=tar "$sha" |
+     ssh petrolyst "install -d -o deploy -g deploy -m 0755 $dir && tar -x --no-same-owner -C $dir && chown -R deploy:deploy $dir"
+   ssh petrolyst "sudo -u deploy $dir/deploy/deploy.sh $sha $dir"
+   ```
+
+4. Check both hosts from outside:
+
+   ```bash
+   test "$(curl -fsS --max-time 20 https://atoma.petrolyst.com/health)" = OK && echo atoma ok
+   curl -fsS --max-time 20 -o /dev/null -w '%{http_code}\n' https://traefik.petrolyst.com/   # 302
+   ```
+
+5. Push `main` to Gitea, so that the repository matches what is live.
+
+`git archive` sends exactly the commit, never uncommitted or ignored files. The server runs
+`deploy.sh` under the host lock it shares with `rollback.sh`, and records the live commit in
+`/opt/petrolyst/state/deployed-sha`.

 **Rollback** (`deploy/rollback.sh`, manual, under the same lock as deploys):

@@ -291,8 +294,7 @@ Results:
 Not verified locally:
 - Let's Encrypt issuance;
 - a real Keycloak sign-in, OTP and logout (they need the client from setup step 2);
- the theme as rendered in a browser;
- the Gitea runner.
+- the theme as rendered in a browser.

 ## Open items