diff --git a/.buildkite/explorer-release.yml b/.buildkite/explorer-release.yml index 752efe742..a1a8424c8 100644 --- a/.buildkite/explorer-release.yml +++ b/.buildkite/explorer-release.yml @@ -1,7 +1,7 @@ # Builds the radicle-httpd and radicle-search release archives from a # `releases/httpd/` tag, and keeps them as Buildkite artifacts. A # delegate downloads them with explorer/crates/radicle-httpd/build/fetch, then -# signs and uploads them, see explorer/RELEASE.md. +# signs, uploads and registers them, see explorer/RELEASE.md. # # garden-broker builds a tag push like a branch push: BUILDKITE_BRANCH is the # tag name and BUILDKITE_COMMIT the tag object. build/ci checks that the build diff --git a/explorer/RELEASE.md b/explorer/RELEASE.md index 98e3d36f6..98061a4c8 100644 --- a/explorer/RELEASE.md +++ b/explorer/RELEASE.md @@ -15,7 +15,9 @@ The release tooling lives in [`crates/radicle-httpd/build/`][build]. Releases cover **only** the `radicle-httpd` and `radicle-search` binaries. The web front-end (`radicle-explorer`) is not released this way: it is deployed continuously (rolling release) and must always remain compatible with the latest -`radicle-httpd`/`radicle-search` release. +`radicle-httpd`/`radicle-search` release. A release also registers the +frontend bundle of the tagged commit with radicle-artifact, see +[Publish](#publish). ## Versioning @@ -35,8 +37,11 @@ for which releases explorer supports. git config set user.signingKey "$(rad self --ssh-key)" ``` - Working `podman` setup (see [Podman Setup](#podman-setup) below), or Docker -- Required tools: `podman` or `docker`, `rad`, `sha256sum`, `curl`, `jq`, - `ssh`, `scp`, `ssh-keygen` +- Required tools: `podman` or `docker`, `rad`, `rad-artifact`, `sha256sum`, + `curl`, `jq`, `ssh`, `scp`, `ssh-keygen`, `node`, `npm`. Install `rad-artifact` with: + ```sh + curl -sSf https://files.radicle.dev/releases/radicle-artifact/install | sh + ``` - To download the CI build: a Buildkite API token with `read_builds` and `read_artifacts` @@ -86,6 +91,13 @@ Normal patches add their highlights to `## Unreleased` in rad sync --announce ``` +5. Optional: create the release in radicle-artifact now, so that it is visible + before its archives are registered. `register` creates it otherwise: + + ```sh + rad-artifact create releases/httpd/ + ``` + ## Build Pushing the tag starts the `📦 httpd release build` step in Buildkite. It builds @@ -154,6 +166,35 @@ the archives. > **Warning:** Uploading immediately publishes the release. +3. Register the archives with radicle-artifact, in the release of the tag. This + records each archive's files.radicle.dev URL and `sha256`. If you publish the + archives that `fetch` downloaded into `artifacts/`, it also records their + Buildkite build. It also builds the explorer frontend bundle from the + checked out tag with `config/default.json`, registers it as the collection + `radicle-explorer` and seeds it from your local artifact node. This needs + `node` and `npm`, in the version of `.nvmrc`, and no untracked files in + `config/`, such as `local.json`. Running it again changes nothing: + + ```sh + crates/radicle-httpd/build/register + ``` + +## Attest (recommended) + +Other delegates check the release by building it themselves. `attest` compares +the CIDs of your build with the registered ones, and attests them only if all +match: + +```sh +git switch --detach releases/httpd/ +crates/radicle-httpd/build/build +crates/radicle-httpd/build/attest +``` + +If your build differs, find out why before anyone uses the release. If the +release is wrong, withdraw its archives with +`rad-artifact redact releases/httpd/ --cid -m `. + ## Announce 1. Mark the previous release as resolved in Zulip `#Announcements` @@ -180,6 +221,10 @@ the archives. - https://files.radicle.dev/releases/radicle-search/ - https://radicle.dev/download + Check a download with + `rad-artifact verify --repo rad:z4V1sjrXqjvFdnCUbxPFqd5p4DtH5 `, + after `rad seed rad:z4V1sjrXqjvFdnCUbxPFqd5p4DtH5`. + ## Changelog section from CHANGELOG.md> @@ -190,8 +235,8 @@ the archives. ``` - :info: This release was built on . Build reproducibility across - platforms is unsupported; checksums may differ if rebuilt elsewhere. + Attested by: `, if + any> ```` ## Notes diff --git a/explorer/crates/radicle-httpd/build/attest b/explorer/crates/radicle-httpd/build/attest new file mode 100755 index 000000000..ee299b34a --- /dev/null +++ b/explorer/crates/radicle-httpd/build/attest @@ -0,0 +1,66 @@ +#!/bin/sh +set -e + +# Attests the registered archives of `releases/httpd/` after checking +# that your own build in `artifacts/` gives the same CIDs. Run `build` first. + +outdir="$(dirname "$0")/artifacts" +version="$("$(dirname "$0")/version")" +tag="releases/httpd/$version" + +for tool in rad rad-artifact jq; do + if ! command -v "$tool" >/dev/null; then + echo "fatal: $tool is not installed" >&2 + exit 1 + fi +done + +release="$(rad-artifact show "$tag" --json)" +me="$(rad self --did)" + +# Registering already counts as the author's attestation. +if echo "$release" | jq -e --arg me "$me" 'any(.[].artifacts[]; .author == $me)' >/dev/null; then + echo "fatal: you registered $tag, so another delegate has to attest it" >&2 + exit 1 +fi + +cids="" +mismatch=false + +for target in $(cat "$(dirname "$0")/TARGETS"); do + for bin in radicle-httpd radicle-search; do + filename="$bin-$version-$target.tar.xz" + filepath="$outdir/$filename" + + if [ ! -f "$filepath" ]; then + echo "fatal: $filepath is missing, run build first" >&2 + exit 1 + fi + + cid="$(rad-artifact cid "$filepath")" + registered="$(echo "$release" | jq -r --arg name "$filename" \ + '[.[].artifacts[] | select(.name == $name) | .cid] | first // empty')" + + if [ "$cid" = "$registered" ]; then + echo "✓ $filename $cid" + cids="$cids $cid" + else + echo "✗ $filename: your build is $cid, the release has ${registered:-nothing}" >&2 + mismatch=true + fi + done +done + +if [ "$mismatch" = true ]; then + echo "fatal: your build differs from the release; nothing was attested" >&2 + echo "If the release is wrong, withdraw it with 'rad-artifact redact $tag --cid -m '" >&2 + exit 1 +fi + +rad auth +for cid in $cids; do + rad-artifact --no-announce --no-input attest "$tag" --cid "$cid" +done + +rad sync --announce +echo "Attestation successful." diff --git a/explorer/crates/radicle-httpd/build/build b/explorer/crates/radicle-httpd/build/build index 81e7f5e2f..ca0d87625 100755 --- a/explorer/crates/radicle-httpd/build/build +++ b/explorer/crates/radicle-httpd/build/build @@ -83,6 +83,8 @@ main() { # Copy build artifacts to output folder. outdir="$(dirname "$0")/artifacts" mkdir -p "$outdir" + # These archives are not from a CI build, see `fetch`. + rm -f "$outdir/build-url" $engine cp "$container:/builds/." "$outdir/" for target in $(cat "$(dirname "$0")/TARGETS"); do diff --git a/explorer/crates/radicle-httpd/build/fetch b/explorer/crates/radicle-httpd/build/fetch index a24b1f959..73a540225 100755 --- a/explorer/crates/radicle-httpd/build/fetch +++ b/explorer/crates/radicle-httpd/build/fetch @@ -78,4 +78,6 @@ for target in $(cat "$(dirname "$0")/TARGETS"); do done done +# `register` records the CI build of these archives. +echo "$web_url" >"$outdir/build-url" echo "Fetch successful." diff --git a/explorer/crates/radicle-httpd/build/register b/explorer/crates/radicle-httpd/build/register new file mode 100755 index 000000000..ab8c416f3 --- /dev/null +++ b/explorer/crates/radicle-httpd/build/register @@ -0,0 +1,74 @@ +#!/bin/sh +set -e + +# Registers the archives in `artifacts/` with radicle-artifact, in the release +# of the tag `releases/httpd/`, with their files.radicle.dev URLs and +# checksums. It also builds the explorer bundle from the checked out tag with +# config/default.json, registers it as a collection and seeds it from the local +# artifact node. Run it after `upload`. Running it again changes nothing. + +outdir="$(dirname "$0")/artifacts" +explorer="$(cd "$(dirname "$0")/../../.." && pwd)" +version="$("$(dirname "$0")/version")" +tag="releases/httpd/$version" + +for tool in git npm rad rad-artifact jq; do + if ! command -v "$tool" >/dev/null; then + echo "fatal: $tool is not installed" >&2 + exit 1 + fi +done + +# node-config also loads untracked files such as config/local.json. +if [ -n "$(git -C "$explorer" ls-files --others config)" ]; then + echo "fatal: $explorer/config has untracked files, remove them" >&2 + exit 1 +fi + +rad auth + +artifact() { + rad-artifact --no-announce --no-input "$@" +} + +for target in $(cat "$(dirname "$0")/TARGETS"); do + for bin in radicle-httpd radicle-search; do + filename="$bin-$version-$target.tar.xz" + filepath="$outdir/$filename" + + if [ ! -f "$filepath" ]; then + echo "fatal: $filepath is missing" >&2 + exit 1 + fi + (cd "$outdir" && sha256sum --check --quiet "$filename.sha256") + sha256="$(cut -d' ' -f1 "$filepath.sha256")" + + echo "Registering $filename.." + cid="$(artifact register "$filepath" --revision "$tag" -n "$filename" --json | jq -r .cid)" + artifact location add --revision "$tag" --cid "$cid" \ + "https://files.radicle.dev/releases/$bin/$version/$filename" + artifact metadata set --revision "$tag" --cid "$cid" sha256 "$sha256" + if [ -f "$outdir/build-url" ]; then + artifact metadata set --revision "$tag" --cid "$cid" ci-build "$(cat "$outdir/build-url")" + fi + done +done + +echo "Building the explorer bundle.." +( + cd "$explorer" + unset NODE_CONFIG NODE_CONFIG_ENV NODE_CONFIG_DIR VITE_RUNTIME_CONFIG + npm ci -w explorer --no-audit --no-fund + npm run build +) + +if ! rad-artifact node status >/dev/null 2>&1; then + rad-artifact node start +fi + +echo "Registering the explorer bundle.." +artifact register "$explorer/build" --collection -n radicle-explorer \ + --revision "$tag" --seed >/dev/null + +rad sync --announce +echo "Registration successful."