Releasing
This page is for maintainers who prepare and tag a release. It describes how the version is kept in one place, how the release workflow turns a tag into a signed, verified and immutable release, what you do for a release candidate and for a final release, and what to do when a job fails. The local gates are in the development guide. How users check a release is in Verify releases. The decision behind the workflow is ADR-0035.
Versioning
Myriarch uses semantic versioning in two forms only: X.Y.Z for a final release and X.Y.Z-rc.N for a
release candidate. Numbers have no leading zeros, and the rc number starts at 1. Tags are the version with a leading v
(v0.1.0-rc.1, v0.1.0). Other forms (-beta.1, -rc.0, +build) are not release tags.
One version source
The file VERSION at the repository root holds the version without the leading v, for example 0.1.0-rc.1. Between releases it
names the last release; the pull request that prepares the next one sets it to the next tag. Each release candidate bumps it in the
pull request that carries the fixes for the previous candidate, and the pull request that prepares a final release sets it to X.Y.Z.
Every other place that states the product version is a copy of VERSION:
| Copy | What it is |
|---|---|
deploy/helm/myriarch/Chart.yaml, version | Version of the Helm chart in the repository. The release workflow packages the chart with --version and --app-version set to the tag without v, so the published OCI chart always has the version of the release. |
deploy/helm/myriarch/Chart.yaml, appVersion | Default image tag of the chart (image.tag is empty by default). |
deploy/helm/myriarch-collector/Chart.yaml, version and appVersion | Version of the collector chart and default tag of the updater image. |
api/openapi.yaml, info.version | Contract version of the REST API. It equals the release version. |
web/package.json, version | Version of the web UI package. |
deploy/examples/production/myriarch-values.yaml, image.tag | Image tag in the production example. |
gitops-template/.gitlab-ci.yml, MYRCTL_IMAGE | Tag of the placeholder myrctl image in the GitOps template (the project publishes no myrctl image; you build it from the myrctl release archive). |
deploy/compose/quickstart.yaml, x-image | Default image tag of the Compose quickstart (QUICKSTART_IMAGE replaces the whole image). |
The list lives in ci/version-copies.py. The binaries do not have a copy: they receive the version from the tag at build time.
Change the version with one command and commit the result:
scripts/set-version.sh 0.1.0-rc.2
The script accepts only X.Y.Z or X.Y.Z-rc.N. It writes VERSION and all copies, then runs make docs-gen and
pnpm gen:api in web/, so the generated REST reference (docs/reference/api/) and the web UI types stay in step. It needs
Go, Node.js and pnpm with the dependencies of web/ installed.
ci/check-versions.sh fails when VERSION is not a valid version or a copy differs from it. It is part of ci/check.sh, so
every pull request runs it, and the tag guard runs it again on the tagged commit. A failure names the file and tells you to run
scripts/set-version.sh.
The same script runs ci/check-stale-versions.py, which fails when a text still promises a release that exists: “planned for” or
“until” a version up to VERSION (for a release candidate, below its X.Y.Z, because that release is still ahead). It is part of
the same pull request check, so the pull request “Prepare vX.Y.Z” cannot be merged while such a sentence is left (see Review the
texts before every tag, step 6). scripts/set-version.sh runs the check last: when it reports
a sentence, the script ends with exit status 1 although VERSION and all copies are written. Reword the sentences and run
ci/check-versions.sh again. ci/tests/stale-versions-test.sh tests the check.
Release candidates and the final release
- A release candidate is published as a GitHub pre-release. A final release is a normal release and is marked as the latest release; its notes carry the support statement for the alpha phase.
- A published tag is never moved or reused. A defect found in
v0.1.0-rc.1is fixed in a pull request that also setsVERSIONto0.1.0-rc.2, and the next tag isv0.1.0-rc.2. A defect in a final release is fixed in the next patch release. - The pull request that prepares a final release sets
VERSIONtoX.Y.Z, turns the “Unreleased” section ofCHANGELOG.mdinto the heading## [X.Y.Z] - YYYY-MM-DDand checksdeploy/release/windows-signing.env(see Tag guard).
Image tags
The container images carry the version without the v:
| Tag pushed | Image tags |
|---|---|
v0.1.0-rc.2 | 0.1.0-rc.2 |
v0.1.0 | 0.1.0 and 0.1 |
There is no latest tag while the major version is 0. Pin a version or, better, a digest.
Tag guard
ci/release/guard.sh runs as the first step of the job guard, the first job of the release workflow. It reads the tag, the
commit, the repository, the API URL and the token from environment variables, matches each
value as a whole (a value with a newline or another control character is refused), never prints the token, and refuses a
release with one clear message per cause. It checks, in this order:
- The tag name is
vX.Y.ZorvX.Y.Z-rc.Nwithout leading zeros, and the commit is a full SHA. - The tag is an annotated tag. The guard fetches the tag object from
originagain, because a checkout can turn an annotated tag into a lightweight one. - The tag points at the commit that the run was started for, the checkout is at that commit and has no changes.
- The commit is on
origin/main: a release is built only from merged history. VERSIONequals the tag without thev, and every copy equalsVERSION(ci/check-versions.sh).- A final tag has a heading
## [X.Y.Z] - YYYY-MM-DDwith a valid date inCHANGELOG.md(a heading inside a code block or an HTML comment does not count), the section under it has at least one list item, and the “Unreleased” section has none left. An rc tag needs at least one list item in the “Unreleased” section; prose and comments are no entries. - The Windows signing rule holds (below).
- The check run
ci-okof GitHub Actions exists for the commit and has finished withsuccess. The guard asks the GitHub REST API withcurl; the token needs onlychecks: read. Any API error, an unexpected answer or a check run from another app refuses the release (fail closed).
The checks that need no network run first, and the API call comes last. After the guard, ci/release/meta.sh derives the outputs
that every later job uses (version, kind, pre-release flag, latest flag, image tags, the previous tag for the generated notes
and the Windows signing settings), and refuses a run whose GITHUB_WORKFLOW_REF is not
myriarch/myriarch/.github/workflows/release.yml@refs/tags/<tag>: a renamed workflow file or a call through a reusable workflow
would sign with another identity (see Never).
Windows signing rule
deploy/release/windows-signing.env holds two settings that the guard reads as data:
WINDOWS_SIGNINGisoff,signpathorkms. Any other value fails closed.WINDOWS_UNSIGNED_FINAL_TAGis empty or names one final tag, for examplev0.2.0.
Release candidates may use off. A final tag with WINDOWS_SIGNING=off passes only when WINDOWS_UNSIGNED_FINAL_TAG is exactly
that tag; an empty value, another tag or a pattern fails. A final tag with signpath or kms needs no exception. The exception
is therefore a reviewed change in the pull request that prepares the release, it covers one tag, and the next final release
needs signing or a new reviewed exception. The release notes of a release under the exception show an “unsigned” banner.
Run the guard locally
The guard needs a checkout of the tag with origin as the remote that has the tag and main, and a token. The test suite
ci/tests/versions-test.sh runs it against temporary repositories and a stub of the API:
ci/tests/versions-test.sh
Release build
The release build (deploy/release/goreleaser.yaml) turns a tagged commit into the archives, the bare updater binaries, one SBOM
per archive and one for the web UI, and checksums.txt. It holds no signing key and publishes nothing: the release workflow
signs keyless, and the Windows files get their Authenticode signature through a hook in stages (MYRIARCH_WINDOWS_STAGE), whose
mode comes from deploy/release/windows-signing.env (see Windows signing rule and
ADR-0034).
deploy/release/README.md is the reference for the build: the assets and the contents of
checksums.txt, the stages off, collect and apply, the contract of the signing hook (ci/release/windows-sign-hook.sh),
the acceptance check of signed files, the release notes template and the commands. Two commands run the build locally:
make release-snapshot # unsigned snapshot of the whole matrix
make release-rehearsal # the path of a release in a snapshot, with a throwaway signing CA
The release workflow
.github/workflows/release.yml runs when a tag v* is pushed, and only then. It builds the files, signs and attests them,
pushes the two images and the two Helm charts (the server chart and the collector chart, both signed like the images) by digest, creates a draft release, verifies everything with the commands that users run, and publishes
only after you approved a second time. The release becomes immutable on publication. After that, smoke jobs download and install
the release the way a user does.
| Job | What it does | Approval |
|---|---|---|
guard | Tag guard and meta.sh (above); the outputs of all later jobs. | |
signing-tools | Builds cosign, crane, gh and helm (and cosign, crane for other platforms) once from ci/tools on a runner of its own, before anything else is installed or run there; no cache. The only tools the privileged jobs execute. | |
build-tools | Builds goreleaser, syft, yq, go-winres and osslsigncode once from ci/tools; no cache. Only jobs with contents: read execute them. | |
ui | Builds the web UI (the only job with Node.js) and runs the release dependency and license gates. | |
build-windows | goreleaser build of the Windows targets in stage collect. | |
sign-windows | Runs the Windows signing hook (mode off in v0.1.0, v0.2.0 and v0.2.1). | Gate 1 |
package | goreleaser release --skip=publish in stage apply, then stages the 40 release files. | |
images | Builds the OCI layouts of both images from the release archives and their SBOMs. | |
sign-blobs | Signs checksums.txt keyless and attests the build provenance of its 39 files. | |
push-images | Pushes both images by digest (no release tags), signs them and attaches provenance and SBOM attestations. | |
chart | Packages the server chart with the digest of the server image written into image.digest and the collector chart with the digest of the updater image written into images.updater.digest (ci/release/chart-package.sh; chart version = app version = the tag without v; the two upstream images of the collector chart stay at the pins of ci/versions.env), points the relative links of each README to the files of the tag (Artifact Hub), checks the renderings, and turns each package into an OCI layout (exactly what helm push writes). It holds contents: read only and hands the layout on as an artifact. | |
push-chart | Pushes the layouts to ghcr.io/myriarch/charts/myriarch and ghcr.io/myriarch/charts/myriarch-collector by digest, without a tag, signs each keyless and attaches the provenance attestation, one chart after the other. It runs after push-images. The version tags of the charts are set by finalize, like the tags of the images. | |
draft | Renders the release notes and creates the draft release with 42 assets. It holds contents: write and runs before gate 2. | |
verify | Verifies signatures, provenance and images, and the files from which the draft was created (not the draft itself); writes the verified manifest and a summary. | |
verify-images | Runs both images on amd64 and arm64. | |
verify-chart | Verifies the pushed chart by digest: that the version tag does not exist yet (or, in a re-run after finalize, points at this digest), the chart layer (every image of the helm template renderings is the image of this run that the chart references: the server image, or for the collector chart the updater image and the two upstream pins; the README has no relative link), the signature and the provenance, for both charts. | |
verify-windows | Checks version resources and Authenticode state of the Windows files on a Windows runner. | |
finalize | Binds the draft to the verified files (GitHub’s asset digests), checks that the tag still points at the tagged commit and that all four packages (both images and both charts) are public, tags the images and the charts (X.Y.Z[-rc.N]), reads the draft again directly before it publishes, publishes the release, compares the published release with the verified files once more and checks that it is immutable. | Gate 2 |
smoke-assets, smoke-linux, smoke-windows, smoke-kind | Anonymous downloads and verification, installers on amd64 and arm64, Windows staging, docker run, the Helm chart on kind. |
Rules that hold for all of it:
- The signing identity is frozen. Every installed verifier checks
https://github.com/myriarch/myriarch/.github/workflows/release.yml@refs/tags/v*. The filerelease.ymlis never renamed, and signing never moves into a reusable workflow or another workflow (ADR-0035, point 10). - Build code never meets a privilege. Rule R13 of
ci/check-workflows.pykeeps Go, Node.js, pnpm, goreleaser and make out of jobs with a write scope,id-token,attestationsor an environment (Security rules of the workflows). Privileged jobs run only the scripts inci/release/. - Artifacts between jobs are tar files, downloaded by artifact ID and checked against the SHA-256 that the producing job reported. They are kept one day. This matters for gate 2 (below).
- Two approvals. In the release workflow, exactly the jobs
sign-windowsandfinalizeuse the environmentrelease. The only other job with that environment ispublishof.github/workflows/artifacthub.yml(Artifact Hub). Each of the two asks for its own approval. A gate is passed once per run, not once per attempt (see Re-runs), and onlyfinalizeis stopped by gate 2: the jobdraftholdscontents: writewithout the environment and runs before it. - The release workflow uses no secrets. The jobs use
GITHUB_TOKENand the OIDC token of the run.
Before the first release
These settings are made by a maintainer in GitHub, once. The workflow cannot check most of them, and an agent never changes them.
- Release immutability. Enable it for the organization (organization settings, Repository, General, Releases, “All
repositories”) or for the repository before you tag the first candidate. It applies only to releases published after you
enable it, and the job token cannot read the setting, so
finalizedetects a missing setting only after it published (release_not_immutable). - Environment
release. You are the required reviewer. Switch “Prevent self-review” off while you are the only reviewer. Administrators cannot bypass the protection rules. Deployment tags:v*only. No wait timer. The environment holds no secret. - Tag ruleset and Actions policy. Only you create
v*tags; they cannot be updated or deleted.GITHUB_TOKENis read-only by default, actions must be pinned to a full SHA, the allowed actions are the GitHub-owned ones plusossf/scorecard-action@*, and Actions cannot create or approve pull requests. - Packages. The organization allows public packages. Do not create
ghcr.io/myriarch/myriarch,ghcr.io/myriarch/myriarch-updateror the charts belowghcr.io/myriarch/charts/by hand: the first push of the first candidate creates them, and linking them to the repository happens through the image labelorg.opencontainers.image.source. - A second owner of the organization exists for emergencies (ADR-0029). Until a second person can approve gate 2, the two gates are a review step by one person.
First candidate: what to observe
Three things cannot be tested before the first candidate, and the first candidate is where they are settled. Record the result in the pull request of After every publication, or in a follow-up pull request if the re-run has not been observed yet.
- Run the kind smoke test once before you tag.
smoke-kindruns only after publication, where a failure can only be fixed forward, and no job ofci.ymlor of the release workflow runs the probe before the tag (e2e-kind.ymlrunsmake kind-probeevery night, outsideci-ok, and it is no gate of the release). Runmake kind-probeon a machine with a container runtime (the same cluster configuration and chart) or read the last green run ofe2e-kind.ymlon the commit, and fix what it finds before the tag.ci/release/smoke.sh kindis not a substitute: it always installs the published image of the tag, which does not exist before the tag. - Observe re-run approvals. After the first candidate went through gate 2, re-run one failed or completed job on a throw-away
candidate if you can, and write down whether “Re-run failed jobs” asks for gate 1 or gate 2 again. The runbook and the security
model state the safe reading (no new approval for the privileged jobs before
finalize) until you have observed it. - Read the
smoke-windowslog for the ACL ofcosign.exe. The job sets the owner and the ACL of the directory that holdscosign.exewithicacls(best effort) so thatTest-AdminOnlyDirectoryof the installer accepts it. Neither theicaclscalls nor that function have run on a Windows runner before the first publication, and no CI row tests them. If the log shows the warningicacls could not set the owner and the ACLor the installer stops withdirectory_unsafe, fix it forward and add a Windows CI step that creates an administrator-only and a user-writable directory and runs the real function on both.
The first published candidate is v0.1.0-rc.3 (v0.1.0-rc.1 and v0.1.0-rc.2 stopped before the publication). Its results:
- Kind:
make kind-probepassed before the first tag, andsmoke-kindpassed onv0.1.0-rc.3. - Re-run approvals: “Re-run failed jobs” after gate 2 re-ran only the failed row of
smoke-linux(HTTP 504 errors of downloads from github.com during a GitHub incident) and asked for no approval. No job of the environmentreleaseran again, so whetherfinalizeasks for gate 2 again after an approved attempt is still open, and the safe reading stays. - The directory of
cosign.exe:smoke-windowslogged noicaclswarning, and the installer acceptedC:\Program Files\cosignon the Windows runner (Test-AdminOnlyDirectoryruns on Windows in staging mode too). The refusal of a directory that unprivileged accounts can write is still tested only with test doubles.
Release a candidate
Every candidate, the first one included, goes through these steps. A candidate is a GitHub pre-release and never the latest release.
Prepare. Open a pull request that runs
scripts/set-version.sh X.Y.Z-rc.N(see One version source) and has the entries of the candidate under “Unreleased” inCHANGELOG.md. In the same pull request, or one merged before it, do the review of the texts: the archives cannot be corrected after the tag. Before you squash-merge it, check the contents of the release:- Changelog. Every change stands once, in its section. Changes to a security control are named under “Security”. The upgrade
notes come first and agree with the page that describes the upgrade from the last final release (for v0.2.0 that is
docs/operations/upgrade-to-v0-2-0.md): the migrations, the new and changed settings, the chart values and the installers. The notes of the release are rendered from this section, so what is not in it is not in the release. If the section has more than 60,000 characters, the notes keep only its “Upgrade notes” and “Security” subsections and link toCHANGELOG.mdfor the rest (toCHANGELOG.mdalone if those two are missing or too long as well), so these two stay complete and short enough. - Charts.
Chart.yamlof both charts (deploy/helm/myriarch,deploy/helm/myriarch-collector) has the version ofVERSION, andmake check-localrenders both with their schemas. The release packages the charts; it does not take them from the repository as they are. - Windows signing.
deploy/release/windows-signing.envsays what you expect (see Windows signing rule):offis allowed for a candidate.
- Changelog. Every change stands once, in its section. Changes to a security control are named under “Security”. The upgrade
notes come first and agree with the page that describes the upgrade from the last final release (for v0.2.0 that is
Wait for
ci-okon the merge commit ofmain. The guard refuses a commit without it.ci-okdoes not contain the two workflows that need real machines. They do not run on a push tomain(only on pull requests that touch their paths, on a schedule and by hand), so after the merge start both by hand (workflow_dispatchonmain,release_tagempty), or use a scheduled run that ran on the merge commit:host-e2e: the installers, the updater and the services on fresh Ubuntu 24.04 runners (amd64 and arm64) and on Windows Server 2025 (Windows PowerShell 5.1 and PowerShell 7). The scenarioone-commandruns everywhere;manualruns on Linux only.e2e-kind: the probe ofhack/kind, and from v0.2.0 the jobcharts(both charts on two Kubernetes versions, the uninstall and the upgrade from the chart of the last release) and the jobquickstart(the Kubernetes Quickstart from a checkout). A red row here is fixed before the tag: the smoke jobs after the publication cannot undo a release.
Tag the merge commit with an annotated tag and push it:
git tag -a vX.Y.Z-rc.N -m vX.Y.Z-rc.N <merge-commit-sha> # -s instead of -a when you have a signing key git push origin vX.Y.Z-rc.NApprove gate 1 (
sign-windows) in the run of the workflowrelease. Nothing is signed or pushed to a registry before you do.Review the draft. When
draft,verify,verify-imagesandverify-windowsare green, open the draft release and the summary of the jobverify, and check:- the draft has 42 assets and the pre-release flag (a candidate) is set.
verifychecked the files and images from which the draft was created, not the draft: the count is yours to check, andfinalizebinds the draft to the verified files by their digests; - the notes show the expected banners (the release candidate banner; the unsigned banner while
WINDOWS_SIGNING=off) and the right changes section (the “Upgrade notes” and “Security” subsections of “Unreleased”, and a link toCHANGELOG.mdfor the rest when the section is longer than 60,000 characters); - the digests of both images in the notes equal the digests in the summary of
verify; - both charts,
ghcr.io/myriarch/charts/myriarchandghcr.io/myriarch/charts/myriarch-collector, passedverify-chart(the job is green for each of them). They are pushed by digest and get their version tag only infinalize, so the version tag must not exist yet. Their digests are the ones you check again after the publication (step 8); - the summary names the identity
https://github.com/myriarch/myriarch/.github/workflows/release.yml@refs/tags/vX.Y.Z-rc.Nand the commit you tagged.
- the draft has 42 assets and the pre-release flag (a candidate) is set.
First release of a package only: make it public. After
push-imagesandpush-chartof the first release that publishes a package and before gate 2, an organization owner opens each new package (organizationmyriarch, Packages:myriarchandmyriarch-updatersince v0.1.0, andcharts/myriarchandcharts/myriarch-collector, the charts, which exist from v0.2.0 on; v0.2.0 is therefore the release for which this step is needed, for the two charts), then Package settings, “Danger Zone”, “Change visibility”, Public. This cannot be undone. While you are there, check that the package is connected to the repositorymyriarch/myriarchand that the repository has Write access in “Manage Actions access”.finalizefails withghcr_not_publicuntil all are public.Approve gate 2 (
finalize) within 24 hours of the start of the run, because the artifacts expire after one day (see Re-runs).finalizetags the images and the charts, publishes the release, and checks that GitHub made it immutable.Check the smoke jobs (
smoke-assets,smoke-linux,smoke-windows,smoke-kind). They run after publication. A failure here is fixed forward (see When something fails). Then check the two charts as a user does, on a machine without credentials: for each chart,crane digestof the version tag,cosign verifyandgh attestation verifyof that digest with the identity of the tag, as in Verify releases; the version tag isX.Y.Z-rc.N(nov), and the chart version andappVersionequal it.Test the published release on real hosts. In Actions, run the workflow
host-e2eby hand (workflow_dispatch, branchmain) withrelease_tagset to the tag. It runs only the scenarioone-command: it verifies thechecksums.txtof the release keyless and installs from the published installer bundle of the release, on Ubuntu 24.04 (amd64 and arm64) and on Windows Server 2025 in both PowerShell versions. A green run is the acceptance of the candidate on real runners; a red one is fixed forward in the next candidate. The run proves the bundle of the release, which the run of step 2 (a bundle built from the commit) cannot.Publish the chart’s metadata on Artifact Hub when the board wants the repository listed or verified: follow Artifact Hub. It needs the published chart and runs on the tag.
Update the status in a pull request after the publication, see After every publication, and decide about the release post.
Release a final version
A final release is a normal release and is marked as the latest release when it is the highest final tag. Its image tags are
X.Y.Z and the moving X.Y (which moves only forward; latest is never set while the major version is 0).
- Open the pull request “Prepare vX.Y.Z”:
scripts/set-version.sh X.Y.Z, the heading## [X.Y.Z] - YYYY-MM-DDinCHANGELOG.mdwith an empty “Unreleased” section above it, and, only for a reviewed exception,WINDOWS_UNSIGNED_FINAL_TAGindeploy/release/windows-signing.env(see Windows signing rule). Remove the exception again in the next pull request after the release. The same pull request turns the texts that describe the phase before the final release into texts for the final one: the “Status” section ofREADME.md, “Supported versions” inSECURITY.mdand the first paragraph of “Unreleased” inCHANGELOG.md(before v0.1.0, all three said that v0.1.0 is published first as release candidates).README.mdandCHANGELOG.mdare in the archives, so they change before the tag, not after it. - Windows signing decides before the tag. A final tag with
WINDOWS_SIGNING=offand no exception is refused by the guard. Before the pull request of step 1 is merged, the maintainers either setsignpathorkms(the signing then has to be set up and tested) or review the exception for that one tag, which puts an “unsigned” banner on the release notes. This page does not choose. The state at the time of writing:WINDOWS_SIGNING=offwith the exception forv0.2.1(the reviewed pull request “Prepare v0.2.1” sets it, ADR-0034 records it, as it records the exception ofv0.2.0). Without an exception,offis enough for release candidates only. - Follow steps 2 to 11 of Release a candidate with the tag
vX.Y.Z. Step 6 is not needed any more, except for a package that no release has published before (v0.2.0: the two chart packages). If the repository on Artifact Hub exists already, step 10 is skipped: the next scan lists the new version by itself, and the workflow runs again only when the metadata file has to change. - A final tag can be used once. If a problem shows up after publication, fix it and release
vX.Y.(Z+1).
Review the texts before every tag
A release archive is immutable. Some repository files are copied into it, and the text in them is read for as long as the release
exists, long after the sentence “no release has been published yet” stopped being true. Read the files below against the
“Unreleased” section of CHANGELOG.md and the changes since the last tag before you tag. deploy/release/goreleaser.yaml
(archives, files) is the authority; this list is its state at the time of writing:
| Archive | Repository files in it |
|---|---|
myriarchd_* (the server image is built from the myriarchd, myrctl and myriarch-sim archives) | README.md, LICENSE, THIRD_PARTY_NOTICES, deploy/helm/, deploy/compose/, deploy/docker/, examples/quickstart/ |
myrctl_* | README.md, CHANGELOG.md, LICENSE, THIRD_PARTY_NOTICES |
myriarch-updater_* | deploy/scripts/ (the installer of the platform), deploy/supervisor/, deploy/systemd/, deploy/updater/, deploy/windows/ (README.md; on Windows also Set-ServiceEnvironment.ps1), LICENSE, THIRD_PARTY_NOTICES |
myriarch-sim_* | examples/sim/, LICENSE, THIRD_PARTY_NOTICES |
The notes of a candidate are rendered from the “Unreleased” section of CHANGELOG.md (ci/release/notes.sh), so that section is
read as part of the release as well.
Compare, do not trust. Read
README.md, the pages ofdocs/and the website against the changelog and the commits since the last tag: commands, flags, defaults, version numbers, platforms, support levels and what is signed. A sentence that claims more than the release delivers (a signed Windows file, a signed macOS binary, an OCI chart) is wrong, whatever the plan says. Read the pages as they are deployed on https://myriarch.dev/ as well, not only their sources: the landing page,/docs/and/project/what-is-myriarch/are what a reader finds first, and a source that is right can still be deployed from an old build.Files in the archives describe the process. Every sentence must be true when the archive is created and still true after publication, until the next version. Write “release candidates are published as GitHub pre-releases”, not “no release has been published yet”, “available from v0.1.0” or “will be published”. Search the files of the table:
grep -rniE "no release|not (yet )?published|first release|available from|planned for|will be (published|released)|/v[0-9]+\.[0-9]+\.[0-9]+(-rc\.[0-9]+)?/|--branch v[0-9]" \ README.md CHANGELOG.md deploy examplesEvery hit is read, not only the ones that look wrong. Commands that fetch a file or a branch of a tag must not name a tag that does not exist yet: use a placeholder such as
v<version>.Other texts may name the state, but only if the update after publication changes them.
SECURITY.md,docs/and the website are not in the archives. Prefer wording that needs no edit; where a page must name the state, it is on the list of After every publication.Upgrade notes and support statements. The upgrade page, the “Upgrade notes” of the changelog, the support levels and the list of tested platforms (
docs/operations/tested-platforms.md) say the same, and what they say about migrations, settings and platforms is what the tests of this release cover. The notes of the release repeat the list of tested platforms.Run the gates for the texts:
make docs-checkandmake website-check.Stale promises.
ci/check-versions.shfails when a text outsidedocs/adr/,CHANGELOG.mdandchangelog.d/says “planned for” or “until” a version up toVERSION, or when a placeholder page hasplannedup toVERSION. Write the page or change the sentence in the pull request that prepares the release. The gate knows only these two phrases (with or without “the”, “release”, “version” or “of” and Markdown or HTML around the version, but not “v0.2”, “v0.2.x” or a range): step 1 still applies. It reports the file and the line; a false alarm is solved by rewording (“before v1.2.3 …” for the past), not by an exception list.
After every publication
When finalize is green and the smoke jobs have been checked, open a pull request that updates the status. These files are not in
the archives, so they can follow the release:
website/data/release.yaml, the single source of the release status. After a candidate setcandidate: "vX.Y.Z-rc.N"(andfinal: ""only while no final release exists; otherwisefinalkeeps the last final release); after a final release setfinal: "vX.Y.Z"andcandidate: "". The status chip of the landing page (website/layouts/_partials/landing/hero.html) is derived from it, andwebsite/scripts/check-landing.pyfails when the chip and the file differ. So is theversionin the structured data of the home page (website/layouts/_partials/seo/), whichwebsite/scripts/check-seo.pycompares with the file. A candidate of a later version does not replace a final release in the chip. The release workflow never writes this file (it has no write access to the repository), andREADME.mdanddocs/cannot read it: they are plain Markdown, which is why they describe the process.The pages that say whether images exist, name a release or state what the release contains:
docs/_index.md,docs/getting-started/_index.md,docs/getting-started/docker-compose.md,docs/getting-started/kubernetes-kind.md,docs/tasks/connect-a-host.md,docs/operations/install-helm.md,docs/operations/install-collector-helm.md,docs/operations/tested-platforms.md,docs/security/verify-releases.md,website/data/landing.yaml,website/data/llms.yaml,website/content/project/faq.md,website/content/project/comparison.mdandwebsite/content/project/what-is-myriarch.md. Check that every tag they name exists, that the commands that name a version use the new one, that their sentences about the images are true, and that the landing page andllms.txtname the core of the release. Then search the previous version and the wording that the release has ended, and read every hit (the ADRs are left out):PREV=0.1.0; NEW=0.2.0 grep -rnE "planned for v?${NEW}|[Bb]efore v${NEW}|not part of v${PREV}|VERSION=${PREV}|--version ${PREV}|first release|v${NEW}-rc\." \ docs deploy/helm website/content website/data ROADMAP.md SECURITY.md --exclude-dir=adrThe deferred observations: after the first candidate, write the results of the three observations of First candidate in this pull request, the one on re-run approvals also in
docs/security/model.md, and remove the wording that defers them. The kind result is known from before the tag and the result of thecosign.exedirectory from the log ofsmoke-windows; the re-run observation may need a throw-away candidate, so it can follow in a pull request of its own.The release post if it is due now, see Release post, and the website pages that name the release.
Texts in the archives (the files in the table of Review the texts before every tag, among
them README.md, CHANGELOG.md and the chart) are not touched here: they are right already, or the next release corrects them.
Other files of the repository, such as deploy/README.md, are not in an archive and can follow the release like the pages above.
Artifact Hub
Artifact Hub lists the server chart (oci://ghcr.io/myriarch/charts/myriarch) and shows it as “Verified publisher” when the chart
repository holds a metadata file under the tag artifacthub.io. The file carries the repository ID, which exists only after the board has
added the repository on Artifact Hub, and that needs a published chart. So the upload is not a job of the release workflow: a workflow of its
own, .github/workflows/artifacthub.yml, does it after the release, started by hand. Do the steps in this order:
- Publish the chart. The release of a candidate or of a final version has finished (
finalizeis green), so the chart repository has a version tag. The workflow refuses to run without one (chart_not_published). - Create the repository on Artifact Hub (the board, once). Sign in at https://artifacthub.io, accept the terms of use and open the control
panel. Add a repository of the kind “Helm charts” with the name
myriarch, the display nameMyriarchand the URLoci://ghcr.io/myriarch/charts/myriarch. - Copy the repository ID from the card of the repository in the control panel. It is a UUID in lower case.
- Run the workflow on the release tag. On GitHub open Actions, “artifacthub”, Run workflow, choose the release tag (for example
v0.2.0) under “Use workflow from”, and enter the ID. A branch,mainincluded, and any other tag are refused: the environmentreleaseadmits only tagsv*, and the script checks the tag shape again. The workflow file has to be onmainand in the tag; a fix to it takes effect with the next tag, because a tag is never moved. The jobtoolsbuilds the pinnedcrane; the jobpublishwaits for the approval in the environmentrelease(three jobs use that environment:sign-windowsandfinalizeof the release workflow andpublishof this one) and shows the digest of the uploaded artifact in its job summary. The ID becomes part of a public file, so enter nothing else. The workflow writes the tagartifacthub.ioofghcr.io/myriarch/charts/myriarchand nothing else. It can run again (it overwrites only that tag) and works the same whether the repository was added after a candidate or after the final version. - Check the result. After the next scan of the repository, Artifact Hub shows “Verified publisher” on the repository card (the control panel shows the time of the last scan and its errors).
The file has exactly one key, repositoryID. It has no owners; an owners entry would put an e-mail address into a public file, and
the board has to ask for it explicitly, in a change of its own.
The page of a chart on Artifact Hub shows the README.md of the chart package. The READMEs are written for GitHub and the website, with links relative to the file (../../../docs/operations/install-helm.md), which do not resolve on artifacthub.io. ci/release/chart-package.sh package therefore rewrites every relative link in the copy of the README that it packages, for both charts: a link to a file becomes https://github.com/myriarch/myriarch/blob/<tag>/<path>, a link to a directory https://github.com/myriarch/myriarch/tree/<tag>/<path>, and an anchor is kept. Links with a scheme, links to a heading of the README itself (#...) and text in fenced code blocks, code spans and HTML comments stay as they are. The packaging stops with chart_readme_link when a link names a path that the tagged commit does not have (the script reads the tree of the commit with git ls-tree, so the sparse checkout of the job chart is enough), leaves the repository, starts with /, is a relative image or has a form that the script cannot rewrite; chart-package.sh verify, and with it the job verify-chart, refuses a package whose README still has a relative link. The READMEs in the repository keep their relative links. A published chart keeps the README it was published with: the listing of 0.2.0 keeps its relative links.
Artifact Hub lists every version tag of the repository, release candidates included. A repository that is added after a candidate
therefore shows the candidate until the final version is published. Artifact Hub documents the chart annotation
artifacthub.io/prerelease for marking a version as a pre-release; the
release flow does not set it yet, so check how the listing shows a candidate after its scan.
Release post
The news section of the website (website/content/project/news/, described in the section “News” of website/README.md) comes with
the branch of the first post; it is not on main before. The post for a release is prepared
in advance on a branch of its own and is not merged before the release exists, because it names the version, the date and the
links of a published release. It carries open markers ([[PLACEHOLDER: …]], [[ONLY IF IN RELEASE: …]], [[ONLY FOR RC: …]]). Whether
the first post comes with a candidate or only with the final version is the maintainers’ decision; the steps are the same, and for a
candidate the post says that it is a release candidate.
- Rebase the branch on the current
origin/main. - Fill the markers. Version, date, the link of the release and of the chart (
oci://ghcr.io/myriarch/charts/myriarch). Keep a line marked “only if in release” if the release contains what it says (check it against the release notes) and delete it otherwise. The command in the section “News” ofwebsite/README.mdlists every open marker and must print nothing. Thenmake website-checkis green. - Review and merge on the day of the tag: a second pair of eyes reads the post against the release notes, and the pull request is merged once the release is published.
rc numbering and re-runs
- Every attempt that needs a code or content change is a new candidate (
rc.N+1), prepared by a pull request that bumpsVERSION. A tag is never moved, deleted or reused. - “Re-run failed jobs” is the answer to a transient failure (a runner, a network or a registry error, an outage of GitHub or
Sigstore). It re-runs only the failed jobs and the jobs that depend on them. A gate is passed once per run: a
sign-windowsthat already passed is not asked again, so the privileged jobssign-blobs,push-imagesanddraftrun again without a new approval, by anyone with write access to Actions. Whetherfinalizeasks again after an approved attempt is not documented by GitHub in a form we could confirm, and the first published candidate,v0.1.0-rc.3, did not re-run it (see First candidate). Stale drafts of the tag are deleted and replaced, a second push of the same digest changes nothing, duplicate signatures and attestations are harmless, andfinalizeresumes after a run in which it had already published. - The resume path of
finalize(an attempt above 1) accepts a release that is already published when its assets are exactly the verified files. It cannot tell a publication byfinalizefrom one made by hand between two attempts, which is forbidden (see Never). After a re-run, check in the log of the failed attempt thatfinalizeitself got as far as the PATCH. - Every run leaves valid credentials for the tag, also a failed or abandoned one and each “Re-run all jobs” attempt: the
signature bundle of
checksums.txt, the signatures and attestations of the images and the build provenance. Deleting a draft does not revoke them. Proof that a file was released isgh release verify-asset <tag> <file>, not a valid signature. - Artifacts expire after one day. If gate 2 waits longer than that, or a job has to be repeated after that, use “Re-run all jobs”: it builds a new draft and new signatures. The image digests stay the same, because the images are reproducible.
When something fails
| Job | Cause | What to do |
|---|---|---|
guard | A guard check or workflow_ref_mismatch | The message names the cause. Fix it in a pull request and tag the next candidate, or wait for ci-ok and re-run. |
signing-tools, build-tools, ui, build-windows, package, images | A build, a gate (dependencies, licenses, notices) or an artifact check failed | Fix it in a pull request and tag the next candidate. |
sign-windows | The signing provider failed (in mode off: only the copy of the files) | Re-run. A provider issue is handled in the provider’s change. |
sign-blobs, push-images | Sigstore or ghcr.io unavailable, a token problem | Re-run failed jobs. |
push-chart | Sigstore or ghcr.io unavailable, a token problem | Re-run failed jobs. The charts are pushed by digest and have no tag yet, so pushing the same digest again changes nothing and no manual deletion is needed. |
draft | release_exists or draft_foreign | A published release of the tag exists, or someone else made a draft of it: tag the next candidate (the next patch version for a final tag), or delete the foreign draft by hand and re-run. |
verify, verify-images, verify-chart, verify-windows | A verification failed | Do not approve gate 2. Fix forward with the next candidate. |
finalize, ghcr_not_public | A package is still private | Make all packages public (step 6 above), then “Re-run failed jobs”. |
finalize, image_tag_conflict | An immutable image tag points to another digest | Investigate. Never move an immutable tag. |
finalize, release_not_immutable | The release is published but not immutable | The release stays mutable. Enable release immutability before the next tag, and announce nothing until it is fixed. |
finalize, tag_moved | refs/tags/<tag> does not exist (any more) or does not resolve to the tagged commit | Nothing was published. The tag was moved, re-created or deleted, against the ruleset: investigate, do not approve again, tag the next candidate. |
finalize, draft_foreign, draft_ambiguous, release_not_draft | The release is not the draft that the workflow made, the tag has a second release, or the draft was published by hand; every re-read of the release repeats these checks | Nothing was published by this job (after the PATCH: see release_assets_changed). Investigate who changed the release, do not approve again, tag the next candidate. |
finalize, release_assets_changed | After the PATCH the release (now immutable) differs from the verified files | Do not use this release and do not announce it. It cannot be changed or deleted into shape; say in its notes that it is void and publish the next version. |
finalize, release_state, release_attestation, release_asset_attestation | The release is published, but GitHub does not (yet) show it as published or attested | “Re-run failed jobs”: finalize resumes because the attempt is above 1, tags nothing twice and repeats the checks. |
smoke-* | A problem found after publication | The release is immutable: fix forward in the next version and describe the problem in the release notes, which stay editable. A [download_failed] whose message says “retry window of 120 s used up” is an outage of github.com, not a problem of the release: use “Re-run failed jobs”. Two kinds of step have a policy of their own and can fail during a shorter outage: the installation of kind, kubectl and helm in smoke-kind (ci/install-tools.sh retries the download of kubectl for up to 120 s and a go build of kind or helm up to 4 times, 5, 10 and 20 s apart, each only after a transient network error; “retry window of 120 s used up” or a go build that failed four times with a network error is an outage of dl.k8s.io or of the Go module proxy), and the service checks of ci/release/smoke.sh (docker pull from ghcr.io, crane, cosign and gh: SMOKE_RETRIES attempts, 3 by default, 10 s apart). Re-run the job in this case too. |
Abandoned candidates. Delete the draft of a candidate that never went through gate 2 with
gh release delete <tag> --repo myriarch/myriarch --yes, without --cleanup-tag. Untagged image versions left by a failed
candidate may stay. Delete a package version only after you checked that no published release uses it, and never delete the
sha256-<hex> tags of a released digest: they hold the signatures and attestations that verification needs.
Never
- Publish the draft by hand.
finalizebinds the publication to the verified files. - Move, re-create or delete a tag, or delete a published release.
- Rename
.github/workflows/release.yml, call it as a reusable workflow, or move signing into another workflow. - Add
workflow_dispatchor any other trigger to the release workflow. - Delete
sha256-<hex>tags or package versions of a released image. - Change the settings of Before the first release from a script or an agent.