Release Process

This is intended to serve as documentation for Wasmtime's release process. It's largely an internal checklist for those of us performing a Wasmtime release, but others might be curious in this as well!

Releasing a major version

Major versions of Wasmtime are released once-a-month. Most of this is automatic and all that needs to be done is to merge GitHub PRs that CI will generate. At a high-level the structure of Wasmtime's release process is:

  • On the 5th of every month a new release-X.Y.Z branch is created with the current contents of main, and X.Y.Z-rc.1 is published as a release candidate.
  • On the 20th of every month this release branch is published to crates.io and release artifacts are built.

This means that Wasmtime releases are always at least two weeks behind development on main and additionally happen once a month. The lag time behind main is intended to give time to fuzz changes on main as well as allow testing for any users using main. It's expected, though, that most consumers will likely use the release branches of wasmtime.

Version numbers move through a lifecycle over the course of this process:

  • The main branch permanently carries a X.Y.Z-dev version. This version is never published anywhere and exists to make it obvious that a checkout of main is in-development.
  • Once a release branch is cut it moves to X.Y.Z-rc.1, which is published to crates.io and as a GitHub pre-release. Further release candidates may be cut as -rc.2, -rc.3, etc.
  • On release day the pre-release suffix is dropped and X.Y.Z is published.

A detailed list of all the steps in the release automation process are below. The steps requiring interactions are bolded, otherwise everything else is automatic and this is documenting what automation does.

  1. On the 5th of every month, (configured via .github/workflows/release-process.yml) a CI job will run and do these steps:
    • Download the current main branch
    • Push the main branch to release-X.0.0
    • Run the wasmtime-publish crate with the bump-major argument
    • Commit the changes
    • Push these changes to a temporary ci/* branch
    • Open a PR with this branch against main
    • Then, starting again from the commit the release branch was cut at, run the wasmtime-publish crate with the bump-rc argument to move to X.0.0-rc.1, add the tag-and-release marker to the commit message, and open a second PR against release-X.0.0
    • This step can also be triggered manually with the main branch and the cut argument.
  2. A maintainer of Wasmtime merges these two PRs
    • It's intended that both PRs can be immediately merged as all they're doing is bumping the version.
    • Merging the release branch PR publishes X.0.0-rc.1 following the same tag-and-publish path as a real release, except that the GitHub release is marked as a pre-release.
  3. Time passes and the release-X.0.0 branch is maintained
    • All changes land on main first, then are backported to release-X.0.0 as necessary.
    • If enough has changed to be worth testing again, a new release candidate can be triggered manually with the release branch selected and the release-rc argument. This opens a PR bumping to the next -rc.N; merging it publishes that candidate and yanks the previous one.
  4. On the 20th of every month (same CI job as before) another CI job will run performing:
    • Reset to release-X.0.0
    • Run the wasmtime-publish crate with the bump-drop-rc argument to drop the -rc.N suffix, leaving the version at X.0.0
    • Update the release date of X.0.0 to today in RELEASES.md
    • Add a special marker to the commit message to indicate a tag should be made.
    • Open a PR against release-X.0.0 for this change
    • This step can also be triggered manually with the main branch and the release-latest argument.
  5. A maintainer of Wasmtime merges this PR
    • When merged, will trigger the next steps due to the marker in the commit message. A maintainer should double-check there are no open security issues, but otherwise it's expected that all other release issues are resolved by this point.
  6. The main CI workflow at .github/workflow/main.yml has special logic at the end such that pushes to the release-* branch will scan the git logs of pushed changes for the special marker added by release-process.yml. If found and CI passes a tag is created and pushed.
  7. Once a tag is created the .github/workflows/publish-* workflows run. One publishes all crates as-is to crates.io and the other will download all artifacts from the main.yml workflow and then upload them all as an official release.

If all goes well you won't have to read up much on this and after hitting the Big Green Button for the automatically created PRs everything will merrily carry on its way.

Releasing a patch version

Wasmtime does not currently have a cadence for patch version nor a strict set of criteria. It's done on an as-needed basis. Requirements, however, are:

  • All changes must land on main first (if applicable) and then get backported to an older branch. Release branches should already exist from the above major release steps.

  • When a patch release is made it must be applied to all supported versions that need the patch. Wasmtime will not release a patch release until all versions have been equally patched to ensure that releases remain consistent.

  • Patch releases must not contain API-breaking changes in public crates. The list of PUBLIC_CRATES in crates/misc/publish/src/main.rs is the list of crates to worry about in terms of breaking changes.

  • Wasm-level ABI-breaking changes are by default not allowed in patch releases. Users are expected to, for example, be able to load *.cwasm artifacts from before the patch release is made. If an ABI-breaking change is necessary then be sure to update crates/wasmtime/src/engine/serialization.rs and the WasmtimeVersion matching strategy to modify the string to ensure older artifacts cannot be loaded.

Making a patch release is somewhat more manual than a major version, but like before there's automation to help guide the process as well and take care of more mundane bits.

This is a list of steps taken to perform a patch release for 2.0.1 for example. Like above human interaction is indicated with bold text in these steps.

  1. Necessary changes are backported to the release-2.0.0 branch from main
    • CI for supported branches is run weekly, even after release, to ensure that it's at most broken for a week. Nevertheless issues come up, so be aware that CI may be green on main but red on a release branch.
    • When merging backports maintainers need to double-check that the PUBLIC_CRATES listed in crates/misc/publish/src/main.rs do not have semver-API-breaking changes (in the strictest sense). All security fixes must be done in such a way that the API doesn't break between the patch version and the original version.
    • Don't forget to write patch notes in RELEASES.md for backported changes.
  2. The patch release process is triggered manually with the release-2.0.0 branch and the release-patch argument
    • This will run the release-process.yml workflow. The wasmtime-publish script will be run with the bump-patch argument.
    • The changes will be committed with a special marker indicating a release needs to be made.
    • A PR will be created from a temporary ci/* branch to the release-2.0.0 branch which, when merged, will trigger the release process.
  3. Review the generated PR and merge it
    • This will resume from step 6 above in the major release process where the special marker in the commit message generated by CI will trigger a tag to get pushed which will further trigger the rest of the release process.
    • Please make sure to update the RELEASES.md at this point to include the Released on date by pushing directly to the branch associated with the PR.

Releasing a security patch

For security releases see the documentation of the vulnerability runbook.

Releasing Notes

Release notes for Wasmtime are written in the RELEASES.md file in the root of the repository. Management of this file looks like:

  • (theoretically) All changes on main bundle an appropriate update of RELEASES.md. In practice this almost never happens.
  • When the main branch gets a version bump the RELEASES.md file is emptied and replaced with ci/RELEASES-template.md. An entry for the upcoming release is added to the bulleted list at the bottom.
  • (realistically) After a release-X.Y.Z branch is created release notes are updated and edited on the release branch directly.

This means that RELEASES.md only has release notes for the release branch that it is on. Historical release notes can be found through links at the bottom to previous copies of RELEASES.md

Keeping Old release branch CI up-to-date

Over time CI configuration goes out of date and may need to be updated. The Wasmtime repository has a cron job via GitHub Actions to run release CI on all supported release branches on a weekly basis to try to weed out these problems. If a release branch CI fails it'll open an issue and maintainers should resolve it expediently.

Where possible old release branch CI should not update software to fix CI. Try to pin to older versions if something wasn't pinned already for example. Sometimes though updates are inevitable and may be required.