Snapd SRU Special Case Documentation
Introduction
This document explains how SnapD updates deviate from the standard SRU policy, outlining the background, key deviations, and the responsibilities of the SnapD, sponsor, and SRU teams in managing quality and risk. It replaces an earlier version that had become outdated and lacked sufficient detail.
Summary
SnapD is a core Ubuntu component that requires a special SRU process.
Key deviations from the standard SRU model include:
- Rolling release model: Each SRU delivers a full upstream release, typically 150–250 pull requests, about half of which are test-related.
- Re-execution model: If a newer SnapD snap is present, the SnapD deb will re-execute into it, ensuring the SnapD runtime is kept up to date with snaps as they are refreshed, but shifting rollback control to the snap store.
- Flexible timing: Updates may land outside normal freeze windows to maintain security and feature currency.
These deviations are necessary to keep Ubuntu systems secure, up-to-date, and consistent across distributions, while enabling developers to use new system capabilities without delay.
Risks are tightly managed through:
- A multi-layered QA process.
- The use of experimental feature flags for high-risk changes.
- Snap revert mechanism for urgent rollbacks.
This process ensures SnapD delivers at the pace required by its role, while maintaining the stability and trust expected of Ubuntu releases.
Background
SnapD is a background service that manages and maintains installed snaps. It is used on classic Ubuntu (and other distributions) as well as on Ubuntu Core. From the perspective of the SnapD team, a release usually involves both a deb and snap package to serve the needs of desktop, server, cloud and IOT use cases.
An important goal of the SnapD project is to always keep systems up-to-date with the latest security fixes, bug fixes and the latest features, while also enabling developers to access system resources that are initially restricted by interfaces but may be opened in a controlled manner to support new solutions. The SnapD value proposition is to allow installing robust software that can move at a different pace than distributions, in a rolling release model. In order to fulfill this, it needs to provide a security-maintained, consistent and up-to-date runtime environment for them across distributions and distribution versions. The SnapD design, development and release processes have been designed to minimize the risks associated with maintaining this level of “up-to-dateness”.
Classic systems
- Released as deb and snap packages.
- Deb package is used to seed classic Ubuntu images.
- Ubuntu (and some additional distros) supports and enables re-execution to SnapD provided by the snap in preference to SnapD provided by a deb if the SnapD snap version is bigger or equal to the SnapD deb version. This means reverts (and other release management relating to SnapD) makes use of store based snap revert as explained here, as opposed to falling under normal SRU processes for deb packages.
- SRU release targets all Ubuntu Releases in Standard Support (not ESM or Legacy Support).
- From Ubuntu Desktop 26.04 onwards, SnapD will be deeply integrated with secure boot on classic hybrid systems using TPM FDE.
Ubuntu Core systems
- Released as a snap package, therefore the SRU process is not relevant to this product.
- SnapD snap is used to seed Ubuntu Core images.
Exceptions
Categorization
SnapD falls into the following overlapping special types of SRU:
- Package-specific non-standard process:
This process deviates from the standard guidelines to accommodate its goal to always keep systems up-to-date with the latest bug fixes and features
- New bugfix-only upstream release:
Updates to SnapD may involve minor upstream releases (dot releases) that focus exclusively on fixing bugs
- New upstream release that adds features without breaking existing behaviour:
SnapD updates may also include upstream releases that introduce new features. These major updates are carefully designed to ensure backward compatibility. It is possible to gate new features using experimental flags that require user opt-in to be enabled. This is used when deemed necessary by the SnapD architect.
Interfaces enable controlled access to a specific set of system resources and other snaps. It regularly happens that new interfaces or extensions are necessary to empower creators with the system resources they need to develop and enhance their applications.
All SnapD releases, regardless of the exact category, are required to comply with the same thorough QA process.
Deviations requiring sign-off
SnapD requires the following deviations from what is acceptable to SRU to support its goal of always keeping systems up-to-date and enable creators to access not-yet-supported system resources.
Process
- Regular major releases, typically every eight weeks, are delivered as a single SRU bug. Each SRU aligns with a full upstream release - often encompassing 150 to 250 pull requests, approximately half of which are dedicated to tests additions and maintenance - and introduces complete or partial features alongside bug fixes and interface extensions.
- Regular minor releases, as required to fix bugs and low risk interface extensions
- Flexibility during the development cycle (LTS or otherwise) to release major or minor releases between feature freeze and BETA freeze. In general SnapD releases may land in stable Ubuntu releases at any time which makes strictly adhering to the feature freeze counterproductive.
- Flexibility with LTS point releases to release up to the release date minus 14 days.
- Sponsor reviews the SnapD source package and resulting deb packages in the SnapD Team owned `ppa:snappy-dev/image` and sponsors upload to -unapproved queue from where the SRU team takes over, including further reviews
- Sponsor, Ubuntu Release and SRU team not required to review every single pull requests (there are often 100+ for major releases)
Package behavior
- SnapD deb version >= 2.40 will automatically install the latest stable SnapD snap along with any non-essential (not: core, base, gadget, kernel or snapd) snap that does not itself require the Core snap. This ensures an up-to-date runtime and additionally enables the use of interfaces (PR-6778).
- SnapD deb version >= 2.66.1 will automatically install the latest stable SnapD snap along with any non-essential (not: core, base, gadget, kernel or snapd) snap. This is required to systematically remove the core snap upgrade path (PR-14173).
- If the SnapD snap version is greater than or equal to the SnapD deb version, the SnapD deb will re-execute to run the SnapD snap. Note that in the case of the same counterpart versions, where SnapD deb version is 2.66.1+24.04 and SnapD snap version is 2.66.1, re-execution will not happen because the postfix +24.04 makes the deb version the higher version.
- With no SnapD snap installed and Core snap installed, if the version of SnapD embedded in the Core snap is greater than or equal to the SnapD deb version, the SnapD deb will re-execute to the SnapD embedded in the Core snap.
- If SnapD snap is installed it will automatically refresh to the latest available version according to the autorefresh schedule.
Quality Assurance
- Complicated or risky changes/features requires a specification for approval by at least the product architect
- Compatibility mindset: any SnapD version should run on any classic Ubuntu release and be able to:
- Upgrade to any new version without breaking existing behavior
- Downgrade to any version without breaking behavior that also exists on the earlier version
- Each change must follow conduct, contribution and coding guidelines
- Each change must clearly indicate which support tickets it addresses. This includes Salesforce Support, Launchpad Bugs and Snapcraft Forum reports.
- Each change must be approved by at least 2 members of the SnapD team
- Each change must be fully tested at unit level
- Each feature must have up-to-date integration test
- Each change must pass required static analysis checks
- Each change must pass the required unit test for amd64 and arm64 on all supported classic Ubuntu releases for all corresponding versions of the Go toolchain
- Each change must pass required integration tests on all supported classic Ubuntu and Ubuntu Core releases. This includes:
- re-execution of SnapD deb to SnapD snap
- upgrade/downgrade from/to previous to candidate version
- Tests that cannot be automated are documented and manually executed when there are changes in the code that can affect the feature
- Features that are actively under development and carry uncertainty in terms of direction or risk must leverage the experimental flag system. This system ensures users must explicitly opt-in to activate and test such functionality. The SnapD architect is responsible for determining which features should be gated by experimental flags and for defining the duration of their experimental status. Existing experimental features must be documented.
- Release: Refer to the SnapD release testing diagram for the full validation flow that also covers the SnapD snap. This section focuses on the SnapD deb part. The diagram serves as a guideline and may change as required.
- Release notes: Release note comments must represent all PRs, excluding test and non-functional changes, and reference all the relevant Launchpad Bugs
- Release testing: The release PR must pass required static checks, unit tests (amd64, arm64) and integration tests (amd64, arm64) on all supported classic Ubuntu releases
- QA testing on SnapD PPA: SnapD debs must be staged to `ppa:snappy-dev/image` and pass required static checks, unit tests (amd64, arm64) and integration tests (amd64, arm64) on all supported classic Ubuntu releases. Apart from QA spread based unit tests, unit tests will also be run at package build time, thereby covering all architectures.
- QA testing on -proposed: SnapD debs must be staged to -proposed and must pass required static checks, unit tests (amd64, arm64) and integration tests (amd64, arm64) similar to development on all supported classic Ubuntu releases. Apart from QA spread based unit tests, unit tests will also be run at package build time, thereby covering all architectures.
- Autopkgtest on -proposed: SnapD debs must undergo autopkgtest testing (all architectures) for all supported classic Ubuntu releases.
- Launchpad Bug testing on -proposed: Each LP bug, relevant to Ubuntu (excludes e.g. other distros and snaps), should implement the SRU Bug template, most importantly the `Impact` and `Test Plan` and the rest as applicable. Each LP bug must be independently verified on all the targeted Ubuntu versions.
- SnapD snap testing: The tests required as part of the SnapD snap release validation must pass, with the exception of failures not applicable to the deb packages
- If a serious regression is discovered after release to -update or -release of the counterpart SnapD snap, and is confirmed by a member of the SnapD team, the SnapD team must request a SnapD snap revert as a matter of urgency. Due to re-execution, a deb package revert alone is not sufficient to solve this situation.
Refer to the SnapD release process for details.
Release Targets
- Development release (LTS and interim)
- All supported interim releases in standard support
- All LTS releases in standard support
Releases for the different targets must share the same source tree, with the only difference being the additional “backport” entry at the top of debian/changelog. See SnapD versioning.
Key Integrations and Interactions
Refer to the SnapD Key Integrations/Interactions on Classic Systems for more details.
- kernel: support for apparmor, seccomp, module loader, u-events, fuse module, etc.
- systemd: snap services, cgroups, mounting
- udev, libudev1: mediate device access
- dbus-x11, dbus-user-session: mediate bus access
- policykit-1: polkit policy
- apparmor: apparmor profiles
- squashfs-tools: packing of snaps
- squashfuse, fuse3, libfuse3-3: to mount squashfs in virtual environments
- mount: mounting and unmounting
- passwd, libc-bin: user creation and lookup
- openssh-client: device key generation
- ca-certificates: secure communications
- xdg-desktop-portals: portal support
- prompting-client, desktop-security-center: apparmor prompting support
- snapd-desktop-integration: SnapD notifications
- snap-store: app-center
Upload Process
Refer to the SRU Bug template for details about the required documentation.
Review/Sponsoring
The SnapD team requires the help of a dedicated Ubuntu Team member (core-dev) to copy the applicable source packages available in `ppa:snappy-dev/image` to -unapproved for all targeted Ubuntu releases. This must be explicitly requested by the SnapD release manager as shown in the SRU bug template. In the future when SnapD Team member(s) qualify as sponsors, this dependency may fall away.
Workflow and responsibilities
Preparation:
- SnapD Team uploads source packages to `ppa:snappy-dev/image`
- SnapD Team provides test results for `ppa:snappy-dev/image`
- SnapD Team provides autopkgtest results for packages in `ppa:snappy-dev/image`
- Sponsor reviews release candidates in `ppa:snappy-dev/image` and test results
Development release:
- Sponsor uploads to Development release -proposed
- SnapD Team provides test results for -proposed
- SnapD Team address autopkgtest failures if any (excuses) or retrigger tests as required
Interim and LTS releases:
- Sponsor uploads to Interim/Stable release -unapproved
- SRU Team reviews -unapproved
- SRU Team promotes to -proposed and applies blocks `block-proposed-<series>`
- SRU Team reviews -proposed
- SnapD Team provides test results for -proposed
- SnapD Team address autopkgtest failures if any (excuses) or retrigger tests as required
- SRU Team reviews test results and removes `block-proposed-<series>`
Review
It is generally not required to review all the individual pull requests, with the exception of packaging and service changes. A review should cover at least the following:
- Review that required documentation has been provided as per the SRU template
- Compare uploaded code to that of the respective release tag to confirm the only significant difference is the additional Go and C vendoring code in the package.
- Review the release notes
- Review the related Launchpad bugs indicated in the release notes. Each LP bug, relevant to Ubuntu (excludes e.g. other distros and snaps), should implement the SRU Bug template, most importantly the `Impact` and `Test Plan` and the rest as applicable. Each LP bug must be independently verified in -proposed.
- Review packaging and service changes
- Review areas of potential regressions
- Review results of all required release testing
- For all source packages for the different target releases
- Ensure builds for all architectures succeeded
- Source tarballs match
- Vendored dependencies match
- Changelogs for all targeted releases match the release notes and each other
SRU Template
Sponsoring Request
The following information is required from the SnapD release manager to initiate the upload process:
This is a new SnapD release.
SnapD [bugfix only] release 2.XX[.X] should be released to:
- [Development Release]
- [Latest LTS]
- ...
- [Oldest LTS]
The SnapD package deviates from the standard SRU process. The following special SRU process was followed: https://documentation.ubuntu.com/sru/en/latest/reference/exception-Snapd-Updates
Release preparation: < Github release PR URL >
Release preparation test results: < Github release PR test results URL >
Release notes: < Changelogs commit URL >
Launchpad bugs addressed: <URL(filter)>
Packaging and Service changes:
- [Packaging changes]
- [Service changes]
Content overview:
- [Features]
- [Bug fixes]
- [Creator support changes]
- [Other]
Areas of potential regressions:
- <Potential regression | <URLs to relevant PR(s)>
...
Source packages on `ppa:snappy-dev/image` for upload to -proposed:
- < Development Release URL>
- < Interim release URL>
- < Latest LTS URL >
- ...
- < Oldest LTS URL >
Validation already completed:
- Release PR test results: <URL to relevant PR >
- QA beta validation: <JIRA URL>
- SnapD deb testing on `ppa:snappy-dev/image`: <JIRA URL>
Validation required before release:
- Autopkgtests on -proposed for all targeted releases
- SnapD deb testing on -proposed
- SnapD snap testing
Final Test Feedback
The following updates from the SnapD QA engineer is required before considering releasing to -updates:
Autopkgtests on -proposed for all targeted releases: <RESULT>
SnapD deb testing on -proposed: <RESULT>
SnapD snap testing: <RESULT>
Open discussion points & board meeting prep (SnapD internal)
- Re-exec taking away revert control from Ubuntu SRU Team
- Robie B: Re-exec behavior takes away Ubuntu developers ability to revert a bad update effectively downgrading quality from Ubuntu users perspective.The claim is https://launchpad.net/bugs/2083890 took 36 hours to revert, this doc exception should include commitment to improve.
- Bug reported until release reverted: 11 hours and 58 minutes
- Robie B: Does not agree with revert statement “QA engineer must be informed and is required to request a SnapD snap” and advises that Ubuntu SRU Team should have control over decision to revert. One idea was SnapD deb revert that disables re-execution.
- Julian K: Proposed the use of SnapD tracks in accordance with this technical board approved specification that was approved after SnapD special case documentation acceptance that would give Ubuntu SRU team a way to target only SnapD on Ubuntu.
- Commitment towards test coverage improvement
- Meeting notes
- “Each bug fix that affects the user interface must have one QA review”
- We rely on test coverage during development
- We need to make a promise to always cover any regression
- SRU LP bug template dictates that we at least have to manually confirm all LP bug fixes
- "most of these tests are automated and executed as part of the autopkgtest suite"
- Our previous manager MVO agreed with Lukasz to move test coverage from autopkgtest to our spread based CI system, since the autopkgtest tests were unstable and took very long to run
- Captured in Review and improve autopkgtest coverage, for coming cycles.
- “when a new version is ready to be proposed, the QA team will perform extensive exploratory testing on the areas that will be changed by the release”
- Design phase problem, to think ahead about whether testing is required
- Agreed to not promise this and see what happens.
Document History