How to update Rust¶

This guide details the process of creating a new versioned rustc Ubuntu package for a new upstream Rust release.

To see the process of backporting Rust, consult the How to backport Rust guide instead.
To see the process of fixing an existing Rust package, consult the How to patch Rust guide instead.

Attention

This is not a guide for updating your system’s Rust toolchain. This guide is intended only for Ubuntu toolchain package maintainers seeking to add new Rust versions to the Ubuntu Archive.

Background¶

The rustc source package, which provides binary packages for the Rust toolchain, is a versioned package. This means that a new source package is created for every Rust release (e.g., rustc-1.83 and rustc-1.84).

These packages are maintained largely in order to support building other Rust packages in the Ubuntu archive. Rust developers seeking to work on their own Rust programs typically use the rustup snap instead.

The default Rust toolchain version used to build Rust packages in the archive is denoted using the rust-defaults package.

High-Level Summary of the Update Process¶

A typical rustc update goes through the following steps:

The source code of the new Rust version is downloaded from upstream, overwriting the old upstream source code.
The existing package patches are refreshed so they apply properly onto the new Rust source code.
Unnecessary vendored dependencies are pruned.
The upstream Rust source is re-downloaded with the new list of files to yank out, overwriting the source code once again, but with the unnecessary files removed.

Substitution Terms¶

From now on, the documentation will contain certain terms within angle brackets which must be replaced with the actual value that applies to your situation.

As an example, let’s assume you are updating rustc-1.84 (upstream version 1.84.0) to rustc-1.85 (upstream version 1.85.1) for Noble Numbat:

<X.Y>: The short Rust version you’re updating to.
- Example: 1.85
<X.Y.Z>: The long Rust version you’re updating to.
- Example: 1.85.1
<X.Y_old>: The short Rust version you’re updating from.
- Example: 1.84
<X.Y.Z_old>: The long Rust version you’re updating from.
- Example: 1.84.0
<release>: The target Ubuntu release adjective.
- Example: noble
<lpuser>: Your Launchpad username. This is also used to refer to your personal Launchpad Git repository’s remote name.
<foundations>: Your local Git remote name for the Foundations rustc Git repository.
<lp_bug_number>: The number of the Launchpad bug associated with this upload.

Setting up the Repository Locally¶

This only needs to be done once when setting up a machine for Rust toolchain maintenance for the first time.

Project directory structure¶

Since the Debian build tools generate files in the parent directory of your package source directory, it’s recommended to keep things organized by placing the cloned repository inside of a fresh directory of its own.

Clone the repository inside an existing rustc directory so your file structure looks like the following:

rustc
├── rustc
│   ├── [...]
│   ├── Cargo.lock
│   ├── Cargo.toml
│   ├── debian
│   └── [...]
└── rustc-<...>.orig.tar.xz

Naturally, your higher-level rustc directory won’t have any .orig.tar.xz files yet, but they will be stored there once you start working on the package.

Cloning the Git repository¶

The main repository for all versioned Rust toolchain packages is the Foundations Launchpad Git repository. A branch exists for every single upstream release and backport and serves as a central place to store all Rust toolchain code, regardless of which versioned package a particular branch belongs to.

Clone the Foundations Git repository within your existing parent directory:

$ git clone git+ssh://<lpuser>@git.launchpad.net/~canonical-foundations/ubuntu/+source/rustc

Then, create your own personal Git repository on Launchpad:

$ git remote add <lpuser> git+ssh://<lpuser>@git.launchpad.net/~<lpuser>/ubuntu/+source/rustc

Generally, it’s recommended to use your personal Git repository as a remote backup throughout the process — the update procedure involves multiple rebases, so it’s best to wait pushing to the Foundations repository until you’re done.

The Rust Update¶

This section details the process of the actual Rust package update, which is repeated every time a new upstream Rust release comes out.

Creating a Bug Report¶

In order to publicly track the progress and status of the update, you must create a bug report on Launchpad. The type of bug report you create depends on whether or not this Rust version will be the default for the current devel release.

Bug report for the default release¶

If this Rust version is the target rust-defaults version for devel, then you should create a bug under rust-defaults. This is because you will eventually need to update rust-defaults to point to this new Rust version. As an example, consult this real-life rust-defaults bug report here.

Bug report for non-default releases¶

If this Rust version is not the target default for devel, then the process is closer to adding a new package to the archive in general. You can create a general Ubuntu bug tagged with needs-packaging and Wishlist importance. A real-life bug report for rustc-1.86 can be found here. Notice that it is targeted to the appropriate series and tagged accordingly.

Setting Up¶

You’re now ready to start work on the next Rust version. To begin, you must create a new Git branch titled merge-<X.Y>. All new Rust branches for devel use this naming convention.

First, make sure you’re on the previous version’s branch:

$ git fetch --all
$ git checkout merge-<X.Y_old>

Create your new branch, then upload it to your personal Git repo:

$ git checkout -b merge-<X.Y>
$ git push <lpuser> merge-<X.Y>

Getting the New Upstream Rust Source¶

In this step, you get the source code of the new Rust version. The watch file, debian/watch, automates this process.

Removing the old vendor prune list¶

First, understand that a great deal of unnecessary vendored crate dependencies must be pruned from the vendor directory.

We tell the Debian packaging tools which files from the Rust team’s source code it can ignore in the copyright file, debian/copyright. The list of ignored vendored dependencies is huge — you’ll automatically generate that list later. For now, we will include all vendored dependencies from the new Rust version to ensure we don’t miss anything.

To do this, remove the DO NOT EDIT (...) AUTOGENERATED chunk from debian/copyright:

- # DO NOT EDIT below, AUTOGENERATED
-  vendor/addr2line-0.17.0
-  vendor/aes-0.8.4
-  vendor/ahash-0.8.10
- [...]
-  vendor/zerocopy-derive-0.8.14
-  vendor/zeroize_derive-1.4.2
- # DO NOT EDIT above, AUTOGENERATED

Updating the changelog and package name¶

You can also update the changelog at debian/changelog, manually setting the version number:

$ dch -v <X.Y.Z>+dfsg0ubuntu0-0ubuntu0

Important

The changelog version string is complicated. It’s strongly recommended to consult the Rust version strings article before making this change to ensure you understand the version string.

Even though the ubuntu suffix in version strings starts at 1, the fact that both suffixes are ubuntu0 here is not a typo. This version string will not be added to the archive — it is simply an interim number to be used temporarily until we’re finished repacking the tarball.

Don’t forget to manually change the versioned package name in the changelog too! (i.e., rustc-<X.Y_old> -> rustc-<X.Y>)

You can also create your first changelog bullet point — the “New upstream version” point. It should look something like the following; consult previous changelog entries for examples:

* New upstream version <X.Y.Z> (LP: #<lp_bug_number>)

Important

Make sure <lp_bug_number> matches the bug number you created earlier!

Getting the new source and orig tarball with uscan¶

We can use uscan(1) (from the devscripts package) to get the new source code and generate the new orig tarball.

The log of this should be saved somewhere because uscan will warn you if any files you’ve excluded from debian/copyright aren’t actually in the original source. You must consult the upstream Rust changes and see what happened to that file, updating debian/copyright accordingly depending on if it was removed, renamed, or refactored.

Download the orig tarball from the upstream Rust source, yanking out all excluded files:

$ uscan --download-version <X.Y.Z> -v 2>&1 | tee <path_to_log_output>

This process can take a while. Once it is complete, you will find a file with an .orig.tar.xz suffix in your parent rustc directory. That is your orig tarball. It contains the new upstream source code for the new Rust version.

You must then rename the orig tarball to match the first part of your package version number, i.e., rustc-<X.Y>_<X.Y.Z>+dfsg0ubuntu0.

Note

This 0ubuntu0 is semantically and functionally important. It not only says “I am not finished repacking the orig tarball yet”, but if you were to start with 0ubuntu1 instead, you’d run into problems in later steps when trying to overlay an updated tarball using the same version number as before.

Updating the source code in your repository¶

uscan just downloads the new Rust source, yanks out the ignored files, and packs the orig tarball. Your actual Git repository hasn’t changed at all yet. To do that, we can use gbp to import the new Rust version onto your existing repository.

Important

This is point at which your actual source code moves from <X.Y.Z_old> to <X.Y.Z>.

We use the experimental branch to store the upstream releases. Normally, this would be where the Debian experimental branch is, but we can’t use that because our Rust package is not downstream from Debian. Make sure you reset this branch beforehand, branching it off from your current Git branch:

$ git branch -D experimental
$ git branch experimental

Now, we’re ready to invoke gbp:

$ gbp import-orig \
    --no-symlink-orig \
    --no-pristine-tar \
    --upstream-branch=experimental \
    --debian-branch=merge-<X.Y> \
    ../rustc-<X.Y>_<X.Y.Z>+dfsg0ubuntu0.orig.tar.xz

Afterwards, you should now see two commits in your Git log stating that your upstream source has been updated.

Initial Patch Refresh¶

Now that the actual upstream source code has changed, many of the different patches in debian/patches won’t apply cleanly. In this step, you must fix these patches.

Getting the next patch to refresh¶

To identify the next patch that fails to apply, try to push all the patches:

$ quilt push -a

quilt will then stop applying patches right before the patch that fails to apply. You can then force-apply this patch, displaying the conflicts in a Git-style merge conflict format:

$ quilt push -f --merge

The response you get from this command will list all of the patch components which failed to apply.

There are several common reasons why a patch fails to apply. These reasons are listed below.

Surrounding code changed¶

The easiest situation is that the surrounding code was changed.

In this case, you must simply verify that the changes don’t impact the patch itself before re-applying it to this new context.

Patch implemented upstream¶

Sometimes, a patch has changes that were incorporated upstream.

In this case, you can drop the patch entirely by deleting the .patch file itself and removing it from debian/patches/series.

Vendored dependency changed¶

Whenever a vendored dependency gets updated, patches which apply to vendored dependencies won’t be able to find their target files because the file paths have changed.

The easiest solution in this case is to manually edit the target file paths within the .patch file, then try to reapply the patch and solve any remaining conflicts from there.

Targeted code refactored¶

The most difficult case to deal with is when the code impacted by a patch is refactored or replaced entirely.

In this case, you must consult the upstream changes to figure out what replaced it, then act accordingly, modifying or dropping the patch if warranted.

Remember, your goal here is to preserve the intent of the patch. If, for example, a patch disables certain tests which require an internet connection, and those tests get refactored completely, it’s your responsibility to track down the tests which require an internet connection and disable them accordingly.

Pruned vendored dependency patch exception¶

Finally, debian/patches/prune/d-0021-vendor-remove-windows-dependencies.patch is a special case. Many of its changes will fail to apply because their targeted vendored crates have changed versions.

You will later have to manually update this patch, so you may simply refresh this patch after force-applying it, dropping all the missing vendored crates.

Pruning Unwanted Dependencies¶

As mentioned above, we don’t want to include unnecessary dependencies, especially Windows-related crates like windows-sys. This pruning ensures adherence to free software principles, reduces the attack surface of the binary packages, and reduces the binary package size on the end user’s hard drive.

Since we removed the autogenerated debian/copyright chunk earlier before getting the upstream source with uscan, our vendor directory will contain everything. We must now remove the dependencies on things we don’t need.

You must prune the unwanted dependencies of both the Rust source code itself and its vendored dependencies.

Get a list of things to prune¶

To assist in this task, you can use two different scripts. They search Cargo.toml files for potentially unwanted terms, then output all the lines which contain said terms.

Script for pruning the Rust code itself. This script needs debian/patches/prune/d-0020-remove-windows-dependencies.patch to be the topmost patch.
Script for pruning the vendored dependencies. This script needs debian/patches/prune/d-0021-vendor-remove-windows-dependencies.patch to be the topmost patch.

Push the proper patch and redirect the output to a file. Example for vendored dependencies:

$ quilt push debian/patches/prune/d-0021-vendor-remove-windows-dependencies.patch
$ win-rustc-vendored-prune-list > <path_to_prune_list>

Note

The majority of this list will be taken up by lines from the Cargo.tomls of various versions of the Windows crates like windows, windows-sys, etc. You don’t need to prune these! Basically, if you know that a given crate will be removed, you don’t need to prune its Cargo.toml.

Prune the Cargo.toml files¶

Your top priority will be removing any dependencies which pull in windows-sys, winapi, ntapi, windows, etc. Go through the prune list you just generated and inspect the flagged lines to see whether or not it’s something that should be pruned.

Here’s an example of something which should definitely be pruned:

--- a/vendor/dbus-0.9.7/Cargo.toml
+++ b/vendor/dbus-0.9.7/Cargo.toml
@@ -63,9 +63,5 @@
 stdfd = []
 vendored = ["libdbus-sys/vendored"]

-[target."cfg(windows)".dependencies.winapi]
-version = "0.3.0"
-features = ["winsock2"]
-
 [badges.maintenance]
 status = "actively-developed"

In this example, the vendored dbus-0.9.7 crate pulls in winapi as a dependency on Windows targets. We obviously aren’t targeting Windows, so we can delete this whole chunk.

Here’s another example of something which should be deleted:

--- a/vendor/opener-0.7.2/Cargo.toml
+++ b/vendor/opener-0.7.2/Cargo.toml
@@ -48,7 +48,6 @@
 reveal = [
     "dep:url",
     "dep:dbus",
-    "windows-sys/Win32_System_Com",
 ]

 [target.'cfg(target_os = "linux")'.dependencies.bstr]
@@ -62,16 +61,5 @@
 version = "2"
 optional = true

-[target."cfg(windows)".dependencies.normpath]
-version = "1"
-
-[target."cfg(windows)".dependencies.windows-sys]
-version = "0.59"
-features = [
-    "Win32_Foundation",
-    "Win32_UI_Shell",
-    "Win32_UI_WindowsAndMessaging",
-]
-
 [badges.maintenance]
 status = "passively-maintained"

In this case, the reveal crate feature relies on something from windows-sys, so we also remove that line. We know that the reveal feature doesn’t actually need windows-sys/Win32_System_Com on non-Windows targets, because windows-sys is a conditional dependency, so it’s safe to remove that line.

Here’s an example of something that win-rustc-vendored-prune-list picks up that you don’t want to remove:

[target.'cfg(any(unix, windows, target_os = "wasi"))'.dependencies.getrandom]
version = "0.3.0"
optional = true
default-features = false

win-rustc-vendored-prune-list just sees windows and flags it, so it’s your job to recognize that it’s also required for unix.

Finally, if you’re not sure about how/if you should prune something, you can take a look at versions of the patch from earlier Rust updates to see if a different version of the vendored crate has been pruned before. Here’s an example of a patch with a large list of vendored crates.

Note

You may notice that windows-bindgen and windows-metadata aren’t included in the exclusion list — they don’t pull in windows-sys and friends, and (at least in earlier versions) they’re necessary for the build process, so it’s not the end of the world if they don’t get pruned.

That said, it’s been possible to prune windows-metadata from more recent rustc packages, so we may potentially be able to consistently prune both in the future. More research is needed on this topic.

Final manual crate checks¶

While the above content focuses on removing Windows dependencies, there are a few specific libraries we target for pruning. Review the following patches, and make sure that nothing else is trying to use the libraries they target:

prune/d-0005-no-jemalloc.patch: tikv-jemalloc-sys and jemalloc-ctl
prune/d-0011-cargo-remove-nghttp2.patch: libnghttp2-sys

Removing Unused Dependencies¶

Once you’ve removed all Cargo.toml lines which pull in unnecessary vendored dependencies, you’re ready to remove said dependencies from the orig tarball and vendor directory entirely.

prune-unused-deps¶

Previous Rust maintainers have been kind enough to create a script for this purpose: debian/prune-unused-deps. This script locates the unneeded vendored crates and adds the list to the Files-Excluded field of d/copyright.

Note

The autogenerated chunk generated by prune-unused-deps is an updated version of the one you deleted at the start of the update process!

In order to use the script, you need the previous Rust toolchain to bootstrap. It’s easiest to get the toolchain using the rustup snap:

$ rustup install <X.Y.Z_old>

You’re now ready to run the script, pointing it to your Rust toolchain:

$ RUST_BOOTSTRAP_DIR=~/.rustup/toolchains/<X.Y.Z_old>-<arch>-unknown-linux-gnu/bin/rustc \
    debian/prune-unused-deps

If you have issues running prune-unused-deps due to features requiring “nightly version[s] of Cargo”, set RUSTC_BOOTSTRAP=1 at the cargo update command within debian/prune-unused-deps:

--- a/debian/prune-unused-deps
+++ b/debian/prune-unused-deps
@@ -24,7 +24,7 @@ done
 find vendor -name .cargo-checksum.json -execdir "$scriptdir/debian/prune-checksums" "{}" +

 for ws in $workspaces; do
-       (cd "$ws" && cargo update --offline)
+       (cd "$ws" && RUSTC_BOOTSTRAP=1 cargo update --offline)
 done

 needed_crates() {

Note

It’s possible the above change can be made permanent, but such a change should be discussed as a team before making that decision.

Committing the right changes¶

After running prune-unused-deps, there will be many changes, almost none of which you actually need.

First, if you needed to edit prune-unused-deps earlier, restore it:

$ git restore debian/prune-unused-deps

Next, commit the changes to debian/control anddebian/source/lintian-overrides. Their version numbers have been updated.

Double-checking your pruning¶

After that, consult the new autogenerated block underneath the Files-Excluded field of debian/copyright. This lists all the crates within vendor that aren’t needed for the source package build.

Make sure that the Windows crates you pruned earlier are included within that list. If they aren’t in that list, you need to go back, check for anything that pulls in those dependencies, and prune them.

I recommend comparing your new list with the previous version’s list as well:

$ git diff merge-<X.Y_old> -- debian/copyright

You shouldn’t see many dramatic changes. Remember, anything removed from the diff means that the crate used to be excluded, but isn’t anymore. This often just means that the version number changed. In this case, you can usually see a crate with the same name and a different version number that’s new to the Files-Excluded list.

Once you’ve checked over your new list of excluded vendored crates, you can commit your debian/copyright changes, restore your Git repository from all the other changes, and continue.

Removing Vendored C Libraries¶

Unlike C, Rust doesn’t have a stable ABI, meaning that dependencies (generally) must be statically linked to the binary. An excellent article by a Gentoo maintainer goes more in depth regarding the conflicts between Linux packaging and static dependencies.

This is relevant because while we must vendor Rust dependencies, we don’t have to vendor the C libraries included within some vendored crates. This can be seen in the Files-Excluded field of debian/copyright:

Files-Excluded:
 ...
# Embedded C libraries
 vendor/blake3-*/c
 vendor/curl-sys-*/curl
 vendor/libdbus-sys-*/vendor
 vendor/libgit2-sys-*/libgit2
 ...

Subdirectories within vendored crates are being pruned from the orig tarball. They are replaced by the system C libraries, thoughtfully provided in the Ubuntu archive. Considering the exclusion of vendor/libgit2-sys-*/libgit2 above, we can consult debian/control:

Build-Depends:
 ...
 libgit2-dev (>= 1.9.0~~),
 libgit2-dev (<< 1.10~~),
 ...

These two changes form the basis of removing vendored C dependencies.

Find vendored C dependencies¶

Search for C source files within the vendor/ directory:

$ cd vendor
$ fdfind -e c

You can now check this list and figure out which of these are bundled C libraries. However, remember that you just pruned some of these! You can cross-reference this list with your autogenerated Files-Excluded field from d/copyright. You only have to worry about the vendored C libraries which aren’t contained within that list.

Removing C dependencies from the next orig tarball¶

Naturally, the process of pruning a vendored C library varies from library to library. As an example, we will use a removal of the bundled oniguruma library from rustc-1.86, which caused some build failures when it wasn’t removed.

Next time we run uscan, we want to make sure that the bundled C libraries we want to remove aren’t included. To do that, simply add the C library directory to Files-Excluded in debian/copyright:

--- a/debian/copyright
+++ b/debian/copyright
@@ -50,6 +50,7 @@ Files-Excluded:
  vendor/libsqlite3-sys-*/sqlcipher
  vendor/libz-sys-*/src/zlib*
  vendor/lzma-sys*/xz-*
+ vendor/onig_sys*/oniguruma
 # Embedded binary blobs
  vendor/jsonpath_lib-*/docs
  vendor/mdbook-*/src/theme/playground_editor

Adding the system library as a build dependency¶

We can’t remove a C library needed by a vendored dependency without providing a proper equivalent of said library in its place. Instead, we can use the oniguruma Ubuntu package, libonig-dev. We do this by adding the package to Build-Depends in d/control AND d/control.in:

--- a/debian/control
+++ b/debian/control
@@ -37,6 +37,7 @@ Build-Depends:
  libgit2-dev (<< 1.10~~),
  libhttp-parser-dev,
  libsqlite3-dev,
+ libonig-dev,
 # test dependencies:
  binutils (>= 2.26) <!nocheck> | binutils-2.26 <!nocheck>,
  git <!nocheck>,

--- a/debian/control.in
+++ b/debian/control.in
@@ -37,6 +37,7 @@ Build-Depends:
  libgit2-dev (<< 1.10~~),
  libhttp-parser-dev,
  libsqlite3-dev,
+ libonig-dev,
 # test dependencies:
  binutils (>= 2.26) <!nocheck> | binutils-2.26 <!nocheck>,
  git <!nocheck>,

Making the vendored crate use the system library instead¶

In all likelihood, you’ll need to adjust the vendored crate so it knows to use the system library instead of the bundled one. This can vary greatly, but it usually involves patching the crate’s Cargo.toml or build.rs, so look in those places first.

In the case of onig_sys, we can simply patch it to use the system library by default:

--- a/vendor/onig_sys-69.8.1/build.rs
+++ b/vendor/onig_sys-69.8.1/build.rs
@@ -219,7 +219,7 @@

 pub fn main() {
     let link_type = link_type_override();
-    let require_pkg_config = env_var_bool("RUSTONIG_SYSTEM_LIBONIG").unwrap_or(false);
+    let require_pkg_config = env_var_bool("RUSTONIG_SYSTEM_LIBONIG").unwrap_or(true);

     if require_pkg_config || link_type == Some(LinkType::Dynamic) {
         let mut conf = Config::new();

Updating the Source Tree Again¶

At this point, you’ve updated the list of files which must be excluded from the upstream source. This means that you need a new upstream tarball and a new Rust source.

We download the upstream Rust source code using uscan again, which will use your new debian/copyright exclusion list to yank out everything you just pruned:

$ uscan --download-version <X.Y.Z> -v 2>&1 | tee <path_to_log_output>

Since we’ve now updated the orig tarball repack, we can rename this new orig tarball by making its last number 1, i.e., rustc-<X.Y>_<X.Y.Z>+dfsg0ubuntu1, symbolizing that we’ve repacked the source once again.

We can also update our debian/changelog version number to <X.Y.Z>+dfsg0ubuntu1-0ubuntu1. Make sure that the portion of the version number before the hyphen matches the new orig tarball. You can use dch -r to automatically bump the changelog timestamp.

Keeping the tree clean¶

To keep the Git tree clean, we must rebase all our changes on top of the newly-packed orig tarball.

First, make a backup just to be safe:

$ git branch backup

Next, start an interactive rebase, dropping the gbp commit where we imported the upstream source:

$ git rebase -i merge-<X.Y_old>

Here’s an example of what the 1.87 rebase todo looked like. Note that we took the opportunity to clean up the Git tree in general: squashing the large list of vendored pruning commits into one, dropping the initial removal of the old autogenerated Files-Excluded chunk, etc.:

drop f24d250953b Remove autogenerated Files-Excluded chunk
pick 222264f2705 Create 1.87.0 changelog entry
drop 93acc98c031 New upstream version 1.87.0+dfsg0ubuntu0
pick ec0e3f69232 Refresh upstream/u-ignore-ppc-hangs.patch
squash 7c73c0c7640 Refresh upstream/u-hurd-tests.patch
squash a7aecdca8cc Refresh upstream/d-ignore-test_arc_condvar_poison-ppc.patch
[...]
squash 8269adcb6d8 Refresh ubuntu/ubuntu-enzyme-use-system-llvm.patch
pick d045017bed6 d/p/series: Drop u-mdbook-robust-chapter-path.patch applied upstream
pick 960395225de Prune termcolor-1.1.2
squash 0c8624920e7 Prune ring-0.16.20
squash b5316157d1f Prune mio-1.0.1
[...]
squash d0696832bf1 Prune openssl-sys from libssh2-sys-0.2.23
pick 1239a9b4309 d/control, d/s/lintian-overrides: Update versioned identifiers
pick 713e26dc275 d/copyright: Update Files-Excluded with newly-pruned vendored crates
pick 3a590ea16cd d/copyright: Remove src/tools/rls from Files-Excluded (dropped upstream)
pick 46d9e994fbd Update changelog

Important

Remember, the only change that’s actually necessary for this step is the dropping of the New upstream version <X.Y.Z>+dfsg0ubuntu0 commit.

After the interactive rebase, we can go back to the previous Rust version and re-import the new orig tarball:

$ git checkout merge-<X.Y_old>
$ git checkout -b import-new-<X.Y>

Recreate the experimental branch:

$ git branch -D experimental
$ git branch experimental

Then, we can merge our newly-pruned upstream source onto the previous Rust version’s source cleanly:

$ gbp import-orig \
    --no-symlink-orig \
    --no-pristine-tar \
    --upstream-branch=experimental \
    --debian-branch=import-new-<X.Y> \
    ../rustc-<X.Y>_<X.Y.Z>+dfsg0ubuntu1.orig.tar.xz

Finally, we can switch back to our actual branch and rebase:

$ git checkout merge-<X.Y>
$ git rebase import-new-<X.Y>

Consulting your git log, you should see the two new gbp commits immediately after the final commit of the last Rust version.

Verifying your Git changes¶

Before you force-push to your personal Launchpad remote, you must be absolutely sure you didn’t miss anything. Compare your branch with your backup branch:

$ git diff backup --name-status

You should only see a huge list of deleted vendor files.

Warning

If there are any other changes apart from deleted vendor files, then you made a mistake during the rebase — reset to your backup branch and try again.

Once you’re absolutely sure everything is good, you may safely delete your import-new-<X.Y> and backup branches. This is also a good time to force-push to your <lpuser> remote.

After-Repack Patch Refreshes¶

Some of the patches will no longer apply now that more files have been removed. You must refresh all the patches so they once again apply cleanly onto the newly-pruned source.

In general, you will follow the same protocol as the initial patch refresh.

Naturally, many lines will be removed from debian/patches/prune/d-0021-vendor-remove-windows-dependencies because many of the vendored crates you pruned were themselves unnecessary.

Updating XS-Vendored-Sources-Rust¶

Inside of debian/control and debian/control.in, there’s a special field called XS-Vendored-Sources-Rust which must be updated. It simply lists all the vendored crate dependencies, along with their version numbers, on a single line.

Luckily, the dh-cargo package contains a script for automatically generating this line. Push all your patches, then run the script:

$ quilt push -a
$ CARGO_VENDOR_DIR=vendor/ /usr/share/cargo/bin/dh-cargo-vendored-sources

Copy-paste the expected value it provides to both debian/control AND debian/control.in.

Attention

Make sure there’s still an empty line after the end of the field! Mistakenly dropping the empty line will result in a build failure right at the end of the test build.

Outdated toolchain issues¶

If you’re running a pre-versioned Rust Ubuntu release, then there’s a decent chance the cargo installation required by dh-cargo will be too old. In this case, don’t use dh-cargo—instead, manually download dh-cargo-vendored-sources (it’s just a Perl script) and use it without deb-based installations of Rust, which ensures that the Rustup snap’s version will be used instead.

Updating debian/copyright¶

All the new vendor files must be added to debian/copyright. Luckily, we can use a script which uses Lintian (lintian(1)) to generate all the missing copyright stanzas.

This script requires pytoml, so we must create and enter a virtual environment.

# apt install python3-venv

Create the virtual environment somewhere convenient, such as ~/.venvs/:

$ python3 -m venv rustc-lintian-to-copyright

After that, enter the virtual environment and install pytoml:

$ source ~/.venvs/rustc-lintian-to-copyright
$ which python3
$ python3 -m pip install pytoml

Temporarily edit debian/lintian-to-copyright.sh to recognize the virtual environment:

@@ -1,5 +1,5 @@
 #!/bin/sh
 # Pipe the output of lintian into this.
 sed -ne 's/.* file-without-copyright-information //p' | cut -d/ -f1-2 | sort -u | while read x; do
-       /usr/share/cargo/scripts/guess-crate-copyright "$x"
+   python3 /home/maxgmr/rustc/rustc/debian/scripts/guess-crate-copyright "$x"
 done

Clean up previous build artifacts, build the source package using dpkg-buildpackage(1), then run Lintian and pipe the output to the script:

$ dpkg-buildpackage -S -I -i -nc -d -sa
$ lintian -i -I -E --pedantic | debian/lintian-to-copyright.sh

Leave the virtual environment afterwards:

$ deactivate

You may need to fill in some fields manually. This is an easy way to find the start date of a GitHub repo.

Keep things clean by adding the new d/copyright stanzas alphabetically. It makes things a lot easier in the long run.

There are also two helper scripts which make it easier to keep d/copyright clean. redundant-copyright-stanzas produces a list of stanzas which are already covered by existing stanzas, and unneeded-copyright-stanzas produces a list of stanzas which don’t apply to any files in the source tree.

Caution

redundant-copyright-stanzas and unneeded-copyright-stanzas are overzealous by design. Don’t delete any stanzas without manually verifying things first!

Local Build and Bugfixing¶

You’re now ready to try to build rustc using sbuild(1).

First, make sure that all previous build artifacts have been cleaned from your upper-level directory:

$ rm -vf ../*.{debian.tar.xz,dsc,buildinfo,changes,ppa.upload}
$ rm -vf debian/files
$ rm -rf .pc

Then, run the build! Depending on your computer, a full build tends to take about 1-3 hours.

$ sbuild -Ad <release>

Using another PPA to bootstrap¶

Not all rustc releases are necessarily in the archive. Perhaps you’re waiting on a previous version to be upload, or you’re creating a backport which isn’t needed by the subsequent Ubuntu release.

If this applies to you, you must add your PPA as an extra repository to your sbuild command:

$ sbuild -Ad <release> \
    --extra-repository="deb [trusted=yes] http://ppa.launchpadcontent.net/<lpuser>/<ppa_name>/ubuntu/ <release> main"

Fixing bugs¶

If the build fails, it’s up to you to figure out why. This will require problem-solving skills and attention to detail.

First, try to find any upstream issues related to the problem on the Rust GitHub page. It’s quite common for non-packaging-related problems to be already known upstream, and you can often find a patch from there.

Searching for failing tests within the build log¶

sbuild saves the build logs to your computer. You can easily jump to the standard output of each failing test by searching for the following within the log:

stdout ----

Running individual tests¶

If the build fails, then sbuild will place you in an interactive shell for debugging. This is extremely useful, as you can change the source code and retry tests without rebuilding the whole thing.

For example, here’s how to re-run all the bootstrap tests:

$ debian/rules override_dh_auto_test-arch RUSTBUILD_TEST_FLAGS="src/bootstrap/"

Here’s how to re-run just the alias_and_path_for_library bootstrap test:

$ debian/rules override_dh_auto_test-arch RUSTBUILD_TEST_FLAGS="src/bootstrap/ --test-args alias_and_path_for_library"

Proper patch header format¶

In order to fix certain bugs, it’s likely you’ll need to create your own patch at some point. It’s important that this patch contains enough information for other people to understand what it’s doing and why it’s doing it.

First, ensure that Debian has not already created an equivalent patch. If so, you can simply use their patch directly. If you need to modify the patch in any way, make sure to add Origin: backport, <Debian VCS patch URL> to the patch header.

Otherwise, you must create your own patch. A template DEP-3 header can be generated using the following command:

$ quilt header -e --dep3 <path/to/patch>

For the most part, you can follow the Debian DEP-3 patch guidelines. However, there are a few extra things you must do:

Debian developers typically don’t use the This patch header follows DEP-3 [...] line added by quilt. Delete this line.
If this patch isn’t something needed to get the new Rust version to build, and you’re instead updating an existing source package, add a Bug-Ubuntu: line linking to the Launchpad bug.

Lintian Checks¶

Lintian (lintian(1)) checks your source package for bugs and Debian policy violations.

Clean up previous build artifacts then build the source package:

$ dpkg-buildpackage -S -I -i -nc -d -sa

First, check the Lintian output with just the warnings and errors:

lintian -i --tag-display-limit 0 2>&1 | tee <path_to_log_file>

Addressing warnings and errors¶

You must address all of these in one way or another. They must either be fixed or added to debian/source/lintian-overrides{,.in}, with a few notable exceptions:

E: rustc-1.86 source: field-too-long Vendored-Sources-Rust
- This is simply the length of the field. While we would like to change this in the future in dh-cargo, there’s nothing that can (or should) be done about this for now.
E: rustc-1.86 source: unknown-file-in-debian-source [debian/source/lintian-overrides.in]
- This is just the file used to generate the Lintian overrides for a given Rust version. It’s completely harmless to have in the source tree.
E: rustc-1.86 source: version-substvar-for-external-package Depends ${binary:Version} cargo-<X.Y> -> rustc [debian/control:*]
- This is just a fallback for a non-versioned rustc package. While it’s unlikely to ever be used, it’s not a typo, so you don’t need to worry about it.
W: rustc-1.86 source: unknown-field Vendored-Sources-Rust
- This is a custom field, not a typo.

As for any other warnings or errors, you must figure out whether the lint should be ignored or remedied. Don’t be afraid to ask for help from more experienced package maintainers, or consult the existing Lintian overrides for precedence.

Extra lints¶

Now you can run Lintian with all the pedantic, experimental, and informational lints enabled. It isn’t typically necessary to fix most of the extra lints, but it’s a good idea to check everything and see if there are some ways to improve the package based on these lints.

lintian -i -I -E --pedantic

Important

Don’t forget to clean and rebuild the source package before re-running Lintian, otherwise your changes to the package won’t apply!

PPA Build¶

Once everything builds on your local machine and Lintian is satisfied, it’s time to test the package on all architectures by uploading it to a PPA.

Creating a new PPA¶

If this is your first PPA upload for this Rust version, you must create a new PPA using the ppa-dev-tools snap. The PPA name depends on whether you are updating Rust, backporting Rust, or patching Rust.

New versioned Rust package:

$ ppa create rustc-<X.Y>-merge

Rust backport:

$ ppa create rustc-<X.Y>-release

Rust patch:

$ ppa create rustc-<X.Y>-lp<lp_bug_number>

The command should return a URL leading to the PPA. You must go to that Launchpad URL and do two things:

“Change Details” -> Enable all “Processors” (Make sure RISC-V is enabled!)
“Edit PPA Dependencies” -> Set Ubuntu dependencies to “Proposed”

If you are using another PPA to bootstrap, then you must explicitly add this PPA as a dependency in the “Edit PPA Dependencies” menu.

PPA changelog entry¶

Next, add a temporary changelog entry, appending ~ppa<N> to your version number so the PPA version isn’t used in favour of the actual version in the archive:

Note

<N> is just the number of the upload. You may have to fix something and re-upload to this PPA, so you should use ~ppa1 for your first PPA upload, ~ppa2 for your second, etc.

$ dch -bv <X.Y.Z>+dfsg0ubuntu1-0ubuntu1\~ppa<N> \
    --distribution "<release>" \
    "PPA upload"

Uploading the source package¶

Make sure that your source directory is clean (especially debian/files), then build the source package:

$ dpkg-buildpackage -S -I -i -nc -d -sa

Finally, upload the newly-created source package:

Note

You can get the source-changes-file script here.

New versioned Rust package:

$ dput ppa:<lpname>/rustc-<X.Y>-merge $(source-changes-file)

Rust backport:

$ dput ppa:<lpname>/rustc-<X.Y>-<release> $(source-changes-file)

Rust patch:

$ dput ppa:<lpname>/rustc-<X.Y>-lp<lp_bug_number> $(source-changes-file)

The PPA will then build the Rust package for all architectures supported by Ubuntu. These builds will highlight any architecture-specific build failures.

Handling early PPA build failures¶

Sometimes, a PPA build on a specific architecture will fail in under 15 minutes with no build log provided. If this happens, there was a Launchpad issue, and you can simply retry the build without consequence.

If the build failed and there is a build log provided, then there was indeed a build failure which you must address.

autopkgtests¶

Checking autopkgtest resource usage locally¶

We must experimentally verify that the default resources allocated to the autopkgtest test bed are insufficient. By default, autopkgtests are run using limited resources, but select packages can be added to the big_packages list to be granted more resources. This can be done locally using autopkgtest(1) and qemu-system-x86_64(1).

Creating local test beds¶

If this is your first time running Rust autopkgtests locally, you must create two local test beds.

There are multiple Openstack flavours used to run autopkgtests. Compare the default m1.small unit resources with the m1.large unit resources:

Unit	RAM Size (MB)	CPU Cores	Disk Size (GB)
`m1.small`	4096	2	20
`m1.large`	8192	4	100

Packages in the big_packages list use m1.large. Therefore, we’ll make one testbed modelled after m1.small and another modelled after m1.large and use them to determine whether or not we must add this Rust package to big_packages.

In a convenient place, e.g. ~/test_beds, create the default test bed using autopkgtest-buildvm-ubuntu-cloud(1) and rename it accordingly:

$ autopkgtest-buildvm-ubuntu-cloud -v -r <release>
$ mv autopkgtest-<release>-<arch>.img autopkgtest-<release>-<arch>-default.img

Then, in the same place, create a big_packages test bed:

$ autopkgtest-buildvm-ubuntu-cloud -s 100G --ram-size=8192 --cpus=4 -v -r <release>
$ mv autopkgtest-<release>-<arch>.img autopkgtest-<release>-<arch>-big.img

Verifying the necessity of big_packages¶

First, run the autopkgtests locally using the default test bed to check if the default Openstack flavour resources are sufficient for this build:

Note

The --log-file option is picky. It doesn’t do bash path expansions and the log file needs to exist already.

$ autopkgtest rustc-<X.Y> \
    --apt-upgrade \
    --shell-fail \
    --add-apt-source=ppa:<lpuser>/rustc-<X.Y>-merge \
    --log-file=<path/to/log/file> \
    -- \
    qemu \
    --ram-size=4096 \
    --cpus=2 \
    <path/to/test/bed/autopkgtest-<series>-<arch>-default.img

If the autopkgtests pass, then you are ready to run the autopkgtests for real. No further action is necessary. Otherwise, consult the log for something like the following:

Did not run successfully: signal: 9 (SIGKILL)
rustc exited with signal: 9 (SIGKILL)

If you see this, then the default autopkgtest resources are insufficient. To verify that adding your package to big_packages will fix the problem, you must now try the autopkgtests again using the big testbed you set up earlier:

$ autopkgtest rustc-<X.Y> \
    --apt-upgrade \
    --shell-fail \
    --add-apt-source=ppa:<lpuser>/rustc-<X.Y>-merge \
    --log-file=<path/to/log/file> \
    -- \
    qemu \
    --ram-size=8192 \
    --cpus=4 \
    <path/to/test/bed/autopkgtest-<series>-<arch>-big.img

If these autopkgtests pass, then you have successfully proven that the default autopkgtest runner will not have enough resources.

Getting more resources for the autopkgtests¶

We need to make sure that when the autopkgtests are run for real, they’re run on the larger test bed profile.

To do this, create a merge proposal in the autopkgtest-package-configs repo adding the new Rust version to the list of big_packages, which are granted the 100GB disk space, 8192MiB of memory, and 4 vCPUs we used earlier.

Launchpad may autofill the incorrect default branch. Make sure you double-check the target repository into which you’re merging — you want to merge into autopkgtest-package-configs, NOT autopkgtest-cloud.

The change itself is trivial:

--- a/big_packages
+++ b/big_packages
@@ -174,6 +174,7 @@ rsass
 ruby-minitest
 ruby-parallel
 rustc
+rustc-<X.Y>
 rust-ahash
 rust-axum/ppc64el
 rust-cargo-c/ppc64el

For an example on how to format this merge proposal, you can see a real-life proposal here.

Running the actual PPA autopkgtests¶

To run the autopkgtests for real, run the following command provided by the ppa-dev-tools snap to get links to all the autopkgtests:

$ ppa tests \
    ppa:<lpuser>/rustc-<X.Y>-merge \
    --release <release> \
    --show-url

Click all of the links except i386 to trigger the autopkgtests for each target architecture.

Re-run the same ppa tests ... command to check the status of the autopkgtests themselves.

The infrastructure can be a little flaky at times. If you get a “BAD” reponse (instead of a “PASS” or “FAIL”), then you just need to retry it.

Uploading the Package¶

You’re nearly ready to request sponsorship. First, it’s your duty to make your sponsor’s job as easy as possible.

Obtaining the right info¶

It’s recommended to create a local draft for your upload comment, so you can make sure everything is correct beforehand.

You need both the Lintian and the autopkgtest results for your package.

Retrieve links to all your passing autopkgtest build logs provided by the autopkgtest command you used earlier.

Get the default Lintian output for the source package by cleaning up all build artifacts, using dpkg-buildpackage to build the source package, then simply running the default Lintian command, redirecting the output to a file for later use:

$ lintian > <path_to_saved_lintian_output_file>

Finally, you can push your merge-<X.Y> branch to the Foundations rustc Launchpad Git remote:

$ git push <foundations> merge-<X.Y>

Compile the following information:

A link to your successfully-built PPA packages
A link to your merge-<X.Y> branch in the Foundations rustc Launchpad Git repository
A list of any notable packaging changes (not upstream changes). These are changes you made to the package, such as the removal or addition of patches. Nothing routine (such as patch refreshes or vendored dependency pruning) should be added here.
The output of lintian you just got
The links to the build logs of all PPA autopkgtests you just got

The i386 allowlist¶

We must ask an Archive Admin to add the new rustc-<X.Y> package to the i386 allowlist so it can be added to the new upload queue.

Usually, the easiest and fastest thing way of doing this is just messaging an Archive Admin directly, politely asking them to add rustc-<X.Y> to the i386 allowlist and providing them a link to the bug report.

Requesting a review¶

Once your comment is ready, subscribe ubuntu-sponsors to your bug to make it visible for sponsorship and upload.

After that, go to the bug report you originally opened and add a comment providing all the necessary info you compiled earlier.

An example of such a comment can be found at the rustc-1.87 bug report here.

Note

You will likely have difficulties finding a sponsor just by subscribing ubuntu-sponsors. You’re uploading a new package and it’s large and complicated. To get a timely sponsorship, it’s better to reach out to the Foundations or Toolchains teams and personally request sponsorship that way.

Toolchain Availability Page¶

After uploading a new Rust version, you must also update the Rust Toolchain Availability Page with the version you have added.