CI Testing in the rust-lang/rust Repository

# Testing with CI The primary goal of our CI system is to ensure that the `master` branch of `rust-lang/rust` is always in a valid state and passes our test suite. From a high-level point of view, when you open a pull request at `rust-lang/rust`, the following will happen: - A small [subset](#pull-request-builds) of tests and checks are run after each push to the PR. This should help catching common errors. - When the PR is approved, the [bors] bot enqueues the PR into a [merge queue]. - Once the PR gets to the front of the queue, bors will create a merge commit and run the [full test suite](#auto-builds) on it. The merge commit either contains only one specific PR or it can be a ["rollup"](#rollups) which combines multiple PRs together, to save CI costs. - Once the whole test suite finishes, two things can happen. Either CI fails with an error that needs to be addressed by the developer, or CI succeeds and the merge commit is then pushed to the `master` branch. If you want to modify what gets executed on CI, see [Modifying CI jobs](#modifying-ci-jobs). ## CI workflow  Our CI is primarily executed on [GitHub Actions], with a single workflow defined in [`.github/workflows/ci.yml`], which contains a bunch of steps that are unified for all CI jobs that we execute. When a commit is pushed to a corresponding branch or a PR, the workflow executes the [`src/ci/citool`] crate, which dynamically generates the specific CI jobs that should be executed. This script uses the [`jobs.yml`] file as an input, which contains a declarative configuration of all our CI jobs. > Almost all build steps shell out to separate scripts. This keeps the CI fairly > platform independent (i.e., we are not overly reliant on GitHub Actions). > GitHub Actions is only relied on for bootstrapping the CI process and for > orchestrating the scripts that drive the process. In essence, all CI jobs run `./x test`, `./x dist` or some other command with different configurations, across various operating systems, targets and platforms. There are two broad categories of jobs that are executed, `dist` and non-`dist` jobs. - Dist jobs build a full release of the compiler for a specific platform, including all the tools we ship through rustup; Those builds are then uploaded to the `rust-lang-ci2` S3 bucket and are available to be locally installed with the [rustup-toolchain-install-master] tool. The same builds are also used for actual releases: our release process basically consists of copying those artifacts from `rust-lang-ci2` to the production endpoint and signing them. - Non-dist jobs run our full test suite on the platform, and the test suite of all the tools we ship through rustup; The amount of stuff we test depends on the platform (for example some tests are run only on Tier 1 platforms), and some quicker platforms are grouped together on the same builder to avoid wasting CI resources. Based on an input event (usually a push to a branch), we execute one of three kinds of builds (sets of jobs). 1. PR builds 2. Auto builds 3. Try builds ### Pull Request builds After each push to a pull request, a set of `pr` jobs are executed. Currently, these execute the `x86_64-gnu-llvm-X`, `x86_64-gnu-tools`, `mingw-check-1`, `mingw-check-2` and `mingw-check-tidy` jobs, all running on Linux. These execute a relatively short (~40 minutes) and lightweight test suite that should catch common issues. More specifically, they run a set of lints, they try to perform a cross-compile check build to Windows mingw (without producing any artifacts) and they test the compiler using a *system* version of LLVM. Unfortunately, it would take too many resources to run the full test suite for each commit on every PR. > **Note on doc comments** > > Note that PR CI as of Oct 2024  by default does not try to > run `./x doc xxx`. This means that if you have any broken intradoc links that > would lead to `./x doc xxx` failing, it will happen very late into the full

This section explains the Continuous Integration (CI) system used for the `rust-lang/rust` repository. The goal is to ensure the `master` branch is always in a valid state by running a test suite. CI is triggered by pull requests, with a subset of tests run on each push, followed by a full test suite run by the bors bot upon approval. The CI workflow is orchestrated through GitHub Actions and the `citool` crate, using `jobs.yml` to define CI jobs. These jobs primarily involve running `./x test` or `./x dist` commands with various configurations. Dist jobs build full compiler releases, while non-dist jobs run the test suite. There are three main types of builds: PR builds, Auto builds, and Try builds. PR builds run a subset of tests on each pull request to catch common issues.