Rust Compiler Testing Tools: Tidy, Formatting, Documentation, and More

All standard library and compiler unit tests are placed in separate `tests` file (which is enforced in [tidy][tidy-unit-tests]). This ensures that when the test file is changed, the crate does not need to be recompiled. For example: ```rust,ignore #[cfg(test)] mod tests; ``` If it wasn't done this way, and you were working on something like `core`, that would require recompiling the entire standard library, and the entirety of `rustc`. `./x test` includes some CLI options for controlling the behavior with these package tests: * `--doc` — Only runs documentation tests in the package. * `--no-doc` — Run all tests *except* documentation tests. ### Tidy Tidy is a custom tool used for validating source code style and formatting conventions, such as rejecting long lines. There is more information in the [section on coding conventions](../conventions.md#formatting). > Examples: `./x test tidy` ### Formatting Rustfmt is integrated with the build system to enforce uniform style across the compiler. The formatting check is automatically run by the Tidy tool mentioned above. Examples: | Command | Description | |-------------------------|--------------------------------------------------------------------| | `./x fmt --check` | Checks formatting and exits with an error if formatting is needed. | | `./x fmt` | Runs rustfmt across the entire codebase. | | `./x test tidy --bless` | First runs rustfmt to format the codebase, then runs tidy checks. | ### Book documentation tests All of the books that are published have their own tests, primarily for validating that the Rust code examples pass. Under the hood, these are essentially using `rustdoc --test` on the markdown files. The tests can be run by passing a path to a book to `./x test`. > Example: `./x test src/doc/book` ### Documentation link checker Links across all documentation is validated with a link checker tool, and it can be invoked so: ```console ./x test linkchecker ``` This requires building all of the documentation, which might take a while. ### Dist check `distcheck` verifies that the source distribution tarball created by the build system will unpack, build, and run all tests. > Example: `./x test distcheck` ### Tool tests Packages that are included with Rust have all of their tests run as well. This includes things such as cargo, clippy, rustfmt, miri, bootstrap (testing the Rust build system itself), etc. Most of the tools are located in the [`src/tools`] directory. To run the tool's tests, just pass its path to `./x test`. > Example: `./x test src/tools/cargo` Usually these tools involve running `cargo test` within the tool's directory. If you want to run only a specified set of tests, append `--test-args FILTER_NAME` to the command. > Example: `./x test src/tools/miri --test-args padding` In CI, some tools are allowed to fail. Failures send notifications to the corresponding teams, and is tracked on the [toolstate website]. More information can be found in the [toolstate documentation]. ### Ecosystem testing Rust tests integration with real-world code to catch regressions and make informed decisions about the evolution of the language. There are several kinds of ecosystem tests, including Crater. See the [Ecosystem testing chapter](ecosystem.md) for more details. ### Performance testing A separate infrastructure is used for testing and tracking performance of the compiler. See the [Performance testing chapter](perf.md) for more details. ### Codegen backend testing See [Codegen backend testing](./codegen-backend-tests/intro.md). ## Miscellaneous information There are some other useful testing-related info at [Misc info](misc.md). ## Further reading The following blog posts may also be of interest: - brson's classic ["How Rust is tested"][howtest]

This section details various testing tools in the Rust compiler project. It covers Tidy for code style, Rustfmt for formatting, book documentation tests, link checker for documentation links, dist check for source distribution validation, and tool tests for included packages like Cargo and Clippy. It also mentions ecosystem testing for real-world integration and performance testing using a separate infrastructure. Example commands and locations of tool tests are provided.