Introduction to Compiletest: Rust's Test Harness

# Compiletest  ## Introduction `compiletest` is the main test harness of the Rust test suite. It allows test authors to organize large numbers of tests (the Rust compiler has many thousands), efficient test execution (parallel execution is supported), and allows the test author to configure behavior and expected results of both individual and groups of tests. > **Note for macOS users** > > For macOS users, `SIP` (System Integrity Protection) [may consistently check > the compiled binary by sending network requests to Apple][zulip], so you may > get a huge performance degradation when running tests. > > You can resolve it by tweaking the following settings: `Privacy & Security -> > Developer Tools -> Add Terminal (Or VsCode, etc.)`. `compiletest` may check test code for compile-time or run-time success/failure. Tests are typically organized as a Rust source file with annotations in comments before and/or within the test code. These comments serve to direct `compiletest` on if or how to run the test, what behavior to expect, and more. See [directives](directives.md) and the test suite documentation below for more details on these annotations. See the [Adding new tests](adding.md) and [Best practies](best-practices.md) chapters for a tutorial on creating a new test and advice on writing a good test, and the [Running tests](running.md) chapter on how to run the test suite. Arguments can be passed to compiletest using `--test-args` or by placing them after `--`, e.g. - `x test --test-args --force-rerun` - `x test -- --force-rerun` Additionally, bootstrap accepts several common arguments directly, e.g. `x test --no-capture --force-rerun --run --pass`. Compiletest itself tries to avoid running tests when the artifacts that are involved (mainly the compiler) haven't changed. You can use `x test --test-args --force-rerun` to rerun a test even when none of the inputs have changed. ## Test suites All of the tests are in the [`tests`] directory. The tests are organized into "suites", with each suite in a separate subdirectory. Each test suite behaves a little differently, with different compiler behavior and different checks for correctness. For example, the [`tests/incremental`] directory contains tests for incremental compilation. The various suites are defined in [`src/tools/compiletest/src/common.rs`] in the `pub enum Mode` declaration. The following test suites are available, with links for more information: ### Compiler-specific test suites | Test suite | Purpose | |-------------------------------------------|---------------------------------------------------------------------------------------------------------------------| | [`ui`](ui.md) | Check the stdout/stderr snapshots from the compilation and/or running the resulting executable | | `ui-fulldeps` | `ui` tests which require a linkable build of `rustc` (such as using `extern crate rustc_span;` or used as a plugin) | | [`pretty`](#pretty-printer-tests) | Check pretty printing | | [`incremental`](#incremental-tests) | Check incremental compilation behavior | | [`debuginfo`](#debuginfo-tests) | Check debuginfo generation running debuggers | | [`codegen`](#codegen-tests) | Check code generation | | [`codegen-units`](#codegen-units-tests) | Check codegen unit partitioning | | [`assembly`](#assembly-tests) | Check assembly output |

Compiletest is the primary test harness for the Rust test suite, designed for organizing, executing, and configuring a large number of tests. It supports parallel execution and allows test authors to specify expected behavior and results using annotations within the test code. The tool can check code for compile-time and run-time success/failure. The document explains how to use compiletest, including how to add new tests, best practices, and how to run tests, and also provides a brief overview of the available test suites.