# Adding new tests

<!-- toc -->

**In general, we expect every PR that fixes a bug in rustc to come accompanied
by a regression test of some kind.** This test should fail in master but pass
after the PR. These tests are really useful for preventing us from repeating the
mistakes of the past.

The first thing to decide is which kind of test to add. This will depend on the
nature of the change and what you want to exercise. Here are some rough
guidelines:

- The majority of compiler tests are done with [compiletest].
  - The majority of compiletest tests are [UI](ui.md) tests in the [`tests/ui`]
    directory.
- Changes to the standard library are usually tested within the standard library
  itself.
  - The majority of standard library tests are written as doctests, which
    illustrate and exercise typical API behavior.
  - Additional [unit tests](intro.md#package-tests) should go in
    `library/${crate}/tests` (where `${crate}` is usually `core`, `alloc`, or
    `std`).
- If the code is part of an isolated system, and you are not testing compiler
  output, consider using a [unit or integration test](intro.md#package-tests).
- Need to run rustdoc? Prefer a `rustdoc` or `rustdoc-ui` test. Occasionally
  you'll need `rustdoc-js` as well.
- Other compiletest test suites are generally used for special purposes:
  - Need to run gdb or lldb? Use the `debuginfo` test suite.
  - Need to inspect LLVM IR or MIR IR? Use the `codegen` or `mir-opt` test
    suites.
  - Need to inspect the resulting binary in some way? Or if all the other test
    suites are too limited for your purposes? Then use `run-make`.
  - Check out the [compiletest] chapter for more specialized test suites.

After deciding on which kind of test to add, see [best
practices](best-practices.md) for guidance on how to author tests that are easy
to work with that stand the test of time (i.e. if a test fails or need to be
modified several years later, how can we make it easier for them?).


## UI test walkthrough

The following is a basic guide for creating a [UI test](ui.md), which is one of
the most common compiler tests. For this tutorial, we'll be adding a test for an
async error message.

### Step 1: Add a test file

The first step is to create a Rust source file somewhere in the [`tests/ui`]
tree. When creating a test, do your best to find a good location and name (see
[Test organization](ui.md#test-organization) for more). Since naming is the
hardest part of development, everything should be downhill from here!

Let's place our async test at `tests/ui/async-await/await-without-async.rs`:

```rust,ignore
// Provide diagnostics when the user writes `await` in a non-`async` function.
//@ edition:2018

async fn foo() {}

fn bar() {
    foo().await
}

fn main() {}
```

A few things to notice about our test:

- The top should start with a short comment that [explains what the test is
  for](#explanatory_comment).
- The `//@ edition:2018` comment is called a [directive](directives.md) which
  provides instructions to compiletest on how to build the test. Here we need to
  set the edition for `async` to work (the default is edition 2015).
- Following that is the source of the test. Try to keep it succinct and to the
  point. This may require some effort if you are trying to minimize an example
  from a bug report.
- We end this test with an empty `fn main` function. This is because the default
  for UI tests is a `bin` crate-type, and we don't want the "main not found"
  error in our test. Alternatively, you could add `#![crate_type="lib"]`.

### Step 2: Generate the expected output

The next step is to create the expected output snapshots from the compiler. This
can be done with the `--bless` option:

```sh
./x test tests/ui/async-await/await-without-async.rs --bless
```

This will build the compiler (if it hasn't already been built), compile the