Output Normalization in UI Tests

- `run.stdout` — stdout when running the test - `64bit.stderr` — compiler stderr with `stderr-per-bitwidth` directive on a 64-bit target - `32bit.stderr` — compiler stderr with `stderr-per-bitwidth` directive on a 32-bit target A simple example would be `foo.stderr` next to a `foo.rs` test. A more complex example would be `foo.my-revision.polonius.stderr`. There are several [directives](directives.md) which will change how compiletest will check for output files: - `stderr-per-bitwidth` — checks separate output files based on the target pointer width. Consider using the `normalize-stderr` directive instead (see [Normalization](#normalization)). - `dont-check-compiler-stderr` — Ignores stderr from the compiler. - `dont-check-compiler-stdout` — Ignores stdout from the compiler. UI tests run with `-Zdeduplicate-diagnostics=no` flag which disables rustc's built-in diagnostic deduplication mechanism. This means you may see some duplicate messages in the output. This helps illuminate situations where duplicate diagnostics are being generated. ### Normalization The compiler output is normalized to eliminate output difference between platforms, mainly about filenames. Compiletest makes the following replacements on the compiler output: - The directory where the test is defined is replaced with `$DIR`. Example: `/path/to/rust/tests/ui/error-codes` - The directory to the standard library source is replaced with `$SRC_DIR`. Example: `/path/to/rust/library` - Line and column numbers for paths in `$SRC_DIR` are replaced with `LL:COL`. This helps ensure that changes to the layout of the standard library do not cause widespread changes to the `.stderr` files. Example: `$SRC_DIR/alloc/src/sync.rs:53:46` - The base directory where the test's output goes is replaced with `$TEST_BUILD_DIR`. This only comes up in a few rare circumstances. Example: `/path/to/rust/build/x86_64-unknown-linux-gnu/test/ui` - Tabs are replaced with `\t`. - Backslashes (`\`) are converted to forward slashes (`/`) within paths (using a heuristic). This helps normalize differences with Windows-style paths. - CRLF newlines are converted to LF. - Error line annotations like `//~ ERROR some message` are removed. - Various v0 and legacy symbol hashes are replaced with placeholders like `[HASH]` or `<SYMBOL_HASH>`. Additionally, the compiler is run with the `-Z ui-testing` flag which causes the compiler itself to apply some changes to the diagnostic output to make it more suitable for UI testing. For example, it will anonymize line numbers in the output (line numbers prefixing each source line are replaced with `LL`). In extremely rare situations, this mode can be disabled with the directive `//@ compile-flags: -Z ui-testing=no`. Note: The line and column numbers for `-->` lines pointing to the test are *not* normalized, and left as-is. This ensures that the compiler continues to point to the correct location, and keeps the stderr files readable. Ideally all line/column information would be retained, but small changes to the source causes large diffs, and more frequent merge conflicts and test errors. Sometimes these built-in normalizations are not enough. In such cases, you may provide custom normalization rules using `normalize-*` directives, e.g. ```rust,ignore //@ normalize-stdout: "foo" -> "bar" //@ normalize-stderr: "foo" -> "bar" //@ normalize-stderr-32bit: "fn $32 bits$" -> "fn $$$PTR bits$" //@ normalize-stderr-64bit: "fn $64 bits$" -> "fn $$$PTR bits$" ``` This tells the test, on 32-bit platforms, whenever the compiler writes `fn() (32 bits)` to stderr, it should be normalized to read `fn() ($PTR bits)` instead. Similar for 64-bit. The replacement is performed by regexes using default regex flavor provided by `regex` crate. The corresponding reference file will use the normalized output to test both 32-bit and 64-bit platforms: ```text ... | = note: source type: fn() ($PTR bits) = note: target type: u16 (16 bits)

Compiler output in UI tests is normalized to handle platform differences. Compiletest replaces paths, line/column numbers in standard library sources, tabs, backslashes, CRLF newlines, and error line annotations. It also anonymizes line numbers in the output using the `-Z ui-testing` flag. The tool normalizes symbol hashes too. Custom normalization rules can be defined using `normalize-*` directives to handle specific cases, employing regular expressions for replacements. These normalizations help ensure consistency and reduce spurious differences in test outputs across different environments and platforms.