Fuzzing Guidelines for Rustc

# Fuzzing  For the purposes of this guide, *fuzzing* is any testing methodology that involves compiling a wide variety of programs in an attempt to uncover bugs in rustc. Fuzzing is often used to find internal compiler errors (ICEs). Fuzzing can be beneficial, because it can find bugs before users run into them and provide small, self-contained programs that make the bug easier to track down. However, some common mistakes can reduce the helpfulness of fuzzing and end up making contributors' lives harder. To maximize your positive impact on the Rust project, please read this guide before reporting fuzzer-generated bugs! ## Guidelines ### In a nutshell *Please do:* - Ensure the bug is still present on the latest nightly rustc - Include a reasonably minimal, standalone example along with any bug report - Include all of the information requested in the bug report template - Search for existing reports with the same message and query stack - Format the test case with `rustfmt`, if it maintains the bug - Indicate that the bug was found by fuzzing *Please don't:* - Don't report lots of bugs that use internal features, including but not limited to `custom_mir`, `lang_items`, `no_core`, and `rustc_attrs`. - Don't seed your fuzzer with inputs that are known to crash rustc (details below). ### Discussion If you're not sure whether or not an ICE is a duplicate of one that's already been reported, please go ahead and report it and link to issues you think might be related. In general, ICEs on the same line but with different *query stacks* are usually distinct bugs. For example, [#109020][#109020] and [#109129][#109129] had similar error messages: ``` error: internal compiler error: compiler/rustc_middle/src/ty/normalize_erasing_regions.rs:195:90: Failed to normalize <[closure@src/main.rs:36:25: 36:28] as std::ops::FnOnce<(Emplacable<()>,)>>::Output, maybe try to call `try_normalize_erasing_regions` instead ``` ``` error: internal compiler error: compiler/rustc_middle/src/ty/normalize_erasing_regions.rs:195:90: Failed to normalize <() as Project>::Assoc, maybe try to call `try_normalize_erasing_regions` instead ``` but different query stacks: ``` query stack during panic: #0 [fn_abi_of_instance] computing call ABI of `<[closure@src/main.rs:36:25: 36:28] as core::ops::function::FnOnce<(Emplacable<()>,)>>::call_once - shim(vtable)` end of query stack ``` ``` query stack during panic: #0 [check_mod_attrs] checking attributes in top-level module #1 [analysis] running analysis passes on this crate end of query stack ``` ## Building a corpus When building a corpus, be sure to avoid collecting tests that are already known to crash rustc. A fuzzer that is seeded with such tests is more likely to generate bugs with the same root cause, wasting everyone's time. The simplest way to avoid this is to loop over each file in the corpus, see if it causes an ICE, and remove it if so. To build a corpus, you may want to use: - The rustc/rust-analyzer/clippy test suites (or even source code) --- though avoid tests that are already known to cause failures, which often begin with comments like `//@ failure-status: 101` or `//@ known-bug: #NNN`.

This document provides guidelines for fuzzing rustc to uncover bugs. It emphasizes the importance of reporting bugs that are still present on the latest nightly rustc, including minimal standalone examples, and avoiding the reporting of bugs that use internal features or are duplicates of existing reports. The guide also suggests building a corpus by avoiding tests that are already known to crash rustc.