We build some C/C++ stuff in various CI jobs, and we rely on [sccache] to cache
the intermediate LLVM artifacts. Sccache is a distributed ccache developed by
Mozilla, which can use an object storage bucket as the storage backend.

With sccache there's no need to calculate the hash key ourselves. Sccache
invalidates the cache automatically when it detects changes to relevant inputs,
such as the source code, the version of the compiler, and important environment
variables.
So we just pass the sccache wrapper on top of cargo and sccache does the rest.

We store the persistent artifacts on the S3 bucket `rust-lang-ci-sccache2`. So
when the CI runs, if sccache sees that LLVM is being compiled with the same C/C++
compiler and the LLVM source code is the same, sccache retrieves the individual
compiled translation units from S3.


## Custom tooling around CI

During the years we developed some custom tooling to improve our CI experience.

### Rust Log Analyzer to show the error message in PRs

The build logs for `rust-lang/rust` are huge, and it’s not practical to find
what caused the build to fail by looking at the logs. To improve the developers’
experience we developed a bot called [Rust Log Analyzer][rla] (RLA) that
receives the build logs on failure and extracts the error message automatically,
posting it on the PR.

The bot is not hardcoded to look for error strings, but was trained with a bunch
of build failures to recognize which lines are common between builds and which
are not. While the generated snippets can be weird sometimes, the bot is pretty
good at identifying the relevant lines even if it’s an error we've never seen
before.


### Toolstate to support allowed failures

The `rust-lang/rust` repo doesn’t only test the compiler on its CI, but also a
variety of tools and documentation. Some documentation is pulled in via git
submodules. If we blocked merging rustc PRs on the documentation being fixed, we
would be stuck in a chicken-and-egg problem, because the documentation's CI