Home Explore Blog CI



docker

4th chunk of `content/manuals/build/cache/garbage-collection.md`
c1dcfdc7e696d6d34613aa76421e3f810b817a5f5b788b910000000100000f96
You can set these options either as number of bytes, a unit string (for
example, `512MB`), or as a percentage of the total disk size. Changing these
options influences the default GC policies used by the BuildKit worker. With
the default thresholds, the GC policies resolve as follows:

```toml
# Global defaults
[worker.oci]
  gc = true
  reservedSpace = "10GB"
  maxUsedSpace = "100GB"
  minFreeSpace = "20%"

# Policy 1
[[worker.oci.gcpolicy]]
  filters = [ "type==source.local", "type==exec.cachemount", "type==source.git.checkout" ]
  keepDuration = "48h"
  maxUsedSpace = "512MB"

# Policy 2
[[worker.oci.gcpolicy]]
  keepDuration = "1440h" # 60 days
  reservedSpace = "10GB"
  maxUsedSpace = "100GB"

# Policy 3
[[worker.oci.gcpolicy]]
  reservedSpace = "10GB"
  maxUsedSpace = "100GB"

# Policy 4
[[worker.oci.gcpolicy]]
  all = true
  reservedSpace = "10GB"
  maxUsedSpace = "100GB"
```

In practical terms, this means:

- Policy 1: If the build cache exceeds 512MB, BuildKit removes cache records
  for local build contexts, remote Git contexts, and cache mounts that haven’t
  been used in the last 48 hours.
- Policy 2: If disk usage exceeds 100GB, unshared build cache older than 60
  days is removed, ensuring at least 10GB of disk space is reserved for cache.
- Policy 3: If disk usage exceeds 100GB, any unshared cache is removed,
  ensuring at least 10GB of disk space is reserved for cache.
- Policy 4: If disk usage exceeds 100GB, all cache—including shared and
  internal records—is removed, ensuring at least 10GB of disk space is reserved
  for cache.

`reservedSpace` has the highest priority in defining the lower limit for build
cache size. If `maxUsedSpace` or `minFreeSpace` would define a lower value, the
minimum cache size would never be brought below `reservedSpace`.

If both `reservedSpace` and `maxUsedSpace` are set, a GC sweep results in a
cache size between those thresholds. For example, if `reservedSpace` is set to
10GB, and `maxUsedSpace` is set to 20GB, the resulting amount of cache after a
GC run is less than 20GB, but at least 10GB.

You can also define completely custom GC policies. Custom policies also let you
define filters, which lets you pinpoint the types of cache entries that a given
policy is allowed to prune.

#### Custom GC policies in BuildKit

Custom GC policies let you fine-tune how BuildKit manages its cache, and gives
you full control over cache retention based on criteria such as cache type,
duration, or disk space thresholds. If you need full control over the cache
thresholds and how cache records should be prioritized, defining custom GC
policies is the way to go.

To define a custom GC policy, use the `[[worker.oci.gcpolicy]]` configuration
block in `buildkitd.toml`. Each policy define the thresholds that will be used
for that policy. The global values for `reservedSpace`, `maxUsedSpace`, and
`minFreeSpace` do not apply if you use custom policies.

Here’s an example configuration:

```toml
# Custom GC Policy 1: Remove unused local contexts older than 24 hours
[[worker.oci.gcpolicy]]
  filters = ["type==source.local"]
  keepDuration = "24h"
  reservedSpace = "5GB"
  maxUsedSpace = "50GB"

# Custom GC Policy 2: Remove remote Git contexts older than 30 days
[[worker.oci.gcpolicy]]
  filters = ["type==source.git.checkout"]
  keepDuration = "720h"
  reservedSpace = "5GB"
  maxUsedSpace = "30GB"

# Custom GC Policy 3: Aggressively clean all cache if disk usage exceeds 90GB
[[worker.oci.gcpolicy]]
  all = true
  reservedSpace = "5GB"
  maxUsedSpace = "90GB"
```

In addition to the `reservedSpace`, `maxUsedSpace`, and `minFreeSpace` threshold,
when defining a GC policy you have two additional configuration options:

- `all`: By default, BuildKit will exclude some cache records from being pruned
  during GC. Setting this option to `true` will allow any cache records to be
  pruned.
- `filters`: Filters let you specify specific types of cache records that a GC
  policy is allowed to prune.

Title: Practical Implications of GC Policies and Customization in BuildKit
Summary
This section explains the practical implications of the default GC policies defined in the `buildkitd.toml` configuration file, outlining how BuildKit removes cache records based on factors like disk usage and the age of the cache. It emphasizes that `reservedSpace` has the highest priority in defining the lower limit for build cache size. The section then introduces custom GC policies, highlighting their importance for fine-tuning cache management with full control over cache retention based on criteria like cache type, duration, or disk space thresholds. It provides an example of custom policies and elaborates on additional configuration options like `all` and `filters` to further customize cache pruning.