dwww Home | Manual pages | Find package

containers-storage-zstd-chunked - Information about zstd:chunked

The   traditional   format   for   container   image   layers  is
application/vnd.oci.image.layer.v1.tar+gzip
⟨https://github.com/opencontainers/image−spec/blob/main/layer.md#gzip−media−types⟩.
More  recently,   the   standard   was   augmented   with   zstd:
application/vnd.oci.image.layer.v1.tar+zstd
⟨https://github.com/opencontainers/image−spec/blob/main/layer.md#zstd−media−types⟩
which is a more modern and efficient compression format.

zstd:chunked        is        a        variant       of       the
application/vnd.oci.image.layer.v1.tar+zstd media type that  uses
zstd                       skippable                       frames
⟨https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md#skippable−frames⟩
to include additional metadata (especially a "table of contents")
that includes the SHA-256 and offsets  of  individual  chunks  of
files.   Additionally  chunks  are  compressed  separately.  This
allows a client  to  dynamically  fetch  only  content  which  it
doesn’t already have using HTTP range requests.

At  the  time  of  this  writing,  support for this is enabled by
default in the code.

You can explicitly enable or disable zstd:chunked with  following
changes to containers-storage.conf:

[storage.overlay.pull_options]  enable_partial_images  = "true" |
"false"

Note that the value of this field must be  a  "string  bool",  it
cannot be a native TOML boolean.

Each  layer  has  an  associated  "big  data" key called chunked-
manifest-cache that is a custom binary format suitable for mmap()
that contains index metadata for each layer with the full  sha256
digest   of   each   file  plus  its  "chunks"  (as  computed  by
zstd:chunked).

When any image is pulled all existing other  layers  are  scanned
using chunked-manifest-cache to see if they contain a file with a
matching digest. If one is found, the other file is hardlinked if
use_hardlinks  =  "true", otherwise it is reflinked (if supported
by the filesystem, or a full physical copy is made). There  is  a
best-effort  attempt to enable fsverity on the file if configured
(see ⟨https://github.com/containers/storage/issues/2017⟩).

For more information, at the current time the file with the  most
information         is        pkg/chunked/internal/compression.go
⟨https://github.com/containers/storage/blob/39d469c34c96db67062e25954bc9d18f2bf6dae3/pkg/chunked/internal/compression.go⟩.
The above is a permanent link for stability, but be sure to check
to see if there are newer changes too.

At the current time the format is not officially standardized  or
documented   beyond  the  comments  and  code  in  the  reference
implementation.

https://github.com/containers/storage/issues?q=is%3Aissue+label%3Aarea%2Fzstd%3Achunked+is%3Aopen

The Containers Storage project is  committed  to  inclusivity,  a
core   value   of  open  source.   The  master  and  slave  mount
propagation  terminology  is  used  in  this  repository.    This
language  is  problematic  and  divisive,  and should be changed.
However, these terms are currently used within the  Linux  kernel
and must be used as-is at this time.  When the kernel maintainers
rectify   this   usage,   Containers  Storage  will  follow  suit
immediately.

Generated by dwww version 1.16 on Mon Nov 2 14:31:17 CET 2026.