Understanding OCI Image Spec
As part of understanding the OCI image spec, I found that a good explanation of the image spec will help others. Explaining a few things here, which I found a little bit difficult to grasp while reading the spec.
Image Format
The spec defines an OCI image. This image spec is almost similar to Docker V2.2 image schema format. Image services like containerd can pull this image, and then create a bundle (more on this as I learn about it).
First, let me explain some of the basics that is used in the image spec so that it is easier to understand once we move on to higher level concepts.
MediaType
This field is used in the spec, so as to identify what an object corresponds to. For example, "application/vnd.oci.image.manifest.v1+json"
,
is used to identify an image manifest. The complete list of mediatypes can be found here.
Digest
The field represents the digest of the content. Normally it will be a sha256 sum. It can be said as a content addressable storage form. Let me explain what is a content addressable form, suppose the following is the content of a file
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 1485,
"digest": "sha256:46331d942d6350436f64e614d75725f6de3bb5c63e266e236e04389820a234c4"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 3208,
"digest": "sha256:7050e35b49f5e348c4809f5eff915842962cb813f32062d3bbdd35c750dd7d01"
}
]
}
and the sha256
sum of this content is 432f982638b3aefab73cc58ab28f5c16e96fdb504e8c134fc58dff4bae8bf338
. Then this
content will be stored in a file with name 432f982638b3aefab73cc58ab28f5c16e96fdb504e8c134fc58dff4bae8bf338
.
root@ubuntuvmw:# sha256sum -b 432f982638b3aefab73cc58ab28f5c16e96fdb504e8c134fc58dff4bae8bf338
432f982638b3aefab73cc58ab28f5c16e96fdb504e8c134fc58dff4bae8bf338 *432f982638b3aefab73cc58ab28f5c16e96fdb504e8c134fc58dff4bae8bf338
So we can identify the file by its content. The file can contain image configuration or sometimes be gzip’ed files.
The full path of the file will be blob/sha256/<sha256>
, where blob
means this is a binary blob, sha256
is the
algorithm used to generate the digest and then the actual file name.
root@ubuntuvmw:/var/lib/containerd/io.containerd.content.v1.content/blobs/sha256# ls -l
total 61788
-r--r--r-- 1 root root 17067249 May 19 17:53 00fcd775d5c6fe8b686bc4cc1c604548a75981a5f4972c552ddf02dd3a592b12
-r--r--r-- 1 root root 3069 May 19 17:53 08b57ba4cbdadace2ff536b8b05808100a54e844215321be541d06e6829c5f64
-r--r--r-- 1 root root 1386 May 19 17:53 0c099543f270a7dded82d2efbdb6ad543959950507228be027225ab95950f2f4
-r--r--r-- 1 root root 2562 May 9 16:56 10d7d58d5ebd2a652f4d93fdd86da8f265f5318c6a73cc5b6a9798ff6d2b2e67
-r--r--r-- 1 root root 1184 May 19 17:51 162d762e6ae2b5e4a938679d2756481d95a3f6b37d45287dbed8b301a091d3c9
-r--r--r-- 1 root root 27169393 May 19 17:51 185e8a4c100571f111d924b5d4399d89f163bf95d71ce2c6a33f656a66c52f0a
-r--r--r-- 1 root root 1363 May 10 10:04 3d674d3f3c0fce3bf2552e02564dc37e1556c0cf68d70a0cb6e143216a1eb510
-r--r--r-- 1 root root 949 May 18 11:07 3e4ac559bd32e387adb8b509b1bbc07f6fff76f78ab77f4e73c8022800044565
-r--r--r-- 1 root root 525 May 9 16:56 432f982638b3aefab73cc58ab28f5c16e96fdb504e8c134fc58dff4bae8bf338
-r--r--r-- 1 root root 1390 May 10 10:04 51f76c0d1bb7de1d4df4afc178599c001c6055f6218589257b6538f7502702be
-r--r--r-- 1 root root 740 May 17 20:08 5401e20b0410d156b3c4f2827a2b335687119cf1fb231455ff5656c2d73941cb
-r--r--r-- 1 root root 1386 May 17 20:19 55d31f3af94c797b65b310569803cacc1c9f4a34bf61afcdc8138f89345c8308
-r--r--r-- 1 root root 18964190 May 19 17:51 5652aee039c525b5a4324443551f5f137d285641af75ee005594c08e901ca0c3
-r--r--r-- 1 root root 741 May 19 17:53 602204011e2f3366bfdd441003998a8f066c81ca9b90493b135d6c2c278f1439
-r--r--r-- 1 root root 949 May 17 20:19 84921482d941f71aa8a8bd399a54d0956a81057332a67ac01e4de04c8c4e57da
-r--r--r-- 1 root root 1386 May 17 20:27 9c2b1dd085f4249cab11cd4d536fd97ecf89a0b752d79a676cff97f3c2889413
-r--r--r-- 1 root root 1386 May 18 11:07 be917c6d0bc015a245741f71966c6a0b1d08bf8b48da6387cad5e7c53c7067ae
-r--r--r-- 1 root root 3546 May 19 17:51 f3118e629e524f427f525ab7b73d78eaecc1a919edb7a0a886d7fafbbb1d6bd1
-r--r--r-- 1 root root 740 May 17 20:27 f901cb1ae59bcd7e4f8d0243037801c80736f4e00a587e670c0f8e9468e09f08
This is the list of all blobs in my directory, each of this can be an image manifest, or root filesystems.
More about the digest implementation in golang can be found here
Descriptors
An OCI image is composed of various types of contents(index, manifest, layers) arranged in Directed Acyclic Graph(DAG). The references between this different types of content are described using a Content Descriptor. The main components are:
mediaType
: It represents what type of content this descriptor refers to. It can be an image index, manifest or tar archivesdigest
: It represents the digest of the content. This digest can be used to access the content from the storage.size
: The size of the content in bytes.
Once you have the above the items, you can parse any content. For example:
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:552d1f2373af9bfe12033568ebbfb0ccbb0de11279f9a415a29207e264d7f4d9",
"size": 2711827
}
means that use the file named 552d1f2373af9bfe12033568ebbfb0ccbb0de11279f9a415a29207e264d7f4d9
from the blob and parse
it as a gzip tar archive
Once the above basic concepts are understood we can look how an image is organized.
- Image Index: Contains the various manifests for the different images. For example.
linux/amd64
andlinux/arm64
will be separate manifests, but will be available under the same index. Depending on the platform requested, the corresponding manifest will be loaded. Manifests will be referenced via content descriptors from an index. - Manifest: Contains the configuration and various layers in the image. Config and Layers will be referenced via content descriptor from the manifest
- Config: The config contains the various configuration data like: rootfs, exposed ports, environment variables.
- Layers: The actual image layers. These layers are applied one by one, on top of the rootfs to create the actual image filesystem. The content descriptor from layers will be pointing to tar/gzip archives on the storage for the image filesytem.
NOTE: This document should only be considered as a primer for reading the image-spec, so that you dont get confused and wont have to jump between links when reading the image spec.