Today we have supply chain artifacts that we didn’t have the last year: gitbom, sbom, claims etc. Today is possible we produce an artifact and that will come with 3x non-deployable artifacts alongside.

deployable_non_deployable_artifacts

How do you store them? Do you make up your own storage service ? Do you need to take in account additional costs to run the infrastructure for the storage (and retrieval) of these artifacts?

What about OCI registries?

Let’s explore what means having an image in a registry and how we can reference them from an OCI artifact.

ORAS

We will use the ORAS CLI to interact with our registry. It provides an interface to interact with OCI artifacts and OCI registries.

On MacOs you can install it via brew

brew install oras

OCI Manifest

Let’s start by taking a look at what is a Manifest.

When we download a Docker Image we’re basically asking for a manifest: a configuration and a set of layers for a single container image for a specific architecture.

Here is an example:

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "size": 7023,
    "digest": "sha256:b5b2b2c507a0944348e0303114d8d93aaaa081732b86451d9bce1f432a537bc7"
  },
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "size": 32654,
      "digest": "sha256:9834876dcfb05cb167a5c24953eba58c4ac89b1adf57f28f2f9d09af107ee8f0"
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "size": 16724,
      "digest": "sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b"
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "size": 73109,
      "digest": "sha256:ec4b8955958665577945c89419d1af06b5f7636b4ac3da7f12184802ad867736"
    }
  ],
  "subject": {
    "mediaType": "application/vnd.oci.image.manifest.v1+json",
    "size": 7682,
    "digest": "sha256:5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270"
  },
  "annotations": {
    "com.example.key1": "value1",
    "com.example.key2": "value2"
  }
}

As you can see it is made up by different sections:

  • schema version: for the current specification is and must be 2
  • mediaType: it must be application/vnd.oci.image.manifest.v1+json and it tells what kind of manifest are we dealing with
  • config: this section defines a configuration object for the container (digest reference). You can retrieve the config for a container image by using oras manifest fetch-config docker.io/library/debian@sha256:749383b0a6d17fb745d397b108d2ea38b5832226586b25c9f5cf7fcde24458ab --pretty. It is a descriptor hence it holds information about: the type of the content of this manifest, a content identifier (digest) and how many bytes the config takes. It also includes optional fields.
  • layers: as the config field, is a descriptor, and it contains reference to the blobs.
  • subject: as above, is a descriptor and it is used to indicate a relationship to the specified manifest of this field.
  • annotations: key value metadata for the manifest.

Manifest relationships and types

Now that we know what are the fields for an image manifest let’s discover the relationship between different kind of manifests. In fact there are several manifest types:

  • Docker manifest: application/vnd.docker.distribution.manifest.v2+json
  • OCI Image Manifest: application/vnd.oci.image.manifest.v1+json
  • Docker manifest: application/vnd.docker.distribution.manifest.list.v2+json
  • OCI Index: application/vnd.oci.image.index.v1+json

and many others!

Image Index

Let’s say we have this image: docker.io/library/debian:10

A TAG usually points to an Image Index a collection of images references (Image manifests) for different platforms

image_spec_1

we can inspect the manifest with: oras manifest fetch docker.io/library/debian:10 --pretty

{
  "manifests": [
    {
      "digest": "sha256:749383b0a6d17fb745d397b108d2ea38b5832226586b25c9f5cf7fcde24458ab",
      "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      },
      "size": 529
    },
    {
      "digest": "sha256:19cde7c8fc75c744ebaacfe625dea8fa0872a112463bdf509ca6d716deddd01f",
      "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "arm",
        "os": "linux",
        "variant": "v5"
      },
      "size": 529
    },
    {
      "digest": "sha256:e514a20691c31e31760fbe7f9b0c3d3e7e19066c8a60a79a6306d494c66689a4",
      "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "arm",
        "os": "linux",
        "variant": "v7"
      },
      "size": 529
    },
    {
      "digest": "sha256:4951d5cf066cd4ff0558a7cc75816dc203eaeb2c634329d3832db76bbc7586b0",
      "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "arm64",
        "os": "linux",
        "variant": "v8"
      },
      "size": 529
    },
    {
      "digest": "sha256:c9c6b79c7caf3e4d0e7fccf71de5ce80ca2ccc48c3b350ca75cd532bc0bb1f17",
      "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "386",
        "os": "linux"
      },
      "size": 529
    },
    {
      "digest": "sha256:8d0057fe4321ef2e46c311f3454261143e8d0f09a3ef4ed2bd83bd7ea6700dc2",
      "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "mips64le",
        "os": "linux"
      },
      "size": 529
    },
    {
      "digest": "sha256:ab9dda324085389607297949872f9cab27e2a86422d1aa9f41fbed62f468e907",
      "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "ppc64le",
        "os": "linux"
      },
      "size": 529
    },
    {
      "digest": "sha256:caba3d8aeec3da35fa3ed6a49c44a58989d184760192874a09f86902d596696f",
      "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "s390x",
        "os": "linux"
      },
      "size": 529
    }
  ],
  "mediaType": "application\/vnd.docker.distribution.manifest.list.v2+json",
  "schemaVersion": 2
}

Image manifest

Then the Image Manifest is the “real” Image that encapsulates the docker layers for that image and the config metadata. Let’s fetch the manifest for the amd64 architecture:

oras manifest fetch docker.io/library/debian:sha256:749383b0a6d17fb745d397b108d2ea38b5832226586b25c9f5cf7fcde24458ab --pretty

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
  "config": {
    "mediaType": "application/vnd.docker.container.image.v1+json",
    "size": 1463,
    "digest": "sha256:54e726b437fbb2dd7b43e4dd5cd79b0181e96a22849b7fc2ffe934fac2d65440"
  },
  "layers": [
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 55046771,
      "digest": "sha256:1e4aec178e0864db93a6f97a20bde3445871a4562c1801185eca1238d3a0e80d"
    }
  ]
}

image_spec_2

Once we have the image, then we need to ask: how can we reference it from an SBOM, signature etc. that image manifest? Artifact Manifest!

Artifact manifest

With the artifact manifest we can specify:

  • a subject: the descripton (sha256) of the image it is referring to
  • artifactType: the type of the content of this artifact
  • annotations: additional metadata
  • blobs: the content of the artifact

image_spec_3

Hands-on

For this demo I will set up a local docker registry from the OCI Playground image:

podman run --rm -it -p 8000:5000 ghcr.io/oci-playground/registry:latest

Now that we have a docker registry running, copy a container image in it

> oras copy docker.io/jpolidor/pino:1.0.0 localhost:8000/pino:1.0.0

Copied docker.io/jpolidor/pino:1.0.0 => localhost:8000/pino:1.0.0
Digest: sha256:814892df0a9c24315bdf647d8f9f9c7f5e0825c21dd1976fd0b11d5dba1a3da0

Once we have this image in our local registry we want to attach to it an OCI Artifact: it can be a text file, zip etc. For this example I will attach a json and a txt file.

> oras attach localhost:8000/pino:1.0.0 --artifact-type example/txt ./file.txt:text/txt
Uploading ad96814c4506 file.txt
Uploaded  ad96814c4506 file.txt
Attached to localhost:8000/pino@sha256:814892df0a9c24315bdf647d8f9f9c7f5e0825c21dd1976fd0b11d5dba1a3da0
Digest: sha256:f8e6b137dbbb340899e8aa09a748e96407fa51d063ab0b36f3fea741cc026038

> oras attach localhost:8000/pino:1.0.0 --artifact-type example/json ./file.json:text/json
Uploading 77ff6c9dc2e8 file.json
Uploaded  77ff6c9dc2e8 file.json
Attached to localhost:8000/pino@sha256:814892df0a9c24315bdf647d8f9f9c7f5e0825c21dd1976fd0b11d5dba1a3da0
Digest: sha256:6b447ac8b51492e88fdd90489de79574acb4af3afc14550327d39cbc8844be2b

Let’s check the manifest for the artifacts above:

> oras manifest fetch localhost:8000/pino@sha256:f8e6b137dbbb340899e8aa09a748e96407fa51d063ab0b36f3fea741cc026038 --pretty

{
  "mediaType": "application/vnd.oci.artifact.manifest.v1+json",
  "artifactType": "example/txt",
  "blobs": [
    {
      "mediaType": "text/txt",
      "digest": "sha256:ad96814c4506fcaa9260233c286480f37fac713700af8220b912d9895c7c39d0",
      "size": 102621,
      "annotations": {
        "org.opencontainers.image.title": "file.txt"
      }
    }
  ],
  "subject": {
    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "digest": "sha256:814892df0a9c24315bdf647d8f9f9c7f5e0825c21dd1976fd0b11d5dba1a3da0",
    "size": 527
  },
  "annotations": {
    "org.opencontainers.artifact.created": "2023-02-17T22:53:11Z"
  }
}

> oras manifest fetch localhost:8000/pino@sha256:6b447ac8b51492e88fdd90489de79574acb4af3afc14550327d39cbc8844be2b --pretty
{
  "mediaType": "application/vnd.oci.artifact.manifest.v1+json",
  "artifactType": "example/json",
  "blobs": [
    {
      "mediaType": "text/json",
      "digest": "sha256:77ff6c9dc2e8a6db2ab82d8ff68b879b5df1acd3e4a6cc24b5cfc629da1ec6e9",
      "size": 38153,
      "annotations": {
        "org.opencontainers.image.title": "file.json"
      }
    }
  ],
  "subject": {
    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "digest": "sha256:814892df0a9c24315bdf647d8f9f9c7f5e0825c21dd1976fd0b11d5dba1a3da0",
    "size": 527
  },
  "annotations": {
    "org.opencontainers.artifact.created": "2023-02-17T23:54:52Z"
  }
}

Here we can see the blob we attached to the container image (the file.txt) and the subject, the container image we’re referring from this artifact (pino:1.0.0).

image_spec_4

Of course we can continue to attach other artifacts to the container image.

But the question now is: how can we access these artifacts? From the OCI distribution spec there is a new API called “referrers” that can give us all the artifact that are associated with a specific digest.

This API returns a OCI Index manifest that contains not multi-arch references but artifact references:

> curl --silent http://localhost:8000/v2/pino/referrers/sha256:814892df0a9c24315bdf647d8f9f9c7f5e0825c21dd1976fd0b11d5dba1a3da0 | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.artifact.manifest.v1+json",
      "digest": "sha256:6b447ac8b51492e88fdd90489de79574acb4af3afc14550327d39cbc8844be2b",
      "size": 533,
      "annotations": {
        "org.opencontainers.artifact.created": "2023-02-17T23:54:52Z"
      },
      "artifactType": "example/json"
    },
    {
      "mediaType": "application/vnd.oci.artifact.manifest.v1+json",
      "digest": "sha256:f8e6b137dbbb340899e8aa09a748e96407fa51d063ab0b36f3fea741cc026038",
      "size": 533,
      "annotations": {
        "org.opencontainers.artifact.created": "2023-02-17T22:53:11Z"
      },
      "artifactType": "example/txt"
    }
  ]
}

or via oras:

> oras discover -o tree localhost:8000/pino@sha256:814892df0a9c24315bdf647d8f9f9c7f5e0825c21dd1976fd0b11d5dba1a3da0
localhost:8000/pino@sha256:814892df0a9c24315bdf647d8f9f9c7f5e0825c21dd1976fd0b11d5dba1a3da0
├── example/json
│   └── sha256:6b447ac8b51492e88fdd90489de79574acb4af3afc14550327d39cbc8844be2b
└── example/txt
    └── sha256:f8e6b137dbbb340899e8aa09a748e96407fa51d063ab0b36f3fea741cc026038

You can also filter for artifactType:

> oras discover localhost:8000/pino@sha256:814892df0a9c24315bdf647d8f9f9c7f5e0825c21dd1976fd0b11d5dba1a3da0 --artifact-type=example/json
Discovered 1 artifact referencing localhost:8000/pino@sha256:814892df0a9c24315bdf647d8f9f9c7f5e0825c21dd1976fd0b11d5dba1a3da0
Digest: sha256:814892df0a9c24315bdf647d8f9f9c7f5e0825c21dd1976fd0b11d5dba1a3da0

Artifact Type   Digest
example/json    sha256:6b447ac8b51492e88fdd90489de79574acb4af3afc14550327d39cbc8844be2b

We can retrieve the blob of the artifact by doing:

> oras blob fetch --output file.txt localhost:8000/pino@sha256:ad96814c4506fcaa9260233c286480f37fac713700af8220b912d9895c7c39d0

where the sha256 is the digest to the blob.

We can also push a blob directly

> oras blob push localhost:8000/myblob file.json

Pushed localhost:8000/myblob
Digest: sha256:77ff6c9dc2e8a6db2ab82d8ff68b879b5df1acd3e4a6cc24b5cfc629da1ec6e9

Then if we try to retrieve it with oras manifest fetch localhost:8000/myblob@sha256:77ff6c9dc2e8a6db2ab82d8ff68b879b5df1acd3e4a6cc24b5cfc629da1ec6e9 we will get the content of this blob!

Conclusion

With the commands above we were able to move across OCI Artifacts and references using ORAS to query the local registry. Keep in mind that at the time of writing the OCI Image Format Specification is still a Release Candidate but Docker Hub has already announced the native support for OCI Artifact while the OCI Artifact Specification is still a RC . It’s just a matter of time until the full OCI specification will be released.