Define naming of docker images for packages + registry

[ruflin]

The Elastic Package Registry (epr.elastic.co) run a docker container that contains the packages + the registry service itself. Today the packages are in the same repository as the registry but this is changing. Packages will be moving to https://github.com/elastic/package-storage Also there is work ongoing to have a Dockerfile available that is independent of both repository.

This means the Docker image will contain data from two different sources. For debugging and support purpose I think it is important that we can identify what exactly went into this Docker image to potentially reproduce issues.

My proposal related to naming each Docker image is as following:

{date}-{package-storage-hash}-{package-registry-hash}

hash in the above can be a commit hash but also a branch or a tag. The date part is for human and sorting purpose to identify when exactly an image was built. Two examples of the above would look as following:

• 202004231234-c890763-c1d2c2c
• 202004231234-c890763-v0.3.0

The package hash will change much more often than the registry one but it also the more important one for debugging purpose. That is why I think it should come first.

To build this image, both the package-storage hash and package-registry hash are passed to the Dockerfile and then used to check out the correct version (https://github.com/elastic/package-registry/blob/master/dev/release/Dockerfile#L19).

Questions

• Is the above possible to build / automate? Does it make sense?
• What is the date/time format we should use?

[ruflin]

@kuisathaverat Would be great to get your feedback / input on the above.

[kuisathaverat]

• Is the above possible to build / automate? Does it make sense?

It depends, 😄 What are the use cases for that tag? I think on the following

• Someone wants to get the latest version
• Someone wants to get a specific version of the package-registry
• Someone wants to get a specific version of the package-registry and specific version of the package-storage

There are some questions about what we will release, or how we will release:

• Would we release several versions of the same package-registry on the same day? probably yes, but would change the package-storage?
• Would we release old versions of the package-registry with new versions of the package-storage?
• What is more important the package-registry version or the package-storage version?

we can have several tags for the same version depending on the use, a date based tag is good for human usage, but again it depends on how we will release the versions it would be the latest version or something else. A tag {package-storage-hash}-{package-registry-hash} or {package-registry-hash}-{package-storag-hash} is really good for developers we should release one of those too.

• What is the date/time format we should use?

no doubt on that ISO-8601, it would be something like 20200423T121930Z

[ruflin]

• Several releases of the package registry in the same day? Probably not. I expect the package registry to change very rarely and if there is a change, it is not urgent.
• Old versions of the package registry with new version of the package storage: No
• Package storage version is more important from my perspective as it changes much more often.

Use cases for the tags are mainly around debugging: A user reports a problem and we want to reproduce the exact same setup locally. For this we need to be able to look at the current running contains and see what exact version is running.

In the future there will also be a part around someone wants to deploy his own registry. But for this we could introduce a tag aligned with the Elastic Stack version. In the background, it would also have a tag like discussed before. Here probably also the "latest" comes into play.

All the above also reminds me it would be nice to have some logs on "when" we rolled out a new version and which one. I assume we get this from the kubernetes logs?

[kuisathaverat]

Old versions of the package registry with new version of the package storage: No
Package storage version is more important from my perspective as it changes much more often.

This make the {package-registry-hash} redundant, a {package-storage-hash} would be only available for a specific {package-registry-hash}.

Several releases of the package registry in the same day? Probably not. I expect the package registry to change very rarely and if there is a change, it is not urgent.

This makes the timestamp a unique tag for each version.

All the above also reminds me it would be nice to have some logs on "when" we rolled out a new version and which one. I assume we get this from the kubernetes logs?

When you create/restart a pod it stores the timestamp in the metadata.

[ruflin]

I think I need to clarify my comment related to the registry and storage. If a new registry is released an no new packages come it, we could still do a release with the same storage of packages as a previous version of the registry. This only applies to the current one, but not any older.

Agree that the timestamp is unique. I think the part I miss is how will a user tell based on the timestamp which version of the registry and package is packaged inside without having to either go into the container or check git logs on what exactly was there at the moment it was built?

[kuisathaverat]

I understand that we will not release more than a version per day, the time stamp has the date and time of the release, so identified a release minute. We do not release older package-registry versions with new package-storage versions. Let's put an example.

• The package registry is in the version 1.0.0 (sha1 AAAAAAA)

• The storage registry is in the version 1.0.0 (sha1 BBBBBBB)

• Today is 202004280000

• we release a version of the release-package+release-storage at 2020-04-20 17:10

• Proposed tags will be 202004281710-BBBBBBB-AAAAAAA and 202004281710-BBBBBBB-v1.0.0

• We detect some mistake in the storage-registry and we have to release a new version 1.0.1 (sha1 CCCCCCC) in the same day at 17:35 (shit happens), it will have the tags 202004281735-CCCCCCC-AAAAAAA and 202004281735-CCCCCCC-v1.0.0

• The next day we release a new version of the package registry 1.0.1 (sha1 DDDDDDD) at 10:12, the tags will be 202004291012-CCCCCCC-DDDDDDD and 202004291012-CCCCCCC-v1.0.1

as you can see the timestamp identifies a tuple of package-storage + package-registry

[ruflin]

I synced up with @kuisathaverat over zoom. Following conclusions.

• Releases are triggered by tags on both Github repositories
• The docker image name is 202004281710-BBBBBBB-AAAAAAA
• The hash versions also as labels to the docker image for debugging purpose
• The hash versions should also be to the k8s pods as labels for debugging purpose
• A release must be approved through Jenkins

A release will look as following:

• A CI job is looking for release-production tag changes in package-registry and package-storage
• If one of the tags is updated, the script will figure out the commit hash of each tag and trigger a build
• This build then needs to be approved in Jenkins. E-Mail is sent out to team to be approved.
• As soon as the process is approved, the image 202004281710-BBBBBBB-AAAAAAA is built. The timestamp is the timestamp of when the build was approved.

@kuisathaverat Any opinion on the tag names?

• release:production
• release/production
• release-production

[ruflin]

@kuisathaverat Just realised the above will work any tag / branch we have. To try out the above, could we already build it based on the master branches for the two repository and use this to deploy our staging environment? I already adjusted the Dockerfile: https://github.com/elastic/package-registry/blob/master/dev/release/Dockerfile It will need some cleanup and optimisations but should work for now. I plan to move this Dockerfile to the package-storage as I think it is more fitting there.

[kuisathaverat]

I was thinking in something like this:

• A CI job is looking for tag creation in package-registry and package-storage, the tag are regular tag versions (v1.0.0)
• This build then needs to be approved in Jenkins. E-Mail is sent out to team to be approved.
• As soon as the process is approved, the image 202004281710-BBBBBBB-AAAAAAA is built. The timestamp is the timestamp of when the build was approved.
• This new release is deployed in the staging environment
• when you are confident enough that this versions is OK you promote that version (Docker image) to the production environment.

About to deploy master in some environment, this environment is an "edge" enviroment with latest changes, not sure if the staging is the correct one.

[ruflin]

The package-storage will have so many releases, the regular versioning would not work here. The nice part about using the tag as I proposed above, it can also be used to roll back. Having just 1 tag for each release type also removes logic on the CI to figure out what is newest. It only must bring the two equal tags together.

I like the idea about promoting to prod. How would this work? Also in Jenkins?

For staging: Yes, at the moment it is our testing environment and we will need to rename / a new one in the future.