Deploy image to docker optionally, as well as GHCR by ljones140 · Pull Request #97 · clearlydefined/operations

ljones140 · 2024-10-03T17:47:47Z

What

To migrate deployment of Production Crawler to CD Azure from Microsoft owned Azure.
Changes to workflow app-build-and-deploy

This PR enables the optional docker image publish to Docker Hub, conditional on providing

Input docker-hub-username
secret DOCKERHUB_TOKEN

Without these the action only deploys to ghcr

Why

Unlocks ability to have production crawlers in Cd Azure means any member of Clearly Defined with access to the account can manage, support and troubleshoot. Currently we need to request help from MSFT if there are any issues.
Our other App Services the api and website are already using ghcr for images so we need to move crawler there as well
MSFT crawlers are deployed using webhook from dockerhub that triggers the deploy. So we still need to publish to dockerhub.

Testing

Using test branch https://github.com/clearlydefined/crawler/tree/deploy-production-cd-azure
connfigured to publish to my dockerhub, and deploy to a inactive prod crawler in CD azure

Successful prod deploy which publshses 2 images docker hub and ghcr: https://github.com/clearlydefined/crawler/actions/runs/11163856034
Sucessful dev deploy that publishes 1 gchr image https://github.com/clearlydefined/crawler/actions/runs/11164059253

notes:

I will squash the commits when I merge as there was a lot of back and forth to test

Without repo orgname

…oy-image-to-docker-optionally

elrayle

Really glad to see deploy to DockerHub coming together. I proposed an alternative approach to generating the image name twice. See what you think about that option.

elrayle · 2024-10-04T03:46:28Z

          echo -e "---- script log\n$script_log\n----"; \
          image_name=$(echo "$script_log" | tail -n 1)
-          echo "DOCKER_IMAGE_NAME_WITH_TAG=$image_name" >> $GITHUB_ENV
+          echo "GCHR_DOCKER_IMAGE_NAME_WITH_TAG=$image_name" >> $GITHUB_ENV


Based on the example for push-multi-registries, it appears that the image-name-with-tag is the same for both, except that in the case of ghcr, it is prepended with ghcr.io. It might be cleaner to determine the image name once for both and then do a prepend when they are used.

This would require removing ghcr.io from the image_base_name in the script and prepending it when the image name is used. This works here because the username for DockerHub is the same as the org (owner) name.

Thanks, I will try your proposal

I like that it reduced having the if statements on prefix in the shell script.

elrayle · 2024-10-04T03:47:52Z

-repo="$1"
-deploy_env="$2"
-image_tag="$3"
+prefix="$1"


With the previous suggestion, the prefix isn't needed. image_base_name will be $repo for both ghcr and Docker Hub.

elrayle · 2024-10-04T04:00:15Z

+      gchr-docker-image-name-with-tag: "${{ env.GCHR_DOCKER_IMAGE_NAME_WITH_TAG }}"
+      dockerhub-image-name-with-tag: "${{ env.DOCKERHUB_IMAGE_NAME_WITH_TAG }}"


There is a proposal below for a way to only get the image-name-with-tag once and then add the prepend at the point where these are used. An alternative would be to still get the image-name-with-tag only once and do the prepend here instead of at the point the tags are used.

Suggested change

gchr-docker-image-name-with-tag: "${{ env.GCHR_DOCKER_IMAGE_NAME_WITH_TAG }}"

dockerhub-image-name-with-tag: "${{ env.DOCKERHUB_IMAGE_NAME_WITH_TAG }}"

ghcr-docker-image-name-with-tag: "ghcr.io/${{ env.DOCKER_IMAGE_NAME_WITH_TAG }}"

dockerhub-image-name-with-tag: "${{ env.DOCKER_IMAGE_NAME_WITH_TAG }}"

elrayle · 2024-10-04T04:09:52Z

+          tags: |
+            ${{ needs.determine-image-name.outputs.gchr-docker-image-name-with-tag }}
+            ${{ needs.determine-image-name.outputs.dockerhub-image-name-with-tag }}


If following the previous proposal to have a single image-name-with-tag, you can either do the prepend here like the following or at the point where the tags are created if you think it is clearer to have separate outputs for the tags (as in the suggestion for job determine-image-name).

Suggested change

tags: |

${{ needs.determine-image-name.outputs.gchr-docker-image-name-with-tag }}

${{ needs.determine-image-name.outputs.dockerhub-image-name-with-tag }}

tags: |

"ghcr.io/${{ needs.determine-image-name.outputs.docker-image-name-with-tag }}"

${{ needs.determine-image-name.outputs.docker-image-name-with-tag }}

This looks great, much simpler than my approach

elrayle · 2024-10-04T04:11:29Z

+          tags: |
+            ${{ needs.determine-image-name.outputs.gchr-docker-image-name-with-tag }}
+            ${{ needs.determine-image-name.outputs.dockerhub-image-name-with-tag }}


One other question related to this, what happens if you have the tag that is expected to go to Docker Hub, but the login to Docker Hub didn't happen, which is what is expected for all but prod crawler deploy? Does it just push to ghcr.io in that case?

The tag is not generated and it's an empty string, As I've added the conditional if: ${{ inputs.docker-hub-username != '' }} in the step Determine Dockerhub Image Name

- name: Determine Dockerhub Image Name if: ${{ inputs.docker-hub-username != '' }} id: determine_dockerhub_name

I've tested this with a dev release and it only publishes the ghcr image.

ljones140 · 2024-10-04T08:26:56Z

@elrayle
Thanks for you review, I will attempt to implement your proposal.

It all depends on wether I can add the conditionals to ensure that dockerhub tag is added only when needed.

Create comma seperated list of tags

ljones140 · 2024-10-04T13:12:44Z

@elrayle I've attempted to implement the strategy you have suggested.

One subtle issue I had to deal with however.
Using the suggestion as documented here couples the dockerhub username with the GitHub repos org name.

          tags: |
            user/app:latest
            user/app:1.0.0
            ghcr.io/user/app:latest
            ghcr.io/user/app:1.0.0

But in order for me to test the action I've added my docker credentials into the crawler vars.
https://hub.docker.com/repository/docker/ljones140/crawler
I have to do this so we don't accidentally publish a production release. As the prod app services are deployed from a docker hub webhook.

So I've altered the determining image script so it doesn't take the org/repo just the repo_name.
And only call it once.

Then only add the dockerhub tag string conditionally

elrayle

This looks great. Thanks for pushing this through. There is a single suggestion to update the documentation of the output of the script.

Co-authored-by: E. Lynette Rayle <elrayle@users.noreply.github.com>

ljones140 added 23 commits October 2, 2024 13:19

Docker args

b3e65a2

Conditionally deploy to docker

e08700f

Dowgrade version as 3 doesn't exist yet

0e9069e

Correct tag

7b16e3f

Try bumping action

0eda6ed

Pass sercret

8c0dc9c

Tag the image so it goes to docker

fa4c959

tweak script to build tags for ghcr.io and docker

3adf6fb

Point to tweaked script

3b6fb3b

Script needs to build dockerhub tag

3b4ec17

Without repo orgname

Point to new script

c6e0730

Move condtions inside steps

b7ddcd8

Merge steps into one

a58e2e2

Only has one output now

971fd41

Correct step naming

71d8734

Correct output

2a0d92f

Attempt to add the tags to the end

86bf7d2

Use seperate env var

d6f322e

Update script

51a2e27

Tidyup add comments

5f1759a

Remove comment

49059bb

Merge branch 'main' of github.com:clearlydefined/operations into depl…

a9db3bd

…oy-image-to-docker-optionally

Update versions

8c5144c

elrayle reviewed Oct 4, 2024

View reviewed changes

ljones140 added 5 commits October 4, 2024 09:40

correct var name

421ecbc

image name script only needs to be run once

5ef9201

Create comma seperated list of tags

Point to branch with script

0cc20cf

Fix ids

e76bff2

Fix

73e1d7a

ljones140 added 4 commits October 4, 2024 12:59

Get correct version

47ac1f5

Fix tag concatination

00641a4

update determine issue script tests

6fc8db2

Update refs to next branch

917a217

elrayle approved these changes Oct 4, 2024

View reviewed changes

Comment thread scripts/app-workflows/determine-image-name.sh Outdated

Update scripts/app-workflows/determine-image-name.sh

6443d56

Co-authored-by: E. Lynette Rayle <elrayle@users.noreply.github.com>

ljones140 merged commit 15aec70 into main Oct 4, 2024

ljones140 deleted the deploy-image-to-docker-optionally branch October 4, 2024 16:28

		gchr-docker-image-name-with-tag: "${{ env.GCHR_DOCKER_IMAGE_NAME_WITH_TAG }}"
		dockerhub-image-name-with-tag: "${{ env.DOCKERHUB_IMAGE_NAME_WITH_TAG }}"

Uh oh!

Conversation

ljones140 commented Oct 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Testing

notes:

Uh oh!

elrayle left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

elrayle Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

ljones140 Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

elrayle Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

elrayle Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

elrayle Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

ljones140 Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

elrayle Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

ljones140 Oct 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ljones140 commented Oct 4, 2024

Uh oh!

ljones140 commented Oct 4, 2024

Uh oh!

elrayle left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ljones140 commented Oct 3, 2024 •

edited

Loading

ljones140 Oct 4, 2024 •

edited

Loading