Skip to content

[CI] Update docker images tag to 20240105-165030-51bdaec6#16344

Merged
vinx13 merged 4 commits intoapache:mainfrom
vinx13:ci-images-20240103-070130-97f6e6507
Jan 7, 2024
Merged

[CI] Update docker images tag to 20240105-165030-51bdaec6#16344
vinx13 merged 4 commits intoapache:mainfrom
vinx13:ci-images-20240103-070130-97f6e6507

Conversation

@vinx13
Copy link
Copy Markdown
Member

@vinx13 vinx13 commented Jan 3, 2024

This is to integrate changes in #16336

@junrushao
Copy link
Copy Markdown
Member

junrushao commented Jan 4, 2024

not sure about the CI failures...

@vinx13 vinx13 changed the title [CI] Update docker images tag to 20240103-070130-97f6e6507 [CI] Update docker images tag to 20240105-165030-51bdaec6 Jan 5, 2024
@vinx13
Copy link
Copy Markdown
Member Author

vinx13 commented Jan 6, 2024 •
edited
Loading

2024-01-06T00:34:29.983Z] docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
[2024-01-06T00:34:29.983Z] nvidia-container-cli: requirement error: unsatisfied condition: cuda>=11.8, please update your driver to a newer version, or use an earlier cuda container: unknown.

this seems related to ci gpu error. CI has cuda 11.7 installed while the docker image has 11.8, which can be run until the Nvidia updates on base image on November. We need to update the AMIs for GPU workers. @tqchen

@vinx13 vinx13 force-pushed the ci-images-20240103-070130-97f6e6507 branch from c2acc27 to 3e22f6e Compare January 6, 2024 22:51
@vinx13 vinx13 force-pushed the ci-images-20240103-070130-97f6e6507 branch from 3e22f6e to 9d23c57 Compare January 7, 2024 03:18
yongwww
yongwww approved these changes Jan 7, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@yongwww yongwww left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

yongwww
yongwww reviewed Jan 7, 2024
View reviewed changes
Comment thread ci/jenkins/generated/gpu_jenkinsfile.groovy

// command to start a docker container
docker_run = 'docker/bash.sh --env CI --env PLATFORM --env TVM_SHARD_INDEX --env TVM_NUM_SHARDS --env RUN_DISPLAY_URL --env PLATFORM --env SKIP_SLOW_TESTS --env TEST_STEP_NAME'
docker_run = 'docker/bash.sh --env CI --env PLATFORM --env TVM_SHARD_INDEX --env TVM_NUM_SHARDS --env RUN_DISPLAY_URL --env PLATFORM --env SKIP_SLOW_TESTS --env TEST_STEP_NAME --env NVIDIA_DISABLE_REQUIRE=true'
Copy link
Copy Markdown
Member

@yongwww yongwww Jan 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this effort! doc about the NVIDIA_DISABLE_REQUIRE variable. We may consider disabling it when upgrading the image/cuda in the future.

@vinx13 vinx13 merged commit 9caa179 into apache:main Jan 7, 2024
vinx13 added a commit to vinx13/tvm that referenced this pull request Jan 8, 2024
@vinx13
[CI] Update docker images tag to 20240105-165030-51bdaec6 (apache#16344)
131e8c8 
* [CI] Update docker images tag to 20240103-070130-97f6e6507

* Update images tag to 20240105-165030-51bdaec6

* Show cuda info in ci runner

* [CI] Set NVIDIA_DISABLE_REQUIRE=true
@vinx13 vinx13 mentioned this pull request Jan 8, 2024
Merged
@ysh329 ysh329 mentioned this pull request Apr 21, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yongwww yongwww yongwww approved these changes

@junrushao junrushao junrushao approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vinx13 @junrushao @yongwww