fix: workaround podman stop/rm timeout in rootless mode with --pid host#1982
Open
xz-dev wants to merge 6 commits into89luca89:mainfrom
Open
fix: workaround podman stop/rm timeout in rootless mode with --pid host#1982xz-dev wants to merge 6 commits into89luca89:mainfrom
xz-dev wants to merge 6 commits into89luca89:mainfrom
Conversation
231511f to
832522a
Compare
In rootless mode with --pid host (distrobox default), podman stop/rm --force times out because "crun kill --all" fails when the container's cgroup-path is empty. Root cause: When using --pid host, the container shares the host's PID namespace and crun doesn't create a dedicated cgroup. The "crun kill --all" command relies on cgroup to find processes, but with empty cgroup-path, no processes are found and killed. Solution: Call "podman kill" before stop/rm, which uses "crun kill" (without --all flag) that sends signals directly to the container's init process PID, bypassing the cgroup lookup issue. Closes: 89luca89#1939 See also: chimera-linux/cports#1718 Signed-off-by: xz-dev <xiangzhedev@gmail.com>
Add documentation and warning about a limitation of --pid host mode: orphaned child processes may remain after container stop/rm. This is a known podman limitation (containers/podman#11888), not something introduced by our workaround. The workaround only kills the init process; child processes that daemonized or backgrounded will persist. Changes: - Add warning in distrobox-create for rootless podman without --unshare-process - Document the limitation in distrobox-stop and distrobox-rm comments - Recommend --unshare-process for full process cleanup Signed-off-by: xz-dev <xiangzhedev@gmail.com>
832522a to
17a0ba9
Compare
Document that processes started via "podman exec" (e.g., distrobox-enter) run in separate process groups and will become orphaned when the container stops. This is in addition to the existing daemonized process limitation. The workaround only kills the container's init process, not exec'd processes. Also only apply the workaround when PidMode is "host", since this is the specific condition that causes the cgroup-path to be empty. Signed-off-by: xz-dev <xiangzhedev@gmail.com>
0751fdd to
900531f
Compare
…etection Instead of checking PidMode=host, now check if crun's cgroup-path is actually empty. This is more accurate because: - With systemd cgroup manager (e.g., Fedora), cgroup delegation works even with --pid host, so the workaround is not needed - With cgroupfs manager, cgroup-path may be empty regardless of PidMode Changes: - Check crun status file for empty cgroup-path instead of PidMode - Move warning from distrobox-create to distrobox-stop/rm (more accurate) - Show warning only when workaround is actually applied - Simplify sed command (single sed instead of grep|sed) The workaround is applied proactively (before stop/rm) rather than as a fallback, to avoid masking other potential failures. Signed-off-by: xz-dev <xiangzhedev@gmail.com>
d66dd6a to
c7f9472
Compare
…nation - Add background explanation of cgroup delegation in code comments - Explain why non-systemd systems (OpenRC/elogind) need manual configuration - Update user warnings to be more helpful with actionable fix reference - Add link to issue 89luca89#1939 for troubleshooting Signed-off-by: xz-dev <xiangzhedev@gmail.com>
c7f9472 to
4091335
Compare
The core issue is whether cgroup is available, not whether running as rootless. This change: - Remove rootless-only restriction from the workaround - Support both rootless (/run/user/UID/crun/) and rootful (/run/crun/) crun status paths - Update comments to clarify that cgroup unavailability can occur in both rootless (without delegation) and rootful (cgroup disabled) modes Signed-off-by: xz-dev <xiangzhedev@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
podman stop/podman rm --forcetimes out when crun's cgroup-path is empty. This commonly happens on non-systemd systems (OpenRC/elogind) where cgroup delegation is not automatically configured. The issue can affect both rootless and rootful modes when cgroup is unavailable.Background: cgroup delegation
In cgroup v2, only root can create sub-cgroups by default. "Delegation" means granting a non-root user write access to a cgroup subtree, allowing them to create child cgroups.
/sys/fs/cgroup/user.slice/user-1000.slice/) to users./sys/fs/cgroup, so podman cannot create cgroups for containers.To fix this properly, configure cgroup delegation:
See: #1939
Why
--pid hostis problematic but--unshare-processis notThis is due to Linux kernel's PID namespace behavior:
With PID namespace isolation (default /
--unshare-process)When a container has its own PID namespace, the Linux kernel provides automatic process cleanup. According to pid_namespaces(7):
This is a kernel-level guarantee that doesn't depend on cgroup:
SIGKILLs all processes in that namespaceWith
--pid hostContainer processes share the host's PID namespace:
Process cleanup mechanisms comparison
zap_pid_ns_processes()--pid host+ cgroup availablecgroup.kill/cgroup.procs--pid host+ no cgroupThis PR provides a workaround for the last case (no reliable cleanup method exists).
Root Cause
Without available cgroup, crun cannot create a cgroup for the container. When cgroup-path is empty (or null),
crun kill --allcannot enumerate processes via cgroup, causing podman stop to timeout.Solution
Check if crun's cgroup-path is actually empty. If so, call
podman killbeforestop/rm --force, which sends signals directly to the container's init process PID, bypassing the cgroup lookup issue.The workaround:
/run/user/UID/crun/) and rootful (/run/crun/) modesLimitation
This workaround only kills the init process, not orphaned child processes:
podman exec(e.g., distrobox-enter) run in separate process groups--pid=hostin rootless mode doesn't clean up backgrounded child processes (including conmon) containers/podman#11888)This is a known limitation without proper cgroup (see containers/podman#11888). To ensure all processes are cleaned up, use
--unshare-processwhen creating the container.A warning is displayed when the workaround is applied, with a link to the fix documentation.
Changes
distrobox-stop: Check crun cgroup-path; if empty, callpodman killfirst and show warningdistrobox-rm: Check crun cgroup-path; if empty, callpodman killfirst and show warningTest Results
Closes #1939
See also: chimera-linux/cports#1718