Repository navigation
Stabilize TestBootstrapClientServerRotation on slow machines - #2787
Open
lechedesnatada23 wants to merge 1 commit into
Open
lechedesnatada23 wants to merge 1 commit into
lechedesnatada23 wants to merge 1 commit into
Conversation
The test issues certificates with a 5s lifetime and TLSRenewer schedules each renewal at ~2/3 of the certificate lifetime, which leaves only ~1-2s of margin between a renewal and the expiry of the certificate in use. On slow or loaded machines a scheduling stall longer than this margin makes the next mTLS handshake present an already-expired certificate; because /renew requires a valid client certificate, the client cannot recover and the test fails (see smallstep#873). Increase the test certificate lifetime to 15s and the renewal waits to 12s. This grows the margin to ~5s while still guaranteeing at least one renewal of both the client and the server certificate on every test phase. Test-only change; no production behavior is modified. Fixes smallstep#2713
github-actions
Bot
added
the
needs triage
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Test-only change to make
TestBootstrapClientServerRotationstable on slow or loaded machines.Root cause: the test issues certificates with a 5s lifetime (
defaultTLSCertDurationinca/testdata/rotate-ca-*.json), andTLSRenewerschedules each renewal at ~2/3 of the certificate lifetime (see #873), leaving only ~1.7–2.9s between the scheduled renewal and the expiry of the certificate in use. On a slow or loaded machine, a scheduling stall longer than that margin means the next mTLS handshake presents an already-expired certificate. Since/renewrequires a valid client certificate (and the bootstrap token cannot be re-used to re-sign, as OTTs are protected against reuse), the client cannot recover and the test fails — which matches the log in #2713 (a handshake at 15:55:08 hitting a certificate that had just expired at 15:55:08). This is also the same class of instability that led to the test being skipped whenCI=true. Fixing the underlying unrecoverable state would be a behavior change (e.g. a grace window on/renew), so it's left to the discussion in #2713/#873; this PR only stabilizes the test.Change: increase the test certificate lifetime to 15s and the renewal waits from 5s to 12s. This grows the margin before expiry to ~5s while still guaranteeing at least one renewal of both the client and the server certificate on every phase (with 15s certificates, renewals happen every ~9–10s, and the final phase runs ~24s after the first reload).
No production behavior is modified. The only cost is the test itself going from ~21s to ~40s, which only affects local runs (in this repository's CI the test is skipped).
Validation
go test ./ca/ -run TestBootstrapClientServerRotation -count=3passes 3/3 with the machine under full CPU saturation (4 busy loops on 4 cores), ~39.5s per run.go test -short -count=1 ./ca/...passes.golangci-lint runwith the project config reports 0 issues;gofmtis clean.Fixes #2713
Transparency note: this contribution was prepared with AI assistance and validated with the commands above.