Skip to content

feat(2.53.5): opt-in fix for export.match from config; ensure ApplyConfig before export#259

Merged
bwplotka merged 1 commit intorelease-2.53.5-gmpfrom
gmpmatchfix
Nov 7, 2025
Merged

feat(2.53.5): opt-in fix for export.match from config; ensure ApplyConfig before export#259
bwplotka merged 1 commit intorelease-2.53.5-gmpfrom
gmpmatchfix

Conversation

@bwplotka
Copy link
Copy Markdown
Collaborator

@bwplotka bwplotka commented Sep 12, 2025

Implements step 1 from go/gmp:matchstuck

NOTE: This PR also expects ApplyConfig to be triggered for export to work. Otherwise it's noop (similar to disable export = true). IMO this is required for reliable startup behaviour for all options (especially matching). Imagine flag is setting = matchAll and config is setting match only "A". We don't want risk random metrics user don't expect (e.g. broken ones) due to startup races.

In practice Prometheus ensures ApplyConfig is triggered before DB is ready, but adding safecheck for other callers.

@bwplotka bwplotka force-pushed the gmpmatchfix branch 2 times, most recently from cfcdbfd to 62b447b Compare September 15, 2025 07:12
@bwplotka bwplotka marked this pull request as ready for review September 15, 2025 07:12
@bwplotka bwplotka requested a review from dashpole September 16, 2025 04:05
Copy link
Copy Markdown
Collaborator

@pintohutch pintohutch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM % questions/comments

We could also use a /prombench run to ensure we don't see any regressions in steady-state performance.

@pintohutch
Copy link
Copy Markdown
Collaborator

/prombench

@gmp-bot
Copy link
Copy Markdown
Collaborator

gmp-bot commented Oct 2, 2025

This benchmark run compares this PR (#259) with version v2.53.4-gmp.0-gke.1 of the GMP Collector.

After a successful deployment, the benchmarking results can be viewed on the Cloud Monitoring Dashboard.

To cancel or stop the benchmark run, comment /prombench cancel on this PR.

To view the results of the profiling process you can run the following commands. After running the commands, go to localhost:31802/profiles to view your profiling results after a few minutes.

gcloud container clusters get-credentials prombench --zone us-east4-a --project gpe-test-1
kubectl config use-context gke_gpe-test-1_us-east4-a_prombench
kubectl port-forward services/parca 31802:80

@pintohutch
Copy link
Copy Markdown
Collaborator

/prombench cancel

@pintohutch
Copy link
Copy Markdown
Collaborator

I couldn't see any results there... maybe our prombench setup needs fixing...

@gmp-bot
Copy link
Copy Markdown
Collaborator

gmp-bot commented Oct 2, 2025

Cancelling the benchmark run for this PR: #259.

@bwplotka
Copy link
Copy Markdown
Collaborator Author

bwplotka commented Oct 9, 2025

I'd be very surprised if it worked given we didn't touch it for ~year.

It might be only one thing though:

image

However, @pintohutch should we spend time fixing it now vs progressing on matcher stuckness? Is there anything you are are worried about in terms of perf in this PR? I am not adding any state and we don't filter on prombench.

Copy link
Copy Markdown
Collaborator Author

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for review!

As per above comment, I am not fixing prombench, but instead prioritize further parts of go/gmp:matchstuck

Copy link
Copy Markdown
Collaborator

@pintohutch pintohutch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great conversations - some more feedback for you (apologies for the delay)

@bwplotka bwplotka changed the title feat(2.53.5): opt-in fix for export.match from config feat(2.53.5): opt-in fix for export.match from config; ensure ApplyConfig before export Oct 27, 2025
@bwplotka
Copy link
Copy Markdown
Collaborator Author

PTAL, this should be now good to @pintohutch

I did add one more tiny change as a safeguard, please check the description note: #259 (comment)

Copy link
Copy Markdown
Collaborator

@pintohutch pintohutch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM % nit

Thanks for doing this!

Implements step 1 from go/gmp:matchstuck

Signed-off-by: bwplotka <bwplotka@gmail.com>
@bwplotka bwplotka merged commit 3c8f741 into release-2.53.5-gmp Nov 7, 2025
5 checks passed
@bernot-dev bernot-dev deleted the gmpmatchfix branch March 2, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants