process_xxx_memory statistics for macOS (cgo) #1616

mharbison72 · 2024-09-04T04:27:37Z

This is built on top of the much easier to get process metrics in PR1600, so only the last commit is interesting here (and the commit message has more details). I don't expect or want this to be landed as-is, but am posting it here to help with any discussion about finishing off #1590.

I've not paid much attention to the RSS values of Go processes on macOS, but after ~5 minutes of banging on my own exporter with curl, I only ever see the value going up. The values agree with the ps command. There should be ~0 code other than from this module running, but we'll see what happens.

@bwplotka

mharbison72 · 2024-09-07T20:46:09Z

As an alternative to this, I have now managed to also get it working without cgo, using github.com/ebitengine/purego. The problem was a native structure had a 32 bit field, followed by a 64 bit field, but was packed. This package doesn't handle that missing padding, but encoding/binary.Read() does. This alternative allows compiling on any platform and always exporting the metrics (whereas the cgo solution here silently ignores the C code in favor of a stub method that causes the metrics to not be exported if cgo is not supported).

The one wrinkle that I noticed is that the command code passed to the native API is a macro that evaluates to 5 on amd64, and 18 on arm64. It's this kind of nuisance stuff that gives me pause using this mechanism, though there's clear benefit instead of silently dropping some metrics. Maybe a reasonable solution is using github.com/ebitengine/purego, plus some _test.go that uses cgo to pass back offsetof(), sizeof(), and any other C macro evaluation that is relevant, so they can be compared against constants defined in Go. It's more or less what I did to validate my reading of the C headers, and would provide sanity checks similar to just using cgo, but without hassling client program build environments. But IDK if there are any macOS test runners available for this project.

So I guess the choices are simpler code, but slightly limited (this cgo PR), or a dependency + a bit more code without C compiler checks by default, but more flexible for users. Let me know if this second approach is reasonable, and I can clean up and submit this alternative in a different PR.

bwplotka · 2024-09-23T14:44:08Z

Epic! I would love to avoid github.com/ebitengine/purego or any dependency. We have a backlash the moment client_golang introduces more dependencies than needed.

Personally I would rather have something optional, perhaps requirement for building with CGO IF you want darwin RSS (otherwise it will not expose that metric) would be ok. If we want something more complex we might want to put process collector as a separate Go module/project (:

mharbison72 · 2024-09-23T16:54:11Z

Epic! I would love to avoid github.com/ebitengine/purego or any dependency. We have a backlash the moment client_golang introduces more dependencies than needed.

Totally understandable, and that's why I opted for cgo first. FTR, the purego version is #1629 (though I used a newer structure that better supports arm64, so this code needs to be updated too).

Personally I would rather have something optional, perhaps requirement for building with CGO IF you want darwin RSS (otherwise it will not expose that metric) would be ok.

Can you explain? In my mind, that's how this code works- if CGO_ENABLED=1 (e.g. go.exe matches the current platform/arch, and the C compiler is available), you get the cgo code, and the metric exposed. If CGO_ENABLED=0 (either manually or the C compiler is unavailable), you get the stub function from prometheus_collector_memory_noop.go, and the 0 value signals the Collector to skip the metric.

Are you suggesting an explicit build tag? One of the reservations expressed in the IRC channel about cgo was people needing to update their build scripts to use it, and it would seem like an explicit build tag would have the same issue. I like how this is able to avoid that concern (though it's almost too magical in that when I make a fat binary by compiling both architectures, only one has CGO_ENABLED=1 by default). Maybe skipping the cgo code only if there's a no-darwin-cgo build tag would get people to investigate and fix their build scripts?

If we want something more complex we might want to put process collector as a separate Go module/project (:

I would like to keep this as simple as possible. I'm not sure if you're suggesting another module in this repo (or another GH/prometheus repo), but that will probably require more knowledge of the project than I have.

bwplotka · 2024-09-23T20:39:23Z

prometheus/process_collector.go

 }

+func init() {
+	// Debugging to show the correct implementation is used based on CGO_ENABLED=0.


Do we want to merge that though?

Do we want to merge that though?

No. This initial PR was just throwing a dart to show it working, and get either a "this cgo is bad, please avoid it", or "3rd party deps are bad, use cgo", or "this isn't workable at all" response to resolve the various discussions about #1590.

Let me review deeper, is this ready to go then?

It sounds like you're agreeable to the general idea here, so give me a day or two to migrate this to the APIs that I used in the purego version for portability. Is there a preferred format for C code, or is anything fine as long as it's reasonable?

bwplotka · 2024-09-23T20:39:37Z

Can you explain? In my mind, that's how this code works- if CGO_ENABLED=1 (e.g. go.exe matches the current platform/arch, and the C compiler is available), you get the cgo code, and the metric exposed. If CGO_ENABLED=0 (either manually or the C compiler is unavailable), you get the stub function from prometheus_collector_memory_noop.go, and the 0 value signals the Collector to skip the metric.

No, this would work fine I guess, it's what I meant. Let me review deeper, is this ready to go then?

prometheus#1600) Unfortunately, these values aren't available from getrusage(2), or any other builtin Go API. Go itself doesn't provide a mechanism (like on Windows) to call into system libraries. Using a 3rd party package[1] to dynamically call system libraries was proposed and rejected, to avoid adding to the number of dependencies. That leaves using cgo, which is used here when available. When not available (either because of cross compiling or explicitly disabling it), a stub function is linked instead, and the metrics are not exported. That way, cross compiling of other platforms is unaffected (and can also still be done with Darwin too, but at the cost of not exporting these metrics). Note that building an amd64 image on an arm64 mac or vice-versa is cross compiling, and will use the stub method by default. This can be avoided by setting `CGO_ENABLED=1` in the environment to force the use of cgo for both architectures. I'm unsure of the usefulness of the potential adjustment made to the virtual memory value after calling `mach_vm_region()`. I've not seen that code get run with a native amd64 or arm64 image, or with an amd64 image running under Rosetta. But that's what the `ps(1)` command does, and I think we should report what the system tools do. When I was testing this on a beta of macOS 15 with Go 1.21.13 (the current minimum support for this module), the amd64 image ran fine under Rosetta, but the arm64 image immediately printed a message that it was killed, even prior to the cgo call. This seems to be a recurring issue on macOS[2][3], and passing `-ldflags -s` to `go build` avoided the issue. Go 1.23.1 worked out of the box, without fiddling with linker flags, so I don't think this is an issue- Go 1.21 is simply too old to support macOS 15, but I thought it was worth noting. I supposed we could gate the cgo code with an additional build flag, if anyone is concerned about this. [1] https://github.com/ebitengine/purego [2] golang/go#19841 (comment) [3] golang/go#11887 (comment) Signed-off-by: Matt Harbison <mharbison72@gmail.com>

mharbison72 · 2024-09-24T22:55:23Z

@bwplotka - Now it's ready for review. I'll close the purego one when this is closed out. Assuming this technique is ok, the network send/receive stats are the only two metrics missing for parity with Linux.

It looks like the changelog is mostly managed outside of code changes, but LMK if you want me to change it in the next round (or a followup PR if this is acceptable). Needing to enable cgo explicitly when making fat binaries is probably sneaky enough to be worthy of calling out.

prometheus/process_collector_cgo_darwin.go

bwplotka

Beautiful, well done!

Tiny nit around naming file consistency, but otherwise LGTM! We will fix changelog later.

bwplotka · 2024-09-25T10:38:48Z

prometheus/process_collector_darwin.c

+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//


can we rename this file to process_collector_cgo_darwin.c too?

mharbison72 · 2024-09-25T16:23:51Z

Sorry about that- I screwed up the copy/paste and trimmed the "prometheus/" directory off. It should be as intended now.

kakkoyun

LGTM! Thank you for your contribution! 🙌

…S using optional cgo (prometheus#1616) Unfortunately, these values aren't available from getrusage(2), or any other builtin Go API. Go itself doesn't provide a mechanism (like on Windows) to call into system libraries. Using a 3rd party package[1] to dynamically call system libraries was proposed and rejected, to avoid adding to the number of dependencies. That leaves using cgo, which is used here when available. When not available (either because of cross compiling or explicitly disabling it), a stub function is linked instead, and the metrics are not exported. That way, cross compiling of other platforms is unaffected (and can also still be done with Darwin too, but at the cost of not exporting these metrics). Note that building an amd64 image on an arm64 mac or vice-versa is cross compiling, and will use the stub method by default. This can be avoided by setting `CGO_ENABLED=1` in the environment to force the use of cgo for both architectures. I'm unsure of the usefulness of the potential adjustment made to the virtual memory value after calling `mach_vm_region()`. I've not seen that code get run with a native amd64 or arm64 image, or with an amd64 image running under Rosetta. But that's what the `ps(1)` command does, and I think we should report what the system tools do. When I was testing this on a beta of macOS 15 with Go 1.21.13 (the current minimum support for this module), the amd64 image ran fine under Rosetta, but the arm64 image immediately printed a message that it was killed, even prior to the cgo call. This seems to be a recurring issue on macOS[2][3], and passing `-ldflags -s` to `go build` avoided the issue. Go 1.23.1 worked out of the box, without fiddling with linker flags, so I don't think this is an issue- Go 1.21 is simply too old to support macOS 15, but I thought it was worth noting. I supposed we could gate the cgo code with an additional build flag, if anyone is concerned about this. [1] https://github.com/ebitengine/purego [2] golang/go#19841 (comment) [3] golang/go#11887 (comment) Signed-off-by: Matt Harbison <mharbison72@gmail.com> Signed-off-by: Eugene <eugene@amberpixels.io>

…23.2 (#9) This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [github.com/prometheus/client_golang](https://redirect.github.com/prometheus/client_golang) | `v1.20.5` → `v1.23.2` | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fprometheus%2fclient_golang/v1.23.2?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fprometheus%2fclient_golang/v1.20.5/v1.23.2?slim=true) | --- ### Release Notes <details> <summary>prometheus/client_golang (github.com/prometheus/client_golang)</summary> ### [`v1.23.2`](https://redirect.github.com/prometheus/client_golang/releases/tag/v1.23.2): - 2025-09-05 [Compare Source](https://redirect.github.com/prometheus/client_golang/compare/v1.23.1...v1.23.2) This release is made to upgrade to prometheus/common v0.66.1, which drops the dependencies github.com/grafana/regexp and go.uber.org/atomic and replaces gopkg.in/yaml.v2 with go.yaml.in/yaml/v2 (a drop-in replacement). There are no functional changes. <details> <summary>All Changes</summary> - \[release-1.23] Upgrade to prometheus/common\@v0.66.1 by [@aknuds1](https://redirect.github.com/aknuds1) in [#1869](https://redirect.github.com/prometheus/client_golang/pull/1869) - \[release-1.23] Cut v1.23.2 by [@aknuds1](https://redirect.github.com/aknuds1) in [#1870](https://redirect.github.com/prometheus/client_golang/pull/1870) </details> **Full Changelog**: <prometheus/client_golang@v1.23.1...v1.23.2> ### [`v1.23.1`](https://redirect.github.com/prometheus/client_golang/releases/tag/v1.23.1): - 2025-09-04 [Compare Source](https://redirect.github.com/prometheus/client_golang/compare/v1.23.0...v1.23.1) This release is made to be compatible with a backwards incompatible API change in prometheus/common v0.66.0. There are no functional changes. <details> <summary>All Changes</summary> - \[release-1.23] Upgrade to prometheus/common v0.66 by [@aknuds1](https://redirect.github.com/aknuds1) in [#1866](https://redirect.github.com/prometheus/client_golang/pull/1866) - \[release-1.23] Cut v1.23.1 by [@aknuds1](https://redirect.github.com/aknuds1) in [#1867](https://redirect.github.com/prometheus/client_golang/pull/1867) </details> **Full Changelog**: <prometheus/client_golang@v1.23.0...v1.23.1> ### [`v1.23.0`](https://redirect.github.com/prometheus/client_golang/releases/tag/v1.23.0): - 2025-07-30 [Compare Source](https://redirect.github.com/prometheus/client_golang/compare/v1.22.0...v1.23.0) - \[CHANGE] Minimum required Go version is now 1.23, only the two latest Go versions are supported from now on. [#1812](https://redirect.github.com/prometheus/client_golang/issues/1812) - \[FEATURE] Add WrapCollectorWith and WrapCollectorWithPrefix [#1766](https://redirect.github.com/prometheus/client_golang/issues/1766) - \[FEATURE] Add exemplars for native histograms [#1686](https://redirect.github.com/prometheus/client_golang/issues/1686) - \[ENHANCEMENT] exp/api: Bubble up status code from writeResponse [#1823](https://redirect.github.com/prometheus/client_golang/issues/1823) - \[ENHANCEMENT] collector/go: Update runtime metrics for Go v1.23 and v1.24 [#1833](https://redirect.github.com/prometheus/client_golang/issues/1833) - \[BUGFIX] exp/api: client prompt return on context cancellation [#1729](https://redirect.github.com/prometheus/client_golang/issues/1729) <details> <summary>All Changes</summary> * Update example test by @SuperQ in prometheus/client_golang#1770 * build(deps): bump golang.org/x/net from 0.34.0 to 0.36.0 in /tutorials/whatsup by @dependabot[bot] in prometheus/client_golang#1776 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1771 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1778 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1779 * build(deps): bump google.golang.org/protobuf from 1.36.5 to 1.36.6 in /exp by @dependabot[bot] in prometheus/client_golang#1782 * build(deps): bump github.com/prometheus/common from 0.62.0 to 0.63.0 in /exp by @dependabot[bot] in prometheus/client_golang#1781 * build(deps): bump github.com/prometheus/common from 0.62.0 to 0.63.0 by @dependabot[bot] in prometheus/client_golang#1783 * build(deps): bump google.golang.org/protobuf from 1.36.5 to 1.36.6 by @dependabot[bot] in prometheus/client_golang#1784 * build(deps): bump github.com/prometheus/procfs from 0.15.1 to 0.16.0 by @dependabot[bot] in prometheus/client_golang#1786 * chore: Upgrade golangci-lint to v2 by @kakkoyun in prometheus/client_golang#1789 * build(deps): bump the github-actions group across 1 directory with 3 updates by @dependabot[bot] in prometheus/client_golang#1790 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1791 * Add `WrapCollectorWith` and `WrapCollectorWithPrefix` by @colega in prometheus/client_golang#1766 * feat(github-actions): add Go file change detection to golangci-lint workflow by @kakkoyun in prometheus/client_golang#1794 * chore(ci): Fix flaky tests by @kakkoyun in prometheus/client_golang#1795 * build(deps): bump golang.org/x/net from 0.36.0 to 0.38.0 in /tutorials/whatsup by @dependabot[bot] in prometheus/client_golang#1799 * test(registry): Add goleak-based goroutine leak detection by @surinkim in prometheus/client_golang#1797 * build(deps): bump go.uber.org/goleak from 1.2.0 to 1.3.0 by @dependabot[bot] in prometheus/client_golang#1806 * build(deps): bump the github-actions group with 2 updates by @dependabot[bot] in prometheus/client_golang#1804 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1809 * Add exemplars for native histograms by @shivanthzen in prometheus/client_golang#1686 * build(deps): bump golang.org/x/sys from 0.30.0 to 0.32.0 by @dependabot[bot] in prometheus/client_golang#1807 * build(deps): bump github.com/prometheus/client_model from 0.6.1 to 0.6.2 by @dependabot[bot] in prometheus/client_golang#1805 * build(deps): bump github.com/prometheus/procfs from 0.16.0 to 0.16.1 by @dependabot[bot] in prometheus/client_golang#1808 * build(deps): bump golang.org/x/net from 0.35.0 to 0.38.0 by @dependabot[bot] in prometheus/client_golang#1800 * Update supported Go versions by @SuperQ in prometheus/client_golang#1812 * Cleaup Go modules by @SuperQ in prometheus/client_golang#1813 * fix: client prompt return on context cancellation by @umegbewe in prometheus/client_golang#1729 * Simplify buf binary install by @SuperQ in prometheus/client_golang#1814 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1815 * build(deps): bump the github-actions group with 5 updates by @dependabot[bot] in prometheus/client_golang#1817 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1821 * exp/api: Bubble up status code from writeResponse by @saswatamcode in prometheus/client_golang#1823 * build(deps): bump github.com/prometheus/common from 0.64.0 to 0.65.0 by @dependabot[bot] in prometheus/client_golang#1827 * build(deps): bump github.com/prometheus/common from 0.64.0 to 0.65.0 in /exp by @dependabot[bot] in prometheus/client_golang#1828 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1831 * Update runtime metrics for Go v1.23 and v1.24 by @aknuds1 in prometheus/client_golang#1833 * Fix `errNotImplemented` reference by @aknuds1 in prometheus/client_golang#1835 * build(deps): bump the github-actions group with 3 updates by @dependabot[bot] in prometheus/client_golang#1826 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1832 * Cut v1.23.0-rc.0 by @vesari in prometheus/client_golang#1837 * cut v1.23.0-rc.1 by @vesari in prometheus/client_golang#1842 </details> #### New Contributors * @surinkim made their first contributi[https://github.com/prometheus/client_golang/pull/1797](https://redirect.github.com/prometheus/client_golang/pull/1797)l/1797 * @umegbewe made their first contributi[https://github.com/prometheus/client_golang/pull/1729](https://redirect.github.com/prometheus/client_golang/pull/1729)l/1729 * @aknuds1 made their first contributi[https://github.com/prometheus/client_golang/pull/1833](https://redirect.github.com/prometheus/client_golang/pull/1833)l/1833 **Full Changelog**: <prometheus/client_golang@v1.22.0...v1.23.0> ### [`v1.22.0`](https://redirect.github.com/prometheus/client_golang/releases/tag/v1.22.0): - 2025-04-07 [Compare Source](https://redirect.github.com/prometheus/client_golang/compare/v1.21.1...v1.22.0) :warning: This release contains potential breaking change if you use experimental `zstd` support introduce in [#1496](https://redirect.github.com/prometheus/client_golang/issues/1496) :warning: Experimental support for `zstd` on scrape was added, controlled by the request `Accept-Encoding` header. It was enabled by default since version 1.20, but now you need to add a blank import to enable it. The decision to make it opt-in by default was originally made because the Go standard library was expected to have default zstd support added soon, [golang/go#62513](https://redirect.github.com/golang/go/issues/62513) however, the work took longer than anticipated and it will be postponed to upcoming major Go versions. e.g.: > ```go > import ( > _ "github.com/prometheus/client_golang/prometheus/promhttp/zstd" > ) > ``` - \[FEATURE] prometheus: Add new CollectorFunc utility [#1724](https://redirect.github.com/prometheus/client_golang/issues/1724) - \[CHANGE] Minimum required Go version is now 1.22 (we also test client\_golang against latest go version - 1.24) [#1738](https://redirect.github.com/prometheus/client_golang/issues/1738) - \[FEATURE] api: `WithLookbackDelta` and `WithStats` options have been added to API client. [#1743](https://redirect.github.com/prometheus/client_golang/issues/1743) - \[CHANGE] ⚠️ promhttp: Isolate zstd support and klauspost/compress library use to promhttp/zstd package. [#1765](https://redirect.github.com/prometheus/client_golang/issues/1765) <details> <summary> All Changes </summary> - build(deps): bump golang.org/x/sys from 0.28.0 to 0.29.0 by [@dependabot](https://redirect.github.com/dependabot) in [#1720](https://redirect.github.com/prometheus/client_golang/pull/1720) - build(deps): bump google.golang.org/protobuf from 1.36.1 to 1.36.3 by [@dependabot](https://redirect.github.com/dependabot) in [#1719](https://redirect.github.com/prometheus/client_golang/pull/1719) - Update RELEASE.md by [@bwplotka](https://redirect.github.com/bwplotka) in [#1721](https://redirect.github.com/prometheus/client_golang/pull/1721) - chore(docs): Add links for the upstream PRs by [@kakkoyun](https://redirect.github.com/kakkoyun) in [#1722](https://redirect.github.com/prometheus/client_golang/pull/1722) - Added tips on releasing client and checking with k8s. by [@bwplotka](https://redirect.github.com/bwplotka) in [#1723](https://redirect.github.com/prometheus/client_golang/pull/1723) - feat: Add new CollectorFunc utility by [@Saumya40-codes](https://redirect.github.com/Saumya40-codes) in [#1724](https://redirect.github.com/prometheus/client_golang/pull/1724) - build(deps): bump google.golang.org/protobuf from 1.36.3 to 1.36.4 by [@dependabot](https://redirect.github.com/dependabot) in [#1725](https://redirect.github.com/prometheus/client_golang/pull/1725) - build(deps): bump the github-actions group with 5 updates by [@dependabot](https://redirect.github.com/dependabot) in [#1726](https://redirect.github.com/prometheus/client_golang/pull/1726) - Synchronize common files from prometheus/prometheus by [@prombot](https://redirect.github.com/prombot) in [#1727](https://redirect.github.com/prometheus/client_golang/pull/1727) - Synchronize common files from prometheus/prometheus by [@prombot](https://redirect.github.com/prombot) in [#1731](https://redirect.github.com/prometheus/client_golang/pull/1731) - build(deps): bump golang.org/x/sys from 0.29.0 to 0.30.0 by [@dependabot](https://redirect.github.com/dependabot) in [#1739](https://redirect.github.com/prometheus/client_golang/pull/1739) - build(deps): bump google.golang.org/protobuf from 1.36.4 to 1.36.5 by [@dependabot](https://redirect.github.com/dependabot) in [#1740](https://redirect.github.com/prometheus/client_golang/pull/1740) - Cleanup dependabot config by [@SuperQ](https://redirect.github.com/SuperQ) in [#1741](https://redirect.github.com/prometheus/client_golang/pull/1741) - Upgrade Golang version v1.24 by [@dongjiang1989](https://redirect.github.com/dongjiang1989) in [#1738](https://redirect.github.com/prometheus/client_golang/pull/1738) - build(deps): bump the github-actions group with 2 updates by [@dependabot](https://redirect.github.com/dependabot) in [#1742](https://redirect.github.com/prometheus/client_golang/pull/1742) - Merging 1.21 release back to main. by [@bwplotka](https://redirect.github.com/bwplotka) in [#1744](https://redirect.github.com/prometheus/client_golang/pull/1744) - Synchronize common files from prometheus/prometheus by [@prombot](https://redirect.github.com/prombot) in [#1745](https://redirect.github.com/prometheus/client_golang/pull/1745) - Add support for undocumented query options for API by [@mahendrapaipuri](https://redirect.github.com/mahendrapaipuri) in [#1743](https://redirect.github.com/prometheus/client_golang/pull/1743) - exp/api: Add experimental exp module; Add remote API with write client and handler. by [@bwplotka](https://redirect.github.com/bwplotka) in [#1658](https://redirect.github.com/prometheus/client_golang/pull/1658) - exp/api: Add accepted msg type validation to handler by [@saswatamcode](https://redirect.github.com/saswatamcode) in [#1750](https://redirect.github.com/prometheus/client_golang/pull/1750) - build(deps): bump the github-actions group with 5 updates by [@dependabot](https://redirect.github.com/dependabot) in [#1751](https://redirect.github.com/prometheus/client_golang/pull/1751) - build(deps): bump github.com/klauspost/compress from 1.17.11 to 1.18.0 by [@dependabot](https://redirect.github.com/dependabot) in [#1752](https://redirect.github.com/prometheus/client_golang/pull/1752) - build(deps): bump github.com/google/go-cmp from 0.6.0 to 0.7.0 by [@dependabot](https://redirect.github.com/dependabot) in [#1753](https://redirect.github.com/prometheus/client_golang/pull/1753) - exp: Reset snappy buf by [@saswatamcode](https://redirect.github.com/saswatamcode) in [#1756](https://redirect.github.com/prometheus/client_golang/pull/1756) - Merge release 1.21.1 to main. by [@bwplotka](https://redirect.github.com/bwplotka) in [#1762](https://redirect.github.com/prometheus/client_golang/pull/1762) - exp: Add dependabot config by [@saswatamcode](https://redirect.github.com/saswatamcode) in [#1754](https://redirect.github.com/prometheus/client_golang/pull/1754) - build(deps): bump peter-evans/create-pull-request from 7.0.7 to 7.0.8 in the github-actions group by [@dependabot](https://redirect.github.com/dependabot) in [#1764](https://redirect.github.com/prometheus/client_golang/pull/1764) - promhttp: Isolate zstd support and klauspost/compress library use to promhttp/zstd package by [@liggitt](https://redirect.github.com/liggitt) in [#1765](https://redirect.github.com/prometheus/client_golang/pull/1765) - Cut 1.22.0-rc.0 by [@kakkoyun](https://redirect.github.com/kakkoyun) in [#1768](https://redirect.github.com/prometheus/client_golang/pull/1768) </details> #### New Contributors * @Saumya40-codes made their first contributi[https://github.com/prometheus/client_golang/pull/1724](https://redirect.github.com/prometheus/client_golang/pull/1724)l/1724 * @mahendrapaipuri made their first contributi[https://github.com/prometheus/client_golang/pull/1743](https://redirect.github.com/prometheus/client_golang/pull/1743)l/1743 * @liggitt made their first contributi[https://github.com/prometheus/client_golang/pull/1765](https://redirect.github.com/prometheus/client_golang/pull/1765)l/1765 **Full Changelog**: <prometheus/client_golang@v1.21.1...v1.22.0-rc.0> ### [`v1.21.1`](https://redirect.github.com/prometheus/client_golang/releases/tag/v1.21.1): / 2025-03-04 [Compare Source](https://redirect.github.com/prometheus/client_golang/compare/v1.21.0...v1.21.1) This release addresses a performance regression introduced in [#1661](https://redirect.github.com/prometheus/client_golang/issues/1661) -- thanks to all who [reported this quickly](https://redirect.github.com/prometheus/client_golang/issues/1748): [@chlunde](https://redirect.github.com/chlunde), [@dethi](https://redirect.github.com/dethi), [@aaronbee](https://redirect.github.com/aaronbee) [@tsuna](https://redirect.github.com/tsuna) [@kakkoyun](https://redirect.github.com/kakkoyun) 💪🏽. This patch release also fixes the iOS build. We will be hardening the release process even further ([#1759](https://redirect.github.com/prometheus/client_golang/issues/1759), [#1761](https://redirect.github.com/prometheus/client_golang/issues/1761)) to prevent this in future, sorry for the inconvenience! The high concurrency optimization is planned to be eventually reintroduced, however in a much safer manner, potentially in a separate API. - \[BUGFIX] prometheus: Revert of `Inc`, `Add` and `Observe` cumulative metric CAS optimizations ([#1661](https://redirect.github.com/prometheus/client_golang/issues/1661)), causing regressions on low concurrency cases [#1757](https://redirect.github.com/prometheus/client_golang/issues/1757) - \[BUGFIX] prometheus: Fix GOOS=ios build, broken due to process\_collector\_\* wrong build tags. [#1758](https://redirect.github.com/prometheus/client_golang/issues/1758) <details> <summary>All commits</summary> - Revert "exponential backoff for CAS operations on floats" and cut 1.21.1 by [@bwplotka](https://redirect.github.com/bwplotka) in [#1757](https://redirect.github.com/prometheus/client_golang/pull/1757) - Fix ios build for 1.21.1 by [@bwplotka](https://redirect.github.com/bwplotka) in [#1758](https://redirect.github.com/prometheus/client_golang/pull/1758) </details> **Full Changelog**: <prometheus/client_golang@v1.21.0...v1.21.1> ### [`v1.21.0`](https://redirect.github.com/prometheus/client_golang/releases/tag/v1.21.0): / 2025-02-19 [Compare Source](https://redirect.github.com/prometheus/client_golang/compare/v1.20.5...v1.21.0) :warning: This release contains potential breaking change if you upgrade `github.com/prometheus/common` to 0.62+ together with client\_golang (and depend on the strict, legacy validation for the label names). New common version [changes `model.NameValidationScheme` global variable](https://redirect.github.com/prometheus/common/pull/724), which relaxes the validation of label names and metric name, allowing all UTF-8 characters. Typically, this should not break any user, unless your test or usage expects strict certain names to panic/fail on client\_golang metric registration, gathering or scrape. In case of problems change `model.NameValidationScheme` to old `model.LegacyValidation` value in your project `init` function. :warning: - \[BUGFIX] gocollector: Fix help message for runtime/metric metrics. [#1583](https://redirect.github.com/prometheus/client_golang/issues/1583) - \[BUGFIX] prometheus: Fix `Desc.String()` method for no labels case. [#1687](https://redirect.github.com/prometheus/client_golang/issues/1687) - \[PERF] prometheus: Optimize popular `prometheus.BuildFQName` function; now up to 30% faster. [#1665](https://redirect.github.com/prometheus/client_golang/issues/1665) - \[PERF] prometheus: Optimize `Inc`, `Add` and `Observe` cumulative metrics; now up to 50% faster under high concurrent contention. [#1661](https://redirect.github.com/prometheus/client_golang/issues/1661) - \[CHANGE] Upgrade prometheus/common to 0.62.0 which changes `model.NameValidationScheme` global variable. [#1712](https://redirect.github.com/prometheus/client_golang/issues/1712) - \[CHANGE] Add support for Go 1.23. [#1602](https://redirect.github.com/prometheus/client_golang/issues/1602) - \[FEATURE] process\_collector: Add support for Darwin systems. [#1600](https://redirect.github.com/prometheus/client_golang/issues/1600) [#1616](https://redirect.github.com/prometheus/client_golang/issues/1616) [#1625](https://redirect.github.com/prometheus/client_golang/issues/1625) [#1675](https://redirect.github.com/prometheus/client_golang/issues/1675) [#1715](https://redirect.github.com/prometheus/client_golang/issues/1715) - \[FEATURE] api: Add ability to invoke `CloseIdleConnections` on api.Client using `api.Client.(CloseIdler).CloseIdleConnections()` casting. [#1513](https://redirect.github.com/prometheus/client_golang/issues/1513) - \[FEATURE] promhttp: Add `promhttp.HandlerOpts.EnableOpenMetricsTextCreatedSamples` option to create OpenMetrics \_created lines. Not recommended unless you want to use opt-in Created Timestamp feature. Community works on OpenMetrics 2.0 format that should make those lines obsolete (they increase cardinality significantly). [#1408](https://redirect.github.com/prometheus/client_golang/issues/1408) - \[FEATURE] prometheus: Add `NewConstNativeHistogram` function. [#1654](https://redirect.github.com/prometheus/client_golang/issues/1654) <details> <summary> All commits </summary> * Merge release-1.20 to main by @bwplotka in prometheus/client_golang#1582 * gocollector: Tiny fix for help message with runtime/metrics source. by @bwplotka in prometheus/client_golang#1583 * ci: bump dagger to the latest version by @marcosnils in prometheus/client_golang#1588 * Merge release-1.20 back to main by @ArthurSens in prometheus/client_golang#1593 * Update linting by @SuperQ in prometheus/client_golang#1603 * Update supported Go versions by @SuperQ in prometheus/client_golang#1602 * build(deps): bump golang.org/x/sys from 0.22.0 to 0.24.0 by @dependabot in prometheus/client_golang#1611 * build(deps): bump github.com/prometheus/common from 0.55.0 to 0.57.0 by @dependabot in prometheus/client_golang#1612 * changed the name of all variables with min/max name by @parthlaw in prometheus/client_golang#1606 * Update Dagger and build. by @SuperQ in prometheus/client_golang#1610 * build(deps): bump github/codeql-action from 3.25.15 to 3.26.6 in the github-actions group across 1 directory by @dependabot in prometheus/client_golang#1614 * examples: Improved GoCollector example. by @bwplotka in prometheus/client_golang#1589 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1615 * process_collector: fill in most statistics on macOS by @mharbison72 in prometheus/client_golang#1600 * ⚡ http client defer CloseIdleConnections by @cuisongliu in prometheus/client_golang#1513 * Set allow-utf-8 in Format during tests to avoid escaping. by @ywwg in prometheus/client_golang#1618 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1622 * Merge Release 1.20 back to main by @ArthurSens in prometheus/client_golang#1627 * examples: Add custom labels example by @ying-jeanne in prometheus/client_golang#1626 * Refactor default runtime metrics tests for Go collector so that default runtime metric set autogenerates by @vesari in prometheus/client_golang#1631 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1628 * process_xxx_memory statistics for macOS (cgo) by @mharbison72 in prometheus/client_golang#1616 * build(deps): bump github.com/klauspost/compress from 1.17.9 to 1.17.10 by @dependabot in prometheus/client_golang#1633 * build(deps): bump golang.org/x/sys from 0.24.0 to 0.25.0 by @dependabot in prometheus/client_golang#1632 * process_collector: Add Platform-Specific Describe for processCollector by @ying-jeanne in prometheus/client_golang#1625 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1635 * build(deps): bump the github-actions group with 4 updates by @dependabot in prometheus/client_golang#1634 * Optionally print OM created lines by @ArthurSens in prometheus/client_golang#1408 * process_collector: merge wasip1 and js into a single implementation by @ying-jeanne in prometheus/client_golang#1644 * Merge release 1.20 to main by @bwplotka in prometheus/client_golang#1647 * Add Arianna as maintainer 💪 by @ArthurSens in prometheus/client_golang#1651 * test add headers round tripper by @Manask322 in prometheus/client_golang#1657 * build(deps): bump github.com/klauspost/compress from 1.17.10 to 1.17.11 by @dependabot in prometheus/client_golang#1668 * build(deps): bump golang.org/x/sys from 0.25.0 to 0.26.0 by @dependabot in prometheus/client_golang#1669 * build(deps): bump github.com/prometheus/common from 0.59.1 to 0.60.1 by @dependabot in prometheus/client_golang#1667 * build(deps): bump google.golang.org/protobuf from 1.34.2 to 1.35.1 by @dependabot in prometheus/client_golang#1670 * Optimize BuildFQName function by @jkroepke in prometheus/client_golang#1665 * fix: use injected now() instead of time.Now() in summary methods by @imorph in prometheus/client_golang#1672 * process_collector: avoid a compiler warning on macOS (fixes #1660) by @mharbison72 in prometheus/client_golang#1675 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1674 * build(deps): bump the github-actions group across 1 directory with 3 updates by @dependabot in prometheus/client_golang#1678 * [chore]: enable perfsprint linter by @mmorel-35 in prometheus/client_golang#1676 * Duplicate of #1662 by @imorph in prometheus/client_golang#1673 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1679 * chore: enable usestdlibvars linter by @mmorel-35 in prometheus/client_golang#1680 * Add: exponential backoff for CAS operations on floats by @imorph in prometheus/client_golang#1661 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1683 * [1617] Add ConstnativeHistogram by @shivanthzen in prometheus/client_golang#1654 * fix: replace fmt.Errorf with errors.New by @kakkoyun in prometheus/client_golang#1689 * Add codeowners by @kakkoyun in prometheus/client_golang#1688 * fix: add very small delay between observations in `TestHistogramAtomicObserve` by @imorph in prometheus/client_golang#1691 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1692 * Fix: handle nil variableLabels in Desc.String() method and add tests for nil label values by @kakkoyun in prometheus/client_golang#1687 * examples: Follow best practices and established naming conventions by @lilic in prometheus/client_golang#1650 * setup OSSF Scorecard workflow by @mmorel-35 in prometheus/client_golang#1432 * build(deps): bump google.golang.org/protobuf from 1.35.1 to 1.35.2 by @dependabot in prometheus/client_golang#1697 * build(deps): bump golang.org/x/sys from 0.26.0 to 0.27.0 by @dependabot in prometheus/client_golang#1696 * build(deps): bump the github-actions group with 5 updates by @dependabot in prometheus/client_golang#1695 * update links to openmetrics to reference the v1.0.0 release by @dashpole in prometheus/client_golang#1699 * build(deps): bump google.golang.org/protobuf from 1.35.2 to 1.36.1 by @dependabot in prometheus/client_golang#1706 * build(deps): bump golang.org/x/sys from 0.27.0 to 0.28.0 by @dependabot in prometheus/client_golang#1705 * build(deps): bump the github-actions group with 5 updates by @dependabot in prometheus/client_golang#1707 * build(deps): bump github.com/prometheus/common from 0.60.1 to 0.61.0 by @dependabot in prometheus/client_golang#1704 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1703 * Synchronize common files from prometheus/prometheus by @prombot in prometheus/client_golang#1708 * Upgrade to prometheus/common 0.62.0 with breaking change by @bwplotka in prometheus/client_golang#1712 * build(deps): bump golang.org/x/net from 0.26.0 to 0.33.0 in /tutorials/whatsup by @dependabot in prometheus/client_golang#1713 * docs: Add RELEASE.md for the release process by @kakkoyun in prometheus/client_golang#1690 * tutorials/whatsup: Updated deps by @bwplotka in prometheus/client_golang#1716 * process collector: Fixed pedantic registry failures on darwin with cgo. by @bwplotka in prometheus/client_golang#1715 * Revert "ci: daggerize test and lint pipelines (#1534)" by @bwplotka in prometheus/client_golang#1717 * Cut 1.21.0-rc.0 by @bwplotka in prometheus/client_golang#1718 * Cut 1.21 by @bwplotka in prometheus/client_golang#1737 </details> #### New Contributors * @parthlaw made their first contributi[https://github.com/prometheus/client_golang/pull/1606](https://redirect.github.com/prometheus/client_golang/pull/1606)l/1606 * @mharbison72 made their first contributi[https://github.com/prometheus/client_golang/pull/1600](https://redirect.github.com/prometheus/client_golang/pull/1600)l/1600 * @cuisongliu made their first contributi[https://github.com/prometheus/client_golang/pull/1513](https://redirect.github.com/prometheus/client_golang/pull/1513)l/1513 * @ying-jeanne made their first contributi[https://github.com/prometheus/client_golang/pull/1626](https://redirect.github.com/prometheus/client_golang/pull/1626)l/1626 * @Manask322 made their first contributi[https://github.com/prometheus/client_golang/pull/1657](https://redirect.github.com/prometheus/client_golang/pull/1657)l/1657 * @jkroepke made their first contributi[https://github.com/prometheus/client_golang/pull/1665](https://redirect.github.com/prometheus/client_golang/pull/1665)l/1665 * @imorph made their first contributi[https://github.com/prometheus/client_golang/pull/1672](https://redirect.github.com/prometheus/client_golang/pull/1672)l/1672 * @mmorel-35 made their first contributi[https://github.com/prometheus/client_golang/pull/1676](https://redirect.github.com/prometheus/client_golang/pull/1676)l/1676 * @shivanthzen made their first contributi[https://github.com/prometheus/client_golang/pull/1654](https://redirect.github.com/prometheus/client_golang/pull/1654)l/1654 * @dashpole made their first contributi[https://github.com/prometheus/client_golang/pull/1699](https://redirect.github.com/prometheus/client_golang/pull/1699)l/1699 **Full Changelog**: <prometheus/client_golang@v1.20.5...v1.21.0> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://redirect.github.com/renovatebot/renovate).  Co-authored-by: renovate-vince-riv[bot] <227500739+renovate-vince-riv[bot]@users.noreply.github.com>

mharbison72 force-pushed the mac-process-mem-stats branch from 78af2fb to eca836e Compare September 6, 2024 16:37

mharbison72 changed the title ~~process_xxx_memory statistics for macOS~~ process_xxx_memory statistics for macOS (cgo) Sep 17, 2024

mharbison72 mentioned this pull request Sep 17, 2024

process_xxx_memory statistics for macOS (without cgo) #1629

Closed

bwplotka reviewed Sep 23, 2024

View reviewed changes

mharbison72 force-pushed the mac-process-mem-stats branch 2 times, most recently from 0bbaeea to 6619b9d Compare September 24, 2024 22:40

bwplotka reviewed Sep 25, 2024

View reviewed changes

prometheus/process_collector_cgo_darwin.go Show resolved Hide resolved

bwplotka approved these changes Sep 25, 2024

View reviewed changes

mharbison72 force-pushed the mac-process-mem-stats branch 2 times, most recently from 7c9e084 to 1a14ac7 Compare September 25, 2024 16:22

kakkoyun approved these changes Sep 27, 2024

View reviewed changes

bwplotka approved these changes Sep 27, 2024

View reviewed changes

bwplotka merged commit 25bda7c into prometheus:main Sep 27, 2024

bwplotka mentioned this pull request Oct 22, 2024

darwin: Building with CGO produces warning #1660

Closed

This was referenced Feb 19, 2025

fix(deps): update module github.com/prometheus/client_golang to v1.21.0 gardener/gardener#11444

Merged

fix(deps): update module github.com/prometheus/client_golang to v1.21.0 gardener/landscaper#1341

Merged

dghubble-renovate bot mentioned this pull request Feb 19, 2025

Bump github.com/prometheus/client_golang from v1.20.5 to v1.21.0 poseidon/fleetlock#306

Merged

1 task

mr-borboto bot mentioned this pull request Feb 19, 2025

fix(go): update module github.com/prometheus/client_golang ( v1.20.5 → v1.21.1 ) mirceanton/external-dns-provider-mikrotik#200

Merged

1 task

pull-bunyan bot mentioned this pull request Feb 19, 2025

Update module github.com/prometheus/client_golang to v1.22.0 - autoclosed sholdee/adguard-exporter#12

Closed

1 task

pb-renovate bot mentioned this pull request Feb 19, 2025

chore(deps): update module github.com/prometheus/client_golang to v1.21.0 - autoclosed tamcore/imagepullsecret-patcher#29

Closed

1 task

theriver-bot bot mentioned this pull request Feb 19, 2025

fix(deps): update module github.com/prometheus/client_golang to v1.21.0 larivierec/cloudflare-ddns#381

Merged

1 task

larsl-renovate bot mentioned this pull request Feb 19, 2025

fix(deps): update module github.com/prometheus/client_golang to v1.22.0 p5xeu/prompage#19

Merged

1 task

renovate-sh-app bot mentioned this pull request Nov 27, 2025

fix(deps): update module github.com/prometheus/client_golang to v1.23.2 grafana/catchpoint-prometheus-exporter#11

Open

1 task

jhcloud-bot mentioned this pull request Nov 29, 2025

chore(deps): update module github.com/prometheus/client_golang to v1.23.2 JHOFER-Cloud/sonnenbatterie-exporter#7

Merged

1 task

This was referenced Dec 1, 2025

fix(deps): update module github.com/prometheus/client_golang to v1.23.2 - autoclosed grafana/k6build#259

Closed

fix(deps): update module github.com/prometheus/client_golang to v1.23.2 grafana/vmware_exporter#52

Open

sudo-Tiz mentioned this pull request Dec 2, 2025

chore(deps): Update Go dependencies (non-major) sudo-Tiz/dns-tester-go#5

Merged

1 task

This was referenced Dec 3, 2025

fix(deps): update module github.com/prometheus/client_golang to v1.23.2 Wikid82/Charon#299

Merged

fix(deps): update module github.com/prometheus/client_golang to v1.23.2 nscuro/dtapac#294

Merged

renovate bot mentioned this pull request Dec 11, 2025

Update module github.com/prometheus/client_golang to v1.23.2 giantswarm/teleport-exporter#7

Merged

1 task

red-hat-konflux-kflux-prd-rh02 bot mentioned this pull request Dec 14, 2025

fix(deps): update module github.com/prometheus/client_golang to v1.23.2 openshift/file-integrity-operator#843

Closed

1 task

doug-piranha-bot bot mentioned this pull request Dec 18, 2025

fix(deps): update module github.com/prometheus/client_golang to v1.23.2 haraldkoch/tplink-plug-exporter#6

Open

1 task

renovate bot mentioned this pull request Dec 20, 2025

Update module github.com/prometheus/client_golang to v1.23.2 cr3/postfix_exporter#11

Open

1 task

red-hat-konflux bot mentioned this pull request Dec 23, 2025

chore(deps): update module github.com/prometheus/client_golang to v1.23.2 (backplane-2.6) stolostron/backplane-operator#2404

Closed

1 task

renovate bot mentioned this pull request Jan 6, 2026

Update module github.com/prometheus/client_golang to v1.23.2 appuio/emergency-credentials-controller#57

Open

1 task

renovate-vince-riv bot mentioned this pull request Jan 7, 2026

chore(deps): update module github.com/prometheus/client_golang to v1.23.2 vince-riv/aks-node-termination-handler#9

Merged

1 task

renovate-sh-app bot mentioned this pull request Jan 7, 2026

fix(deps): update module github.com/prometheus/client_golang to v1.23.2 grafana/prometheus-alertmanager-tutorial#9

Open

1 task

red-hat-konflux bot mentioned this pull request Jan 21, 2026

chore(deps): update module github.com/prometheus/client_golang to v1.23.2 redhat-appstudio/dora-metrics#116

Closed

1 task

renovate bot mentioned this pull request Jan 21, 2026

Update module github.com/prometheus/client_golang to v1.23.2 maypok86/gatekeeper#86

Open

1 task

red-hat-konflux bot mentioned this pull request Jan 22, 2026

fix(deps): update module github.com/prometheus/client_golang to v1.23.2 (backplane-2.8) stolostron/discovery#1604

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

process_xxx_memory statistics for macOS (cgo) #1616

process_xxx_memory statistics for macOS (cgo) #1616

mharbison72 commented Sep 4, 2024

Uh oh!

mharbison72 commented Sep 7, 2024

Uh oh!

bwplotka commented Sep 23, 2024 •

edited

Loading

Uh oh!

mharbison72 commented Sep 23, 2024

Uh oh!

bwplotka Sep 23, 2024

Uh oh!

mharbison72 Sep 23, 2024

Uh oh!

bwplotka commented Sep 23, 2024

Uh oh!

mharbison72 commented Sep 24, 2024

Uh oh!

Uh oh!

bwplotka left a comment

Uh oh!

bwplotka Sep 25, 2024

Uh oh!

mharbison72 commented Sep 25, 2024

Uh oh!

kakkoyun left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

process_xxx_memory statistics for macOS (cgo) #1616

process_xxx_memory statistics for macOS (cgo) #1616

Conversation

mharbison72 commented Sep 4, 2024

Uh oh!

mharbison72 commented Sep 7, 2024

Uh oh!

bwplotka commented Sep 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mharbison72 commented Sep 23, 2024

Uh oh!

bwplotka Sep 23, 2024

Choose a reason for hiding this comment

Uh oh!

mharbison72 Sep 23, 2024

Choose a reason for hiding this comment

Uh oh!

bwplotka commented Sep 23, 2024

Uh oh!

mharbison72 commented Sep 24, 2024

Uh oh!

Uh oh!

bwplotka left a comment

Choose a reason for hiding this comment

Uh oh!

bwplotka Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

mharbison72 commented Sep 25, 2024

Uh oh!

kakkoyun left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bwplotka commented Sep 23, 2024 •

edited

Loading