POST: storing and using httr2 cache tokens on CI by drmowinckels · Pull Request #1173 · ropensci/roweb3

drmowinckels · 2025-10-14T10:36:16Z

Maëlle and I describe how we set up meetupr package for CI/CD compatibility. Intended to be cross posted to R-Ladies blog, awaiting their blog team's decision.

drmowinckels · 2025-10-14T10:36:47Z

@maelle first draft ready for your feedback!

maelle

Thank you!! I made some comments. I wonder whether a diagram would help. And a link to a real GHA example?

yabellini · 2026-01-19T16:49:35Z

Hi @drmowinckels and @maelle, no rush, just let us know when this is ready to review for our editorial team or if you need any help from us to move forward.

drmowinckels · 2026-01-20T21:48:21Z

I'll get to this next week I think. We ended up with a completely different approach in the end, so needs a pretty big rewrite.

yabellini · 2026-01-20T23:49:17Z

I'll get to this next week I think. We ended up with a completely different approach in the end, so needs a pretty big rewrite.

Thanks for letting me know. No pressure and no rush. I was just going through our pending posts and want to get a sense of timing to schedule publications.

drmowinckels · 2026-03-18T15:58:13Z

@maelle did some updates, added what you suggested :)

maelle

Yay!! I find it very clear especially knowing the pain and confusion that the Meetup API and OAuth create.

maelle · 2026-03-19T09:17:33Z

+title: "Cache Me If You Can – httr2 OAuth Token Management for CI/CD"
+author:
+- Athanasia Mo Mowinckel
+- Maëlle Salmon


I wonder whether I should be an editor instead, given what my input was.

I leave that to you, but I think of you as co-auther if that counts for something.

maelle · 2026-03-19T09:20:51Z

+
+## Understanding httr2's OAuth flow
+
+httr2's `req_oauth_auth_code()` handles [OAuth2](https://blog.r-hub.io/2021/01/25/oauth-2.0/) authentication for local development:


I obviously like that R-hub post but maybe link to https://httr2.r-lib.org/articles/oauth.html instead/as well?

Co-authored-by: Maëlle Salmon <maelle.salmon@yahoo.se>

drmowinckels · 2026-04-03T17:58:29Z

+title: "Cache Me If You Can – httr2 OAuth Token Management for CI/CD"
+author:
+- Athanasia Mo Mowinckel
+- Maëlle Salmon


I leave that to you, but I think of you as co-auther if that counts for something.

drmowinckels

minor changes

drmowinckels · 2026-04-04T10:24:14Z

To whomever will be editor, we want to crosspost this to the R-Ladies blog, but have not set up the post there yet. I thought it would be best to get it ready here, and then set a publishing date together with the @rladies/blog team

yabellini · 2026-04-06T11:20:32Z

@drmowinckels, that sounds good. Is this ready for editorial review? If yes, then @steffilazerte will be the editor 😸

drmowinckels · 2026-04-08T19:27:18Z

ready for review @steffilazerte .

steffilazerte

Hi @drmowinckels!

This is a great post and a fascinating idea! I'm already thinking about how to incorporate it into my promoutils package so I don't have to deal with the httr2 token storage, thanks!

Overall, this is a very clear and well written post. I have one large suggestion regarding moving the "What we tried first" section to earlier, and most of the other comments are small wording tweaks, or add a bit of context to remind the reader what you're talking about.

For moving the "What we tried first" section, see my comments for specifics, but broadly, my suggestion is to:

Move it to right before "How it works"
Rename to "Why encrypted tokens?"
Add the first paragraph of the "How it works" to the end of this section so the two sections flow from one into the other.

These are just suggestions, however, so pick and chose what works best for you!

We have a couple of options for dates, depending what @maelle thinks and how long you need to make any edits.

I'm away until April 24th, so we could publish April 28th, when I'm back, or if you're ready earlier, @maelle could publish on the 21st (or anytime next week that sounds good). Just let me know what works best for you (and @maelle, if that works for you!)

steffilazerte · 2026-04-15T15:58:17Z

+2. _Encrypted token file_ — for regular OAuth apps in CI
+3. _Interactive OAuth_ — the standard browser flow for local development
+
+![Flowchart showing the authentication decision tree. Starting from an API request, the package first checks if JWT credentials are available. If yes, it uses JWT bearer token auth. If not, it checks for an encrypted token file. If found, it decrypts and uses that token. If neither is available, it falls back to the interactive browser OAuth flow.](auth-decision-flow.png "Authentication decision flow — the package tries each method in order and uses the first one available.")


Should the "\n" be New Lines? Looks like that didn't work in the figure...

steffilazerte · 2026-04-15T16:00:23Z

+  req <- req |> 
+  httr2::req_oauth_bearer_jwt(
+        client = meetupr_client(...),
+        claim = httr2::jwt_claim(...)
+      ))
+    return(req)


Just to match the rest

Suggested change

req <- req |>

httr2::req_oauth_bearer_jwt(

client = meetupr_client(...),

claim = httr2::jwt_claim(...)

))

return(req)

req <- req |>

httr2::req_oauth_bearer_jwt(

client = meetupr_client(...),

claim = httr2::jwt_claim(...)

))

return(req)

steffilazerte · 2026-04-15T18:38:00Z

+Our first approach — suggested by [Noam Ross](https://www.noamross.net/) on the rOpenSci Slack — was to base64-encode the raw httr2 cache file and store it as a CI environment variable.
+This worked, but had drawbacks: environment variables have size limits on some CI providers, you had to store both the token content _and_ httr2's hash-based filename as separate secrets, and there was no natural way to write back a rotated token.
+
+It is a bit surprising! 


Was it really surprising? The next sentence seems to argue that this is expected? Perhaps either remove the "It is a bit surprising!" Statement, or tweak it to something like "It was a bit surprising, but made sense once we thought about it a bit more." (or similar).

steffilazerte · 2026-04-15T18:40:00Z

+Instead of the encrypted file going stale after the access token expires, it stays current.
+The caveat being, that every token refresh needs to be committed and pushed to the repo by the CI.
+
+### What we tried first


I wonder if this section should go right before you describe the encryption part (i.e. before the 'How it works' part). It seems to break the flow a bit here and seems out of order to talk about what you did first after you talk about what you did second 😁

If you do that, consider remaming this section to "Why encrypted tokens?"

steffilazerte · 2026-04-15T18:46:58Z

+The encrypted file approach is simpler for users — commit one file, set one secret — and gives CI a place to write back the refreshed token.
+By using an encrypted file in the repository, you sidestep this limitation. 
+The file acts as a renewable credential that CI can update and push, while the decryption password remains protected as a CI secret.
+This pattern is not common in most documentation, but it works well for R packages and similar use cases.


If moving this section to earlier, consider adding a bit of the "How it works" introduction here so it flows directly into that section.

Suggested change

The encrypted file approach is simpler for users — commit one file, set one secret — and gives CI a place to write back the refreshed token.

By using an encrypted file in the repository, you sidestep this limitation.

The file acts as a renewable credential that CI can update and push, while the decryption password remains protected as a CI secret.

This pattern is not common in most documentation, but it works well for R packages and similar use cases.

So we thought of another approach, using encrypted token files. The idea is simple: authenticate locally once, then encrypt and commit the token so CI can use it. Only the decryption password goes into CI secrets — the encrypted file itself lives in your repository.

This is simpler for users — commit one file, set one secret — and gives CI a place to write back the refreshed token.

By using an encrypted file in the repository, you sidestep the limitations of CI environmental variables.

The file acts as a renewable credential that CI can update and push, while the decryption password remains protected as a CI secret.

This pattern is not common in most documentation, but it works well for R packages and similar use cases.

steffilazerte · 2026-04-15T19:06:15Z

Again I don't see this referenced... I think it can be used for the social media card, but then you need the YAML key socialImg

steffilazerte · 2026-04-15T19:15:51Z

+JWT is straightforward — if you have the credentials, httr2 handles the rest.
+The interesting part is the encrypted token path.
+The one we wanted to use in our CI workflows!
+


I suggest moving the "What we tried first" section here, and renaming it "Why encrypted tokens?" (see my other comments below)

steffilazerte · 2026-04-15T19:29:32Z

@@ -0,0 +1,328 @@
+---
+title: "Cache Me If You Can – httr2 OAuth Token Management for CI/CD"


Great title!

steffilazerte · 2026-04-15T19:31:55Z

+author:
+- Athanasia Mo Mowinckel
+- Maëlle Salmon
+date: '2026-03-30'


Remember to adjust the date here and in the folder name

steffilazerte · 2026-04-15T19:35:18Z

+output: hugodown::md_document
+tags:
+  - oAuth
+  - tech notes


Suggested change

- tech notes

- tech notes

- CI

- API

drmowinckels added 2 commits October 13, 2025 20:21

add httr2 token cache draft

8214b84

add more details

a21bc63

maelle reviewed Oct 14, 2025

View reviewed changes

add better examples of the ci setup

ddaa8c0

yabellini added the blog post Blog posts to be published when merged label Dec 3, 2025

drmowinckels and others added 4 commits March 17, 2026 21:48

update post to new workflow

2b2229a

add meetup archive link

dd8ba10

Merge branch 'main' into main

7edbc42

Merge branch 'main' into main

b81a92a

drmowinckels requested a review from maelle March 18, 2026 15:58

maelle reviewed Mar 19, 2026

View reviewed changes

drmowinckels and others added 9 commits April 3, 2026 19:59

Update content/blog/2025-10-30-httr2-cache/index.md

9dd219c

Co-authored-by: Maëlle Salmon <maelle.salmon@yahoo.se>

Update content/blog/2025-10-30-httr2-cache/index.md

2b22c4c

Co-authored-by: Maëlle Salmon <maelle.salmon@yahoo.se>

Update content/blog/2025-10-30-httr2-cache/index.md

8eb8723

Co-authored-by: Maëlle Salmon <maelle.salmon@yahoo.se>

Update content/blog/2025-10-30-httr2-cache/index.md

420a720

Co-authored-by: Maëlle Salmon <maelle.salmon@yahoo.se>

Update content/blog/2025-10-30-httr2-cache/index.md

0700d8e

Co-authored-by: Maëlle Salmon <maelle.salmon@yahoo.se>

Update content/blog/2025-10-30-httr2-cache/index.md

a848301

Co-authored-by: Maëlle Salmon <maelle.salmon@yahoo.se>

Merge branch 'main' into main

854a516

update post with @MaelleS comments

35abdd7

more fixes

f72443b

drmowinckels commented Apr 3, 2026

View reviewed changes

fix editor list

be3c10f

drmowinckels commented Apr 4, 2026

View reviewed changes

drmowinckels added 2 commits April 4, 2026 12:16

Apply suggestion from @drmowinckels

6ea66ea

Apply suggestion from @drmowinckels

fc276f8

drmowinckels added 2 commits April 4, 2026 12:17

Apply suggestion from @drmowinckels

37a33fd

Apply suggestion from @drmowinckels

40d5712

drmowinckels marked this pull request as ready for review April 4, 2026 10:22

steffilazerte reviewed Apr 15, 2026

View reviewed changes


		## Understanding httr2's OAuth flow

		httr2's `req_oauth_auth_code()` handles [OAuth2](https://blog.r-hub.io/2021/01/25/oauth-2.0/) authentication for local development:

		@@ -0,0 +1,328 @@
		---
		title: "Cache Me If You Can – httr2 OAuth Token Management for CI/CD"

Conversation

drmowinckels commented Oct 14, 2025

Uh oh!

drmowinckels commented Oct 14, 2025

Uh oh!

maelle left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yabellini commented Jan 19, 2026

Uh oh!

drmowinckels commented Jan 20, 2026

Uh oh!

yabellini commented Jan 20, 2026

Uh oh!

drmowinckels commented Mar 18, 2026

Uh oh!

maelle left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

drmowinckels left a comment

Choose a reason for hiding this comment

Uh oh!

drmowinckels commented Apr 4, 2026

Uh oh!

yabellini commented Apr 6, 2026

Uh oh!

drmowinckels commented Apr 8, 2026

Uh oh!

steffilazerte left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!