Repository navigation
Conversation
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
4 of 5 tasks
…y TS `env.kv` already exempts a value that is code rather than a literal (`isSourceExpression`, garrytan#3048), and that function's own `CODE_NAME_CHAIN` accepts a Dart member access such as `request.password`. It never ran, because the exemption was gated to TypeScript/JSX by `TS_SOURCE = /\.(?:[cm]?tsx?|jsx)$/i`, so every other language kept the false positive. Found on a Dart backend where a refactor moved two existing lines into new files. Both forward the request's own password field by name, one as a `requireFields` map entry and one as a named argument, and neither holds a literal. Because the scan is over ADDED lines, a file move re-presents long-standing content as new, so the pre-gate's `secrets` check failed and refused publication on two lines that were byte-identical to the base branch. The pre-push hook reported them too. Demonstrated with one byte-identical line under three extensions: `.ts` clean, `.dart` fires, `.py` fires. `lib/pregate/checks/secrets.ts` writes each hunk to a temp file with `path.basename(part.file)`, so the real extension does reach the validator. Dart qualifies on the same ground TypeScript does, and the ground is a property of the language rather than a judgement about any repo: Dart has no unquoted string literal, so a bare name chain after `:` or `=` cannot be a value, only a variable read. Renamed the constant to `CODE_SOURCE` since it is no longer TS-only. Widening the gate alone WOULD have cost a real detection, so it does not go in alone. `withFlags` forces the `m` flag, so a `password:` line inside a triple-quoted string is matched on its own, and the `before` check reads only the character preceding the span — it cannot see a quote that opened three lines earlier. An embedded config block is exactly where the value after the colon IS a literal, and a `'''` fixture or inline manifest is ordinary Dart. Measured: that line reports on `main` and stopped reporting once `.dart` joined the gate. `insideTripleQuotedString` refuses the exemption there. It scans left to right for whichever delimiter opened rather than counting delimiters, because a `'''` inside a `"""` block is content and a combined parity count would call such text open when it is closed. Its two failure modes both land on the behaviour that already shipped: a diff hunk starting mid-string shows no opener and stays exempt, and a stray `'''` inside a single-line string costs an exemption rather than a report. TS/JSX template literals have the same hole and have had it since garrytan#3048; closing that changes behaviour no language added here is responsible for, so it is left alone. So detection holds. A credential in a Dart file still fires when quoted in either Dart quote style, when assigned with `=` (the separator check takes `:` alone, so an `=` value is never exempt), when the unquoted token begins with a digit, and inside a triple-quoted block. One thing this does NOT claim, because measuring it proved otherwise: an unquoted token is not reliably caught. `CODE_NAME_CHAIN` rejects a token only when it does not start with an identifier character, which a random base62 token does about 84% of the time, so `password: Fk2pQ9…` is exempt while `password: 8Fk2pQ…` fires. That is true of `.ts` on `main` today and is garrytan#3048's existing bargain — an unquoted token after `:` in a code file is a name, and code that pastes a raw secret there does not run. This change neither widens nor narrows it. Eight cases are added to the garrytan#3048 contract test: three that must now be exempt (the two Dart shapes, plus a closed `'''` block, which must not cost the exemption) and five that must still fire (both quote styles, an `=` assignment, a digit-first unquoted token, and a config line inside an open `'''` block). The last uses a letter-first token deliberately, so the chain matches and the guard is the only thing making it report; with `v`, which starts with a digit, it would have passed with or without the guard. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…forms Follows the Dart change in this stack, which widened `isSourceExpression`'s file gate from TS/JSX to Dart. Python is the next language with the same grammar property, and it gets that test applied to Python rather than Dart's conclusion carried over: Python has no unquoted string literal either, so a bare name chain after `:` is a name, not a value. Two shapes in practice — a dict entry `"password": request.password` in a handler, and an annotation `password: pydantic.SecretStr` on a model. A `.pyi` stub holds annotations and `...` and nothing else, so it cannot carry a value at all; both extensions join the gate. Measured rather than assumed, which changes what this change claims. Four shapes go from firing to clean: the dict entry, a dict entry reading settings, a dotted annotation in a `.py`, and the same annotation in a `.pyi`. Seven do not move: three quoted literals, the `.env` and YAML colon forms, and — the limitation worth naming — Python's keyword argument and plain assignment, `password=request.password` and `password = request.password`. Those forward a name exactly as the exempt dict entry does, but the separator check takes `:` alone, and `=` is most of how Python forwards a value by name. So Python gains materially less here than Dart did. Lifting the `:` restriction would change TS/JSX/Dart too, so it wants its own argument rather than a ride on this one; the contract test pins the firing half as well, so the gap stays visible instead of being assumed shut. Detection does not move either, because `insideTripleQuotedString` from the previous commit already covers `"""` as well as `'''`. That matters more for Python than it did for Dart: inline YAML in a `"""` block is ordinary fixture and manifest idiom, so a `password:` line inside one is a literal in a config block that happens to live in a `.py` file, and `main` reports it. Without that guard, adding `.py` to the gate would have stopped reporting it. A case for exactly that shape is added here. Nine cases are added to the garrytan#3048 contract test: the four now-exempt shapes and five that must still fire — a quoted dict value, a quoted assignment, the keyword argument and the assignment that make up the limitation, and a config line inside an open `"""` block. The annotation cases use `pydantic.SecretStr` rather than a bare `SecretStr` deliberately: the bare name scores 2.28 Shannon entropy, under the pattern's own 3.0 gate, so it is already clean without this change and would have pinned nothing about the exemption. The `"""` case uses the letter-first fixture for the same reason — with the digit-first one it would report whether the guard existed or not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
hbarney
force-pushed
the
fix/env-kv-python-dict-and-annotation
branch
from
October 26, 2026 16:04
c77a30b to
a713831
Compare
Contributor
|
Thank you for this fix. A crew lane is picking it up, together with your other redact PR, to land on current main with Co-authored-by credit to you. |
Contributor
|
Thanks @hbarney. This landed in #3742 (merged as 64fb6ec, v1.92.8.0), together with #3418 and #3605. Python's |
2 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why (in your own words)
#3418 widened
isSourceExpression's file gate from TypeScript/JSX to Dart. In thatPR I said Python has the identical hole and that I would not widen the gate to it
without making the argument for Python on its own terms. This is that work — and
making it changed what I think the change is worth, so the headline is narrower
than "Python too".
Python does share the grammar property the exemption rests on: it has no unquoted
string literal, so a bare name chain after
:is a name, not a value. That coverstwo real shapes — a dict entry
"password": request.passwordin a handler, and anannotation
password: pydantic.SecretStron a model. A.pyistub is annotationsand
...and nothing else, so it cannot hold a value at all; both extensions jointhe gate.
But Python gains materially less than Dart did, and the reason is worth stating
plainly. Dart's two live shapes — a
requireFieldsmap entry and a namedargument — both use
:. Python's keyword argument and plain assignment use=,and the separator check in
isSourceExpressiontakes:alone:So
password=request.passwordandpassword = request.passwordkeep firing afterthis change, and that is most of how Python forwards a value by name. I am not
lifting the
:restriction here: it would change TS/JSX/Dart behaviour too, and itdeserves its own argument rather than a ride on this one. What I did instead is pin
the firing half in the contract test, so the gap is visible in the suite rather
than something the next reader assumes is shut.
Live evidence
Every shape I could think of, measured before and after. Run against this
repo's own engine — "before" is #3418's head (
2933165), "after" is this branch:Four fixed, seven unmoved, nothing else moved. The two marked "the limitation" are
the
=forms above; they are in the suite now as must-still-fire cases.Detection does not move, and that took a guard. A secret on a
password:lineinside a triple-quoted Python string is a literal in a config block that happens to
live in a
.pyfile, andmainreports it. Adding.pyto the gate on its own wouldhave stopped reporting it.
insideTripleQuotedString, added in #3418, covers """ aswell as ''', so the shape holds — and this PR adds a case for exactly it:
It matters more for Python than it did for Dart: inline YAML in a """ block is
ordinary fixture and manifest idiom, so
tests/conftest.pyis a realistic place tolose a key. The case uses the letter-first fixture deliberately — with the digit-first
vit would report whether the guard existed or not, since a digit failsCODE_NAME_CHAINoutright.The guard's own limits are in #3418: a diff hunk starting mid-string shows no opener
and the line stays exempt, and a stray delimiter inside a single-line string costs an
exemption rather than a report. Both fail toward what already shipped. TS/JSX template
literals keep the hole they have had since #3048; neither PR touches it.
The new tests are not vacuous, proven rather than asserted. With the eight new
cases in place I reverted the one regex back to
/\.(?:[cm]?tsx?|jsx|dart)$/iandre-ran:
One case I deliberately did not use. A bare
password: SecretStrannotation isthe obvious Python test, and it is worthless here:
shannonEntropy("SecretStr")is2.28, under the pattern's own
>= 3.0gate, so it is already clean onmainand would have passed whatever this change did. The annotation cases use
pydantic.SecretStr(entropy 3.46), which genuinely fired before. Both numbers arein the probe output above and the reason is in a comment beside the cases.
Test runs, same fresh full clone,
bun install --frozen-lockfile:Scope
lib/redact-patterns.ts—CODE_SOURCEgainspyi?, plus a commentmaking the Python argument and naming the
=limitation.test/redact-engine.test.ts— 9 cases on the existing fix(redact): ignore TypeScript parameter types and dynamic JSX values #3048 contract test (4 nowexempt, 5 that must still fire: two of them the
=limitation, one the embeddedconfig block).
revert-the-regex vacuity check; the four test/typecheck commands.
unlike fix(redact): env.kv's code-expression exemption reaches Dart, not only TS #3418 this one is not backed by a false positive I personally hit. The
shapes come from reading the validator and from ordinary FastAPI/pydantic idiom,
and every claim about them is measured against the engine rather than recalled.
Also untouched: Go, Ruby, Kotlin, the
:separator restriction itself, and everyother
env.kvvalidator.Liveness proof (required for external contributors)
gh api user --jq .login→hbarney.Same as #3418: I am Claude Code, an agent working on @hbarney's machine at their
direction, so I am reporting this rather than claiming it — the
GSTACK PRscreenshot is not something an agent can honestly produce, since the point of it
is that a human typed it live. @hbarney will attach it. If it is absent, treat the
requirement as unmet rather than satisfied.
Checklist
GSTACK PRtyped live into a real surface) — to be attached by @hbarney, see aboveA correction to something I wrote in #3418
Working this up turned up an error in #3418's own description, so I am flagging it
here and on that PR rather than leaving it to stand. I wrote there that a credential
in a Dart file still fires "even unquoted, because
CODE_NAME_CHAINrequires anidentifier first character and a random high-entropy token rarely has one." The
clause is wrong twice over. A base62 token starts with a letter about 84% of the
time, not rarely; and I checked —
password: Fk2pQ9vXz4wL7mN3rT6yB1cD5eG0hJxin a.dartfile is exempt, while the suite's fixture8Fk2pQ…fires only because ithappens to begin with a digit. The same is true in
.tstoday, so this is #3048behaviour that Dart and now Python inherit, not something either PR introduces.
I think the outcome is right for the same grammar reason the rest of the design
rests on — an unquoted token after
:in a code file is a name, and code that pastesa raw secret there does not run — but that is the argument, not the one I gave, and
the test case I advertised as covering unquoted values really only covers digit-first
ones. I went and worked up whether
CODE_NAME_CHAIN's single-segment case should benarrowed, and I now think it should not, so there is no third PR for it. Dropping
the single-segment branch re-fires
session: SessionState,in a.tsfile — the#3048 false positive itself, which survives today only because of that branch (the
captured span includes the trailing comma, so it clears the 3.0 entropy gate at
3.05). And there is no clean entropy discriminator to narrow it with: legitimate
single-segment type names measure 2.92 for
SessionState, 3.51 forSharedPreferencesTokenStore, 3.85 forAuthenticationCredentialsProviderand 3.95for
GoogleServiceAccountCredentials, against 4.95 for a random base62 token — theranges are adjacent, and a threshold placed between them would be fitted to my six
samples rather than to anything real. The embedded-string miss above is the defect
worth fixing; it is a different lever.
🤖 Generated with Claude Code