Commit 6855012
fix(core): PIIDetector streaming — eliminate per-chunk LLM calls, add buffered detection for context-dependent PII (#17377)
## Description
Fixes the PIIDetector making an LLM API call for every single
`text-delta` chunk during streaming, causing astronomical costs,
latency, and rate-limit issues.
**Two streaming modes based on configured detection types:**
- **Regex-only** (email, phone, SSN, credit card, IP, UUID, URL, etc.):
Static regex patterns run in microseconds with zero LLM calls and zero
latency. Used when only pattern-detectable types are configured.
- **Buffered** (name, address, date-of-birth): Chunks are buffered and
periodically flushed through the LLM at sentence boundaries (`[.!?]`) or
a configurable `bufferSize` threshold (default: 200 chars). This catches
context-dependent PII *before* it reaches the user, while limiting LLM
calls to ~3-5 per response instead of 50-100.
Key implementation details:
- `detectPIILocal(text)` — regex-based detection with `PII_PATTERNS`
map, zero API cost
- `flushLLMBuffer(state, abort)` — periodic LLM call on buffered text
- `applyStreamStrategy(part, result, abort)` — centralized strategy
logic for block/warn/filter/redact
- `hasLLMOnlyTypes` getter — determines which mode to use
- `bufferSize` option — configurable buffer threshold (default: 200),
higher = more LLM context but more latency
- Non-text chunks trigger immediate buffer flush via
`REPROCESS_PART_KEY`
- `processOutputResult` still runs LLM on the complete text as a final
safety net
## Related issue(s)
Fixes #16466
## Type of change
- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Performance improvement
## Checklist
- [x] I have linked the related issue(s) in the description above
- [x] I have added tests that prove my fix is effective or that my
feature works (73 tests passing)
- [x] I have addressed all Coderabbit comments on this PR
Link to Devin session:
https://app.devin.ai/sessions/213292456554414b915711fdb8c3abad
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Abhi Aiyer <abhi@mastra.ai>1 parent 4c02027 commit 6855012
3 files changed
Lines changed: 1163 additions & 80 deletions
File tree
- .changeset
- packages/core/src/processors/processors
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
0 commit comments