Skip to content

Commit 6855012

Browse files
devin-ai-integration[bot]Abhi Aiyer
andauthored
fix(core): PIIDetector streaming — eliminate per-chunk LLM calls, add buffered detection for context-dependent PII (#17377)
## Description Fixes the PIIDetector making an LLM API call for every single `text-delta` chunk during streaming, causing astronomical costs, latency, and rate-limit issues. **Two streaming modes based on configured detection types:** - **Regex-only** (email, phone, SSN, credit card, IP, UUID, URL, etc.): Static regex patterns run in microseconds with zero LLM calls and zero latency. Used when only pattern-detectable types are configured. - **Buffered** (name, address, date-of-birth): Chunks are buffered and periodically flushed through the LLM at sentence boundaries (`[.!?]`) or a configurable `bufferSize` threshold (default: 200 chars). This catches context-dependent PII *before* it reaches the user, while limiting LLM calls to ~3-5 per response instead of 50-100. Key implementation details: - `detectPIILocal(text)` — regex-based detection with `PII_PATTERNS` map, zero API cost - `flushLLMBuffer(state, abort)` — periodic LLM call on buffered text - `applyStreamStrategy(part, result, abort)` — centralized strategy logic for block/warn/filter/redact - `hasLLMOnlyTypes` getter — determines which mode to use - `bufferSize` option — configurable buffer threshold (default: 200), higher = more LLM context but more latency - Non-text chunks trigger immediate buffer flush via `REPROCESS_PART_KEY` - `processOutputResult` still runs LLM on the complete text as a final safety net ## Related issue(s) Fixes #16466 ## Type of change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Performance improvement ## Checklist - [x] I have linked the related issue(s) in the description above - [x] I have added tests that prove my fix is effective or that my feature works (73 tests passing) - [x] I have addressed all Coderabbit comments on this PR Link to Devin session: https://app.devin.ai/sessions/213292456554414b915711fdb8c3abad --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Abhi Aiyer <abhi@mastra.ai>
1 parent 4c02027 commit 6855012

3 files changed

Lines changed: 1163 additions & 80 deletions

File tree

.changeset/tough-coins-start.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
'@mastra/core': patch
3+
---
4+
5+
Improved PIIDetector streaming performance.
6+
7+
- Removed per-chunk LLM calls during streaming PII checks.
8+
- Added local regex detection for common PII types (email, phone, SSN, credit card, IP address, API keys, URLs, UUIDs, crypto wallets, and IBAN).
9+
- Added regex carryover buffer across chunk boundaries to catch split PII patterns.
10+
- Buffered context-dependent PII types (names, addresses, dates of birth) with periodic LLM calls at configurable thresholds.
11+
- Added `bufferSize` option (default: 200) to control LLM buffer flush threshold.
12+
- Reduced streaming API cost, latency, and rate-limit pressure.
13+
14+
Closes #16466.

0 commit comments

Comments
 (0)