Skip to content

fix(pr-management-code-review): never live-@-mention anyone in posted reviews - #915

Merged
potiuk merged 1 commit into
apache:mainfrom
shahar1:fix/pr-review-no-live-mentions
Jul 26, 2026
Merged

potiuk merged 1 commit into
apache:mainfrom
shahar1:fix/pr-review-no-live-mentions

Conversation

@shahar1

@shahar1 shahar1 commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • A posted review already notifies the PR author and every subscriber; a live @-mention additionally summons the named person without their consent onto a thread where nothing is asked of them yet. Seen in the wild: a review-queue comment needlessly notifying a maintainer who was merely named in an unresolved-threads observation (Pass dag_id to Dag bundle construction on task execution airflow#69697 (comment)).
  • Adopts the review-side counterpart of pr-management-triage's author-only notification rule: a new normative Mention policy section in posting.md — nothing the skill posts (review body, inline comments, adversarial fold-in, quoted contributor text) ever live-@-mentions anyone; handles render backtick-quoted (`@login`), and the Step 4.5 "Worth a second look from" suggestions are explicitly named, not notified.
  • Adds a mechanical Step 8 mention scan right before posting: any live mention that slipped through (quoted contributor text, adversarial fold-in, maintainer edit) is flagged with a [K]eep / [E]scape prompt — escape is the default, and a live mention reaches GitHub only through an explicit [K]eep, never silently.

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • prek run --all-files passes (all hooks green, including markdownlint, lychee, check-placeholders, skill-and-tool-validate)
  • For Python packages touched: uv run pytest / ruff check / mypy passes — n/a, no Python touched
  • For Groovy bridges touched: command-line invocation tested end-to-end — n/a
  • For skill changes: eval suite passes for the affected skill — new suite not yet executed against a live model (authored in a sandbox without model access); fixtures follow the step-4.5-suggested-reviewers structure and README updated (116 → 119 cases)
  • For skill behaviour changes: a new or updated eval fixture is included in this PR — new step-8-mention-scan suite, 3 cases: clean backtick-escaped body posts silently; stray live mention fires the [K]eep/[E]scape prompt; @ tokens inside code spans/fences, cron aliases, and decorators do not fire

RFC-AI-0004 compliance

  • HITL — the new mention scan gates any surviving live mention on an explicit [K]eep; nothing is escaped or posted silently
  • Sandbox — no change to host/network access
  • Vendor neutrality — new prose uses existing placeholder conventions; check-placeholders passes
  • Conversational + correctable — no new adopter-tunable behaviour
  • Write-access discipline — strengthens it: posted reviews can no longer notify third parties as a drafting side effect
  • Privacy LLM — n/a

Linked issues

Refs the incident thread: apache/airflow#69697 (comment)

Notes for reviewers (optional)

  • Handle style chosen: backticked with the @ (`@login`) — copy-pasting it into a comment produces a working mention. pr-management-triage is internally inconsistent on this (its hard-rule section says `@login`, its placeholder table says `login`); normalizing triage is deliberately left to a follow-up so this PR stays one-skill-one-concern.
  • The blanket policy covers the two incidental leak paths (quoted contributor text and adversarial fold-in), not just the deliberate Step 4.5 rendering — the incident above leaked exactly through rendered prose, not through a suggested-reviewers feature.

🤖 Generated with Claude Code

… reviews

A posted review already notifies the PR author and subscribers; a live
@-mention additionally summons the named person without their consent
onto a thread where nothing is asked of them yet (seen in the wild:
a review-queue comment needlessly notifying a maintainer named in an
unresolved-threads observation).

Adopt the review-side counterpart of pr-management-triage's author-only
notification rule:

- New normative "Mention policy" section in posting.md: nothing the
  skill posts — review body, inline comments, adversarial fold-in,
  quoted contributor text — ever live-@-mentions anyone. Handles render
  backtick-quoted (`@login`), which reads as a handle without
  notifying.
- Step 4.5 suggested reviewers are now explicitly named-not-notified;
  the "Worth a second look from" template renders backtick handles and
  says so.
- New Step 8 mention scan right before posting: any live mention that
  slipped through (quoted text, adversarial fold-in, maintainer edit)
  is flagged with a [K]eep / [E]scape prompt — escape is the default,
  a live mention is never posted silently.
- SKILL.md: the "does NOT do" list now covers notifying reviewers, not
  just formally requesting them.
- New step-8-mention-scan eval suite (3 cases): clean escaped body,
  stray live mention fires the prompt, @ tokens inside code spans /
  fences / decorators do not fire.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed - there were still some left overs it seems

@potiuk
potiuk merged commit 1b94202 into apache:main Jul 26, 2026
10 checks passed
@potiuk potiuk added capability:review Deep per-item code review or contributor mentoring family:docs Docs, MISSION.md, READMEs family:pr-management pr-management-* skills labels Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capability:review Deep per-item code review or contributor mentoring family:docs Docs, MISSION.md, READMEs family:pr-management pr-management-* skills

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants