Skip to content

test(hooks): validate source entrypoints and typecheck in CI - #1785

Closed
zack-dev-cm wants to merge 2 commits into
rohitg00:mainfrom
zack-dev-cm:fix/1738-capture-hook-checks
Closed

zack-dev-cm wants to merge 2 commits into
rohitg00:mainfrom
zack-dev-cm:fix/1738-capture-hook-checks

Conversation

@zack-dev-cm

@zack-dev-cm zack-dev-cm commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

CI currently excludes capture hooks from its TypeScript project. Add a separate strict hook project and run it in each existing OS/Node test cell, preserving the main project's exclusions.

Add one direct-source test file for each of session-start, prompt-submit, pre-tool-use, post-tool-use and stop. Isolated Node children read real stdin with controlled fetch responses. The tests check complete requests and context output, malformed input, SDK child guards, transport refusal, unreachable and hanging requests, spooling, and bounded successful exit. The request observer also records missing and non-JSON bodies, so unexpected requests cannot disappear from a no-fetch assertion. Runtime capture code and existing bundled-hook tests stay unchanged.

Validation:

  • All 51 focused checks pass. The original 49 behavior oracles and deadlines are unchanged; two added observer regressions fail on the first recorder and pass after the correction.
  • All 24 hook sources typecheck; the same CI command rejects a deliberately mistyped additional hook. The seven new TypeScript test/harness files also pass a strict check.
  • Required build, skills generation/checks, documentation sync/check and evaluation typecheck pass. Full local suite: 4,969 passed, 13 existing mesh failures, 7 skipped. Every prior test outcome and complete failure message matches the preceding full run.

Fixes #1738.

First fork CI, checked at 21:00 UTC on 13 November 2026: Ubuntu Node 20/22/24/26 passed the full suite, all 51 new checks and the hook typecheck. The four macOS cells had not executed at that check. CI run.

Summary by CodeRabbit

  • Tests
    • Added automated coverage for hook behavior, including request contents, context handling, spooling, and failure scenarios.
    • Added checks for fetch request recording and handling of unreachable or delayed requests.
  • Chores
    • CI now runs a TypeScript check for hooks as part of the test workflow.

First fork CI is now complete: all seven executed cells passed the full 4,989-test suite, all 51 new checks and the hook typecheck. macOS Node 26 was cancelled before a runner was allocated; its annotations report macOS runner capacity. The overall workflow conclusion is failure; that cell remains unvalidated.

Signed-off-by: zack-dev-cm <112114097+zack-dev-cm@users.noreply.github.com>
Signed-off-by: zack-dev-cm <112114097+zack-dev-cm@users.noreply.github.com>
@vercel

vercel Bot commented Nov 13, 2026

Copy link
Copy Markdown

@zack-dev-cm is attempting to deploy a commit to the rohitg00's projects Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The CI workflow now typechecks source hooks. New test helpers run hooks in child processes and capture requests and spool records. Tests cover the fetch observer and five hook entrypoints, including configured failure cases.

Changes: Hook validation

Layer / File(s) Summary
Hooks typecheck
tsconfig.hooks.json, package.json, .github/workflows/ci.yml
Adds a no-emit TypeScript check for hook files and runs it in CI.
Source-hook test harness
test/helpers/hook-source.ts, test/fixtures/hook-source-fetch.mjs
Adds child-process execution, result and request capture, and a fetch mock that supports successful, unreachable, and hanging requests.
Hook entrypoint tests
test/*-source.test.ts, test/hook-source-fetch.test.ts
Adds request, output, and failure-case assertions for the fetch observer and the post-tool-use, pre-tool-use, prompt-submit, session-start, and stop hooks.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other · Severity of issue fixed: Medium

Merge Risk: Minimal · up to 5862d

This PR adds CI typechecking for the capture hooks and direct tests for five hooks, and leaves runtime behavior unchanged. No actionable merge-blocking risk is identified.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the test(hooks) scope and the CI typecheck addition, which are the two main parts of the changeset. The changeset adds direct source-entrypoint tests for five capture hooks and a separ…
Linked Issues check Passed Issue #1738 has two coding acceptance criteria. (1) src/hooks is typechecked in CI: tsconfig.hooks.json extends ./tsconfig.json, sets noEmit, and includes src/hooks/**/*.ts. package.json adds typechec…
Out of Scope Changes check Passed All changed files support issue #1738. Changes: .github/workflows/ci.yml (adds typecheck:hooks step); package.json (adds typecheck:hooks script); tsconfig.hooks.json (new strict hook project); test/he…
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Full details: Title check

Explanation

The title names the test(hooks) scope and the CI typecheck addition, which are the two main parts of the changeset. The changeset adds direct source-entrypoint tests for five capture hooks and a separate strict tsconfig for src/hooks that CI runs. The title does not name every test file, but it covers the primary change.

Full details: Linked Issues check

Explanation

Issue #1738 has two coding acceptance criteria. (1) src/hooks is typechecked in CI: tsconfig.hooks.json extends ./tsconfig.json, sets noEmit, and includes src/hooks/**/*.ts. package.json adds typecheck:hooks, and ci.yml runs it in the test job. The main tsconfig exclusions are unchanged. (2) The five main hooks each have a source-level test file: session-start, prompt-submit, pre-tool-use, post-tool-use, stop. The shared helper test/helpers/hook-source.ts runs each hook as a child Node process with real stdin and a controlled fetch preload. It covers: valid payload posts to the expected endpoint with the expected auth and content-type headers (per-hook files assert body and path); malformed stdin (broken JSON, null, 7) exits 0 with no requests and no spool; SDK child guard via environment and payload; unreachable and hanging fetch exit 0 under 5 s, with spool records for capture hooks; non-loopback http with a secret is refused with 'must use HTTPS' and no fetch or spool. The request observer records missing and non-JSON bodies, so unexpected requests fail no-fetch assertions. The per-hook test files were reviewed only through the summary and the helper source. The author reports 51 passing checks and Ubuntu CI passes. I did not run the tests. MacOS CI cells had not run at the time of the PR description. The 7 s helper deadline and the 5 s budget match the issue's time-budget intent, but the hook's own timeout value was not inspected directly. Stop-hook behavior tests are kept separate, as the issue allows.

Full details: Out of Scope Changes check

Explanation

All changed files support issue #1738. Changes: .github/workflows/ci.yml (adds typecheck:hooks step); package.json (adds typecheck:hooks script); tsconfig.hooks.json (new strict hook project); test/helpers/hook-source.ts (shared test harness); test/fixtures/hook-source-fetch.mjs (fetch preload for the harness); test/hook-source-fetch.test.ts (tests the observer that the harness depends on, so that unexpected requests cannot pass unnoticed); and five test/*-source.test.ts files, one per target hook. The PR states that runtime capture code and existing bundled-hook tests are unchanged. The changed-file list in the summary shows no runtime source changes under src/. No unrelated change was found.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 08ddeaeb-6eac-4c3e-88bc-a17814ca4d82
Commits

Reviewing files that changed from the base of the PR and between cf9a738 and 5862d55.

Files selected for processing: 11

  • .github/workflows/ci.yml
  • package.json
  • test/fixtures/hook-source-fetch.mjs
  • test/helpers/hook-source.ts
  • test/hook-source-fetch.test.ts
  • test/post-tool-use-source.test.ts
  • test/pre-tool-use-source.test.ts
  • test/prompt-submit-source.test.ts
  • test/session-start-source.test.ts
  • test/stop-source.test.ts
  • tsconfig.hooks.json

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@rohitg00

Copy link
Copy Markdown
Owner

Merged as part of #1811, which kept your commits and added review fixes on top (details in its description). Thanks for the contribution.

@rohitg00 rohitg00 closed this Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Capture hooks have no direct tests and are excluded from typecheck

2 participants