Skip to content

test: add SDK-to-CLI coverage audit snapshot tests#27

Draft
romitg2 wants to merge 1 commit intomainfrom
devin/1773226412-sdk-cli-coverage-audit
Draft

test: add SDK-to-CLI coverage audit snapshot tests#27
romitg2 wants to merge 1 commit intomainfrom
devin/1773226412-sdk-cli-coverage-audit

Conversation

@romitg2
Copy link
Member

@romitg2 romitg2 commented Mar 11, 2026

test: add SDK-to-CLI coverage audit snapshot tests

Summary

Adds vitest snapshot tests that track which of the 272 generated SDK endpoint functions the CLI actually imports (214, ~79% coverage) and which 59 remain unused. When openapi.json changes — endpoints added, removed, or renamed — the snapshot diffs will surface the change and whether the CLI needs updating.

Changes:

  • packages/cli/src/__tests__/sdk-cli-coverage-audit.test.ts — 6 snapshot tests:
    1. All exported SDK endpoint functions
    2. CLI-imported SDK functions
    3. Unused SDK functions (generated but not used by any CLI command)
    4. SDK imports by file (which command files import which functions)
    5. Hard assertion: no CLI imports reference non-existent SDK functions
    6. Coverage metrics (total, imported, unused, percent)
  • packages/cli/package.json — adds vitest devDependency, test and test:update scripts
  • bun.lock — updated with vitest dependency tree

This is PR 3 of 3:

Review & Testing Checklist for Human

  • Merge order matters. All 3 PRs independently add vitest + test scripts to packages/cli/package.json (branched off main). Merging the second/third will produce conflicts in package.json and bun.lock — plan the merge order accordingly.
  • Regex-based parsing is fragile. getExportedSdkFunctions uses /^export const (\w+)\s*=/gm and getImportedSdkFunctions uses a regex to match import { ... } from '...generated/sdk.gen'. Verify these match the actual patterns in sdk.gen.ts and the CLI command files. Edge cases: multi-line imports with inline comments, or future changes to @hey-api/openapi-ts output format could silently produce wrong results rather than failing.
  • client is hardcoded as the only non-endpoint export to skip. If @hey-api/openapi-ts introduces other infrastructure exports, this filter will need updating — consider whether a more robust heuristic is preferable. Note: client still appears in the "CLI-imported" snapshot (tracked for completeness) but is excluded from the hard assertion that checks for non-existent SDK functions.
  • Coverage metric (79%) is a snapshot, not a threshold. The test will diff on any change but won't fail if coverage drops. Confirm this is the desired behavior vs. enforcing a minimum.

Suggested test plan

  1. cd packages/cli && bun run test — all 6 tests should pass
  2. Add a fake export const fooBarController = ... line to sdk.gen.ts → re-run → verify the "all SDK functions" and "unused functions" snapshots fail
  3. Remove an import from a CLI command file → re-run → verify the "CLI-imported" and "imports by file" snapshots fail
  4. Import a non-existent function from sdk.gen in a command file → re-run → verify the hard assertion (importedButMissing) fails

Notes

Adds vitest snapshot tests that track which SDK functions the CLI
imports vs what's available. Detects new endpoints, removed endpoints,
and changes in CLI coverage as openapi.json evolves.

Snapshots cover:
- All exported SDK endpoint functions
- CLI-imported SDK functions
- Unused SDK functions (available but not used by CLI)
- SDK imports by file (which CLI files import which functions)
- Coverage metrics (total, imported, unused, percent)

Co-Authored-By: romitgabani1 <romitgabani1.work@gmail.com>
@devin-ai-integration
Copy link
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant