Skip to content

feat(publish): add --dry-run validation endpoint and CLI option#451

Merged
dongmucat merged 6 commits into
mainfrom
feature/publish-dry-run
May 19, 2026
Merged

feat(publish): add --dry-run validation endpoint and CLI option#451
dongmucat merged 6 commits into
mainfrom
feature/publish-dry-run

Conversation

@dongmucat

Copy link
Copy Markdown
Collaborator

概述

skillhub publish 增加 --dry-run 选项,调用后端新增的"仅校验"接口,让开发者在本地即可获知包结构和元数据是否合法,无需实际发布。

变更内容

后端实现

  • SkillPublishService.validateOnly() — 提取验证逻辑(包结构、SKILL.md 解析、credential 扫描、slug 冲突检测),只读事务不持久化
  • CliSkillController 新增 POST /api/cli/v1/skills/{namespace}/publish/validate 端点
  • CliSkillAppService.validatePublish() — 编排层方法
  • CliDryRunResponse DTO — 返回 valid/errors/warnings/resolvedSlug/resolvedVersion

CLI 实现

  • SkillHubClient.validatePublish() — 调用新端点
  • publishCommand 增加 --dry-run 分支,调用 validate 而非 publish
  • index.ts 注册 --dry-run 选项

测试覆盖

  • 后端:CliDryRunValidateTest — 3 个用例(成功、失败、未认证)
  • CLI:publish-dry-run.test.ts — 6 个集成测试(成功/失败/JSON/namespace/不实际发布/认证)

质量门禁

  • make typecheck-web 通过 (0 errors)
  • make lint-web 通过 (0 errors, 0 warnings)
  • CLI 全量测试通过 (188 tests, 0 failures)
  • make test-backend-app 通过 (468 tests, 0 failures)
  • make generate-api 已执行且 schema.d.ts 已提交

安全考虑

  • 新端点复用现有认证机制(Bearer token + PlatformPrincipal)
  • 使用 @Transactional(readOnly = true) 确保不会意外写入
  • Rate limit 与 publish 端点一致(10 req/authenticated user)

相关 Issue

Closes #429

测试说明

本地验证步骤

  1. make dev-all 启动本地环境
  2. skillhub login --registry http://localhost:8080 --token <your-token>
  3. 创建一个包含 SKILL.md 的目录
  4. skillhub publish ./my-skill --dry-run — 应输出 "Validation passed" + slug/version
  5. 删除 SKILL.md 后再次执行 — 应输出 "Validation failed" + 错误信息
  6. skillhub publish ./my-skill --dry-run --json — 应输出结构化 JSON

回归测试范围

  • publish 正常流程不受影响(已通过现有测试验证)
  • CLI 其他命令不受影响(188 tests 全部通过)

Add a validate-only endpoint (POST /api/cli/v1/skills/{namespace}/publish/validate)
that runs the full pre-publish validation chain without persisting anything.
This allows developers to check their package locally before actual publishing.

The validation covers:
- SKILL.md existence and frontmatter parsing (name, description required)
- File extension whitelist and size limits
- Credential leak scanning with line-number precision
- Slug generation and name conflict detection

CLI usage: `skillhub publish <path> --dry-run`

Closes #429

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a dry-run feature for skill publishing, allowing users to validate their packages via the CLI using a new --dry-run flag. The implementation includes a new backend endpoint, service logic for validation without persistence, and comprehensive integration tests. Feedback highlights the need for explicit UTF-8 charset handling when parsing metadata, the inclusion of missing validation checks for archived skills and existing versions to ensure consistency with the actual publish flow, and the requirement for the CLI to return a non-zero exit code upon validation failure to support CI/CD pipelines.


SkillMetadata metadata;
try {
metadata = skillMetadataParser.parse(new String(skillMd.content()));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using the platform's default charset when converting byte[] to String is risky and can lead to data corruption if the SKILL.md file contains non-ASCII characters (e.g., UTF-8 encoded Chinese characters) and the server environment uses a different default charset. Always specify StandardCharsets.UTF_8 explicitly.

Suggested change
metadata = skillMetadataParser.parse(new String(skillMd.content()));
metadata = skillMetadataParser.parse(new String(skillMd.content(), java.nio.charset.StandardCharsets.UTF_8));

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in bdc94ad. Changed to new String(skillMd.content(), StandardCharsets.UTF_8).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in bdc94ad. Changed to new String(skillMd.content(), StandardCharsets.UTF_8).

Comment on lines +216 to +229
if (resolvedSlug != null && errors.isEmpty()) {
List<Skill> existingSkills = skillRepository.findByNamespaceIdAndSlug(namespace.getId(), resolvedSlug);
for (Skill existing : existingSkills) {
if (!existing.getOwnerId().equals(publisherId)) {
boolean hasPublished = !skillVersionRepository
.findBySkillIdAndStatus(existing.getId(), SkillVersionStatus.PUBLISHED)
.isEmpty();
if (hasPublished) {
errors.add("Name conflict: slug \"" + resolvedSlug + "\" is already published by another user");
break;
}
}
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The validateOnly logic is missing two critical checks that are performed during the actual publish flow:

  1. Archived Skill Check: It should verify if the publisher already owns a skill with the same slug that is currently archived. Publishing to an archived skill is prohibited.
  2. Existing Version Check: It should verify if the specific version being validated has already been published for an existing skill owned by the publisher.

Without these checks, a dry run might incorrectly report success for a package that will fail during actual publication.

        // 6. Slug conflict and version existence check
        if (resolvedSlug != null && errors.isEmpty()) {
            List<Skill> existingSkills = skillRepository.findByNamespaceIdAndSlug(namespace.getId(), resolvedSlug);
            for (Skill existing : existingSkills) {
                if (existing.getOwnerId().equals(publisherId)) {
                    if (existing.getStatus() == SkillStatus.ARCHIVED) {
                        errors.add("Cannot publish to archived skill: " + resolvedSlug);
                    }
                    var existingVersion = skillVersionRepository.findBySkillIdAndVersion(existing.getId(), resolvedVersion);
                    if (existingVersion.isPresent() && existingVersion.get().getStatus() == SkillVersionStatus.PUBLISHED) {
                        errors.add("Version already exists and is published: " + resolvedVersion);
                    }
                } else {
                    boolean hasPublished = !skillVersionRepository
                            .findBySkillIdAndStatus(existing.getId(), SkillVersionStatus.PUBLISHED)
                            .isEmpty();
                    if (hasPublished) {
                        errors.add("Name conflict: slug \"" + resolvedSlug + "\" is already published by another user");
                        break;
                    }
                }
            }
        }
References
  1. Ensure there is an archival mechanism for auditability when dealing with unique identifiers like slugs.
  2. Multiple existence checks are acceptable for low-frequency operations due to short-circuiting.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in bdc94ad. Added both checks:

  1. Archived skill: existing.getStatus() == SkillStatus.ARCHIVED → error
  2. Version exists: findBySkillIdAndVersion + status == PUBLISHED → error

Comment on lines +60 to +92
if (options.dryRun) {
const result = await client.validatePublish(namespace, archiveBlob, archiveName)

if (options.json) {
return JSON.stringify(result)
}

const lines: string[] = []
if (result.valid) {
lines.push('Validation passed')
} else {
lines.push('Validation failed')
}
if (result.resolvedSlug) {
lines.push(` Slug: ${result.resolvedSlug}`)
}
if (result.resolvedVersion) {
lines.push(` Version: ${result.resolvedVersion}`)
}
if (result.errors.length > 0) {
lines.push('Errors:')
for (const error of result.errors) {
lines.push(` - ${error}`)
}
}
if (result.warnings.length > 0) {
lines.push('Warnings:')
for (const warning of result.warnings) {
lines.push(` - ${warning}`)
}
}
return lines.join('\n')
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When --dry-run is used, the command currently returns a formatted string and exits with code 0 even if validation fails (result.valid === false). For a CLI tool, a validation failure should signal a non-zero exit code so that it can be correctly handled in CI/CD pipelines or git hooks. Consider throwing a CliError when validation fails.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in bdc94ad. Validation failure now exits with code 6 (EXIT.validation). Both --json and human-readable output are written to stdout before the non-zero exit, so CI pipelines can parse the details.

dongmucat added 5 commits May 18, 2026 10:57
- Exit non-zero (code 6) when --dry-run validation fails, enabling
  CI/CD pipeline integration
- Add archived skill check: dry-run now detects when the publisher's
  own skill is archived
- Add version-exists check: dry-run now detects when the resolved
  version is already published
- Use StandardCharsets.UTF_8 for SKILL.md content parsing
Fix three blockers and one contract drift issue surfaced in code review:

1. API token policy: add skill:publish scope policy and authentication
   policy for /api/cli/v1/skills/*/publish/validate. Without these the
   AntPathMatcher pattern /publish would not cover /publish/validate,
   so Bearer-token requests would be rejected by the scope filter.

2. Warnings semantics: dry-run now treats warnings as making valid=false.
   The CLI publish flow uses confirmWarnings=false, so the real publish
   rejects any warnings; dry-run must mirror that to avoid false positives.

3. Visibility parameter: validate endpoint now accepts the same
   visibility multipart field as publish. The CLI forwards --visibility
   so invalid values are caught at dry-run time rather than at publish.

4. Schema drift: resolvedSlug and resolvedVersion are nullable in
   practice (returned as null when validation fails before resolution).
   Updated schema.d.ts to reflect string | null instead of optional string.

Tests added:
- RouteSecurityPolicyRegistryTest: validate endpoint scope check
- CliDryRunValidateTest: custom + invalid visibility cases
- publish-dry-run.test.ts: --visibility forwarded to server
ApiTokenAuthenticationFilter authenticates /api/cli/** Bearer tokens
but ApiTokenScopeFilter.shouldNotFilter() previously skipped them.
The result: API token requests on CLI routes were authenticated and
authorization-policy-checked, but scope enforcement never ran. Tokens
without skill:publish or skill:delete could call /publish, /publish/validate,
and DELETE despite the policy table requiring those scopes.

Add /api/cli/ to the scope filter's covered prefixes and a filter-level
test that confirms a token missing skill:publish is rejected on the new
validate endpoint. Update the existing CLI controller tests to grant
the appropriate SCOPE_* authorities to their api_token principals so
they continue to pass under enforced scopes.
- Fix test fixture: warnings-only response now uses valid=false to
  match real backend behavior (warnings make dry-run invalid)
- Distinguish 403 from 401 in CLI error messages: 403 now says
  "access denied — token may lack required scope" with a hint to
  regenerate the token, rather than the generic "authentication failed"
Covers the new "access denied — token may lack required scope" error
path with a fake-registry 'forbidden' failure mode. Prevents the
improved 403 message from regressing silently.
@dongmucat dongmucat merged commit baf84ac into main May 19, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] 增强预发布校验与 publish --dry-run 功能

1 participant