feat(publish): add --dry-run validation endpoint and CLI option#451
Conversation
Add a validate-only endpoint (POST /api/cli/v1/skills/{namespace}/publish/validate)
that runs the full pre-publish validation chain without persisting anything.
This allows developers to check their package locally before actual publishing.
The validation covers:
- SKILL.md existence and frontmatter parsing (name, description required)
- File extension whitelist and size limits
- Credential leak scanning with line-number precision
- Slug generation and name conflict detection
CLI usage: `skillhub publish <path> --dry-run`
Closes #429
There was a problem hiding this comment.
Code Review
This pull request introduces a dry-run feature for skill publishing, allowing users to validate their packages via the CLI using a new --dry-run flag. The implementation includes a new backend endpoint, service logic for validation without persistence, and comprehensive integration tests. Feedback highlights the need for explicit UTF-8 charset handling when parsing metadata, the inclusion of missing validation checks for archived skills and existing versions to ensure consistency with the actual publish flow, and the requirement for the CLI to return a non-zero exit code upon validation failure to support CI/CD pipelines.
|
|
||
| SkillMetadata metadata; | ||
| try { | ||
| metadata = skillMetadataParser.parse(new String(skillMd.content())); |
There was a problem hiding this comment.
Using the platform's default charset when converting byte[] to String is risky and can lead to data corruption if the SKILL.md file contains non-ASCII characters (e.g., UTF-8 encoded Chinese characters) and the server environment uses a different default charset. Always specify StandardCharsets.UTF_8 explicitly.
| metadata = skillMetadataParser.parse(new String(skillMd.content())); | |
| metadata = skillMetadataParser.parse(new String(skillMd.content(), java.nio.charset.StandardCharsets.UTF_8)); |
There was a problem hiding this comment.
Fixed in bdc94ad. Changed to new String(skillMd.content(), StandardCharsets.UTF_8).
There was a problem hiding this comment.
Fixed in bdc94ad. Changed to new String(skillMd.content(), StandardCharsets.UTF_8).
| if (resolvedSlug != null && errors.isEmpty()) { | ||
| List<Skill> existingSkills = skillRepository.findByNamespaceIdAndSlug(namespace.getId(), resolvedSlug); | ||
| for (Skill existing : existingSkills) { | ||
| if (!existing.getOwnerId().equals(publisherId)) { | ||
| boolean hasPublished = !skillVersionRepository | ||
| .findBySkillIdAndStatus(existing.getId(), SkillVersionStatus.PUBLISHED) | ||
| .isEmpty(); | ||
| if (hasPublished) { | ||
| errors.add("Name conflict: slug \"" + resolvedSlug + "\" is already published by another user"); | ||
| break; | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
The validateOnly logic is missing two critical checks that are performed during the actual publish flow:
- Archived Skill Check: It should verify if the publisher already owns a skill with the same slug that is currently archived. Publishing to an archived skill is prohibited.
- Existing Version Check: It should verify if the specific version being validated has already been published for an existing skill owned by the publisher.
Without these checks, a dry run might incorrectly report success for a package that will fail during actual publication.
// 6. Slug conflict and version existence check
if (resolvedSlug != null && errors.isEmpty()) {
List<Skill> existingSkills = skillRepository.findByNamespaceIdAndSlug(namespace.getId(), resolvedSlug);
for (Skill existing : existingSkills) {
if (existing.getOwnerId().equals(publisherId)) {
if (existing.getStatus() == SkillStatus.ARCHIVED) {
errors.add("Cannot publish to archived skill: " + resolvedSlug);
}
var existingVersion = skillVersionRepository.findBySkillIdAndVersion(existing.getId(), resolvedVersion);
if (existingVersion.isPresent() && existingVersion.get().getStatus() == SkillVersionStatus.PUBLISHED) {
errors.add("Version already exists and is published: " + resolvedVersion);
}
} else {
boolean hasPublished = !skillVersionRepository
.findBySkillIdAndStatus(existing.getId(), SkillVersionStatus.PUBLISHED)
.isEmpty();
if (hasPublished) {
errors.add("Name conflict: slug \"" + resolvedSlug + "\" is already published by another user");
break;
}
}
}
}References
- Ensure there is an archival mechanism for auditability when dealing with unique identifiers like slugs.
- Multiple existence checks are acceptable for low-frequency operations due to short-circuiting.
There was a problem hiding this comment.
Fixed in bdc94ad. Added both checks:
- Archived skill:
existing.getStatus() == SkillStatus.ARCHIVED→ error - Version exists:
findBySkillIdAndVersion+ status == PUBLISHED → error
| if (options.dryRun) { | ||
| const result = await client.validatePublish(namespace, archiveBlob, archiveName) | ||
|
|
||
| if (options.json) { | ||
| return JSON.stringify(result) | ||
| } | ||
|
|
||
| const lines: string[] = [] | ||
| if (result.valid) { | ||
| lines.push('Validation passed') | ||
| } else { | ||
| lines.push('Validation failed') | ||
| } | ||
| if (result.resolvedSlug) { | ||
| lines.push(` Slug: ${result.resolvedSlug}`) | ||
| } | ||
| if (result.resolvedVersion) { | ||
| lines.push(` Version: ${result.resolvedVersion}`) | ||
| } | ||
| if (result.errors.length > 0) { | ||
| lines.push('Errors:') | ||
| for (const error of result.errors) { | ||
| lines.push(` - ${error}`) | ||
| } | ||
| } | ||
| if (result.warnings.length > 0) { | ||
| lines.push('Warnings:') | ||
| for (const warning of result.warnings) { | ||
| lines.push(` - ${warning}`) | ||
| } | ||
| } | ||
| return lines.join('\n') | ||
| } |
There was a problem hiding this comment.
When --dry-run is used, the command currently returns a formatted string and exits with code 0 even if validation fails (result.valid === false). For a CLI tool, a validation failure should signal a non-zero exit code so that it can be correctly handled in CI/CD pipelines or git hooks. Consider throwing a CliError when validation fails.
There was a problem hiding this comment.
Fixed in bdc94ad. Validation failure now exits with code 6 (EXIT.validation). Both --json and human-readable output are written to stdout before the non-zero exit, so CI pipelines can parse the details.
- Exit non-zero (code 6) when --dry-run validation fails, enabling CI/CD pipeline integration - Add archived skill check: dry-run now detects when the publisher's own skill is archived - Add version-exists check: dry-run now detects when the resolved version is already published - Use StandardCharsets.UTF_8 for SKILL.md content parsing
Fix three blockers and one contract drift issue surfaced in code review: 1. API token policy: add skill:publish scope policy and authentication policy for /api/cli/v1/skills/*/publish/validate. Without these the AntPathMatcher pattern /publish would not cover /publish/validate, so Bearer-token requests would be rejected by the scope filter. 2. Warnings semantics: dry-run now treats warnings as making valid=false. The CLI publish flow uses confirmWarnings=false, so the real publish rejects any warnings; dry-run must mirror that to avoid false positives. 3. Visibility parameter: validate endpoint now accepts the same visibility multipart field as publish. The CLI forwards --visibility so invalid values are caught at dry-run time rather than at publish. 4. Schema drift: resolvedSlug and resolvedVersion are nullable in practice (returned as null when validation fails before resolution). Updated schema.d.ts to reflect string | null instead of optional string. Tests added: - RouteSecurityPolicyRegistryTest: validate endpoint scope check - CliDryRunValidateTest: custom + invalid visibility cases - publish-dry-run.test.ts: --visibility forwarded to server
ApiTokenAuthenticationFilter authenticates /api/cli/** Bearer tokens but ApiTokenScopeFilter.shouldNotFilter() previously skipped them. The result: API token requests on CLI routes were authenticated and authorization-policy-checked, but scope enforcement never ran. Tokens without skill:publish or skill:delete could call /publish, /publish/validate, and DELETE despite the policy table requiring those scopes. Add /api/cli/ to the scope filter's covered prefixes and a filter-level test that confirms a token missing skill:publish is rejected on the new validate endpoint. Update the existing CLI controller tests to grant the appropriate SCOPE_* authorities to their api_token principals so they continue to pass under enforced scopes.
- Fix test fixture: warnings-only response now uses valid=false to match real backend behavior (warnings make dry-run invalid) - Distinguish 403 from 401 in CLI error messages: 403 now says "access denied — token may lack required scope" with a hint to regenerate the token, rather than the generic "authentication failed"
Covers the new "access denied — token may lack required scope" error path with a fake-registry 'forbidden' failure mode. Prevents the improved 403 message from regressing silently.
概述
为
skillhub publish增加--dry-run选项,调用后端新增的"仅校验"接口,让开发者在本地即可获知包结构和元数据是否合法,无需实际发布。变更内容
后端实现
SkillPublishService.validateOnly()— 提取验证逻辑(包结构、SKILL.md 解析、credential 扫描、slug 冲突检测),只读事务不持久化CliSkillController新增POST /api/cli/v1/skills/{namespace}/publish/validate端点CliSkillAppService.validatePublish()— 编排层方法CliDryRunResponseDTO — 返回 valid/errors/warnings/resolvedSlug/resolvedVersionCLI 实现
SkillHubClient.validatePublish()— 调用新端点publishCommand增加--dry-run分支,调用 validate 而非 publishindex.ts注册--dry-run选项测试覆盖
CliDryRunValidateTest— 3 个用例(成功、失败、未认证)publish-dry-run.test.ts— 6 个集成测试(成功/失败/JSON/namespace/不实际发布/认证)质量门禁
make typecheck-web通过 (0 errors)make lint-web通过 (0 errors, 0 warnings)make test-backend-app通过 (468 tests, 0 failures)make generate-api已执行且schema.d.ts已提交安全考虑
@Transactional(readOnly = true)确保不会意外写入相关 Issue
Closes #429
测试说明
本地验证步骤
make dev-all启动本地环境skillhub login --registry http://localhost:8080 --token <your-token>skillhub publish ./my-skill --dry-run— 应输出 "Validation passed" + slug/versionskillhub publish ./my-skill --dry-run --json— 应输出结构化 JSON回归测试范围