Skip to content

Commit c40789b

Browse files
cameroncookecodexcursoragent
authored
test(snapshot): Add structured JSON fixture targets (#338)
* test(snapshot): Add structured JSON fixture targets Add a structured JSON fixture corpus for the planned CLI json output mode. Define shared envelope and schema-family targets in docs/dev/STRUCTURED_JSON_OUTPUT_PLAN.md and capture the expected payloads across simulator, device, macOS, SwiftPM, debugging, discovery, and session workflows. This keeps the current producer unchanged while giving the upcoming implementation a concrete fixture contract to build against. Co-Authored-By: OpenAI Codex <noreply@openai.com> * Fix schema consistency in JSON fixtures - Add missing isAvailable field to device-list schema in resources/devices--success.json - Add missing diagnostics field to simulator success fixtures (stop, install, launch-app) Applied via @cursor push command * fix(fixtures): Align simulator and resources fixtures with cross-platform schema contract - Add missing summary.durationMs and summary.target to all simulator fixtures - Fix singular "Test failed" to "Tests failed" in simulator failure fixtures - Normalize counts key ordering to passed/failed/skipped in simulator test failure - Align resources/devices field ordering with device/list fixture * fix(fixtures): Add missing durationMs and target to simulator error fixtures * fix(fixtures): Replace string placeholders with mock numeric values across all fixtures - durationMs: "<DURATION>" -> 1234 - processId/pid: "<PID>" -> 99999 - Test failure counts: "<PASS_COUNT>"/"<FAIL_COUNT>"/"<SKIP_COUNT>" -> mock integers matching discovered totals - session-status: uptimeMs/rssBytes/heapUsedBytes -> mock integers - doctor: manifest counts, simulatorVideoCaptureSupported -> proper types - stack: threadId/index -> integers - swift-package/list: rename uptime -> uptimeSeconds - Fix remaining "Test failed" -> "Tests failed" in macos and swift-package error fixtures * fix(fixtures): Restructure variables scopes, remove output sections, drop resource fixtures - Convert debug variables scopes from array to keyed object, remove displayName - Remove output.stdout/stderr from build-and-run success fixtures - Remove remainingCount from test failure discovered items - Delete resource fixtures (devices, doctor, session-status, simulators) * test(snapshot): Add JSON fixture schema validation Add a dedicated schema corpus for structured JSON fixtures and validate each discovered fixture against its declared schema and version. This gives the hand-crafted fixture contracts an executable check without coupling them to the existing text snapshot harness. Co-Authored-By: OpenAI Codex <noreply@openai.com> * ci(schemas): Add website publish workflow Document schema versioning and deployment policy and add a workflow that syncs published schema files into the website repository for Vercel deployment. This keeps schema authoring in this repository while making the published schema URLs durable and real. Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix(ci): Harden schema publish workflow Limit the publish workflow token to read-only contents access and avoid interpolating the deploy key secret directly inside the shell guard. This keeps the workflow aligned with GitHub Actions security expectations and removes a brittle secret-handling pattern. Co-Authored-By: OpenAI Codex <noreply@openai.com> --------- Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com>
1 parent dbb2d3b commit c40789b

File tree

173 files changed

+8341
-56
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

173 files changed

+8341
-56
lines changed

‎.github/workflows/publish-schemas.yml‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
name: Publish Schemas
2+
3+
permissions:
4+
contents: read
5+
6+
on:
7+
push:
8+
branches: [main]
9+
paths:
10+
- 'schemas/**'
11+
- '.github/workflows/publish-schemas.yml'
12+
workflow_dispatch:
13+
14+
jobs:
15+
publish:
16+
runs-on: ubuntu-latest
17+
env:
18+
WEBSITE_REPO: getsentry/xcodebuildmcp.com
19+
WEBSITE_BRANCH: main
20+
TARGET_DIR: public/schemas/structured-output
21+
steps:
22+
- name: Checkout source repository
23+
uses: actions/checkout@v4
24+
25+
- name: Fail if deploy key is missing
26+
env:
27+
DEPLOY_KEY: ${{ secrets.XCODEBUILDMCP_WEBSITE_DEPLOY_KEY }}
28+
run: |
29+
set -euo pipefail
30+
if [ -z "$DEPLOY_KEY" ]; then
31+
echo "XCODEBUILDMCP_WEBSITE_DEPLOY_KEY is required to publish schemas." >&2
32+
exit 1
33+
fi
34+
35+
- name: Configure SSH for website repository
36+
uses: webfactory/ssh-agent@v0.9.0
37+
with:
38+
ssh-private-key: ${{ secrets.XCODEBUILDMCP_WEBSITE_DEPLOY_KEY }}
39+
40+
- name: Clone website repository
41+
run: |
42+
set -euo pipefail
43+
git clone "git@github.com:${WEBSITE_REPO}.git" website-repo
44+
cd website-repo
45+
git checkout "$WEBSITE_BRANCH"
46+
git pull --ff-only origin "$WEBSITE_BRANCH"
47+
48+
- name: Sync schema files into website public directory
49+
run: |
50+
set -euo pipefail
51+
mkdir -p "website-repo/${TARGET_DIR}"
52+
rsync -a --delete schemas/structured-output/ "website-repo/${TARGET_DIR}/"
53+
54+
- name: Commit and push website update
55+
run: |
56+
set -euo pipefail
57+
cd website-repo
58+
git config user.name "github-actions[bot]"
59+
git config user.email "github-actions[bot]@users.noreply.github.com"
60+
git add "$TARGET_DIR"
61+
if git diff --cached --quiet; then
62+
echo "Schema publish target already up to date."
63+
exit 0
64+
fi
65+
git commit -m "Publish structured output schemas from ${GITHUB_REPOSITORY}@${GITHUB_SHA}"
66+
git push origin "$WEBSITE_BRANCH"

‎docs/SCHEMA_VERSIONING.md‎

Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
# Structured JSON Schema versioning and publishing
2+
3+
This document defines how XcodeBuildMCP versions and publishes the JSON Schemas for
4+
structured output fixtures and runtime payloads.
5+
6+
## Goals
7+
8+
- Keep schema contracts stable and predictable for external consumers.
9+
- Make published schema URLs real, durable, and safe to reference.
10+
- Let the website serve schemas directly from `https://xcodebuildmcp.com/schemas/...`.
11+
- Avoid ambiguous compatibility rules.
12+
13+
## Canonical schema identity
14+
15+
Each schema has two stable identifiers:
16+
17+
1. The payload metadata:
18+
```json
19+
{
20+
"schema": "xcodebuildmcp.output.build-result",
21+
"schemaVersion": "1"
22+
}
23+
```
24+
2. The published schema file path:
25+
```text
26+
https://xcodebuildmcp.com/schemas/structured-output/xcodebuildmcp.output.build-result/1.schema.json
27+
```
28+
29+
The in-payload `schema` and `schemaVersion` values must always match the published
30+
schema document that validates that payload.
31+
32+
## Version format
33+
34+
`schemaVersion` uses integer strings only:
35+
36+
- `"1"`
37+
- `"2"`
38+
- `"3"`
39+
40+
Do not use semver-style schema versions such as `"1.1"` or `"2.0"`.
41+
42+
The version number is a contract version, not a release number.
43+
44+
## Versioning rules
45+
46+
### Published versions are immutable
47+
48+
Once a schema version is published, do not make breaking changes to that file.
49+
50+
Breaking changes include:
51+
52+
- removing a property
53+
- making an optional property required
54+
- narrowing allowed values or enums
55+
- changing object shape incompatibly
56+
- changing field meaning in a way that could break clients
57+
58+
If any of those changes are needed, publish a new version instead:
59+
60+
```text
61+
schemas/structured-output/xcodebuildmcp.output.build-result/2.schema.json
62+
```
63+
64+
and emit:
65+
66+
```json
67+
"schemaVersion": "2"
68+
```
69+
70+
### Old versions remain available
71+
72+
Previously published schema files must continue to be hosted.
73+
74+
Do not remove old schema versions from the website once consumers may rely on them.
75+
76+
### Additive changes
77+
78+
Additive, optional fields can be compatible, but use caution.
79+
80+
If a new field is truly optional and old clients can safely ignore it, it may remain
81+
within the same schema version. If there is any doubt about compatibility or meaning,
82+
bump the schema version.
83+
84+
Bias toward a new version when the contract meaning changes.
85+
86+
## Directory layout
87+
88+
Source schemas in this repository live under:
89+
90+
```text
91+
schemas/structured-output/
92+
```
93+
94+
Published schemas on the website live under:
95+
96+
```text
97+
public/schemas/structured-output/
98+
```
99+
100+
A source file such as:
101+
102+
```text
103+
schemas/structured-output/xcodebuildmcp.output.build-result/1.schema.json
104+
```
105+
106+
is published to:
107+
108+
```text
109+
public/schemas/structured-output/xcodebuildmcp.output.build-result/1.schema.json
110+
```
111+
112+
which is then served at:
113+
114+
```text
115+
https://xcodebuildmcp.com/schemas/structured-output/xcodebuildmcp.output.build-result/1.schema.json
116+
```
117+
118+
## Publishing workflow
119+
120+
Schema publishing is handled from this repository by a GitHub Actions workflow.
121+
122+
Trigger conditions:
123+
124+
- push to `main` when files under `schemas/**` change
125+
- manual `workflow_dispatch`
126+
127+
Publishing steps:
128+
129+
1. Check out this repository.
130+
2. Clone `getsentry/xcodebuildmcp.com` over SSH.
131+
3. Sync `schemas/structured-output/` from this repository into
132+
`public/schemas/structured-output/` in the website repository.
133+
4. Commit the website change if the published files changed.
134+
5. Push to the website repository `main` branch.
135+
6. Let Vercel deploy the website normally.
136+
137+
This keeps schema authoring in the main project repository while using the website
138+
repository as the deployment surface.
139+
140+
## Required secret
141+
142+
The publishing workflow requires this repository secret:
143+
144+
```text
145+
XCODEBUILDMCP_WEBSITE_DEPLOY_KEY
146+
```
147+
148+
This secret must contain an SSH private key with write access to:
149+
150+
```text
151+
git@github.com:getsentry/xcodebuildmcp.com.git
152+
```
153+
154+
The corresponding public key should be installed as a deploy key on the website
155+
repository with write access.
156+
157+
### Deploy key setup
158+
159+
1. Generate a dedicated SSH key pair for schema publishing.
160+
```bash
161+
ssh-keygen -t ed25519 -C "schema-publisher" -f ./xcodebuildmcp-website-deploy-key
162+
```
163+
2. In `getsentry/xcodebuildmcp.com`, add the public key as a deploy key with write
164+
access.
165+
- GitHub, Settings, Deploy keys
166+
- Add `xcodebuildmcp-website-deploy-key.pub`
167+
- Enable write access
168+
3. In `getsentry/XcodeBuildMCP`, add the private key as an actions secret named:
169+
```text
170+
XCODEBUILDMCP_WEBSITE_DEPLOY_KEY
171+
```
172+
4. Trigger the `Publish Schemas` workflow manually once to verify SSH access and sync.
173+
5. Confirm that the website repository receives the commit and Vercel deploys it.
174+
6. Confirm a final URL resolves, for example:
175+
```text
176+
https://xcodebuildmcp.com/schemas/structured-output/xcodebuildmcp.output.build-result/1.schema.json
177+
```
178+
179+
Use a dedicated deploy key for this workflow only. Do not reuse a personal SSH key.
180+
181+
## Consumer guidance
182+
183+
Consumers should branch on both `schema` and `schemaVersion`.
184+
185+
Example:
186+
187+
```ts
188+
switch (`${payload.schema}@${payload.schemaVersion}`) {
189+
case "xcodebuildmcp.output.build-result@1":
190+
// validate using v1 schema
191+
break
192+
case "xcodebuildmcp.output.build-result@2":
193+
// validate using v2 schema
194+
break
195+
default:
196+
throw new Error("Unsupported schema version")
197+
}
198+
```
199+
200+
These JSON Schemas describe payload shapes. They are not an OpenAPI description by
201+
themselves. If an HTTP API is introduced later, OpenAPI should reference the schema
202+
files as component schemas instead of trying to infer endpoints from them.
203+
204+
## Maintenance checklist
205+
206+
When updating schemas:
207+
208+
1. Decide whether the change is compatible or breaking.
209+
2. If breaking, add a new versioned schema file instead of changing the old one.
210+
3. Update fixture payloads to emit the correct `schemaVersion`.
211+
4. Run:
212+
```bash
213+
npm run test:schema-fixtures
214+
```
215+
5. Merge to `main`.
216+
6. Confirm the publish workflow updated the website repo.
217+
7. Confirm the final schema URL resolves on the website.

0 commit comments

Comments
 (0)