Add azure.ai.training extension

[saanikaguptamicrosoft]

Notes

• Relevant PRs (refer PR overview for details)
  • Add artifact resolution for code and inputs in job create #7153
  • Implement cancel job command for custom training #7272
  • Implement delete job command for custom training #7273
  • Add validate command for custom training #7407 , Share offline validation between job validate and job submit commands #8068
  • Implement download command for custom training #7453
  • Implement stream command for training job #7939
  • Implement connect-ssh to job node #7964
  • Implement download command for training job #8029
  • Add support for gpuCount for partial SKU scenario #8067
  • Bugfix: Honor --subscription and --project-endpoint flags over stored env values #8093
  • Implement show services command #8121
  • Add validations for UAMI requirement #8122
  • Add template flag in init command #8123

Testing

• Help command

• Init command

• Job submit

• Job show

• Jobs list

Other operations like validate, download, stream, delete, cancel are also supported.

[jongio]

New extension for Azure AI Foundry training jobs. Solid foundation overall - clean command tree, good offline validation, safe download handling (path traversal protection, atomic writes). A few items worth addressing before merge:

TenantId auth bug: init.go uses Subscription.TenantId (resource tenant) from PromptSubscription() instead of Subscription.UserTenantId (user access tenant). The azure.ai.finetune extension already handles this correctly. Multi-tenant/guest users will hit auth failures with the current code.

CODEOWNERS scope: This PR adds the training extension CODEOWNERS line (expected), but also adds @jongio and @wbreza to ~15 unrelated ownership lines (installer, ext/, .github/, eng/, schemas/, etc.). Those changes should be a separate PR to keep review scope clean.

No PR description: The PR body is empty. For a 14K+ line new extension, some context on the design, what training jobs are, and how this relates to the existing finetune extension would help reviewers.

[jongio]

Addresses my prior feedback - TenantId fix, go version bump, SDK v2, and semver pin all look correct. One remaining item: the PR body is still empty. For a 14K-line new extension, even a brief summary (what training jobs are, how this relates to the finetune extension, key design decisions) would help reviewers and show up in the git log.

[wbreza]

Test comment

[wbreza]

Code review — supplementing prior reviews

Thanks for the substantial work here, @saanikaguptamicrosoft. Below are findings that aren't already covered by Copilot's or @jongio's earlier comments. Verified that the UserTenantId fix, go 1.26.1 bump, semver pin, and CODEOWNERS scope are all in place. ✅

🟧 High

1. --debug flag prints HTTP request URLs & bodies to stdout — risk of leaking SAS URIs / sensitive payloads
pkg/client/client.go:91–103. SetDebugBody() is called from production commands (job_submit.go:93, job_stream.go, job_download.go, job_connect_ssh.go) when --debug is passed. Logs include URLs (which for dataset blob/SAS operations may include SAS tokens) and request bodies that flow through GetDatasetCredentials / GetModelCredentials. Repo guidance forbids logging credentials/tokens.
Fix: Redact sasUri, Authorization, sig, signature, and known credential fields before printing; or move this behind a separate developer-only env var and document that it may expose tokens.

🟨 Medium

2. SSH proxy stdin goroutine intentionally untracked
internal/cmd/job_ssh_proxy.go:167–182. The upstream stdin → websocket goroutine is not added to the sync.WaitGroup and blocks on os.Stdin.Read(). The comment acknowledges the Windows limitation and relies on process exit. Works for the short-lived ProxyCommand model, but wg.Wait() at line 211 cannot guarantee clean shutdown on graceful cancellation.
Fix (optional): Close os.Stdin from the cancellation watcher to force Read() to return, or document the lifecycle expectation more visibly.

3. azcopy runner silently drops malformed JSON and parse errors
internal/azcopy/runner.go:167 (JSON unmarshal) and :175–177 (ParseInt). On azcopy output format drift, progress reporting silently degrades to "0 bytes" instead of surfacing the problem.
Fix: Log a one-time warning on first unparseable line; preserve raw text for diagnostics.

4. Nil-guard / pattern duplication on job.Properties.Services access
internal/service/stream_service.go:100, internal/cmd/job_download.go:350 (extractTrackingEndpoint), internal/cmd/job_show_services.go:45. Three inline reimplementations of "extract service endpoint" without consistent nil guards. Sparse API responses can omit Properties or Services.
Fix: Centralize as internal/utils/ExtractServiceEndpoint(job, name) with proper nil checks; replace the three call sites.

5. Critical untested code paths
Production files with zero test coverage: internal/service/upload_service.go (dedup + zombie recovery + sentinel tagging), internal/service/hash.go (deterministic directory hashing — drives dedup correctness), pkg/client/download.go (artifact/output downloads). The team's existing test files (resolver_test.go, job_validator_test.go, job_show_services_test.go) demonstrate solid patterns with fakes — this is a coverage gap on the most failure-prone code.
Fix (recommended before merge): Unit tests for hash.go (determinism + symlink + cross-platform path normalization) and upload_service.go (hash collision fallback, missing SAS URI, azcopy failure).

6. PR description is still empty
Reiterating @jongio's request — for a 14K-line new first-party extension, the empty description hurts git log archaeology, future reviewers, and release notes. A brief summary of what training jobs are, how this relates to azure.ai.finetune, key design decisions, and a pointer to the included Design/ docs would help significantly.

7. PR description references job get, but the actual command is job show
internal/cmd/job.go:32–42 registers newJobShowCommand() (defined in job_get.go). There is no newJobGetCommand. Commit history shows a rename from job get → job show. The PR description and screenshots still reference job get.
Fix: Update PR description and any docs (including content in Design/command-job-guide.md) to match the shipped command names.

🟦 Low

8. extension.yaml is missing an example for the metadata capability (other first-party extensions list examples for every declared capability).
9. proxyEndpointPattern (job_ssh_proxy.go) allows %; if any downstream consumer URL-decodes the endpoint, this could enable host/scheme bypass. Quick audit of buildTunnelWSURL recommended.
10. internal/azcopy/runner.go sourceArg lacks defense-in-depth shell-metachar validation (#nosec G204 only covers azcopyPath).
11. AdditionallyAllowedTenants: []string{"*"} in init.go:86 and job_ssh_proxy.go:265 is intentional for multi-tenant/guest support but should carry an inline security comment explaining the wide scope.
12. Missing godoc on exported symbols in pkg/client/ and pkg/models/ per repo standards.
13. ~140 KB of Design/ markdown shipped in the extension source tree (design.md 75 KB, execution-plan.md 40 KB, command-job-guide.md 25 KB). Will drift from code without a sync mechanism. Consider moving to repo-level docs/ or wiki, or trimming to a README link.
14. Client init boilerplate duplicated across ~15 command files. Extract newDefaultJobClient(ctx) helper to centralize credential/endpoint/client construction.

✅ Verified clean / resolved

• Subscription.UserTenantId correctly used at init.go:105.
• CODEOWNERS scope creep: only the training-extension line added — no co-owner additions to unrelated paths.
• go.mod is go 1.26.1 with semver-pinned azd dependency v1.25.1, aligned with extension.yaml's requiredAzdVersion.
• safeJoin in download.go correctly prevents zip-slip.
• resolver_test.go, job_validator_test.go, job_show_services_test.go, init_template_test.go are solid examples.
• All 9 required CI checks passing.

Posted by Copilot CLI on behalf of the reviewer.