experimental: import ai-dev-kit skills into experimental/ directory#73
experimental: import ai-dev-kit skills into experimental/ directory#73jamesbroadhead wants to merge 16 commits into
Conversation
2035bab to
2a45b83
Compare
8387654 to
a0fbb21
Compare
The default DATABRICKS_SKILLS_REF pin is a release tag that pre-dates the experimental_skills manifest section (see databricks/databricks-agent-skills#73). Users who pass --experimental against that ref today silently get no experimental skills installed. Log a Warnf at install time pointing them at the env var override (=main, or a future release that includes the section). Helper: manifestHasExperimental(), unit-tested in source_test.go. Co-authored-by: Isaac
Replaces the previous import (a-d-k commit 2228c3e on add_appkit) with the head of a-d-k PR #533 (commit 9c7a5b3 on appkit-on-experimental), which targets a-d-k's experimental branch. Changes: - Refresh 23 experimental skill directories from the new source. - Drop databricks-lakebase-provisioned — removed on a-d-k experimental. - databricks-apps-python: rename + SKILL.md now leads with AppKit (TypeScript + React SDK) and demotes Python frameworks to alternatives; 6-mcp-approach.md replaced with 6-cli-approach.md. - databricks-lakebase-autoscale/references/connection-patterns.md: change placeholder `user:password` to `<user>:<password>` so the secret scanner doesn't flag the doc-only example. Cosmetic only. - Continue to exclude databricks-model-serving and databricks-spark-declarative-pipelines (PR #73 TODOs #1b and #5). - Regenerate manifest.json and agents/openai.yaml stubs via scripts/skills.py generate. - Update experimental/README.md provenance section with the new SHA, branch, and divergence notes. Co-authored-by: Isaac
Merges the comprehensive jobs reference content from experimental/databricks-jobs/ into skills/databricks-jobs/ and removes the experimental copy. What's new in stable databricks-jobs (v0.2.0): - Full task-types reference (9 types: notebook, spark_python, python_wheel, sql, dbt, pipeline, spark_jar, run_job, for_each) - All 6 trigger types with examples (cron, periodic, file_arrival, table_update, continuous, manual) + combining + pause/resume - Notifications + health rules + retries + timeouts + queues - 7 end-to-end worked examples (ETL, warehouse refresh, event-driven, ML training, multi-env, streaming, cross-job orchestration) - run_if conditions, environments (serverless deps), permissions What's retained from the prior stable skill: - parent: databricks-core hierarchy - Compatibility note + version metadata (bumped 0.1.0 → 0.2.0) - Scaffolding workflow (databricks bundle init + CLAUDE.md/AGENTS.md template + project structure) - Unit testing + development workflow sections - agents/openai.yaml + assets/ Cleanups during the merge: - Replaced the trigger-spam description with a terse one - Normalized hard-coded /Workspace/Users/user@example.com/ paths in the imported reference files to /Workspace/Shared/ scripts/skills.py: updated SKILL_METADATA description for jobs to reflect the broader scope. Manifest regenerated; experimental count drops from 23 to 22. Resolves PR #73 TODO #1a. Co-authored-by: Isaac
Adds an experimental/ directory containing the 26 agent skills from databricks-solutions/ai-dev-kit. These are imported as a snapshot on a best-effort basis — they are not officially supported skills and follow a looser contract than skills/ (no agents/openai.yaml, no shared-asset sync, no SKILL_METADATA gate). The manifest now exposes them under a new top-level experimental_skills map so consumers can distinguish them from stable skills and skip them by default. scripts/skills.py handles the new directory; the existing generate / validate flow is unchanged for stable skills. Co-authored-by: Isaac
Owners are the top contributors to databricks-solutions/ai-dev-kit (>=10 commits at import time). The cross-org team @databricks-solutions/ai-dev-kit-maintainers is the canonical owner; this line can be replaced with that team handle if the team gets write access to this repo. Co-authored-by: Isaac
Adds a Provenance & sync model section to experimental/README.md: - Transition phase: source of truth is upstream ai-dev-kit; this dir gets periodic manual re-syncs. - Post-lock: source of truth is this repo; ai-dev-kit's databricks-skills/ becomes read-only. Co-authored-by: Isaac
Adds two helpers to scripts/skills.py that run as part of `generate`:
- `ensure_experimental_codex_metadata` copies the shared
assets/databricks.{svg,png} into each experimental skill (mirroring
what stable skills get via sync_assets).
- `synthesize_openai_yaml` writes agents/openai.yaml from each
experimental skill's SKILL.md frontmatter (display_name from the
skill name, short_description from the first sentence of the
frontmatter description, brand_color and icon paths fixed).
Both run only when the destination file is absent, so upstream
ai-dev-kit can override by shipping its own openai.yaml or assets.
This closes the cosmetic gap that experimental skills installed into
Codex CLI would render without an icon or marketplace metadata.
Co-authored-by: Isaac
Same pattern as databricks-model-serving: the experimental version covers a different surface than the stable databricks-pipelines skill (workflow / scaffolding / DLT-migration / per-language performance vs feature reference / decision tree / common traps / format options). The DAB-coupled scaffolding workflow is also the specific concern Dustin flagged in the Apr 28 Slack thread for demo-generator flows. Removed experimental/databricks-spark-declarative-pipelines/; manifest regenerated (24 experimental skills). Follow-up TODO: port the high-value pieces (DLT migration guide, workflow A/B/C decision matrix, per-language performance reference, language-selection rules) into skills/databricks-pipelines/, stripping MCP-tool refs. Co-authored-by: Isaac
Adds the actual a-d-k commit hash (2228c3e on the add_appkit branch, 5 commits ahead of origin/main at import time) along with a note about the local deltas vs public main. Surfaces the key one: the databricks-app-python -> databricks-apps-python rename hadn't merged upstream, and pulling from the renamed version is what avoids a 3rd skill-name collision with d-a-s's stable databricks-apps. Co-authored-by: Isaac
extract_version_from_skill() falls back to a synthetic version when a skill's SKILL.md frontmatter has no version: field. The previous fallback was 0.0.0, which several install tools treat as "unset" rather than "first release". Bump to 0.0.1. Affects all 24 experimental skills (imported from ai-dev-kit without versions) plus the stable databricks-dabs skill. Skills with an explicit version are unchanged. Co-authored-by: Isaac
- experimental/README.md: install examples now use the -experimental suffix on the skill name + the --experimental flag (matching the install-path behaviour landed in databricks/cli#5243). Adds a short note explaining why the in-repo dir name and the install dir name differ. - experimental/README.md: drop databricks-model-serving from the collision example (it was removed from this PR earlier). - experimental/README.md: update the (also available as stable skill) note for databricks-jobs to point at the open TODO #1a. - Root README: clarify the suffixed install name in the by-name install example. Co-authored-by: Isaac
Replaces the previous import (a-d-k commit 2228c3e on add_appkit) with the head of a-d-k PR #533 (commit 9c7a5b3 on appkit-on-experimental), which targets a-d-k's experimental branch. Changes: - Refresh 23 experimental skill directories from the new source. - Drop databricks-lakebase-provisioned — removed on a-d-k experimental. - databricks-apps-python: rename + SKILL.md now leads with AppKit (TypeScript + React SDK) and demotes Python frameworks to alternatives; 6-mcp-approach.md replaced with 6-cli-approach.md. - databricks-lakebase-autoscale/references/connection-patterns.md: change placeholder `user:password` to `<user>:<password>` so the secret scanner doesn't flag the doc-only example. Cosmetic only. - Continue to exclude databricks-model-serving and databricks-spark-declarative-pipelines (PR #73 TODOs #1b and #5). - Regenerate manifest.json and agents/openai.yaml stubs via scripts/skills.py generate. - Update experimental/README.md provenance section with the new SHA, branch, and divergence notes. Co-authored-by: Isaac
The previous regex-only parser in extract_description_from_skill() captured the YAML block-scalar indicator (`>-`) verbatim, so any SKILL.md that wrote `description: >-\n multi-line content` produced a manifest entry of `">-"`. The new ai-dev-kit import (PR #533) brought two such files — databricks-dbsql and databricks-execution-compute — which landed corrupted descriptions in manifest.json and corrupted short_description / default_prompt in agents/openai.yaml. Walk the frontmatter line by line: if the value is a block-scalar indicator (|, |-, |+, >, >-, >+), aggregate the indented continuation lines (folded with spaces for `>`-style, newlines for `|`-style). Regenerate manifest.json and the two affected agents/openai.yaml stubs. Co-authored-by: Isaac
Replaces the hand-rolled block-scalar walker (added one commit ago) with PyYAML's safe_load. PyYAML's default SafeLoader is pure-Python — no C extension required — and handles every YAML edge case for free instead of reimplementing them. Side-benefit: also fixes a second latent bug. The regex parser stripped the outer YAML quotes but left inner `\"` escapes intact as literal backslash-quote characters, so descriptions like `"... mentions \"switch workspace\"..."` ended up in manifest.json with the backslashes preserved. yaml.safe_load resolves these correctly. Regenerated manifest reflects the fix for databricks-config. Co-authored-by: Isaac
Merges the comprehensive jobs reference content from experimental/databricks-jobs/ into skills/databricks-jobs/ and removes the experimental copy. What's new in stable databricks-jobs (v0.2.0): - Full task-types reference (9 types: notebook, spark_python, python_wheel, sql, dbt, pipeline, spark_jar, run_job, for_each) - All 6 trigger types with examples (cron, periodic, file_arrival, table_update, continuous, manual) + combining + pause/resume - Notifications + health rules + retries + timeouts + queues - 7 end-to-end worked examples (ETL, warehouse refresh, event-driven, ML training, multi-env, streaming, cross-job orchestration) - run_if conditions, environments (serverless deps), permissions What's retained from the prior stable skill: - parent: databricks-core hierarchy - Compatibility note + version metadata (bumped 0.1.0 → 0.2.0) - Scaffolding workflow (databricks bundle init + CLAUDE.md/AGENTS.md template + project structure) - Unit testing + development workflow sections - agents/openai.yaml + assets/ Cleanups during the merge: - Replaced the trigger-spam description with a terse one - Normalized hard-coded /Workspace/Users/user@example.com/ paths in the imported reference files to /Workspace/Shared/ scripts/skills.py: updated SKILL_METADATA description for jobs to reflect the broader scope. Manifest regenerated; experimental count drops from 23 to 22. Resolves PR #73 TODO #1a. Co-authored-by: Isaac
PR #533 has merged into upstream a-d-k experimental. The
databricks-skills/ tree is byte-identical between the previous
import SHA (9c7a5b3) and the merge commit (7b07f18) — only
install.{sh,ps1} changed, which we don't import. README updated
to point at the now-authoritative branch + SHA and drop the "one
commit ahead of origin/experimental" caveat.
Co-authored-by: Isaac
50467c3 to
5719156
Compare
Stable ↔ experimental skill overlapSurfacing the overlap explicitly per Quentin's question. There are 6 stable/experimental pairs with non-trivial topical overlap; the remaining 16 experimental skills cover surfaces with no stable equivalent.
No stable equivalent (16 skills — no overlap)
TL;DRThree pairs are real duplicates that should converge over time: dabs/bundles (drop experimental), lakebase/lakebase-autoscale (drop experimental, port connection-patterns content first), core/config (drop experimental). Two pairs are complementary and worth keeping side-by-side: apps/apps-python (TS vs Python audiences), serverless-migration/execution-compute (migration vs runtime selection). One pair is mostly disjoint: core/python-sdk (CLI vs SDK). Intentional for this PR per the rationale in the description — port adk mostly intact, then resolve the duplicates as a series of single-skill PRs against stable, with the experimental copies removed in the same change. Happy to start that series if folks agree on the deltas above. This comment was written by Claude. |
Removes three experimental skills that duplicate stable equivalents without adding net new content: - databricks-bundles → use stable `databricks-dabs` - databricks-lakebase-autoscale → use stable `databricks-lakebase` - databricks-config → use stable `databricks-core` Cross-references in the surviving experimental skills (apps-python, python-sdk, zerobus-ingest, synthetic-data-gen, 4-deployment.md) now point at the stable names by bare name — matching the convention already used in stable skills' "Related Skills" / "Product Skills" sections. `experimental/README.md` records the removals and points readers at the stable replacements. Manifest regenerated — experimental skill count drops from 22 to 19. Constraint per James: root must not depend on experimental, but experimental may depend on root. Honored — all new references go in that direction, and no stable skill content was changed by this commit. Co-authored-by: Isaac
User-walkthrough testing of experimental/README.md surfaced 13 broken cross-reference links in 9 SKILL.md files, all in "Related Skills" sections. Each pointed at ../<name>/SKILL.md for a skill that doesn't exist in experimental/ — they reference earlier-resolved name collisions (databricks-jobs, databricks-model-serving) or experimental-only content that was excluded from this import (databricks-spark-declarative-pipelines). Replacement strategy mirrors c4daa14: use the bare skill name without a link, matching the convention stable skills use in their own cross-references. Targets: - databricks-spark-declarative-pipelines (excluded per TODO #5) → databricks-pipelines (stable analogue) - databricks-model-serving (dropped per TODO #1b) → databricks-model-serving (bare; stable exists at same name) - databricks-jobs (merged per TODO #1a) → databricks-jobs (bare; stable exists at same name) Also rewrites the stale "collision handling for jobs and model-serving" sentence in experimental/README.md — those collisions were resolved via merge + drop respectively in this PR, not via the suffix mechanism the sentence implied. Co-authored-by: Isaac
…ix subskill example
User-walkthrough testing of the root README surfaced three doc
issues:
1. "Available Skills" listed only databricks-apps. The repo ships
8 stable skills — added all of them with one-line descriptions.
2. Install section mixed two valid command forms without
explanation: `databricks experimental aitools install`
(agent-detection wrapper, used in the Claude Code section) and
`databricks experimental aitools skills install` (skills-only
subcommand, used in the Experimental section). Settled on the
wrapper as the primary form and explained the subcommand as
the power-user path with --experimental support.
3. The "Adding New Skills" example used a name (ai-databricks-apps)
and a prose mechanism ("First, load the base ... skill") that
do not match the actual subskill pattern stable skills use.
Rewritten to demonstrate the real mechanism: parent:
<skill-name> in frontmatter, matching the CLAUDE.md convention
and what every stable subskill does today.
Co-authored-by: Isaac
Summary
Adds an
experimental/directory containing 23 agent skills from databricks-solutions/ai-dev-kitdatabricks-skills/, imported as a snapshot on a best-effort basis. Excluded:databricks-model-serving(TODO #1b — different surface than stable, heavy MCP coupling) anddatabricks-spark-declarative-pipelines(TODO #5 — different surface than stabledatabricks-pipelines).databricks-lakebase-provisionedis not in the upstreamexperimentalbranch either, so absent here.The manifest now exposes them under a new top-level
experimental_skillsmap alongside the existingskillsmap so consumers can skip them by default.Paired with databricks/cli#5243 which teaches
databricks experimental aitools skills installto:experimental_skillsby default,--experimental,--experimentalrequired and-experimentalsuffix on the install dir).Source
Import is from
9c7a5b3on the
appkit-on-experimentalbranch ofjamesbroadhead/ai-dev-kit— the head of a-d-k PR #533,which targets a-d-k's
experimentalbranch (notmain). 1 commit ahead oforigin/experimentalat import time.Landing dependency: a-d-k PR #533 should merge before this PR so the first periodic sync from a-d-k doesn't conflict. The rename (
databricks-app-python→databricks-apps-python) is load-bearing — it's what prevents a 3rd skill-name collision with d-a-s's stabledatabricks-apps.Direction caveat — please read
In the Apr 28 thread (Slack link), Dustin's stated plan was to move
databricks-agent-skillsskills intoai-dev-kit'sexperimentalbranch as defaults. This PR goes the other direction (a-d-k content → d-a-s/experimental). I don't see any d-a-s commits from Dustin yet, and the timing has slipped. Opening this so we have something concrete to iterate on — happy to drop it if the original direction is still preferred.TODOs / caveats for iteration
Name collisions. Resolved in this PR:
databricks-jobs— merged into stable. Imported the comprehensive reference content from a-d-k'sdatabricks-jobsskill intoskills/databricks-jobs/, bumping version to0.2.0. The merged skill keeps stable's scaffolding workflow +parent: databricks-corehierarchy + Codexagents/openai.yaml+ compatibility note, and adds the experimental's full task-types reference (9 types), trigger types (6), notifications/health/retries/queues, and 7 worked end-to-end examples. Layered structure: SKILL.md as overview + four reference files (task-types.md,triggers-schedules.md,notifications-monitoring.md,examples.md). Cleanups during merge: dropped trigger-spam description, normalized/Workspace/Users/user@example.com/...paths to/Workspace/Shared/.... The experimental copy is removed; the install-time-experimentalsuffix collision-handling on the cli side becomes unnecessary for jobs.databricks-model-serving— dropped from this PR. After a deep compare, the two skills cover almost entirely different surfaces: stable is ops-focused (manage existing endpoints via CLI:serving-endpoints create/get/query/update-config/build-logs/put-ai-gateway/get-permissions/..., AI Gateway, traffic config, app integration viadatabricks-appsskill); experimental is dev-focused (build & ship MLflow models / GenAI agents: autolog →mlflow.pyfunc.log_model→databricks.agents.deploy()→ query, with full Classical ML / Custom PyFunc /ResponsesAgent+ LangGraph / UCFunctionToolkit / VectorSearchRetrieverTool coverage). Near-zero content overlap. Experimental version also has heavy MCP-tool dependency (60+ refs to ai-dev-kit'smanage_serving_endpoint,manage_workspace_files,manage_jobs,manage_job_runs,execute_codethat don't exist in the d-a-s/databricks experimental aitoolsflow). Removedexperimental/databricks-model-serving/from this PR; manifest regenerated. Follow-up: port the high-value dev-side content into the stable skill — classical-ml autolog patterns (mlflow.{sklearn,xgboost,lightgbm,pytorch,tensorflow,spark}.autolog()), Custom PyFunc signatures,ResponsesAgentpattern with thecreate_text_output_itemhelper-method gotcha,UCFunctionToolkit+VectorSearchRetrieverToolwith resource passthrough for auth, the Foundation Model API endpoint table. Strip MCP refs; replace with CLI/SDK equivalents. Owners: @databricks/eng-apps-devex (per CODEOWNERS).CODEOWNERS for
experimental/— open. Discussing access model offline.No sync mechanism with upstream a-d-k.Resolved with a paired RFC. Two-part plan:ai-dev-kitintoexperimental/. Documented inexperimental/README.md.databricks-skills/imported/in a-d-k is agit subtreeof this repo'sexperimental/. RFC PR opened against a-d-k: RFC: subtree-sync skills from databricks-agent-skills/experimental databricks-solutions/ai-dev-kit#530 (draft). To make subtree work, d-a-s needs to publish anexperimental-onlybranch viagit subtree split --prefix=experimentalafter every push to main — that's a small workflow to add here in a follow-up PR. A one-shot preview branchexperimental-only-previewwas pushed to this repo to enable the RFC demo and should be deleted once the auto-publish workflow lands.No agent metadata.Resolved. Imported skills install fine on Codex CLI — the missingagents/openai.yamlwas a cosmetic gap, not a functional blocker (skill files still get copied; only the marketplace UI metadata is absent).scripts/skills.pynow auto-generatesagents/openai.yaml+ copies shared assets for each experimental skill ongenerate, using SKILL.md frontmatter as the source. Stubs are only written when missing, so upstream a-d-k can override by shipping its own files in the skill. The auto-generated names are titlecased from the skill key — most look good (Databricks Iceberg,Databricks Genie); a few degrade gracefully (Databricks Aibi Dashboards). Refining those is a follow-up.Resolved. a-d-k doesn't ship adatabricks-pipelineswas deliberately excluded.databricks-pipelinesskill under that name, but it does shipdatabricks-spark-declarative-pipelinescovering the same product. After a deep compare, that experimental version covers a different surface than stable: scaffolding (databricks pipelines init+ bundle/MCP workflow A/B/C), DLT migration guide, language-selection rules, per-language performance reference. The stable skill covers feature reference (decision tree, common traps, format options, fine-grained per-feature × per-language refs). Partial overlap; experimental's DAB-coupled workflow is the exact concern Dustin flagged in the Apr 28 Slack thread for demo-generator flows. Removedexperimental/databricks-spark-declarative-pipelines/from this PR (24 experimental skills now). Follow-up TODO (post-merge): port the high-value pieces into stableskills/databricks-pipelines/— DLT migration guide, workflow A/B/C decision matrix, per-language performance reference, language-selection rules. Strip MCP-tool refs. Owners: @lennartkats-db / @camielstee-db (per CODEOWNERS).Kept as-is. The skill is about the OSS Apache Spark 4+ PySpark DataSource API (building custom connector libraries), not a Databricks product — only lightly flavored with Databricks idioms. The convention break is acceptable given the content.spark-python-data-sourcenaming exception.Versioning.Resolved. Bumped theextract_version_from_skillfallback inscripts/skills.pyfrom0.0.0→0.0.1so the manifest never reports0.0.0(which some tools treat as "unset"). Applies to all 24 experimental skills + the stabledatabricks-dabsskill, which currently have no explicitversion:in their SKILL.md frontmatter. Skills with an explicit version are unchanged. The change is sync-safe: when upstream a-d-k eventually adds version fields, those win; until then, the manifest reports the floor.All experimental skills install under ainstalled_dirfor experimental skills.-experimentalsuffix. Every experimental skill installs to~/.claude/skills/<name>-experimental/regardless of whether there's a stable skill with the colliding base name. Implemented in databricks/cli#5243 via a newSourceNamefield onSkillMeta: the manifest key (and install dir) carry the-experimentalsuffix;SourceNamepreserves the unsuffixed name for fetching fromexperimental/<name>/in this repo. Users see at a glance which installed skills are experimental.Excluded a-d-k content.Confirmed scope. Excluded:TEMPLATE/(template, not a skill),install_skills.sh+install_genie_code_skills.py(a-d-k's installers — we use the cli installer instead),databricks-builder-app/(a Python app for a-d-k's builder UI),databricks-mcp-server/(the a-d-k MCP server — separate concern from skills),databricks-tools-core/(Python lib used by a-d-k tooling — no experimental skill references it),hooks/hooks.json(a-d-k plugin lifecycle hooks tied to${CLAUDE_PLUGIN_ROOT}/.claude-plugin/setup.sh/check_update.sh— plugin-specific, not skill content), plus top-level repo metadata (.github/,LICENSE.md,README.md,VERSION,install.{sh,ps1}, etc.). Verified no experimental skill cross-references any excluded path.README placement.Verified.experimental/README.mdretains the adapted a-d-k skill list with a top warning block; the rootREADME.mdhas an "Experimental Skills" section with an install-by-name example. Three concrete fixes applied during the verification pass: (a) dropped the staledatabricks-model-servingcollision example since that skill was removed from the PR, (b) install commands updated to include the-experimentalsuffix + flag per TODO Add CODEOWNERS #8's resolution, (c) added a short note inexperimental/README.mdexplaining why the in-repo dir names don't carry the suffix (it's added at install time).Test plan
python3 scripts/skills.py generateregenerates the manifest cleanly.python3 scripts/skills.py validatepasses.databricks experimental aitools skills install(no flag) installs only stable skills.databricks experimental aitools skills install --experimentalinstalls both.databricks experimental aitools skills install databricks-icebergerrors because it's experimental.databricks experimental aitools skills install databricks-iceberg --experimentalinstalls that one skill.This pull request and its description were written by Claude.