Skip to content

api: route per-call against unified hosts #5137

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
simonfaltum merged 20 commits into from
May 5, 2026
Merged

api: route per-call against unified hosts #5137

Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
eb20678
api: generate workspace-proxy deny-list from SDK source
simonfaltum Apr 30, 2026
a775d99
genpaths: fix lint findings
simonfaltum Apr 30, 2026
28fbf39
api: replace generated proxy paths with manual list
simonfaltum Apr 30, 2026
cf65ef8
api: route per-call against unified hosts
simonfaltum Apr 30, 2026
4769fa7
api: route account detection through proxy helper
simonfaltum Apr 30, 2026
116914e
acceptance: stabilize cmd/api recorded UA across CI and OS
simonfaltum Apr 30, 2026
f548898
Merge branch 'main' into simonfaltum/cli-api-spog-routing
simonfaltum May 1, 2026
9526263
Merge branch 'main' into simonfaltum/cli-api-spog-routing
simonfaltum May 4, 2026
29a2fb6
api: treat ?o=<workspace-id> as per-call workspace override
simonfaltum May 4, 2026
b60ab18
Merge remote-tracking branch 'origin/simonfaltum/cli-api-spog-routing…
simonfaltum May 4, 2026
adf58b6
acceptance: fix Windows path conversion and refresh out.test.toml
simonfaltum May 4, 2026
f63d896
Merge remote-tracking branch 'origin/main' into simonfaltum/cli-api-s…
simonfaltum May 4, 2026
55a83d5
Merge remote-tracking branch 'origin/main' into simonfaltum/cli-api-s…
simonfaltum May 4, 2026
1ea4ffc
api: drop {account_id} path substitution
simonfaltum May 4, 2026
e9e0b4a
acceptance/cmd/api: assert X-Databricks-Org-Id presence/absence expli…
simonfaltum May 4, 2026
f7793ce
Merge branch 'main' into simonfaltum/cli-api-spog-routing
simonfaltum May 4, 2026
556b552
acceptance/cmd/api: drop --keep so the request lives only in output.txt
simonfaltum May 4, 2026
1de3f95
Merge remote-tracking branch 'origin/main' into simonfaltum/cli-api-s…
simonfaltum May 5, 2026
2400ff0
acceptance/cmd/api: scope MSYS_NO_PATHCONV inline to CLI invocations
simonfaltum May 5, 2026
5f4e552
Merge branch 'main' into simonfaltum/cli-api-spog-routing
simonfaltum May 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions NEXT_CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

### CLI

* `databricks api` now works against unified hosts. Adds `--account` to scope a call to the account API and `--workspace-id` to override the workspace routing identifier per call. A `?o=<workspace-id>` query parameter on the path (the SPOG URL convention used by the Databricks UI) is also recognized as a per-call workspace override, so URLs pasted from the browser route correctly.

### Bundles

### Dependency updates
21 changes: 21 additions & 0 deletions acceptance/cmd/api/account-flag/out.requests.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"headers": {
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]"
]
},
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
]
},
"method": "GET",
"path": "/api/2.0/clusters/list"
}
3 changes: 3 additions & 0 deletions acceptance/cmd/api/account-flag/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 15 additions & 0 deletions acceptance/cmd/api/account-flag/output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{}

>>> print_requests.py --get --keep //api/2.0/clusters/list
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
]
},
"method": "GET",
"path": "/api/2.0/clusters/list"
}
2 changes: 2 additions & 0 deletions acceptance/cmd/api/account-flag/script
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
$CLI api get /api/2.0/clusters/list --account
Copy link
Copy Markdown
Contributor

@pietern pietern May 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal is that we don't see any org ID in the request log, correct?

If so, it's worth capturing in another call here that confirms that assertion.

Easy to miss the intent otherwise.

Copy link
Copy Markdown
Member Author

@simonfaltum simonfaltum May 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in e9e0b4a. Each of the seven cmd/api acceptance tests now uses print_requests.py --get --keep | contains.py to assert the recorded request either does or does not carry X-Databricks-Org-Id (with a specific value where one is expected). The intent is now visible in the script and in output.txt, not implicit in out.requests.txt.

Copy link
Copy Markdown
Contributor

@pietern pietern May 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now dups the requests between out.requests.txt and output.txt. Can you keep one of them?

Copy link
Copy Markdown
Member Author

@simonfaltum simonfaltum May 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped --keep in 556b552 — print_requests.py now consumes out.requests.txt (deleted from the test directories) and the recorded request lives in output.txt only.

trace print_requests.py --get --keep //api/2.0/clusters/list | contains.py "!X-Databricks-Org-Id"
21 changes: 21 additions & 0 deletions acceptance/cmd/api/account-path/out.requests.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"headers": {
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]"
]
},
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
]
},
"method": "GET",
"path": "/api/2.0/accounts/abc-123/network-policies"
}
3 changes: 3 additions & 0 deletions acceptance/cmd/api/account-path/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 15 additions & 0 deletions acceptance/cmd/api/account-path/output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{}

>>> print_requests.py --get --keep //api/2.0/accounts/abc-123/network-policies
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
]
},
"method": "GET",
"path": "/api/2.0/accounts/abc-123/network-policies"
}
2 changes: 2 additions & 0 deletions acceptance/cmd/api/account-path/script
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
$CLI api get /api/2.0/accounts/abc-123/network-policies
trace print_requests.py --get --keep //api/2.0/accounts/abc-123/network-policies | contains.py "!X-Databricks-Org-Id"
46 changes: 46 additions & 0 deletions acceptance/cmd/api/test.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
RecordRequests = true
IncludeRequestHeaders = ["Authorization", "User-Agent", "X-Databricks-Org-Id"]

[Env]
# MSYS2 (Git Bash on Windows) auto-converts a leading "/" argument to a
# Windows path (e.g. /api/2.0/... becomes C:/Program Files/Git/api/2.0/...).
# Disable that so the CLI receives the path verbatim.
MSYS_NO_PATHCONV = "1"

# Normalize OS-dependent and CI-only User-Agent segments so the recorded
# requests are stable across local macOS/Linux runs and GitHub Actions.
[[Repls]]
Old = '(linux|darwin|windows)'
New = '[OS]'

[[Repls]]
Old = " cicd/[A-Za-z0-9.-]+"
New = ""

[[Repls]]
Old = " upstream/[A-Za-z0-9.-]+"
New = ""

[[Repls]]
Old = " upstream-version/[A-Za-z0-9.-]+"
New = ""

# Common stubs for paths used across variants. Each returns an empty JSON
# object; the variants assert on the *recorded request* (out.requests.txt),
# not on the response body, so any well-formed JSON is fine.

[[Server]]
Pattern = "GET /api/2.0/clusters/list"
Response.Body = '{}'

[[Server]]
Pattern = "GET /api/2.0/accounts/abc-123/network-policies"
Response.Body = '{}'

[[Server]]
Pattern = "GET /api/2.0/accounts/abc-123/oauth2/published-app-integrations"
Response.Body = '{}'

[[Server]]
Pattern = "GET /api/2.0/preview/accounts/access-control/rule-sets"
Response.Body = '{}'
24 changes: 24 additions & 0 deletions acceptance/cmd/api/workspace-id-flag/out.requests.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"headers": {
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]"
]
},
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
],
"X-Databricks-Org-Id": [
"999"
]
},
"method": "GET",
"path": "/api/2.0/clusters/list"
}
3 changes: 3 additions & 0 deletions acceptance/cmd/api/workspace-id-flag/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 18 additions & 0 deletions acceptance/cmd/api/workspace-id-flag/output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{}

>>> print_requests.py --get --keep //api/2.0/clusters/list
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
],
"X-Databricks-Org-Id": [
"999"
]
},
"method": "GET",
"path": "/api/2.0/clusters/list"
}
2 changes: 2 additions & 0 deletions acceptance/cmd/api/workspace-id-flag/script
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
$CLI api get /api/2.0/clusters/list --workspace-id 999
trace print_requests.py --get --keep //api/2.0/clusters/list | contains.py "X-Databricks-Org-Id" "999"
27 changes: 27 additions & 0 deletions acceptance/cmd/api/workspace-id-from-query/out.requests.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
{
"headers": {
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]"
]
},
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
],
"X-Databricks-Org-Id": [
"999"
]
},
"method": "GET",
"path": "/api/2.0/clusters/list",
"q": {
"o": "999"
}
}
3 changes: 3 additions & 0 deletions acceptance/cmd/api/workspace-id-from-query/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 21 additions & 0 deletions acceptance/cmd/api/workspace-id-from-query/output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{}

>>> print_requests.py --get --keep //api/2.0/clusters/list
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
],
"X-Databricks-Org-Id": [
"999"
]
},
"method": "GET",
"path": "/api/2.0/clusters/list",
"q": {
"o": "999"
}
}
2 changes: 2 additions & 0 deletions acceptance/cmd/api/workspace-id-from-query/script
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
$CLI api get "/api/2.0/clusters/list?o=999"
trace print_requests.py --get --keep //api/2.0/clusters/list | contains.py "X-Databricks-Org-Id" "999"
21 changes: 21 additions & 0 deletions acceptance/cmd/api/workspace-id-none/out.requests.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"headers": {
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]"
]
},
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
]
},
"method": "GET",
"path": "/api/2.0/clusters/list"
}
3 changes: 3 additions & 0 deletions acceptance/cmd/api/workspace-id-none/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 15 additions & 0 deletions acceptance/cmd/api/workspace-id-none/output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{}

>>> print_requests.py --get --keep //api/2.0/clusters/list
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
]
},
"method": "GET",
"path": "/api/2.0/clusters/list"
}
13 changes: 13 additions & 0 deletions acceptance/cmd/api/workspace-id-none/script
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Profile with workspace_id = none overrides the host-metadata back-fill.
# The CLI must strip the sentinel before the header decision; the recorded
# request should not carry the routing identifier.
sethome "./home"
cat > "./home/.databrickscfg" <<EOF
[spog-account]
host = $DATABRICKS_HOST
token = $DATABRICKS_TOKEN
workspace_id = none
EOF

$CLI api get /api/2.0/clusters/list --profile spog-account
trace print_requests.py --get --keep //api/2.0/clusters/list | contains.py "!X-Databricks-Org-Id"
3 changes: 3 additions & 0 deletions acceptance/cmd/api/workspace-id-none/test.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Ignore = [
"home"
]
24 changes: 24 additions & 0 deletions acceptance/cmd/api/workspace-path/out.requests.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"headers": {
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]"
]
},
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
],
"X-Databricks-Org-Id": [
"[NUMID]"
]
},
"method": "GET",
"path": "/api/2.0/clusters/list"
}
3 changes: 3 additions & 0 deletions acceptance/cmd/api/workspace-path/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 18 additions & 0 deletions acceptance/cmd/api/workspace-path/output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{}

>>> print_requests.py --get --keep //api/2.0/clusters/list
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
],
"X-Databricks-Org-Id": [
"[NUMID]"
]
},
"method": "GET",
"path": "/api/2.0/clusters/list"
}
2 changes: 2 additions & 0 deletions acceptance/cmd/api/workspace-path/script
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
$CLI api get /api/2.0/clusters/list
trace print_requests.py --get --keep //api/2.0/clusters/list | contains.py "X-Databricks-Org-Id"
24 changes: 24 additions & 0 deletions acceptance/cmd/api/workspace-proxy-regression/out.requests.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"headers": {
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]"
]
},
"method": "GET",
"path": "/.well-known/databricks-config"
}
{
"headers": {
"Authorization": [
"Bearer [DATABRICKS_TOKEN]"
],
"User-Agent": [
"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
],
"X-Databricks-Org-Id": [
"[NUMID]"
]
},
"method": "GET",
"path": "/api/2.0/preview/accounts/access-control/rule-sets"
}
3 changes: 3 additions & 0 deletions acceptance/cmd/api/workspace-proxy-regression/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading