Skip to content

Commit 27b1848

Browse files
authored
feat: introduce Lakebase Autoscaling driver (#98)
1 parent f7bc2eb commit 27b1848

46 files changed

Lines changed: 3503 additions & 146 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
name: Release @databricks/lakebase
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
dry-run:
7+
description: "Dry run (no actual release)"
8+
required: false
9+
type: boolean
10+
default: false
11+
12+
jobs:
13+
release:
14+
runs-on:
15+
group: databricks-protected-runner-group
16+
labels: linux-ubuntu-latest
17+
18+
environment: release
19+
20+
permissions:
21+
contents: write
22+
id-token: write
23+
24+
steps:
25+
- name: Checkout
26+
uses: actions/checkout@v4
27+
with:
28+
fetch-depth: 0
29+
token: ${{ secrets.GITHUB_TOKEN }}
30+
31+
- name: Setup Git
32+
run: |
33+
git config user.name "github-actions[bot]"
34+
git config user.email "github-actions[bot]@users.noreply.github.com"
35+
36+
- name: Setup pnpm
37+
uses: pnpm/action-setup@v4
38+
39+
- name: Setup Node.js
40+
uses: actions/setup-node@v4
41+
with:
42+
node-version: 24
43+
registry-url: "https://registry.npmjs.org"
44+
cache: "pnpm"
45+
46+
- name: Update npm
47+
run: npm install -g npm@latest
48+
49+
- name: Install dependencies
50+
run: pnpm install --frozen-lockfile
51+
52+
- name: Release
53+
working-directory: packages/lakebase
54+
run: |
55+
if [ "${{ inputs.dry-run }}" == "true" ]; then
56+
pnpm release:dry
57+
else
58+
pnpm release:ci
59+
fi
60+
env:
61+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

CLAUDE.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -233,6 +233,39 @@ The AnalyticsPlugin provides SQL query execution:
233233
- Built-in caching with configurable TTL
234234
- Databricks SQL Warehouse connector for execution
235235

236+
### Lakebase Autoscaling Connector
237+
238+
**Location:** `packages/appkit/src/connectors/lakebase/`
239+
240+
AppKit provides `createLakebasePool()` - a factory function that returns a standard `pg.Pool` configured with automatic OAuth token refresh for Databricks Lakebase (OLTP) databases.
241+
242+
**Key Features:**
243+
- Returns standard `pg.Pool` (compatible with all ORMs)
244+
- Automatic OAuth token refresh (1-hour tokens, 2-minute buffer)
245+
- Token caching to minimize API calls
246+
- Battle-tested pattern (same as AWS RDS IAM authentication)
247+
248+
**Quick Example:**
249+
```typescript
250+
import { createLakebasePool } from '@databricks/appkit';
251+
252+
// Reads from PGHOST, PGDATABASE, LAKEBASE_ENDPOINT env vars
253+
const pool = createLakebasePool();
254+
255+
// Standard pg.Pool API
256+
const result = await pool.query('SELECT * FROM users');
257+
```
258+
259+
**ORM Integration:**
260+
Works with Drizzle, Prisma, TypeORM - see [Lakebase Integration Docs](docs/docs/integrations/lakebase.md) for examples.
261+
262+
**Architecture:**
263+
- Connector files: `packages/appkit/src/connectors/lakebase/`
264+
- `pool.ts` - Pool factory with OAuth token refresh
265+
- `types.ts` - TypeScript interfaces (`LakebasePoolConfig`)
266+
- `utils.ts` - Helper functions (`generateDatabaseCredential`)
267+
- `auth-types.ts` - Lakebase v2 API types
268+
236269
### Frontend-Backend Interaction
237270

238271
```
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Enumeration: RequestedClaimsPermissionSet
2+
3+
Permission set for Unity Catalog table access
4+
5+
## Enumeration Members
6+
7+
### READ\_ONLY
8+
9+
```ts
10+
READ_ONLY: "READ_ONLY";
11+
```
12+
13+
Read-only access to specified UC tables
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Function: createLakebasePool()
2+
3+
```ts
4+
function createLakebasePool(config?: Partial<LakebasePoolConfig>): Pool;
5+
```
6+
7+
Create a Lakebase pool with appkit's logger integration.
8+
Telemetry automatically uses appkit's OpenTelemetry configuration via global registry.
9+
10+
## Parameters
11+
12+
| Parameter | Type | Description |
13+
| ------ | ------ | ------ |
14+
| `config?` | `Partial`\<[`LakebasePoolConfig`](Interface.LakebasePoolConfig.md)\> | Lakebase pool configuration |
15+
16+
## Returns
17+
18+
`Pool`
19+
20+
PostgreSQL pool with appkit integration
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Function: generateDatabaseCredential()
2+
3+
```ts
4+
function generateDatabaseCredential(workspaceClient: WorkspaceClient, request: GenerateDatabaseCredentialRequest): Promise<DatabaseCredential>;
5+
```
6+
7+
Generate OAuth credentials for Postgres database connection using the proper Postgres API.
8+
9+
This generates a time-limited OAuth token (expires after 1 hour) that can be used
10+
as a password when connecting to Lakebase Postgres databases.
11+
12+
## Parameters
13+
14+
| Parameter | Type | Description |
15+
| ------ | ------ | ------ |
16+
| `workspaceClient` | `WorkspaceClient` | Databricks workspace client for authentication |
17+
| `request` | [`GenerateDatabaseCredentialRequest`](Interface.GenerateDatabaseCredentialRequest.md) | Request parameters including endpoint path and optional UC claims |
18+
19+
## Returns
20+
21+
`Promise`\<[`DatabaseCredential`](Interface.DatabaseCredential.md)\>
22+
23+
Database credentials with OAuth token and expiration time
24+
25+
## See
26+
27+
https://docs.databricks.com/aws/en/oltp/projects/authentication
28+
29+
## Examples
30+
31+
```typescript
32+
// Format: projects/{project-id}/branches/{branch-id}/endpoints/{endpoint-id}
33+
// Note: Use actual IDs from Databricks (project-id is a UUID)
34+
const credential = await generateDatabaseCredential(workspaceClient, {
35+
endpoint: "projects/6bef4151-4b5d-4147-b4d0-c2f4fd5b40db/branches/br-sparkling-tree-y17uj7fn/endpoints/ep-restless-pine-y1ldaht0"
36+
});
37+
38+
// Use credential.token as password
39+
const conn = await pg.connect({
40+
host: "ep-abc123.database.us-east-1.databricks.com",
41+
user: "user@example.com",
42+
password: credential.token
43+
});
44+
```
45+
46+
```typescript
47+
// Format: projects/{project-id}/branches/{branch-id}/endpoints/{endpoint-id}
48+
const credential = await generateDatabaseCredential(workspaceClient, {
49+
endpoint: "projects/6bef4151-4b5d-4147-b4d0-c2f4fd5b40db/branches/br-sparkling-tree-y17uj7fn/endpoints/ep-restless-pine-y1ldaht0",
50+
claims: [{
51+
permission_set: RequestedClaimsPermissionSet.READ_ONLY,
52+
resources: [{ table_name: "catalog.schema.users" }]
53+
}]
54+
});
55+
```
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Function: getLakebaseOrmConfig()
2+
3+
```ts
4+
function getLakebaseOrmConfig(config?: Partial<LakebasePoolConfig>): {
5+
password: string | () => string | () => Promise<string> | undefined;
6+
ssl: | boolean
7+
| {
8+
rejectUnauthorized: boolean | undefined;
9+
};
10+
username: string | undefined;
11+
};
12+
```
13+
14+
Get Lakebase connection configuration for ORMs that don't accept pg.Pool directly.
15+
16+
Designed for ORMs like TypeORM and Sequelize that need connection parameters
17+
rather than a pre-configured pool instance.
18+
19+
Returns connection config with field names compatible with common ORMs:
20+
- `username` instead of `user`
21+
- Simplified SSL config
22+
- Password callback support for OAuth token refresh
23+
24+
## Parameters
25+
26+
| Parameter | Type | Description |
27+
| ------ | ------ | ------ |
28+
| `config?` | `Partial`\<[`LakebasePoolConfig`](Interface.LakebasePoolConfig.md)\> | Optional configuration (reads from environment if not provided) |
29+
30+
## Returns
31+
32+
```ts
33+
{
34+
password: string | () => string | () => Promise<string> | undefined;
35+
ssl: | boolean
36+
| {
37+
rejectUnauthorized: boolean | undefined;
38+
};
39+
username: string | undefined;
40+
}
41+
```
42+
43+
ORM-compatible connection configuration
44+
45+
### password
46+
47+
```ts
48+
password: string | () => string | () => Promise<string> | undefined;
49+
```
50+
51+
### ssl
52+
53+
```ts
54+
ssl:
55+
| boolean
56+
| {
57+
rejectUnauthorized: boolean | undefined;
58+
};
59+
```
60+
61+
### username
62+
63+
```ts
64+
username: string | undefined = user;
65+
```
66+
67+
## Example
68+
69+
```typescript
70+
// TypeORM
71+
const dataSource = new DataSource({
72+
type: 'postgres',
73+
...getLakebaseOrmConfig(),
74+
entities: [User],
75+
synchronize: true,
76+
});
77+
78+
// Sequelize
79+
const sequelize = new Sequelize({
80+
dialect: 'postgres',
81+
...getLakebaseOrmConfig(),
82+
logging: false,
83+
});
84+
```
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Function: getLakebasePgConfig()
2+
3+
```ts
4+
function getLakebasePgConfig(
5+
config?: Partial<LakebasePoolConfig>,
6+
telemetry?: DriverTelemetry,
7+
logger?: Logger): PoolConfig;
8+
```
9+
10+
Get Lakebase connection configuration for PostgreSQL clients.
11+
12+
Returns pg.PoolConfig with OAuth token authentication configured.
13+
Best used with pg.Pool directly or ORMs that accept pg.Pool instances (like Drizzle).
14+
15+
For ORMs that need connection parameters (TypeORM, Sequelize), use getLakebaseOrmConfig() instead.
16+
17+
Used internally by createLakebasePool().
18+
19+
## Parameters
20+
21+
| Parameter | Type | Description |
22+
| ------ | ------ | ------ |
23+
| `config?` | `Partial`\<[`LakebasePoolConfig`](Interface.LakebasePoolConfig.md)\> | Optional configuration (reads from environment if not provided) |
24+
| `telemetry?` | `DriverTelemetry` | Optional pre-initialized telemetry (created internally if not provided) |
25+
| `logger?` | `Logger` | Optional logger (silent if not provided) |
26+
27+
## Returns
28+
29+
`PoolConfig`
30+
31+
PostgreSQL pool configuration with OAuth token refresh
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Function: getWorkspaceClient()
2+
3+
```ts
4+
function getWorkspaceClient(config: Partial<LakebasePoolConfig>): Promise<WorkspaceClient>;
5+
```
6+
7+
Get workspace client from config or SDK default auth chain
8+
9+
## Parameters
10+
11+
| Parameter | Type |
12+
| ------ | ------ |
13+
| `config` | `Partial`\<[`LakebasePoolConfig`](Interface.LakebasePoolConfig.md)\> |
14+
15+
## Returns
16+
17+
`Promise`\<`WorkspaceClient`\>
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Interface: DatabaseCredential
2+
3+
Database credentials with OAuth token for Postgres connection
4+
5+
## Properties
6+
7+
### expire\_time
8+
9+
```ts
10+
expire_time: string;
11+
```
12+
13+
Token expiration time in UTC (ISO 8601 format)
14+
Tokens expire after 1 hour from generation
15+
16+
#### Example
17+
18+
```ts
19+
"2026-02-06T17:07:00Z"
20+
```
21+
22+
***
23+
24+
### token
25+
26+
```ts
27+
token: string;
28+
```
29+
30+
OAuth token to use as the password when connecting to Postgres

0 commit comments

Comments
 (0)