|
| 1 | +--- |
| 2 | +title: Electric Collection |
| 3 | +--- |
| 4 | + |
| 5 | +# Electric Collection |
| 6 | + |
| 7 | +Electric collections provide seamless integration between TanStack DB and ElectricSQL, enabling real-time data synchronization with your Postgres database through Electric's sync engine. |
| 8 | + |
| 9 | +## Overview |
| 10 | + |
| 11 | +The `@tanstack/electric-db-collection` package allows you to create collections that: |
| 12 | +- Automatically sync data from Postgres via Electric shapes |
| 13 | +- Support optimistic updates with transaction matching and automatic rollback on errors |
| 14 | +- Handle persistence through customizable mutation handlers |
| 15 | + |
| 16 | +## Installation |
| 17 | + |
| 18 | +```bash |
| 19 | +npm install @tanstack/electric-db-collection @tanstack/react-db |
| 20 | +``` |
| 21 | + |
| 22 | +## Basic Usage |
| 23 | + |
| 24 | +```typescript |
| 25 | +import { createCollection } from '@tanstack/react-db' |
| 26 | +import { electricCollectionOptions } from '@tanstack/electric-db-collection' |
| 27 | + |
| 28 | +const todosCollection = createCollection( |
| 29 | + electricCollectionOptions({ |
| 30 | + shapeOptions: { |
| 31 | + url: '/api/todos', |
| 32 | + }, |
| 33 | + getKey: (item) => item.id, |
| 34 | + }) |
| 35 | +) |
| 36 | +``` |
| 37 | + |
| 38 | +## Configuration Options |
| 39 | + |
| 40 | +The `electricCollectionOptions` function accepts the following options: |
| 41 | + |
| 42 | +### Required Options |
| 43 | + |
| 44 | +- `shapeOptions`: Configuration for the ElectricSQL ShapeStream |
| 45 | + - `url`: The URL of your proxy to Electric |
| 46 | + |
| 47 | +- `getKey`: Function to extract the unique key from an item |
| 48 | + |
| 49 | +### Optional |
| 50 | + |
| 51 | +- `id`: Unique identifier for the collection |
| 52 | +- `schema`: Schema for validating items. Any Standard Schema compatible schema |
| 53 | +- `sync`: Custom sync configuration |
| 54 | + |
| 55 | +### Persistence Handlers |
| 56 | + |
| 57 | +- `onInsert`: Handler called before insert operations |
| 58 | +- `onUpdate`: Handler called before update operations |
| 59 | +- `onDelete`: Handler called before delete operations |
| 60 | + |
| 61 | +## Persistence Handlers |
| 62 | + |
| 63 | +Handlers can be defined to run on mutations. They are useful to send mutations to the backend and confirming them once Electric delivers the corresponding transactions. Until confirmation, TanStack DB blocks sync data for the collection to prevent race conditions. To avoid any delays, it’s important to use a matching strategy. |
| 64 | + |
| 65 | +The most reliable strategy is for the backend to include the transaction ID (txid) in its response, allowing the client to match each mutation with Electric’s transaction identifiers for precise confirmation. If no strategy is provided, client mutations are automatically confirmed after three seconds. |
| 66 | + |
| 67 | +```typescript |
| 68 | +const todosCollection = createCollection( |
| 69 | + electricCollectionOptions({ |
| 70 | + id: 'todos', |
| 71 | + schema: todoSchema, |
| 72 | + getKey: (item) => item.id, |
| 73 | + shapeOptions: { |
| 74 | + url: '/api/todos', |
| 75 | + params: { table: 'todos' }, |
| 76 | + }, |
| 77 | + |
| 78 | + onInsert: async ({ transaction }) => { |
| 79 | + const newItem = transaction.mutations[0].modified |
| 80 | + const response = await api.todos.create(newItem) |
| 81 | + |
| 82 | + return { txid: response.txid } |
| 83 | + }, |
| 84 | + |
| 85 | + // you can also implement onUpdate and onDelete handlers |
| 86 | + }) |
| 87 | +) |
| 88 | +``` |
| 89 | + |
| 90 | +On the backend, you can extract the `txid` for a transaction by querying Postgres directly. |
| 91 | + |
| 92 | +```ts |
| 93 | +async function generateTxId(tx) { |
| 94 | + // The ::xid cast strips off the epoch, giving you the raw 32-bit value |
| 95 | + // that matches what PostgreSQL sends in logical replication streams |
| 96 | + // (and then exposed through Electric which we'll match against |
| 97 | + // in the client). |
| 98 | + const result = await tx.execute( |
| 99 | + sql`SELECT pg_current_xact_id()::xid::text as txid` |
| 100 | + ) |
| 101 | + const txid = result.rows[0]?.txid |
| 102 | + |
| 103 | + if (txid === undefined) { |
| 104 | + throw new Error(`Failed to get transaction ID`) |
| 105 | + } |
| 106 | + |
| 107 | + return parseInt(txid as string, 10) |
| 108 | +} |
| 109 | +``` |
| 110 | + |
| 111 | +### Electric Proxy Example |
| 112 | + |
| 113 | +Electric is typically deployed behind a proxy server that handles shape configuration, authentication and authorization. This provides better security and allows you to control what data users can access without exposing Electric to the client. |
| 114 | + |
| 115 | + |
| 116 | +Here is an example proxy implementation using TanStack Starter: |
| 117 | + |
| 118 | +```js |
| 119 | +import { createServerFileRoute } from "@tanstack/react-start/server" |
| 120 | +import { ELECTRIC_PROTOCOL_QUERY_PARAMS } from "@electric-sql/client" |
| 121 | + |
| 122 | +// Electric URL |
| 123 | +const baseUrl = 'http://.../v1/shape' |
| 124 | + |
| 125 | +const serve = async ({ request }: { request: Request }) => { |
| 126 | + // ...check user authorization |
| 127 | + const url = new URL(request.url) |
| 128 | + const originUrl = new URL(baseUrl) |
| 129 | + |
| 130 | + // passthrough parameters from electric client |
| 131 | + url.searchParams.forEach((value, key) => { |
| 132 | + if (ELECTRIC_PROTOCOL_QUERY_PARAMS.includes(key)) { |
| 133 | + originUrl.searchParams.set(key, value) |
| 134 | + } |
| 135 | + }) |
| 136 | + |
| 137 | + // set shape parameters |
| 138 | + // full spec: https://github.com/electric-sql/electric/blob/main/website/electric-api.yaml |
| 139 | + originUrl.searchParams.set("table", "todos") |
| 140 | + // Where clause to filter rows in the table (optional). |
| 141 | + // originUrl.searchParams.set("where", "completed = true") |
| 142 | + |
| 143 | + // Select the columns to sync (optional) |
| 144 | + // originUrl.searchParams.set("columns", "id,text,completed") |
| 145 | + |
| 146 | + const response = await fetch(originUrl) |
| 147 | + const headers = new Headers(response.headers) |
| 148 | + headers.delete("content-encoding") |
| 149 | + headers.delete("content-length") |
| 150 | + |
| 151 | + return new Response(response.body, { |
| 152 | + status: response.status, |
| 153 | + statusText: response.statusText, |
| 154 | + headers, |
| 155 | + }) |
| 156 | +} |
| 157 | + |
| 158 | +export const ServerRoute = createServerFileRoute("/api/todos").methods({ |
| 159 | + GET: serve, |
| 160 | +}) |
| 161 | +``` |
| 162 | + |
| 163 | +## Optimistic Updates with Explicit Transactions |
| 164 | + |
| 165 | +For more advanced use cases, you can create custom actions that can do multiple mutations across collections transactionally. In this case, you need to explicitly await for the transaction ID using `utils.awaitTxId()`. |
| 166 | + |
| 167 | +```typescript |
| 168 | +const addTodoAction = createOptimisticAction({ |
| 169 | + onMutate: ({ text }) => { |
| 170 | + // optimistically insert with a temporary ID |
| 171 | + const tempId = crypto.randomUUID() |
| 172 | + todosCollection.insert({ |
| 173 | + id: tempId, |
| 174 | + text, |
| 175 | + completed: false, |
| 176 | + created_at: new Date(), |
| 177 | + }) |
| 178 | + |
| 179 | + // ... mutate other collections |
| 180 | + }, |
| 181 | + |
| 182 | + mutationFn: async ({ text }) => { |
| 183 | + const response = await api.todos.create({ |
| 184 | + data: { text, completed: false } |
| 185 | + }) |
| 186 | + |
| 187 | + await todosCollection.utils.awaitTxId(response.txid) |
| 188 | + } |
| 189 | +}) |
| 190 | +``` |
| 191 | + |
| 192 | +## Utility Methods |
| 193 | + |
| 194 | +The collection provides these utility methods via `collection.utils`: |
| 195 | + |
| 196 | +- `awaitTxId(txid, timeout?)`: Manually wait for a specific transaction ID to be synchronized |
| 197 | + |
| 198 | +```typescript |
| 199 | +todosCollection.utils.awaitTxId(12345) |
| 200 | +``` |
| 201 | + |
| 202 | +This is useful when you need to ensure a mutation has been synchronized before proceeding with other operations. |
0 commit comments