Skip to content

feat: retry mechanism for known status codes #71

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mohamedveron merged 3 commits into from
May 7, 2026
Merged

feat: retry mechanism for known status codes #71

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3fd504a
feat: retry mechanism for known status codes
christianalfoni May 6, 2026
fe9365a
chore: docs and some fixes
christianalfoni May 6, 2026
6594160
chore: merge
christianalfoni May 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 73 additions & 4 deletions docs/python-sdk.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,11 @@ The main entry point for the SDK.
sdk = TogetherSandbox(api_key=None, base_url="https://api.codesandbox.io")
```

| Parameter | Type | Description |
| ---------- | ------------- | ------------------------------------------------------------------ |
| `api_key` | `str \| None` | Together AI API key. Falls back to the `TOGETHER_API_KEY` env var. |
| `base_url` | `str` | Management API base URL. Defaults to `https://api.codesandbox.io`. |
| Parameter | Type | Description |
| ---------- | --------------------- | ---------------------------------------------------------------------- |
| `api_key` | `str \| None` | Together AI API key. Falls back to the `TOGETHER_API_KEY` env var. |
| `base_url` | `str` | Management API base URL. Defaults to `https://api.codesandbox.io`. |
| `retry` | `RetryConfig \| None` | Retry configuration for transient failures. See [Retry](#retry) below. |

`TogetherSandbox` supports use as an async context manager:

Expand Down Expand Up @@ -489,6 +490,74 @@ await Sandbox.shutdown("sandbox-id", api_key="your-key")

---

## Retry

All operations automatically retry on transient failures. By default:

- **HTTP status codes** `408`, `429`, `500`, `502`, `503`, `504` trigger a retry.
- **Network-level errors** — `httpx.TimeoutException` (timeout), `httpx.ConnectError` (connection refused), `httpx.RemoteProtocolError` (closed connection) — trigger a retry.
- **3 total attempts** (1 initial + 2 retries).
- **Exponential backoff**: starts at 0.5 s, doubles each attempt, plus up to 0.25 s of random jitter.

Pass a `RetryConfig` to `TogetherSandbox(retry=...)` to customise this behaviour.

### `RetryConfig`

`RetryConfig` is a dataclass — instantiate it with keyword arguments.

| Field | Type | Default | Description |
| -------------- | ----------------------------------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `max_attempts` | `int` | `3` | Total number of attempts (including the first). Set to `1` to disable retries. |
| `should_retry` | `Callable[[RetryContext], bool \| float] \| Awaitable[...]` | `None` | Override the retry decision. Return `False` to abort immediately, `True` to retry with the default backoff delay, or a `float` (seconds) to retry after a custom delay. May be a coroutine function. |
| `on_retry` | `Callable[[RetryContext], None] \| Awaitable[None]` | `None` | Called before each retry. Use for logging, metrics, or UI progress updates. May be a coroutine function. |

### `RetryContext`

| Field | Type | Description |
| ----------- | ------------- | -------------------------------------------------------------------------------------------- |
| `operation` | `str` | The operation that failed, e.g. `'startSandbox'`, `'files.read'`. |
| `attempt` | `int` | 1-based number of the attempt that just failed. |
| `error` | `Exception` | The exception that was raised or the parsed error model. |
| `status` | `int \| None` | HTTP status code, or `None` for network-level errors. |
| `delay` | `float` | **Seconds** to wait before the next attempt (default computed, override via `should_retry`). |

### Example

```python
import asyncio
from together_sandbox import TogetherSandbox, RetryConfig, RetryContext

async def main():
sdk = TogetherSandbox(
api_key="...",
retry=RetryConfig(
max_attempts=4,
should_retry=lambda ctx: (
# Never retry snapshot creation — it is not idempotent
False if ctx.operation == "snapshots.create"
# Give up on auth errors
else False if ctx.status in (401, 403)
# Retry everything else with default backoff
else True
),
on_retry=lambda ctx: print(
f"[retry] {ctx.operation} attempt {ctx.attempt} failed "
f"(HTTP {ctx.status or 'network'}) — retrying in {ctx.delay:.2f}s"
),
),
)

asyncio.run(main())
```

> **Note — `snapshots.create` is not idempotent.** Retrying after a transient 500 once the snapshot has already been created will register a duplicate. Exclude it via `should_retry` as shown above, or use the shorthand:
>
> ```python
> RetryConfig(should_retry=lambda ctx: ctx.operation != "snapshots.create")
> ```

---

## Environment variables

| Variable | Description |
Expand Down
75 changes: 71 additions & 4 deletions docs/typescript-sdk.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,11 @@ const sdk = new TogetherSandbox(config: TogetherSandboxConfig);

### `TogetherSandboxConfig`

| Property | Type | Required | Description |
| --------- | -------- | -------- | ------------------------------------------------------------------------------- |
| `apiKey` | `string` | Yes | Together AI API key. Falls back to `TOGETHER_API_KEY` env var if not set. |
| `baseUrl` | `string` | No | Override the management API base URL. Defaults to `https://api.codesandbox.io`. |
| Property | Type | Required | Description |
| --------- | ------------- | -------- | ------------------------------------------------------------------------------- |
| `apiKey` | `string` | Yes | Together AI API key. Falls back to `TOGETHER_API_KEY` env var if not set. |
| `baseUrl` | `string` | No | Override the management API base URL. Defaults to `https://api.codesandbox.io`. |
| `retry` | `RetryConfig` | No | Retry configuration for transient failures. See [Retry](#retry) below. |

### `sdk.sandboxes`

Expand Down Expand Up @@ -459,6 +460,72 @@ await Sandbox.shutdown("your-sandbox-id", {

---

## Retry

All operations automatically retry on transient failures. By default:

- **HTTP status codes** `408`, `429`, `500`, `502`, `503`, `504` trigger a retry.
- **Network-level errors** (connection refused, timeout, closed connection) — any `TypeError` thrown by the fetch layer — trigger a retry.
- **3 total attempts** (1 initial + 2 retries).
- **Exponential backoff**: starts at 500 ms, doubles each attempt, plus up to 250 ms of random jitter.

Pass a `RetryConfig` to `new TogetherSandbox({ retry: ... })` to customise this behaviour.

### `RetryConfig`

| Property | Type | Default | Description |
| ------------- | ------------------------------------------------------------------------ | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `maxAttempts` | `number` | `3` | Total number of attempts (including the first). Set to `1` to disable retries. |
| `shouldRetry` | `(ctx: RetryContext) => boolean \| number \| Promise<boolean \| number>` | — | Override the retry decision. Return `false` to abort immediately, `true` to retry with the default backoff delay, or a `number` (milliseconds) to retry after a custom delay. |
| `onRetry` | `(ctx: RetryContext) => void \| Promise<void>` | — | Called before each retry. Use for logging, metrics, or UI progress updates. |

### `RetryContext`

| Field | Type | Description |
| ----------- | --------------------- | ------------------------------------------------------------------------------------------------ |
| `operation` | `string` | The operation that failed, e.g. `'startSandbox'`, `'files.read'`. |
| `attempt` | `number` | 1-based number of the attempt that just failed. |
| `error` | `unknown` | The error that was thrown or the parsed error model. |
| `status` | `number \| undefined` | HTTP status code, or `undefined` for network-level errors. |
| `delay` | `number` | **Milliseconds** to wait before the next attempt (default computed, override via `shouldRetry`). |

### Example

```typescript
import {
TogetherSandbox,
type RetryConfig,
type RetryContext,
} from "@together-sandbox/sdk";

const sdk = new TogetherSandbox({
apiKey: process.env.TOGETHER_API_KEY!,
retry: {
maxAttempts: 4,
shouldRetry: ({ operation, attempt, status }: RetryContext) => {
// Never retry snapshot creation — it is not idempotent
if (operation === "snapshots.create") return false;
// Give up after attempt 3 on auth errors
if (status === 401 || status === 403) return false;
return true; // retry everything else with default backoff
},
onRetry: ({ operation, attempt, status, delay }: RetryContext) => {
console.warn(
`[retry] ${operation} attempt ${attempt} failed (HTTP ${status ?? "network"}) — retrying in ${delay} ms`,
);
},
},
});
```

> **Note — `snapshots.create` is not idempotent.** Retrying after a transient 500 once the snapshot has already been created will register a duplicate. Exclude it via `shouldRetry` as shown above, or use the shorthand:
>
> ```typescript
> shouldRetry: ({ operation }) => operation !== "snapshots.create";
> ```

---

## Environment variables

| Variable | Description |
Expand Down
Loading
Loading