Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,7 @@ Atomic Agent drives a full desktop tool surface. Dangerous actions are routed th
| Area | Capabilities |
|---|---|
| **Browser** | Navigate, click, type, search, manage tabs, scroll, and read compact ARIA state via `playwright-core` (Chrome / Edge / Brave / Chromium). |
| **Web & HTTP** | Web search with configurable providers (Exa, DuckDuckGo, Brave, SearXNG); fetch and extract pages or make arbitrary HTTP requests, both SSRF-guarded, separate from the browser. |
| **Web & HTTP** | Web search with configurable providers (Exa, DuckDuckGo, Context, Brave, SearXNG); fetch and extract pages or make arbitrary HTTP requests, both SSRF-guarded, separate from the browser. |
| **Filesystem & shell** | Read, write, edit, patch, glob, grep, diff, watch, hash, list, archive extract, create editable PowerPoint decks, run approved shell commands, and inspect or kill processes. |
| **Desktop** | Clipboard read/write, desktop notifications, and window list/focus. |
| **Documents** | Extract text locally from PDF, DOC, DOCX, XLSX, PPTX, ODT, RTF, and plain text. |
Expand Down Expand Up @@ -887,6 +887,7 @@ NOTION_API_KEY=ntn_xxxxxxxx
GITHUB_TOKEN=ghp_xxxxxxxx
TELEGRAM_BOT_TOKEN=123456789:AA-your-bot-token
EXA_API_KEY=exa_xxxxxxxx
CONTEXT_DEV_API_KEY=your_context_api_key
OBSIDIAN_VAULT_PATH=/Users/me/Documents/Obsidian Vault
```

Expand All @@ -896,6 +897,8 @@ Commands the agent runs (`os.shell.run`, `os.git.*`, skill scripts) do not inher

`EXA_API_KEY` is optional. Without it, web search uses Exa's free keyless tier, which runs out quickly, and then DuckDuckGo. When the free limit is reached, or no search provider answers, the search result says so and asks for a free key from <https://dashboard.exa.ai/api-keys>. Apps get the same request as `details.keyRequired` on the `os.web.search` result (`{ provider: "exa", reason: "rate_limited" | "unreachable" | "missing", getKeyUrl, envVar }`); the HTTP stream forwards it as `key_required` on the `tool_result` event. In the TUI, the **Web search (Exa)** entry of the Integrations tab saves the key (as `EXA_API_KEY`), and the next search uses it without a restart; the search error there points at `/integrations` instead of the `.env` file. The entry always writes `EXA_API_KEY`: when `web.search.exa.apiKeyEnv` names another variable, or Exa is neither `web.search.provider` nor in `web.search.fallback`, it says the key will not be used, and the search advice falls back to the `.env` file. A search site that cannot be reached from your network (DNS, refused, no answer to the connection within 5 s) is skipped by the searches that start after the failure, for two minutes, instead of each of them waiting for it again. That skip lives only in the running agent, and when every search site was unreachable the first site is tried again at most every 30 s.

To use Context, set `web.search.provider` to `"context"` in `config.json` and add `CONTEXT_DEV_API_KEY` to `<stateDir>/.env`. Searches request query-relevant Highlights and use your Context.dev account credits; send only public queries. Existing search defaults and fallback settings are unchanged.

The Integrations tab (`/integrations`) writes these for you. Its **GitHub** entry stores `GITHUB_TOKEN` and carries the **Remote sync** switch (`git.remoteSync`, off by default): while it is off, a repository the agent versions stays on this machine (`git push`, `fetch`, `pull`, `clone` and `remote add` are refused, through the git tools and through the shell alike), so you keep full local history without publishing anything. Turn it on when a project should reach GitHub; every sync then goes through the approval ladder. The dedicated tools (`os.git.remote`, `fetch`, `pull`, `push`, `clone`) honour the same switch and hand the token to git only for `github.com`, only through the child process's environment (never in a URL, in argv, or in `.git/config`) and scrub it from every line of output.

</details>
Expand Down
4 changes: 2 additions & 2 deletions docs/user/settings-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ In the TUI, `/settings <words>` searches the same list. The desktop app shows it
| Key | Type | Default | Description |
| --- | --- | --- | --- |
| `web.search.enabled` | `boolean` | `true` | |
| `web.search.provider` | `'duckduckgo' \| 'searxng' \| 'exa' \| 'brave'` | `"exa"` | Primary provider. The default `exa` uses its API endpoint when its `apiKeyEnv` resolves to a key, and its free keyless tier otherwise, with `duckduckgo` appended to the chain when it lacks it. The free tier runs out fast; a 429 parks it and the search result asks for a key (ATO-327). Existing configs need no rewrite for that. |
| `web.search.provider` | `'duckduckgo' \| 'searxng' \| 'context' \| 'exa' \| 'brave'` | `"exa"` | Primary provider. The default `exa` uses its API endpoint when its `apiKeyEnv` resolves to a key, and its free keyless tier otherwise, with `duckduckgo` appended to the chain when it lacks it. The free tier runs out fast; a 429 parks it and the search result asks for a key (ATO-327). Existing configs need no rewrite for that. |
| `web.search.maxResults` | `number` | `8` | |
| `web.search.timeoutMs` | `number` | `15000` | |
| `web.search.cacheTtlMinutes` | `number` | `60` | Per-runtime result cache TTL in minutes. `0` disables caching. The cache is the primary defence against provider rate-limiting on repeated queries. |
Expand Down Expand Up @@ -180,7 +180,7 @@ In the TUI, `/settings <words>` searches the same list. The desktop app shows it
| `browser.headless` | `boolean \| 'auto'` | `"auto"` | The browser runs without a window for every turn (an explicit `true`). |
| `browser.profile` | `'agent' \| 'temp'` | `"agent"` | `UserConfigFile.browser.profile`: the agent's profile or a throwaway one per chat. Absent: `"agent"`. |
| `browser.maxTabs` | `number` | `8` | `UserConfigFile.browser.maxTabs`. Absent: 8. |
| `browser.workerIsolation` | `'tabs' \| 'context'` | `"tabs"` | `UserConfigFile.browser.workerIsolation`. Absent: `"tabs"`. |
| `browser.workerIsolation` | `'context' \| 'tabs'` | `"tabs"` | `UserConfigFile.browser.workerIsolation`. Absent: `"tabs"`. |

## sessions

Expand Down
4 changes: 2 additions & 2 deletions src/config/settings-docs.generated.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ export const SETTINGS_DOCS: Readonly<Record<string, SettingDoc>> = {
"http.maxResponseBytes": { type: "number", description: "" },
"http.defaultTimeoutMs": { type: "number", description: "" },
"web.search.enabled": { type: "boolean", description: "" },
"web.search.provider": { type: "'duckduckgo' | 'searxng' | 'exa' | 'brave'", description: "Primary provider. The default `exa` uses its API endpoint when its `apiKeyEnv` resolves to a key, and its free keyless tier otherwise, with `duckduckgo` appended to the chain when it lacks it. The free tier runs out fast; a 429 parks it and the search result asks for a key (ATO-327). Existing configs need no rewrite for that." },
"web.search.provider": { type: "'duckduckgo' | 'searxng' | 'context' | 'exa' | 'brave'", description: "Primary provider. The default `exa` uses its API endpoint when its `apiKeyEnv` resolves to a key, and its free keyless tier otherwise, with `duckduckgo` appended to the chain when it lacks it. The free tier runs out fast; a 429 parks it and the search result asks for a key (ATO-327). Existing configs need no rewrite for that." },
"web.search.maxResults": { type: "number", description: "" },
"web.search.timeoutMs": { type: "number", description: "" },
"web.search.cacheTtlMinutes": { type: "number", description: "Per-runtime result cache TTL in minutes. `0` disables caching. The cache is the primary defence against provider rate-limiting on repeated queries." },
Expand Down Expand Up @@ -123,7 +123,7 @@ export const SETTINGS_DOCS: Readonly<Record<string, SettingDoc>> = {
"browser.headless": { type: "boolean | 'auto'", description: "The browser runs without a window for every turn (an explicit `true`)." },
"browser.profile": { type: "'agent' | 'temp'", description: "`UserConfigFile.browser.profile`: the agent's profile or a throwaway one per chat. Absent: `\"agent\"`." },
"browser.maxTabs": { type: "number", description: "`UserConfigFile.browser.maxTabs`. Absent: 8." },
"browser.workerIsolation": { type: "'tabs' | 'context'", description: "`UserConfigFile.browser.workerIsolation`. Absent: `\"tabs\"`." },
"browser.workerIsolation": { type: "'context' | 'tabs'", description: "`UserConfigFile.browser.workerIsolation`. Absent: `\"tabs\"`." },
"sessions.retention.enabled": { type: "boolean", description: "Default `false`: the operator opts in." },
"sessions.retention.maxAgeDays": { type: "number | null", description: "Age cutoff in days against `updated_at`. Default 90. A positive integer, or `null` for no age rule (leaving `maxRows` as the only thing that prunes). Note that `undefined` takes the default and an explicit `null` does not — clearing the rule is a choice, not an omission." },
"sessions.retention.maxRows": { type: "number | null", description: "Hard cap on stored sessions; anything past it goes oldest first. Default `null` (no cap) — a row count means nothing without knowing how the operator works, so age is the rule that ships on." },
Expand Down
2 changes: 1 addition & 1 deletion src/config/web-config.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ describe("web configuration ownership and compatibility", () => {
expect(parseWebSearchProviderName(provider, "synthetic.provider")).toBe(provider);
}
expect(thrownError(() => parseWebSearchProviderName("EXA", "synthetic.provider")))
.toMatchObject({ field: "synthetic.provider", reason: 'expected one of duckduckgo|searxng|exa|brave, got "EXA"' });
.toMatchObject({ field: "synthetic.provider", reason: 'expected one of duckduckgo|searxng|context|exa|brave, got "EXA"' });
});

it("creates fresh mutable nested defaults and retains property order", () => {
Expand Down
5 changes: 3 additions & 2 deletions src/config/web-config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import {
parseNonEmptyString,
} from "./config-primitives.js";

export type WebSearchProviderName = "duckduckgo" | "searxng" | "exa" | "brave";
export type WebSearchProviderName = "duckduckgo" | "searxng" | "context" | "exa" | "brave";

/** Main-content extractor `os.web.fetch` runs first on an HTML page. */
export type WebFetchExtractorName = "readability" | "defuddle";
Expand Down Expand Up @@ -250,14 +250,15 @@ export function parseWebSearchProviderName(
if (
raw === "duckduckgo" ||
raw === "searxng" ||
raw === "context" ||
raw === "exa" ||
raw === "brave"
) {
return raw;
}
throw new ConfigValidationError(
field,
`expected one of duckduckgo|searxng|exa|brave, got ${JSON.stringify(raw)}`,
`expected one of duckduckgo|searxng|context|exa|brave, got ${JSON.stringify(raw)}`,
);
}

Expand Down
130 changes: 130 additions & 0 deletions src/tools/os/web-search/providers/context-provider.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
import { assertProviderStatus } from "./assert-provider-status.js";
import { isConnectFailure } from "./search-failure-report.js";
import { searchHttp } from "../transport/search-http.js";
import { WebSearchRateLimitedError } from "../web-search-errors.js";
import type {
WebSearchHttpDeps,
WebSearchProvider,
WebSearchResult,
} from "../web-search-provider.js";

const CONTEXT_SEARCH_URL = "https://api.context.dev/v1/web/search";

interface ContextResult {
title?: unknown;
url?: unknown;
description?: unknown;
highlights?: { code?: unknown; highlights?: unknown };
}

export function createContextProvider(
deps: WebSearchHttpDeps = {},
): WebSearchProvider {
return {
name: "context",
async search(options) {
const apiKey = process.env.CONTEXT_DEV_API_KEY?.trim();
if (!apiKey) {
throw new Error(
"Context search requires CONTEXT_DEV_API_KEY in the process environment.",
);
}
if (options.query.length > 500) {
throw new Error("Context search queries must be at most 500 characters.");
}
let status: number | undefined;
try {
const response = await searchHttp({
url: CONTEXT_SEARCH_URL,
method: "POST",
headers: {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify({
query: options.query,
numResults: options.maxResults,
highlightsOptions: { enabled: true, maxCharacters: 1_000 },
timeoutOpts: {
milliseconds: Math.min(300_000, Math.max(1_000, options.timeoutMs - 1_000)),
behavior: "fail",
},
}),
timeoutMs: options.timeoutMs,
cwd: options.cwd,
signal: options.signal,
maxRedirects: 0,
runCommand: deps.runCommand,
lookup: deps.lookup,
});
status = response.status;
assertProviderStatus(response, "context", "Context search");
const parsed = JSON.parse(response.body) as {
results?: unknown;
partial?: unknown;
} | null;
if (
!parsed || !Array.isArray(parsed.results) || response.truncated ||
(parsed.partial !== undefined && parsed.partial !== false)
) {
throw new Error("Invalid Context search response");
}
const results: WebSearchResult[] = [];
for (const raw of parsed.results as (ContextResult | null)[]) {
if (!raw || typeof raw.title !== "string" || typeof raw.url !== "string") {
continue;
}
if (
raw.highlights !== undefined &&
(!raw.highlights || typeof raw.highlights !== "object" ||
typeof raw.highlights.code !== "string" ||
(raw.highlights.code === "SUCCESS" &&
!Array.isArray(raw.highlights.highlights)))
) {
throw new Error("Invalid Context Highlights response");
}
const title = raw.title.trim();
let url: URL;
try {
url = new URL(raw.url.trim());
} catch {
continue;
}
if (!title || !["http:", "https:"].includes(url.protocol) ||
url.username || url.password) continue;
const highlights =
raw.highlights?.code === "SUCCESS" &&
Array.isArray(raw.highlights.highlights)
? raw.highlights.highlights
.filter((value): value is string => typeof value === "string")
.join("\n").trim()
: "";
results.push({
title,
url: raw.url.trim(),
snippet: highlights ||
(typeof raw.description === "string"
? `[Search snippet; Highlights unavailable] ${raw.description.trim()}`
: ""),
});
if (results.length >= options.maxResults) break;
}
if (parsed.results.length > 0 && results.length === 0) {
throw new Error("Invalid Context search results");
}
return results;
} catch (err) {
if (err instanceof WebSearchRateLimitedError) throw err;
if (status === undefined && isConnectFailure(err)) {
throw new Error("Context search failed to connect.");
}
const detail = status !== undefined && status >= 400 ? ` (HTTP ${status})` : "";
throw new Error(
`Context search failed${detail}. No complete valid result was returned; ` +
"check the API key, account credits, or provider availability.",
);
}
},
};
}
3 changes: 3 additions & 0 deletions src/tools/os/web-search/providers/provider-registry.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import type { AtomicAgentConfig } from "../../../../config/index.js";
import { createBraveProvider } from "./brave-provider.js";
import { createContextProvider } from "./context-provider.js";
import { createDuckDuckGoProvider } from "./duckduckgo-provider.js";
import { createExaProvider } from "./exa-provider.js";
import { createSearxngProvider } from "./searxng-provider.js";
Expand Down Expand Up @@ -31,6 +32,8 @@ export function resolveProviderByName(
return createDuckDuckGoProvider(deps);
case "searxng":
return createSearxngProvider(search.searxng, deps);
case "context":
return createContextProvider(deps);
case "exa":
return createExaProvider(search.exa, deps);
case "brave":
Expand Down
1 change: 1 addition & 0 deletions src/tools/os/web-search/providers/search-failure-report.ts
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ export function describeUnusable(
name: string,
search: AtomicAgentConfig["web"]["search"],
): string {
if (name === "context") return "no CONTEXT_DEV_API_KEY set";
if (name === "brave") return `no ${search.brave.apiKeyEnv} set`;
if (name === "searxng") return "no `web.search.searxng.instanceUrl` set";
return "not configured";
Expand Down
2 changes: 2 additions & 0 deletions src/tools/os/web-search/providers/search-orchestrator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,8 @@ function isProviderUsable(
switch (name) {
case "searxng":
return Boolean(search.searxng.instanceUrl);
case "context":
return hasEnvKey(env, "CONTEXT_DEV_API_KEY");
case "brave": {
const key = env[search.brave.apiKeyEnv];
return typeof key === "string" && key.length > 0;
Expand Down
6 changes: 3 additions & 3 deletions src/tools/os/web-search/tool/warn-missing-search-key.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import type { WebSearchProviderName } from "../web-search-provider.js";
* Startup diagnostic for a primary search provider that cannot run without
* its API key.
*
* That is Brave: it has no keyless tier, so a keyless Brave primary is
* That is Brave or Context: it has no keyless tier, so a keyless primary is
* skipped outright and every search goes to the fallback chain. The
* operator configured one provider and is silently getting another, which
* is the failure mode this warning exists to break (#179).
Expand All @@ -19,7 +19,7 @@ import type { WebSearchProviderName } from "../web-search-provider.js";
*/

/** Providers that are skipped entirely when their `apiKeyEnv` is unset. */
const KEY_REQUIRED_PROVIDERS = new Set<WebSearchProviderName>(["brave"]);
const KEY_REQUIRED_PROVIDERS = new Set<WebSearchProviderName>(["context", "brave"]);

export interface MissingSearchKeyWarning {
provider: WebSearchProviderName;
Expand All @@ -46,7 +46,7 @@ export function checkMissingSearchKey(input: {
const provider = search.provider;
if (!KEY_REQUIRED_PROVIDERS.has(provider)) return null;

const apiKeyEnv = search.brave.apiKeyEnv;
const apiKeyEnv = provider === "context" ? "CONTEXT_DEV_API_KEY" : search.brave.apiKeyEnv;
const key = input.env[apiKeyEnv]?.trim();
if (typeof key === "string" && key.length > 0) return null;

Expand Down
2 changes: 1 addition & 1 deletion src/tools/os/web-search/tool/web-search-tool.ts
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ export function buildOsWebSearchTool(
"Search the web through the configured provider and return compact " +
"title/url/snippet results. Default provider is Exa (its free " +
"keyless tier, or an environment API key when set), with " +
"DuckDuckGo as the fallback; SearXNG is keyless, Brave needs an " +
"DuckDuckGo as the fallback; SearXNG is keyless, Context and Brave need an " +
"environment API key. " +
"Use os.web.fetch to read a chosen result page.",
readonly: true,
Expand Down
5 changes: 3 additions & 2 deletions src/tools/os/web-search/transport/search-http.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ export interface SearchHttpRequest {
cwd: string;
signal: AbortSignal;
maxResponseBytes?: number;
maxRedirects?: number;
runCommand?: typeof defaultRunCommand;
lookup?: HostLookup;
/** Overrides the 429 retry schedule; `maxRetries: 0` disables retrying. */
Expand Down Expand Up @@ -172,8 +173,8 @@ async function sendOnce(
REDIRECT_STATUSES.has(response.status) &&
response.redirectUrl.length > 0
) {
if (hop >= MAX_REDIRECTS) {
throw new Error(`too many redirects (> ${MAX_REDIRECTS})`);
if (hop >= (request.maxRedirects ?? MAX_REDIRECTS)) {
throw new Error(`too many redirects (> ${request.maxRedirects ?? MAX_REDIRECTS})`);
}
currentUrl = parseHttpUrl(response.redirectUrl);
continue;
Expand Down
2 changes: 1 addition & 1 deletion src/tools/os/web-search/web-search-provider.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import type { runCommand as defaultRunCommand } from "../../../sandbox/command-runner.js";
import type { HostLookup } from "../web/web-fetch-ssrf-guard.js";

export type WebSearchProviderName = "duckduckgo" | "searxng" | "exa" | "brave";
export type WebSearchProviderName = "duckduckgo" | "searxng" | "context" | "exa" | "brave";

export interface WebSearchResult {
title: string;
Expand Down