Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 72 additions & 24 deletions docs/vendor-neutrality.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

- [How Magpie achieves vendor neutrality](#how-magpie-achieves-vendor-neutrality)
- [Two questions that get conflated](#two-questions-that-get-conflated)
- [The five axes of neutrality](#the-five-axes-of-neutrality)
- [The six axes of neutrality](#the-six-axes-of-neutrality)
- [The mechanism: skills, tools, capabilities](#the-mechanism-skills-tools-capabilities)
- [Skills target the abstraction, never a vendor's client](#skills-target-the-abstraction-never-a-vendors-client)
- [Tools are the only place vendor-specific code lives](#tools-are-the-only-place-vendor-specific-code-lives)
Expand All @@ -13,8 +13,9 @@
- [1. LLM backend](#1-llm-backend)
- [2. Agentic runtime](#2-agentic-runtime)
- [3. Forge and tracker](#3-forge-and-tracker)
- [4. Source control (VCS)](#4-source-control-vcs)
- [5. Project governance](#5-project-governance)
- [4. Communication channels](#4-communication-channels)
- [5. Source control (VCS)](#5-source-control-vcs)
- [6. Project governance](#6-project-governance)
- [What keeps it neutral over time](#what-keeps-it-neutral-over-time)
- [The contribution model — neutrality as an invitation](#the-contribution-model--neutrality-as-an-invitation)
- [Status at a glance](#status-at-a-glance)
Expand Down Expand Up @@ -74,20 +75,21 @@ documented contract — not a fork of the workflows.** That is what
"vendor neutral" means here, and it is the only definition that is both
achievable and useful.

## The five axes of neutrality
## The six axes of neutrality

"Vendor" is not one thing. Magpie is neutral across five independent
"Vendor" is not one thing. Magpie is neutral across six independent
axes, and a backend choice on one axis never constrains the others:

| Axis | The "vendor" | Neutral by… |
|---|---|---|
| **LLM backend** | Anthropic, OpenAI, Google, Bedrock, local Ollama/vLLM, a future ASF endpoint | Skills written against a model *contract* (capability floor), not a client; a privacy gate that keys on endpoint identity, not on who hosts it |
| **Agentic runtime** | Claude Code, Codex, Cursor, Gemini CLI, Copilot, OpenCode, Kiro, … | Skills are [`AGENTS.md`](https://agents.md/)-standard markdown under a shared `.agents/skills/` home that every runtime reads |
| **Forge / tracker** | GitHub, GitLab, Gitea, Forgejo, Pagure, Bitbucket, Jira, mailing lists | Per-interface **tools** behind capability contracts; many tools are pure adapter *specs* with pluggable backends |
| **Forge / tracker** | GitHub, GitLab, Gitea, Forgejo, Pagure, Bitbucket, Jira, Bugzilla | Per-interface **tools** behind capability contracts; many tools are pure adapter *specs* with pluggable backends |
| **Communication channels** | Mailing lists, GitHub Discussions, Discourse, Zulip, Matrix, IRC | Mail-archive / mail-source adapter contracts; chat and forum bridges as sibling tools |
| **Source control (VCS)** | Git, Mercurial, Subversion, Jujutsu, Fossil, Perforce, … | A single `VCSBackend` contract; skills call the abstract operation, the backend is detected from the working copy |
| **Project governance** | ASF PMC, foundation-hosted, single-vendor, informal maintainer group | Modes and thresholds are adopter config; non-ASF adopters are first-class ([`PRINCIPLES.md` §3](../PRINCIPLES.md#3-project-autonomy-is-the-structural-starting-point)) |

The rest of this page walks the mechanism that makes all five true,
The rest of this page walks the mechanism that makes all six true,
then states exactly where each axis stands today.

## The mechanism: skills, tools, capabilities
Expand Down Expand Up @@ -154,6 +156,15 @@ with pluggable backends already include:
| [`tools/scan-format`](../tools/scan-format/) | security-scanner report formats (ASVS reference) |
| [`tools/vcs`](../tools/vcs/) | Git (complete), Mercurial, Subversion, … (extension points) |

The security-team surface follows the same pattern: CNA backends live
behind [`tools/cve-tool`](../tools/cve-tool/) (the ASF Vulnogram adapter
[`tools/cve-tool-vulnogram`](../tools/cve-tool-vulnogram/) is the landed
reference), inbound report relays behind
[`tools/forwarder-relay`](../tools/forwarder-relay/), scanner formats
behind [`tools/scan-format`](../tools/scan-format/), and an OSV.dev
vulnerability cross-reference bridge is the open extension point
([#311](https://github.com/apache/magpie/issues/311)).

The distinction Magpie enforces: **vendor-specific *integrations* are
expected and welcome; vendor-specific *workflows* are forbidden.** A
GitHub tool is fine. A skill that only works because it assumes GitHub
Expand Down Expand Up @@ -247,7 +258,7 @@ issue`, not hypothetical:
[Codeberg / Gitea / Forgejo](https://github.com/apache/magpie/issues/310),
[Pagure](https://github.com/apache/magpie/issues/312) (Fedora /
`pagure.io`),
[Bitbucket](https://github.com/apache/magpie/issues/306) (deep Jira
[Bitbucket](https://github.com/apache/magpie/issues/606) (deep Jira
pairing), and
[SourceHut](https://github.com/apache/magpie/issues/607) (email-patch
review). Tracker-only surfaces are tracked the same way — e.g.
Expand All @@ -257,7 +268,33 @@ tracker capability and the source-control capability are *separable*: a
project can pair, say, GitHub issues with a Subversion working copy, or
a Bitbucket forge over Git, because each is a distinct contract.

### 4. Source control (VCS)
### 4. Communication channels

Project conversation does not all live on the forge. The intake and
mentoring skills read mailing-list archives, and a project may run its
discussion on a forum or chat system instead of (or alongside) a list.
Both surfaces sit behind adapter contracts:

- [`tools/mail-archive`](../tools/mail-archive/) — public archive reads
across PonyMail, Hyperkitty, Discourse, Google Groups, GitHub
Discussions.
- [`tools/mail-source`](../tools/mail-source/) — raw mail ingestion
across mbox, IMAP, Mailman 3.

The open extension points are labelled `good first issue`:
mail-source backends —
[mbox](https://github.com/apache/magpie/issues/304),
[IMAP](https://github.com/apache/magpie/issues/303),
[Mailman 3 / Hyperkitty](https://github.com/apache/magpie/issues/306);
and chat / forum bridges —
[Discourse](https://github.com/apache/magpie/issues/307),
[Zulip](https://github.com/apache/magpie/issues/308),
[Matrix / Element](https://github.com/apache/magpie/issues/309). A
project on IRC, Slack, or any other channel plugs in the same way — a
sibling tool fulfilling the read capability the mentoring / intake
skills declare, with no skill change.

### 5. Source control (VCS)

The newest axis to be abstracted, and a good worked example of the
mechanism. Earlier, dev-loop skills inlined `git …` calls. Two changes
Expand All @@ -271,19 +308,29 @@ fixed that:
(`magpie-vcs`) runs the *abstract* operation and detects the active
backend from the working copy.

Today: **Git is complete** (the default binding); Mercurial and
Subversion are real, detected extension points that raise an actionable
error naming their tracking issue until the full binding lands. Adding
a backend means replacing one `_UnimplementedBackend` with a concrete
`VCSBackend` subclass — detection, dispatch, the CLI, and every skill
that calls `magpie-vcs` pick it up automatically. Nothing else changes.

Tracking issues exist for the non-Git systems — Mercurial, Subversion
(ASF-critical: `svn.apache.org` and the release `dist.apache.org` area
are SVN), Jujutsu, Fossil, Perforce, Bitbucket, SourceHut — so the
extension points are public and labelled, not hypothetical.

### 5. Project governance
Today: **Git is complete** (the default binding); Mercurial
([#601](https://github.com/apache/magpie/issues/601)) and Subversion
([#602](https://github.com/apache/magpie/issues/602)) are real, detected
extension points that raise an actionable error naming their tracking
issue until the full binding lands. Adding a backend means replacing one
`_UnimplementedBackend` with a concrete `VCSBackend` subclass —
detection, dispatch, the CLI, and every skill that calls `magpie-vcs`
pick it up automatically. Nothing else changes.

Tracking issues exist, labelled `good first issue`, for the rest of the
non-Git systems:
[Mercurial](https://github.com/apache/magpie/issues/601),
[Subversion](https://github.com/apache/magpie/issues/602) (ASF-critical:
`svn.apache.org` and the release `dist.apache.org` area are SVN; the
full ASF SVN surface is [#608](https://github.com/apache/magpie/issues/608)),
[Jujutsu](https://github.com/apache/magpie/issues/603),
[Fossil](https://github.com/apache/magpie/issues/604), and
[Perforce](https://github.com/apache/magpie/issues/605) — so the
extension points are public and labelled, not hypothetical. (The
Bitbucket and SourceHut forges, which carry their own VCS, are tracked
under the forge axis above.)

### 6. Project governance

Vendor neutrality extends to *how a project is run*, not just to its
tooling. Each adopting project picks which modes run and how much
Expand Down Expand Up @@ -344,8 +391,9 @@ coverage without pretending one team can implement an open-ended set.
|---|---|---|---|
| LLM backend | ✅ by construction | Claude Code, Ollama, vLLM, Apache-hosted, Bedrock, direct Anthropic | Any endpoint meeting the capability floor + privacy gate |
| Agentic runtime | ✅ by construction (`AGENTS.md` standard) | Claude Code; community use under Codex, Cursor, Gemini CLI, Copilot, OpenCode, Kiro | Runtime adapters [#313–#322](https://github.com/apache/magpie/issues?q=is%3Aissue+state%3Aopen+adapter+in%3Atitle) |
| Forge / tracker | ✅ by construction | GitHub, Jira; mail/CVE/scan/relay via adapter contracts | GitLab [#305](https://github.com/apache/magpie/issues/305), Forgejo/Gitea [#310](https://github.com/apache/magpie/issues/310), Pagure [#312](https://github.com/apache/magpie/issues/312), Bitbucket [#306](https://github.com/apache/magpie/issues/306), SourceHut [#607](https://github.com/apache/magpie/issues/607), Bugzilla [#302](https://github.com/apache/magpie/issues/302) |
| Source control (VCS) | ✅ by construction | **Git (complete)** | Mercurial, Subversion (detected); Jujutsu, Fossil, Perforce, Bitbucket, SourceHut (tracked) |
| Forge / tracker | ✅ by construction | GitHub, Jira; CVE/scan/relay via adapter contracts | GitLab [#305](https://github.com/apache/magpie/issues/305), Forgejo/Gitea [#310](https://github.com/apache/magpie/issues/310), Pagure [#312](https://github.com/apache/magpie/issues/312), Bitbucket [#606](https://github.com/apache/magpie/issues/606), SourceHut [#607](https://github.com/apache/magpie/issues/607), Bugzilla [#302](https://github.com/apache/magpie/issues/302) |
| Communication channels | ✅ by construction | PonyMail / mail-archive reads | mbox [#304](https://github.com/apache/magpie/issues/304), IMAP [#303](https://github.com/apache/magpie/issues/303), Mailman 3 [#306](https://github.com/apache/magpie/issues/306); Discourse [#307](https://github.com/apache/magpie/issues/307), Zulip [#308](https://github.com/apache/magpie/issues/308), Matrix [#309](https://github.com/apache/magpie/issues/309) |
| Source control (VCS) | ✅ by construction | **Git (complete)** | Mercurial [#601](https://github.com/apache/magpie/issues/601), Subversion [#602](https://github.com/apache/magpie/issues/602)/[#608](https://github.com/apache/magpie/issues/608) (detected); Jujutsu [#603](https://github.com/apache/magpie/issues/603), Fossil [#604](https://github.com/apache/magpie/issues/604), Perforce [#605](https://github.com/apache/magpie/issues/605) (tracked) |
| Project governance | ✅ by construction | ASF + non-ASF adopter profiles | Adopter config (modes, thresholds) |

✅ "by construction" means the workflows carry no vendor assumption;
Expand Down
90 changes: 90 additions & 0 deletions tools/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [AGENTS — conventions for `tools/`](#agents--conventions-for-tools)
- [Every tool is a directory with a README](#every-tool-is-a-directory-with-a-readme)
- [Refresh the cross-references when tools change](#refresh-the-cross-references-when-tools-change)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->

# AGENTS — conventions for `tools/`

Operational context for everything under `tools/`. It is loaded in
addition to the repository-wide [`/AGENTS.md`](../AGENTS.md), which still
governs everything (commit trailers, placeholder convention,
privacy/security posture). Where the two overlap, the repo-wide
`AGENTS.md` wins.

A **tool** is the only layer that is allowed to know a specific vendor
exists — the generic skills target capabilities, and each tool fulfils a
capability for one concrete backend. See
[`docs/vendor-neutrality.md`](../docs/vendor-neutrality.md) for how the
skills / tools / capabilities split delivers vendor neutrality, and
[`docs/labels-and-capabilities.md`](../docs/labels-and-capabilities.md)
for the capability taxonomy.

## Every tool is a directory with a README

Each tool lives at `tools/<name>/` and **must** have a `README.md` that
declares, up front:

1. **Its capability** — a line of the exact form

```markdown
**Capability:** capability:NAME
```

(multi-value: `capability:NAME + capability:NAME`), with NAME drawn
from the taxonomy in `docs/labels-and-capabilities.md`. Substrate
bridges default to `capability:setup`.

2. **Its prerequisites** — a `## Prerequisites` section stating what the
tool needs *before it can run*, so an adopter never discovers a
missing dependency at first invocation. Cover the bullets that apply:

- **Runtime** — e.g. `Python 3.11+ run via uv`, `Bash + coreutils`,
`Python stdlib only`, `Node.js 20+`. State it even when trivial.
- **CLIs** — external commands the tool shells out to (`gh`, `git`,
`svn`, `jq`, `groovy`, `docker`/`podman`, `bubblewrap`, `socat`, …),
or `None beyond the runtime`.
- **Credentials / auth** — `gh auth status`, an OAuth token at a
home-dir path, an API-token env var, …, or `None`.
- **Network** — the hosts it reaches (`api.github.com`,
`*.apache.org`, a JIRA host, …), or state that it runs fully
offline / on local files.
- **Optional** — real optional dependencies or features only.

Keep it factual and tight (≈3–6 bullets); never invent a dependency.
A pure interface-spec tool (an adapter *contract* with no executable
code) says so and defers concrete prerequisites to its adapters.

Both are **HARD** checks in
[`tools/skill-and-tool-validator`](skill-and-tool-validator/) — a tool
README missing either the `**Capability:**` line or the
`## Prerequisites` section fails `skill-and-tool-validate` (and the
`prek` / pre-commit hook that runs it).

## Refresh the cross-references when tools change

The tool inventory is referenced from a few hand-maintained places.
When you **add, rename, remove, or re-scope** a tool — or change which
backends it supports or the tracking issues behind its extension
points — review and refresh:

- [`docs/labels-and-capabilities.md`](../docs/labels-and-capabilities.md)
— the *Capability to tool map* row. The validator's `capability-sync`
check enforces that every tool with a `**Capability:**` declaration has
a matching row, so this one fails the build if you forget it.
- [`docs/vendor-neutrality.md`](../docs/vendor-neutrality.md) — the
per-axis extension-point citations (the axis prose, the
*contract tool* table, and the *Status at a glance* table). This list
is **not** machine-checked; it is the public explanation of which
backends work today and which are open extension points, so it has to
be refreshed by hand whenever a tool or its tracking issue changes.

When in doubt about whether a doc references the tool you touched, grep
`docs/` for the tool name before opening the PR.
8 changes: 8 additions & 0 deletions tools/agent-guard/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [agent-guard](#agent-guard)
- [Prerequisites](#prerequisites)
- [Guards](#guards)
- [Per-command overrides](#per-command-overrides)
- [Wiring](#wiring)
Expand All @@ -28,6 +29,13 @@ It is **stdlib-only** and is invoked directly as
few milliseconds for any command that is not a guarded `gh` / `git commit` /
`git push`.

## Prerequisites

- **Runtime:** Python stdlib only — the hook runs as `python3 .../agent_guard/__init__.py` (3.11+), never via `uv`, so it needs no built/installed environment. The test suite runs under `uv run --project tools/agent-guard pytest`.
- **CLIs:** `git` and `gh` — the guards shell out (via `ctx.run`) to inspect commits, branch state, and GitHub Actions runs. None otherwise.
- **Credentials / auth:** None. The guards read local `git` / `gh` state; `gh` must be on `PATH` for the `mark-ready` guard's Actions lookup.
- **Network:** None in the hot path; the `mark-ready` guard reaches `api.github.com` (via `gh`) when it checks for awaiting-approval Actions runs.

## Guards

**Bundled** (shipped with the engine — universal `git` hygiene, on for every
Expand Down
8 changes: 8 additions & 0 deletions tools/agent-isolation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [`tools/agent-isolation/` — secure agent setup helpers](#toolsagent-isolation--secure-agent-setup-helpers)
- [Prerequisites](#prerequisites)
- [Files](#files)
- [Usage at a glance](#usage-at-a-glance)
- [Referenced by](#referenced-by)
Expand All @@ -23,6 +24,13 @@ under `tools/cve-tool-vulnogram/` and `tools/gmail/oauth-draft/`) — these are
plain shell scripts plus a TOML manifest of pinned upstream
versions.

## Prerequisites

- **Runtime:** Bash + coreutils — this directory is plain shell scripts plus a TOML manifest, not a Python project (the `pyproject.toml` ships only the test harness, which runs under Python 3.11+ via `uv`). `claude-term-bg.sh` uses `python3` / `python` for one heuristic and falls back to calm when absent.
- **CLIs:** `jq` (required by `check-tool-updates.sh` and the status-line scripts), `curl` (the update check), `git` (status line / git hooks), and `gh` (optional — status-line PR title). The secure setup itself installs the pinned `bubblewrap` and `socat` (via `apt-get`) and `@anthropic-ai/claude-code` (via `npm`).
- **Credentials / auth:** None for these helpers; the wrapped `claude` session authenticates on its own (and `claude-iso.sh` deliberately strips credential-shaped env vars).
- **Network:** `api.github.com` and `www.dest-unreach.org` (the release checks in `check-tool-updates.sh`); the install step also reaches the apt and npm registries.

## Files

| File | Purpose |
Expand Down
8 changes: 8 additions & 0 deletions tools/apache-projects/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [`tools/apache-projects/`](#toolsapache-projects)
- [Prerequisites](#prerequisites)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

Expand All @@ -23,3 +24,10 @@ roster-resolution paths in the security skills. For ASF projects it
is a mandatory pre-flight prerequisite, installed from the latest
`main` of `apache/comdev`. See [`tool.md`](tool.md) for the
operation catalogue, setup, and the track-`main` install contract.

## Prerequisites

- **Runtime:** Node.js 20+ — the backing tool is the official `apache/comdev` `apache-projects-mcp` server (a Node.js package), run as `node .../index.js`. This directory ships no code of its own.
- **CLIs:** `git` (clone the `apache/comdev` checkout), `npm` (install the server's deps), `node` (run it).
- **Credentials / auth:** None — the server is read-only and unauthenticated; every field it returns is already public.
- **Network:** `projects.apache.org` (the public JSON feeds the server fetches at run time); `github.com` to clone and track `apache/comdev` at `main`.
Loading