Skip to content

Add KCP manifest and TL;DR summaries: 76% fewer agent navigation calls#4658

Open
totto wants to merge 6 commits intocrewAIInc:mainfrom
totto:main
Open

Add KCP manifest and TL;DR summaries: 76% fewer agent navigation calls#4658
totto wants to merge 6 commits intocrewAIInc:mainfrom
totto:main

Conversation

@totto
Copy link

@totto totto commented Mar 1, 2026

What this adds

A knowledge.yaml manifest at the repo root, plus four pre-built TL;DR summary files for the highest-traffic documentation sections.

KCP (Knowledge Context Protocol) is a lightweight standard that tells AI agents what a repository contains and how to navigate it efficiently — a structured index with intents and trigger phrases for each documentation unit, rather than having agents explore by trial and error.


Benchmark: 76% fewer tool calls

8 queries, same Haiku model, tool counts from the Anthropic API (not agent self-reports):

Query Baseline KCP Saved
Flows vs Crews — what's the difference? 14 2 12
Create my first agent and assign a task? 7 3 4
Create a custom tool? 8 3 5
Add memory to a crew? 7 3 4
Which LLM providers are supported? 17 5 12
Build a flow that triggers a crew? 15 2 13
Hierarchical crew with manager agent? 22 9 13
Add knowledge (RAG) to a crew? 33 3 30
Total 123 30 93

76% reduction. The standout is the RAG query (33 → 3, 91% reduction) — the baseline agent traversed the entire docs tree before finding knowledge.mdx, while KCP directed it straight to tools-memory-tldr.mdx via the rag crew trigger. The hierarchical crew query had the smallest gain (22 → 9) because answering it genuinely requires reading large reference files — KCP removes exploration overhead but cannot compress inherently dense content.


How it works

Without the manifest, an agent asked "how do I build a flow that triggers a crew?" typically:

  1. Reads README.md
  2. Globs docs/en/ directory
  3. Reads introduction.mdx
  4. Reads concepts/crews.mdx
  5. Reads concepts/flows.mdx
  6. ... 15 reads total

With knowledge.yaml, the agent:

  1. Reads knowledge.yaml — sees flows-tldr unit with triggers ["flow triggers crew", "start listen", "event driven workflow"]
  2. Reads docs/en/concepts/flows-tldr.mdx — gets @start/@listen/@router anatomy + a minimal Flow+Crew example in ~500 tokens
  3. Done. 2 tool calls.

Files added

knowledge.yaml                                  ← 16-unit KCP v0.3 manifest
docs/en/getting-started-tldr.mdx               ← Flows vs Crews, minimal working crew, installation
docs/en/concepts/agents-tasks-tldr.mdx         ← Agent role/goal/backstory, task structure, process types
docs/en/concepts/flows-tldr.mdx                ← @start/@listen/@router anatomy, minimal Flow+Crew example
docs/en/concepts/tools-memory-tldr.mdx         ← Built-in tools, @tool decorator, memory types, RAG quickstart
BENCHMARK.md                                    ← Full results with methodology
benchmark.py                                    ← Benchmark runner (reproduces the numbers above)

Static files only — zero runtime overhead, no new dependencies, nothing changes for existing users.


KCP spec

Full specification: github.com/cantara/knowledge-context-protocol (v0.3).

This is the fifth repo we have benchmarked with KCP. Results across all five:

Repo Type Reduction
Application codebase (plugin wizard) app code 74%
Documentation repo (infra agents guide) pure docs 53%
smolagents (HuggingFace) Python framework 73%
AutoGen (Microsoft) Python framework 80%
CrewAI Python framework 76%

Happy to adjust trigger phrases, add coverage for enterprise integrations or observability platforms, or restructure files to match your documentation conventions.


Note

Low Risk
Documentation-only additions plus an optional standalone benchmarking script; no changes to runtime library behavior, but the new script invokes local file/grep access when run.

Overview
Adds a repo-root knowledge.yaml Knowledge Context Protocol (KCP) manifest that indexes key CrewAI docs by intent/triggers and points to smaller summary units to reduce agent “search” reads.

Introduces several new TL;DR MDX pages (getting started, agents/tasks, flows, tools/memory/knowledge) and a BENCHMARK.md + benchmark.py runner to measure the reduction in tool calls when using the manifest.

Written by Cursor Bugbot for commit f532932. This will update automatically on new commits. Configure here.

- knowledge.yaml: 16-unit KCP v0.3 manifest with triggers, hints, and summary cross-links
- docs/en/getting-started-tldr.mdx: architecture summary + minimal crew example
- docs/en/concepts/agents-tasks-tldr.mdx: agent/task quick reference
- docs/en/concepts/flows-tldr.mdx: Flow decorators and state management reference
- docs/en/concepts/tools-memory-tldr.mdx: tools, memory, and knowledge/RAG reference
- benchmark.py: reproducible tool-call efficiency benchmark (baseline vs KCP)
- BENCHMARK.md: results — 76% reduction (123 → 30 tool calls across 8 queries)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace hardcoded /src/totto/crewAI paths in benchmark.py with a
REPO_ROOT variable derived from os.path.dirname(os.path.abspath(__file__)).
This makes the benchmark script reproducible on any machine.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… REPO_ROOT

BENCHMARK.md methodology section referenced /src/totto/crewAI — replaced
with generic description. benchmark.py read_file and grep_content handlers
now validate that requested paths fall within REPO_ROOT, preventing the
agent from reading files outside the repository.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ob_files

Previous startswith() check was vulnerable to path traversal
(/repo-name-suffix/ would pass /repo-name prefix). Replace with
pathlib.Path.relative_to() which correctly checks containment.

Also restricts glob_files to REPO_ROOT by filtering all results
through the same _within_repo() helper — the handler previously
had no path restriction at all.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
glob_files was silently discarding the declared base_dir parameter.
Now reads base_dir from tool_input and validates it is within REPO_ROOT
before using it (falls back to REPO_ROOT if not).

grep_content now uses -e flag so LLM-provided patterns starting with
'-' are treated as patterns rather than grep flags.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

agents-tasks-tldr and tools-memory-tldr each cover two concepts but
summary_of can only reference one unit ID. The previous manifest had
summary_unit on both tasks and memory pointing to the combined TL;DRs,
but those TL;DRs declared summary_of for only the first topic (agents,
tools). A strict KCP agent could skip the TL;DR when starting from
tasks or memory, defeating the optimisation.

Fix: remove summary_available/summary_unit from tasks and memory.
Instead add a relationships section with context links from each
combined TL;DR to its secondary topic unit, so agents can still
discover the TL;DR via the relationship graph without misusing
the summary_of field.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant