Agncty CSIT system design needs to meet continuously expanding requirements of Agntcy projects including Slim Protocol, Agent Directory and many more.
The directory structure of the CSIT:
csit
├── benchmarks # Benchmark tests
│ ├── agntcy-slim # Benchmark tests for Slim
│ │ ├── Taskfile.yml # Tasks for Slim benchmark tests
│ │ └── tests
│ ├── agntcy-dir # Benchmark tests for ADS
│ │ ├── Taskfile.yml # Tasks for ADS benchmark tests
│ │ └── tests
│ ├── go.mod
│ ├── go.sum
│ └── Taskfile.yml
├── integrations # Integration tests
│ ├── agntcy-a2a # Integration tests for Rust/Go A2A interoperability
│ │ ├── fixtures
│ │ ├── Taskfile.yml # Tasks for A2A interoperability tests
│ │ └── tests
│ ├── agntcy-slim # Integration tests for [agntcy/slim](https://github.com/agntcy/slim)
│ │ └── topology # KinD multicluster topology tests
│ │ ├── config
│ │ ├── examples
│ │ ├── Taskfile.yml
│ │ └── tests
│ ├── agntcy-dir # Integration tests for [agntcy/dir](https://github.com/agntcy/dir)
│ │ ├── components
│ │ ├── examples
│ │ ├── manifests
│ │ ├── Taskfile.yml # Tasks for ADS integration tests
│ │ └── tests
│ ├── environment # Test environment helpers
│ │ └── kind
│ ├── Taskfile.yml # Tasks for integration tests
│ └── testutils # Go test utils
└── Taskfile.yml # Repository level task definitions
In the Taskfiles, all required tasks and steps are defined in a structured manner. Each CSIT component contains its necessary tasks within dedicated Taskfiles, with higher-level Taskfiles incorporating lower-level ones to efficiently leverage their defined tasks.
You can list all the task defined in the Taskfiles using the task -l or simply run task.
Representative entries from task -l include:
For the A2A suite, wildcard-backed task families are shown with * to keep the listing compact. Concrete invocations such as task integrations:a2a:test:rust-go:jsonrpc:go-go still work unchanged.
task: Available tasks for this project:
* benchmarks:directory:test: All ADS benchmark test
* benchmarks:slim:test: All Slim benchmark test
* integrations:a2a:test: Default A2A interoperability tests
* integrations:a2a:test:behavior:*: Shared behavior slice task. Use test:behavior:<core|unary-streaming|lifecycle|push-config|parity>.
* integrations:a2a:test:go-dotnet: Go and .NET interoperability test matrix for the current JSON-RPC and HTTP+JSON slices
* integrations:a2a:test:go-dotnet:*: Go and .NET transport task. Use test:go-dotnet:<jsonrpc|rest>.
* integrations:a2a:test:go-dotnet:*:*: Go and .NET per-case task. Use test:go-dotnet:<jsonrpc|rest>:<go-go|go-dotnet|dotnet-go|dotnet-dotnet>.
* integrations:a2a:test:python-go: Python v1.0 and Go interoperability test matrix across JSON-RPC, HTTP+JSON, and gRPC
* integrations:a2a:test:python-go:*: Python and Go transport task. Use test:python-go:<jsonrpc|rest|grpc>.
* integrations:a2a:test:python-go:*:*: Python and Go per-case task. Use test:python-go:<jsonrpc|rest|grpc>:<go-go|go-python|python-go|python-python>.
* integrations:a2a:test:rust-go: Rust and Go interoperability test matrix across JSON-RPC, HTTP+JSON, and gRPC
* integrations:a2a:test:rust-go:*: Rust and Go transport task. Use test:rust-go:<jsonrpc|rest|grpc>.
* integrations:a2a:test:rust-go:*:*: Rust and Go per-case task. Use test:rust-go:<jsonrpc|rest|grpc>:<go-go|go-rust|rust-go|rust-rust>.
* integrations:a2a:test:rust-python: Rust and Python v1.0 interoperability test matrix across JSON-RPC, HTTP+JSON, and gRPC
* integrations:slim:spire:deploy: Deploy SPIRE server
* integrations:slim:spire:remove: Remove SPIRE server
* integrations:slim:test:topology:cleanup:clusters: Undeploy agntcy slim test env for each values file in config/.gen
* integrations:slim:test:topology:cleanup:controller: Remove slim controller test env
* integrations:slim:test:topology:deploy:clusters: Deploy agntcy slim test env for each values file in config/.gen
* integrations:slim:test:topology:deploy:controller: Deploy slim controller
* integrations:slim:test:topology:generate:cluster-configs: Generates test environment configuration(s) for agntcy slim based on the provided test-setup descriptor
* integrations:slim:test:topology:run: Slim topology test
* integrations:slim:test:topology:report: Render HTML dashboard from topology test Ginkgo JSON report
* integrations:kind:create: Create kind cluster
* integrations:kind:destroy: Destroy kind cluster
* integrations:version: Get versionFocuses on testing interactions between integrated components.
Inside csit integrations directory contains the tasks that creating the test environment, deploying the components that will be tested, and running the tests.
├── agntcy-slim # Integration tests for [agntcy/slim](https://github.com/agntcy/slim)
│ └── topology # KinD multicluster topology tests
│ ├── config
│ ├── examples
│ ├── Taskfile.yml
│ └── tests
├── agntcy-dir # Integration tests for [agntcy/dir](https://github.com/agntcy/dir)
│ ├── components
│ ├── examples
│ ├── manifests
│ ├── Taskfile.yml # Tasks for ADS integration tests
│ └── tests
├── environment # Test environment helpers
│ └── kind
├── Taskfile.yml # Tasks for integration tests
└── testutils # Go test utils
We can launch tests using taskfile locally or in GitHub actions. Some suites are self-contained and run directly on the host, while others need a Kubernetes-based test environment.
Suites that deploy components on Kubernetes require creating a test cluster and deploying the test environment before running the tests. They require the following tools to be installed on the local machine:
task integrations:kind:create
task integrations:directory:test-env:deploy
task integrations:directory:testWe can focus on specified tests:
task integrations:directory:test:compilerAfter we finish the tests we can destroy the test cluster
task integratons:kind:destroyThe integrations/agntcy-a2a suite is self-contained and does not require a
Kind cluster, Helm, or repository sibling checkouts. It builds and runs small
Go, Rust, .NET, and Python fixtures locally across these suite slices:
| SDK pair | JSON-RPC | HTTP+JSON | gRPC | Root task |
|---|---|---|---|---|
| Rust/Go | Yes | Yes | Yes | task integrations:a2a:test:rust-go |
| Rust/.NET | Yes | Yes | No | task integrations:a2a:test:rust-dotnet |
| Go/.NET | Yes | Yes | No | task integrations:a2a:test:go-dotnet |
| Python/Go | Yes | Yes | Yes | task integrations:a2a:test:python-go |
| Rust/Python | Yes | Yes | Yes | task integrations:a2a:test:rust-python |
| Python/.NET | Yes | Yes | No | task integrations:a2a:test:python-dotnet |
To run it from the repository root you need:
- Taskfile
- Rust and Cargo
- Go for the Rust/Go, Python/Go, and Go/.NET slices
- Python 3.10+ for the Python/Go, Rust/Python, and Python/.NET slices
- .NET 8 SDK for the Rust/.NET, Go/.NET, and Python/.NET slices
task integrations:a2a:test
task integrations:a2a:test:rust-go:jsonrpc
task integrations:a2a:test:go-dotnet
task integrations:a2a:test:python-go:grpc
task integrations:a2a:test:python-go
task integrations:a2a:test:rust-python
task integrations:a2a:test:python-dotnet
task integrations:a2a:test:rust-python:jsonrpc:python-rustBecause the A2A Taskfile is now wildcard-backed, task -l shows families like integrations:a2a:test:rust-go:*: and integrations:a2a:test:rust-go:*:*: instead of one line per expanded case. Exact commands like task integrations:a2a:test:rust-go:jsonrpc:go-go still work.
The suite writes Ginkgo JSON and JUnit reports under integrations/agntcy-a2a/reports/ and refreshes an HTML dashboard at integrations/agntcy-a2a/reports/index.html. You can rebuild the dashboard from existing raw reports with task integrations:a2a:reports:dashboard.
The task commands above run on your machine. In contrast, gh workflow run
dispatches a workflow on GitHub-hosted runners using a remote branch, not your
local checkout.
Manual dispatch requires GitHub Actions to be enabled and an authenticated
account with write access to the target repository. Use agntcy/csit only
if you have that access; otherwise run locally or use your fork with Actions
enabled. Replace OWNER/csit below with the repository where you can dispatch.
Choose an existing suite-specific workflow:
| Workflow | Test environment | Manual inputs |
|---|---|---|
test-a2a |
Host-based A2A interoperability tests; no Kind cluster | None |
test-slim-integration |
SLIM topology tests; the workflow creates a Kind cluster | kind_version, override_slim_image_tag, override_slim_chart_tag, override_slim_controller_image_tag, override_slim_controller_chart_tag, override_slim_bindings_version |
Neither workflow accepts testenv. All SLIM inputs are optional:
kind_version defaults to 0.24.0, and the overrides default to empty strings.
Empty controller image/chart overrides fall back to override_slim_image_tag
and override_slim_chart_tag, respectively. Otherwise, omitted overrides leave
the defaults in the SLIM Taskfile
and bindings example image
in effect.
Run A2A with no inputs on the target repository's default branch:
gh workflow run test-a2a.yaml --repo OWNER/csitTo select another branch, replace BRANCH with a branch already present in that
repository. The workflow must support workflow_dispatch on the repository's
default branch and on the selected branch. --ref selects remote code; it does
not upload local changes. This SLIM example explicitly sets the current
topology image, chart, and bindings versions:
gh workflow run test-slim-integration.yaml --repo OWNER/csit --ref BRANCH \
-f kind_version=0.24.0 \
-f override_slim_image_tag=2.0.0 \
-f override_slim_chart_tag=v2.0.0 \
-f override_slim_bindings_version=2.0.0In the GitHub web UI, open Actions, select the suite workflow, then choose Run workflow, the branch, and any supported inputs.
Find the dispatched run and inspect its logs, replacing RUN_ID with its run ID:
gh run list --repo OWNER/csit --workflow test-slim-integration.yaml \
--branch BRANCH --event workflow_dispatch
gh run view RUN_ID --repo OWNER/csit --logOpen the run's summary and Artifacts section in the web UI to download reports when available:
| Workflow | Artifact name | Report directory in the runner workspace |
|---|---|---|
test-a2a |
a2a-interop-test-result-<matrix-id>, for example a2a-interop-test-result-rust-go |
integrations/agntcy-a2a/reports/ |
test-slim-integration |
slim-integration-test-result |
integrations/agntcy-slim/topology/reports/ |
These directories contain Ginkgo JSON, JUnit XML, and an index.html dashboard
when generated. For example, download the SLIM reports with:
gh run download RUN_ID --repo OWNER/csit --name slim-integration-test-result \
--dir ./reports/slim-integrationIf a job fails before generating or uploading reports, check its logs instead.
The publish-pages job only publishes from main; use the run artifacts for
feature-branch results.
Contributing your own tests to our project is a great way to improve the robustness and coverage of our testing suite. Follow these steps to add your tests.
- Fork and Clone the Repository
Fork the repository to your GitHub account. Clone your fork to your local machine.
git clone https://github.com/your-username/repository.git
cd repository- Create a New Branch
Create a new branch for your test additions to keep your changes organized and separate from the main codebase.
git checkout -b add-new-test- Navigate to the Integrations Directory
Locate the integrations directory where the test components are organized.
cd integrations- Add Your Test
Create a new sub-directory for your test if necessary, following the existing structure. For example, integrations/new-component. Add all necessary test files, such as scripts, manifests, and configuration files.
- Update Taskfile
Modify the appropriate suite's Taskfile.yml to include tasks for deploying and running your new test.
tasks:
test:env:new-component:deploy:
desc: Desription of deployig new component elements
cmds:
- # Command for deploying your components if needed
test:env:new-component:cleanup:
desc: Desription of cleaning up component elements
cmds:
- # Command for cleaning up your components if needed
test:new-component:
desc: Desription of the test
cmds:
- # Commands to set up and run your testReuse the existing suite workflow rather than creating a new workflow for each
test. For example, integrations/Taskfile.yml includes
agntcy-slim/topology/Taskfile.yml
as slim. The SLIM workflow calls
integrations:slim:test:topology:setup, integrations:slim:test:topology:run, and
integrations:slim:test:topology:report from the repository root. Add topology
cases and configuration to that suite and extend its tasks as needed. If a new
test needs a separate task file, expose it through the existing Taskfile
includes and add the task invocation and required configuration to the
corresponding workflow. Keep reports in the suite's configured artifact
directory so the existing upload step can collect them.
- Test Locally
Before pushing your changes, test them locally to ensure everything works as expected.
task integrations:kind:create
task integrations:new-componet:test-env:deploy
task integrations:new-component:test
task integrations:new-componet:test-env:cleanup
task integrations:kind:destroy- Document Your Test
Update the documentation in the docs folder to include details about your new test. Explain the purpose of the test, any special setup instructions, and how it fits into the overall testing strategy.
- Commit and Push Your Changes
Commit your changes with a descriptive message and push them to your fork.
git add .
git commit -m "feat: add new test for component X"
git push origin add-new-test- Submit a Pull Request
Go to the original repository on GitHub and submit a pull request from your branch. Provide a detailed description of what your test covers and any additional context needed for reviewers.
If we want to update the integrations/agntcy-dir/examples/dir/e2e/testdata directory we will need to add agntcy/dir as a remote and create a patch for it by diffing with the agntcy/dir repo
# add agntcy/dir as remote
git remote add -f dir https://github.com/agntcy/dir.git
# fetch dir
git fetch dir
# example of updating the integrations/agntcy-dir/examples/dir/e2e/testdata directory to the agntcy/dir main
git diff --binary HEAD:integrations/agntcy-dir/examples/dir/e2e/testdata dir/main:e2e/testdata | git apply --directory=integrations/agntcy-dir/examples/dir/e2e/testdataDistributed under Apache 2.0 License. See LICENSE for more information. Copyright AGNTCY Contributors (https://github.com/agntcy)