Repository navigation
Description
Important
- We had already decided to use contributor-docs as source of truth here instead of solely based on Breeze CLI: [GSOC 2026] Airflow Contribution & Verification Agent Skills #62500 (comment)
- Please add the link to your PoC (draft PR, feature-branch link on your fork, etc.) if you have at the very beginning of your proposal so that we don’t overlook it.
Body
Background
Apache Airflow’s Breeze environment is the de facto way to reproduce CI, run tests, and verify changes locally. It encapsulates complex tooling (Docker, integrations, static checks, tests, system verification) behind a single, consistent developer interface.
However, modern AI coding tools (e.g. Claude Code, Gemini CLI, GitHub Copilot–style agents) currently treat Airflow’s repo like any generic Python project. They rarely:
- Understand whether they are running inside or outside Breeze.
- Choose the correct commands for host vs. container.
- Follow the same workflows that Airflow contributors actually use (e.g.
prek,breeze shell,breeze start-airflow).
We already expose some information through docs (e.g. AGENTS.md), but this mostly inflates the context window rather than giving agents a structured, machine-usable interface to Breeze.
This project aims to bridge that gap by creating an “Airflow Breeze Contribution / Contribution Verification” AI skill (final name TBD) that systematically encodes common contribution workflows and makes them reliably executable and testable by AI agents.
Goal
The overarching goal is to make AI tools:
Breeze-aware: able to detect whether they are running inside or outside Breeze and act accordingly.
In practice, this means that for a typical contributor PR, an AI agent can:
- Run the right static checks.
- Run the right subset of tests in Breeze.
- Spin up Airflow and verify system behavior for a Dag representing the change (nice-to-have).
- Do all of the above while respecting host/container boundaries.
Additionally, the solution should be consistency-focused, meaning that we want to keep Breeze CLI as the single source of truth for agent skills. This can be achieved by auto-syncing CLI docstrings and behaviors into the AI skill using existing tooling (e.g. prek), ensuring that the skill definitions always reflect the current state of the Breeze CLI.
Core Tasks
1. Environment Awareness & Detection
- Design and implement a simple, robust mechanism for the agent skills to detect:
- “Host” vs “inside Breeze container”.
- Relevant environment variables, markers, or file paths that indicate context.
- Encode decision logic for when to run:
- Host-only commands (e.g.
breeze shell,breeze start-airflow,gitoperations). - Container-only commands (e.g.
pytest,airflow ...).
- Host-only commands (e.g.
- Provide a clear API/contract that AI tools can call to query current context and get recommended commands.
Note: Maybe we need to add some explicit markers, files in the repo, or write a small helper script that can be called to determine context in a reliable way. Or maybe we can rely on existing environment variables or filesystem cues. This is an open design question to explore.
2. Modeling Core Contributor Workflows as Skills
Based on the three scenarios described, define and implement skills that represent common contribution flows:
Scenario 1: Static checks pass
- Stage changes (
git add ...). - Run
prek. - Collect and surface failures in a structured way so that an agent can fix them.
Scenario 2: Unit tests in Breeze
- Start or attach to a Breeze container with
breeze shellorbreeze exec. - Run
pytestwith a targeted module/test path (not the whole suite). - Then the agent can inspect results and decide on next steps (e.g. fix code, exit Breeze).
3. Syncing with Breeze CLI as Source of Truth (via prek)
- Investigate existing Breeze CLI docstrings and structure.
- Define a mapping from Breeze commands (and their docstrings) to skill definitions, paths, and parameters.
- Implement a
prekhook that:- Generates or updates the agent skills definition files from Breeze CLI docstrings.
- Fails when drift is detected (e.g. a command changed but the skill spec was not updated).
- Integrate these checks into existing static check pipelines so the skills stay in sync automatically.
4. Evaluation & Test Harness
- Design a testable user scenario or “exam” that simulates a typical contribution workflow (e.g. fixing a simple bug, adding a small feature) to verify that the added skills work as intended.
- Add unit tests for any additional scripts or helper functions created.
5. Documentation & Developer Guide
- Add or extend documentation (e.g.
AGENTS.md, Breeze docs) to:- Describe the new Breeze-aware skills.
- Show example workflows for human contributors and AI tools.
- Document how other tools can integrate with the skills (e.g. path to spec file, key commands).
Advanced Tasks (Optional / Stretch Goals)
Scenario: System behavior verification
- Write a Dag representing the feature/bugfix being contributed (or use an existing one).
- Run
breeze start-airflow(with--integrationwhen needed). - Trigger the Dag via CLI (instead of UI) and wait for completion.
- Inspect logs/status to determine success/failure from the TaskInstance logs.
- Inspect logs/status from all the component services (scheduler, api-server, triggerer, etc) to determine if there are any underlying issues.
- The agent can then decide to fix code, fix the Dag, or exit Breeze based on the results.
Expected Outcome
By the end of the project, we expect:
- A Breeze-aware AI skill that can:
- Detect host vs. container context.
- Choose appropriate commands and environment transitions.
- The AI toolings will be "smart-enough" to handle the core workflows for contributions, including:
- Static checks with
prek. - Targeted unit tests in Breeze.
- Continue iterating based on results (e.g. fix code, fix tests, exit).
- Static checks with
- A sync mechanism (likely using
prek) that:- Keeps Breeze CLI and the skill definitions in sync.
- Fails CI when they diverge, ensuring Breeze remains the single source of truth.
- Initial evaluation “exam(s)” and test harnesses that:
- Verify that an implementation of the skill behaves correctly on at least the core scenarios.
- Updated documentation explaining how contributors and AI tools can make use of the new capability.
A successful project will make it much easier for future AI tooling (IDEs, CLIs, bots) to interact with Breeze in a reliable and Airflow-native way, increasing contributor productivity and lowering the barrier to entry.
Recommended Skills
- Programming & Tooling
- Solid Python skills (CLI tools, packaging, basic testing).
- Familiarity with Docker and containerized development environments.
- Experience with writing or using CLIs and handling subprocesses.
- Dev Workflow & CI
- Understanding of typical open source contribution workflows (git, PRs, static checks, unit tests, pre-commit).
- Exposure to CI systems and concepts of reproducible environments.
- AI/Agents
- Interest in or experience with AI coding assistants, Agent Skills, tool-calling, or agent frameworks.
- Comfort reasoning about what “smart enough” means in terms of concrete, testable behaviors.
- Airflow/Breeze (Nice to Have)
- Basic knowledge of Apache Airflow concepts (Dags, tasks, operators).
- Prior use of Breeze for development or testing is a plus, but not strictly required.
Motivation to work at the intersection of developer experience, tooling, and AI is more important than prior deep expertise in all of these areas.
Mentors
- Jason Liu (GitHub: @jason810496, Slack: Zhe-You(Jason) Liu)
- Jarek Potiuk (GitHub: @potiuk, Slack: Jarek Potiuk)
#gsocSlack Channel in Apache Airflow workspace
Learning Materials
- Airflow Breeze documentation: https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst
- Recent Airflow Dev Mailing List discussion regarding Agent Skills/ Agents:
- Airflow
prek(pre-commit) hooks entrypoint: https://github.com/apache/airflow/blob/main/.pre-commit-config.yaml - Modern Python monorepo for Apache Airflow (by Jarek): https://medium.com/apache-airflow/modern-python-monorepo-for-apache-airflow-part-1-1fe84863e1e1
pre-commit: https://pre-commit.com/prek: https://github.com/j178/prek
Committer
- I acknowledge that I am a maintainer/committer of the Apache Airflow project.
Activity
Pyasma commentedon Feb 26, 2026
hi @jason810496,
I’m interested in the “Airflow Breeze Contribution / Contribution Verification” AI skill project and wanted to share my current understanding before going deeper.
My reading is that the core goal is to encode existing Airflow contributor workflows (static checks, targeted tests, environment transitions) into Breeze-aware, machine-readable skills that AI tools (Gemini, Claude Code, GPT, etc.) can reliably execute.
The main challenge seems to be making the agent aware of the host vs Breeze-container environment and enforcing correct command boundaries, while keeping the Breeze CLI as the single source of truth via auto-syncing (for example, through prek).
System-level DAG execution for end-to-end verification, where an agent can trigger and validate real Airflow behavior in the same way a contributor would, using these skills.
I want to confirm that this framing matches the intent of the project before drafting a more detailed proposal.
jason810496 commentedon Feb 27, 2026
Hi @Pyasma , yes, your understanding is correct.
Pyasma commentedon Feb 27, 2026
@jason810496 thanks for confirming
I will work on the proposal
Where i can find template for the proposal ? ( is there a specific template for it )
https://community.apache.org/gsoc/ -> there is template in this page can i use that?
jason810496 commentedon Feb 27, 2026
Sure, feel free to use any template as long as your proposal is clear. The format doesn’t really matter, but the thinking behind the solution is important. Thanks.
KatalKavya96 commentedon Feb 27, 2026
Hello @jason810496,
I have carefully gone through the full description of the Airflow Contribution & Verification Agent Skills project and the related discussions. From my understanding, the core challenge is not just exposing Breeze commands to AI tools, but designing a structured, Breeze-aware abstraction layer that:
Having worked on Apache Airflow previously (including Breeze-based development, local testing, and debugging inside containers) I can directly relate to the host/container boundary issues and the importance of reproducible contributor workflows.
I am currently working on a structured proposal that focuses on:
I would really appreciate staying in close touch while refining the proposal to ensure it aligns well with the project’s expectations.
Looking forward to contributing meaningfully to this effort.
Thank You
Abhishekmishra2808 commentedon Mar 8, 2026
@jason810496 @potiuk, I have few questions from the community before making a proposal,
jason810496 commentedon Mar 9, 2026
The main question here is the long-term maintainability of the Airflow project itself. That’s why it would be better to have a single source of truth for agent skills, to avoid agent skills becoming outdated over time.
It’s totally fine to have an intermediate schema for Breeze and agent skills, as long as we consider the long-term maintainability effort and make sure the mechanism itself is easy enough for maintainers to keep the agent skills synchronized with the Breeze CLI.
For the second question, in most cases, Airflow contributors/maintainers will use the Breeze CLI directly installed from the
mainbranch. The Breeze CLI supports the --use-airflow-versionflag for some common commands.To sum up, the latest Breeze CLI itself should always be able to work with any Airflow version. We just need to make sure the mechanism is able to keep agent skills up to date with changes to the Breeze CLI -- or, more accurately, that changes to the Breeze CLI and changes to the agent skills are part of the same atomic Git commit history from my perspective.
sonianuj287 commentedon Mar 9, 2026
Hi, I’m exploring this issue and would like to work on it.
I’ve started looking into the relevant module and will
share an implementation soon. Any pointers are welcome.
Abhishekmishra2808 commentedon Mar 9, 2026
@jason810496 Thanks
Looking at the Breeze structure, it seems feasible to derive the skill layer directly from the CLI definitions rather than maintaining a separate manually curated spec. A prek hook could generate the skill definitions from the Breeze command metadata and fail when the generated output diverges from the committed files. That would effectively enforce synchronization and ensure that any CLI change requires updating the corresponding agent skill in the same commit.
This also keeps the Breeze CLI as the authoritative interface while allowing the skill layer to remain a thin, generated abstraction over it.
One point I’m curious about: should the skill layer stay close to individual Breeze commands, or should it represent higher-level contributor workflows (e.g., static checks, targeted tests, environment transitions) that internally orchestrate multiple Breeze commands depending on context?
potiuk commentedon Mar 9, 2026
I think we should aim for something that will be both Human and Agents usable.
I would even go as far as to say that we should use the contributing guides as a single source of truth rather than breeze CLI. If anything I would focus on making sure that the workflows described for humans are always updated when we add new breeze command.
One thing that is important in those workflows. NOT EVERYTHING should be done with breeze. While breeze is good for repeatable environment and making sure that "works for me" syndrom does not kick in, a lot of the contributor's workflows - such us running tests or pre-commit hooks can be done without breeze.
In vast majority of cases this is enough to run tests for example:
Only when this is not working because some system dependency is missing (mysql librarires for example) we should fall back to breeze way of running tests. Also in case we see discrepancy between the CI and local tests.
And I think skills should reflect that.
And it's faster and easier to debug as well - as you can run it from the IDE (and agent can also debug it better I guess).
So I am wondering if the intermediate layer that should be the "source of truth" should be contributing docs.
Pyasma commentedon Mar 9, 2026
@potiuk oh this makes more sense. AGENTS.md has something similar mentioned as well.
i was thinking of something similar too when i was reading AGENTS.md (you have this mentioned there):
If uv tests fail with missing system dependencies, run the tests with breeze: breeze run pytest <tests> -xvsthis might also be easier to maintain long term since the agent skills would reflect the contributing docs, just in a more structured way for agents
Abhishekmishra2808 commentedon Mar 9, 2026
From the discussion, it seems the important distinction is that the skill layer should represent contributor workflows, not just mirror the Breeze CLI commands. In that sense, Breeze becomes an execution mechanism for reproducibility, rather than the primary abstraction exposed to agents.So instead of a CLI-driven model, the system would likely look like a workflow-driven execution model, where the agent decides whether to run locally or fall back to Breeze depending on context.
For example, a typical “run tests” workflow could follow something like:
flowchart TD A[Code Change or PR Update] --> B[Run Static Checks] B --> C{Static Checks Pass?} C -- No --> D[Fix Lint or Formatting Issues] D --> B C -- Yes --> E[Run Tests Locally] E --> F{Dependencies Available?} F -- Yes --> G[Run Local Tests<br/>uv run --project distribution_folder pytest] F -- No --> H[Fallback to Breeze] H --> I[Start Breeze Environment<br/>breeze shell or breeze exec] I --> J[Run pytest inside container] G --> K[Analyze Test Results] J --> K K --> L{Tests Pass?} L -- No --> M[Fix Code or Tests] M --> E L -- Yes --> N[Ready for PR / CI Verification]Conceptually the architecture could then look like:
flowchart LR A[Contributing Guides] --> B[Workflow Definitions] B --> C[Agent Skill Layer] C --> D{Execution Strategy} D --> E[Local Execution] D --> F[Breeze Execution] E --> G[Local Commands<br/>uv run pytest, prek, etc.] F --> H[Breeze Commands<br/>breeze shell, breeze exec, airflow services]In this model:
One possible implication is that Breeze CLI metadata could still be used to derive or validate the Breeze execution layer, while the workflow abstraction remains aligned with the contributing documentation. right?
72 remaining items
Pyasma commentedon Apr 21, 2026
@gsemet I agree with your point about keeping things simple and using AGENTS.md as an entrypoint, that makes sense from a usability perspective.
But the main problem here is long term maintainability and whether contributors can actually trust the agent’s behavior. It is not just about exposing information, it is about making sure the agent consistently gets correct and minimal context.
We need to make sure:
So the challenge is finding a balance between keeping things simple and having a mechanism that keeps everything reliable and maintainable as the project evolves.
feat: Breeze-aware AI agent skills system (GSoC 2026 PoC)
feat: Breeze-aware AI agent skills system (GSoC 2026 PoC)
feat: Breeze-aware AI agent skills system (GSoC 2026 PoC)
feat: Breeze-aware AI agent skills system (GSoC 2026 PoC)
feat: Breeze-aware AI agent skills system (GSoC 2026 PoC)
jason810496 commentedon Apr 23, 2026
Just noted down related discussion in dev list that focus on user-facing docs and the actual code implementation. It's out of scope for this project, but it could be considered as post-gsoc project or follow-up project.
https://lists.apache.org/thread/zswfsdgg2qw6bnk8yhxbvpzzookr8pnx
It's also yet another kind of reflect changes from the source of truth, but the output will be user-facing docs or ADR.
Subham-KRLX commentedon Apr 24, 2026
Got it @jason810496 User docs/ADRs are explicitly marked as post GSoC in the proposal Keeping this PoC focused on agent skills only.
gsemet commentedon Apr 24, 2026
That is the new science of harness engineering !
It is pretty simple to have a mental representation of what is going on: each time you start a session, you bring a contractor on your project that knows nothing of your internal working (actually for an opensource project it does a bit), and you want to give him the minimal information map (the "starter kit" doc) and it will jump to the right section.
And you have a "budget" of ~100 rules you can give to your contractor on top of its internal model to adapt to your project, at any given point. That is why splitting task by tech or module is key.
That is half the job. The other half is to actually verify the job done (usually you have a coding agent that take a task and code it, and a second subagent that verify it)
Yesterday I did a simple change, I changed the place in these task files where the guidelines/rules are place (was after the acceptance criteria, now it is on top), and I get a huge bump in accuracy. That means the tasks needs to be organized correctly for agent to understand.
Everybody learns at the same time, that is a bit odd. But it does the job.
You can use superpowers or some other SDD framework, even plan mode works fine, but the agents.md priming is key for accuracy (respect of your rules).
Subham-KRLX commentedon Apr 24, 2026
That makes sense We will keep this PoC focused on the agent skills workflow for now and treat the docs/ADR reflection idea as a follow up after GSoC.
alok-kumar0421 commentedon Apr 24, 2026
Hi, I’ve added a PoC in my proposal and shared the repo as well.
Right now it shows how workflows can run differently based on host vs Breeze.
I’m improving it step by step and planning to make the workflows more structured.
Would really appreciate your feedback!
feat: Breeze-aware AI agent skills system (GSoC 2026 PoC)
feat: Breeze-aware AI agent skills system (GSoC 2026 PoC)
feat: Breeze-aware AI agent skills system (GSoC 2026 PoC)
cschanhniem commentedon Jul 8, 2026
Great project on structured agent interfaces for Breeze. Since this defines what agents can do in the Airflow repo, AGENTOWNERS could complement this as the policy layer — defining which agents may act on which paths, what requires human approval, and what's blocked entirely. AGENTS.md + AGENTOWNERS.yml together cover both the how (skills) and the what's-allowed (policy).