Skip to content

[v3-2-test] Docs: Add JWT authentication docs and strengthen security model (#64760)#64849

Merged
potiuk merged 1 commit intov3-2-testfrom
backport-0a03b4e-v3-2-test
Apr 8, 2026
Merged

[v3-2-test] Docs: Add JWT authentication docs and strengthen security model (#64760)#64849
potiuk merged 1 commit intov3-2-testfrom
backport-0a03b4e-v3-2-test

Conversation

@github-actions
Copy link
Copy Markdown

@github-actions github-actions bot commented Apr 7, 2026

  • Docs: Add JWT authentication docs and strengthen security model

Add comprehensive JWT token authentication documentation covering both
the REST API and Execution API flows, including token structure, timings,
refresh mechanisms, and the DFP/Triggerer in-process bypass.

Update the security model to:

  • Document current isolation limitations (DFP/Triggerer DB access,
    shared Execution API resources, multi-team not guaranteeing task-level
    isolation)
  • Add deployment hardening guidance (per-component config, asymmetric
    JWT keys, env vars with PR_SET_DUMPABLE protection)
  • Add "What is NOT a security vulnerability" section covering all
    categories from the security team's response policies
  • Fix contradicting statements across docs that overstated isolation
    guarantees or recommended sharing all config across components

Update AGENTS.md with security model awareness so AI agents performing
security research distinguish intentional design choices from actual
vulnerabilities.

  • Fix spelling errors and use 'potentially' for DFP/Triggerer access
  • Add dumpable, sandboxing, unsanitized, XSS to spelling wordlist
  • Use 'potentially' consistently when describing Dag File Processor
    and Triggerer database access and JWT authentication bypass, since
    these are capabilities that Dag author code could exploit rather
    than guaranteed behaviors of normal operation
  • Add prek hook to validate security doc constants against config.yml

New hook check-security-doc-constants validates that:

  • [section] option references in security RST files match config.yml
  • AIRFLOW__X__Y env var references correspond to real config options
  • Default values in doc tables match config.yml defaults
  • Sensitive config variables are listed (warning, not error, since
    the list is documented as non-exhaustive)

Loads both airflow-core config.yml and provider.yaml files to cover
all config sections (including celery, sentry, workers, etc.).

Runs automatically when config.yml or security RST docs are modified.

  • Expand sensitive vars to full list with component mapping and auto-update

Update security_model.rst sensitive config variables section:

  • List ALL sensitive vars from config.yml and provider.yaml files
  • Core vars organized in a table with "Needed by" column mapping each
    var to the components that require it (API Server, Scheduler, Workers,
    Dag File Processor, Triggerer)
  • Provider vars in a separate table noting they should only be set where
    the provider functionality is needed
  • Tables are auto-generated between AUTOGENERATED markers

Update prek hook to auto-update the sensitive var tables:

  • Reads config.yml and all provider.yaml files
  • Generates RST list-table content for core and provider sensitive vars
  • Replaces content between markers on each run
  • Warns when new sensitive vars need component mapping added to the hook
  • Validates [section] option and AIRFLOW__X__Y references against config
  • Skips autogenerated sections when checking env var references
  • Clarify software guards vs intentional access in DFP/Triggerer

Address issues raised in security discussion about the gap between
Airflow's isolation promises and reality:

  • Clearly distinguish software guards (prevent accidental DB access)
    from the inability to prevent intentional malicious access by code
    running as the same Unix user as the parent process
  • Document the specific mechanisms: /proc/PID/environ, config files,
    _CMD commands, secrets manager credential reuse
  • Clarify that worker isolation is genuine (no DB credentials at all)
    while DFP/Triggerer isolation is software-level only
  • Add Unix user impersonation as a deployment hardening measure
  • Document strategic (API-based DFP/Triggerer) and tactical (user
    impersonation) planned improvements
  • Add warning about sensitive config leakage through task logs
  • Add guidance to restrict task log access
  • Docs: Improve security docs wording, extract workload isolation, recommend DagBundle
  • Reword DFP/Triggerer descriptions to clarify software guards vs intentional bypass
  • Extract workload isolation section from jwt_token_authentication into workload.rst
  • Recommend Dag Bundle mechanism (GitDagBundle) for DAG synchronization
  • Fix typo in public-airflow-interface.rst and broken backtick in jwt_token_authentication.rst
  • Update cross-references between security docs
    (cherry picked from commit 0a03b4e)

Co-authored-by: Jarek Potiuk jarek@potiuk.com

… model (#64760)

* Docs: Add JWT authentication docs and strengthen security model

Add comprehensive JWT token authentication documentation covering both
the REST API and Execution API flows, including token structure, timings,
refresh mechanisms, and the DFP/Triggerer in-process bypass.

Update the security model to:
- Document current isolation limitations (DFP/Triggerer DB access,
  shared Execution API resources, multi-team not guaranteeing task-level
  isolation)
- Add deployment hardening guidance (per-component config, asymmetric
  JWT keys, env vars with PR_SET_DUMPABLE protection)
- Add "What is NOT a security vulnerability" section covering all
  categories from the security team's response policies
- Fix contradicting statements across docs that overstated isolation
  guarantees or recommended sharing all config across components

Update AGENTS.md with security model awareness so AI agents performing
security research distinguish intentional design choices from actual
vulnerabilities.

* Fix spelling errors and use 'potentially' for DFP/Triggerer access

- Add dumpable, sandboxing, unsanitized, XSS to spelling wordlist
- Use 'potentially' consistently when describing Dag File Processor
  and Triggerer database access and JWT authentication bypass, since
  these are capabilities that Dag author code could exploit rather
  than guaranteed behaviors of normal operation

* Add prek hook to validate security doc constants against config.yml

New hook `check-security-doc-constants` validates that:
- [section] option references in security RST files match config.yml
- AIRFLOW__X__Y env var references correspond to real config options
- Default values in doc tables match config.yml defaults
- Sensitive config variables are listed (warning, not error, since
  the list is documented as non-exhaustive)

Loads both airflow-core config.yml and provider.yaml files to cover
all config sections (including celery, sentry, workers, etc.).

Runs automatically when config.yml or security RST docs are modified.

* Expand sensitive vars to full list with component mapping and auto-update

Update security_model.rst sensitive config variables section:
- List ALL sensitive vars from config.yml and provider.yaml files
- Core vars organized in a table with "Needed by" column mapping each
  var to the components that require it (API Server, Scheduler, Workers,
  Dag File Processor, Triggerer)
- Provider vars in a separate table noting they should only be set where
  the provider functionality is needed
- Tables are auto-generated between AUTOGENERATED markers

Update prek hook to auto-update the sensitive var tables:
- Reads config.yml and all provider.yaml files
- Generates RST list-table content for core and provider sensitive vars
- Replaces content between markers on each run
- Warns when new sensitive vars need component mapping added to the hook
- Validates [section] option and AIRFLOW__X__Y references against config
- Skips autogenerated sections when checking env var references

* Clarify software guards vs intentional access in DFP/Triggerer

Address issues raised in security discussion about the gap between
Airflow's isolation promises and reality:

- Clearly distinguish software guards (prevent accidental DB access)
  from the inability to prevent intentional malicious access by code
  running as the same Unix user as the parent process
- Document the specific mechanisms: /proc/PID/environ, config files,
  _CMD commands, secrets manager credential reuse
- Clarify that worker isolation is genuine (no DB credentials at all)
  while DFP/Triggerer isolation is software-level only
- Add Unix user impersonation as a deployment hardening measure
- Document strategic (API-based DFP/Triggerer) and tactical (user
  impersonation) planned improvements
- Add warning about sensitive config leakage through task logs
- Add guidance to restrict task log access

* Docs: Improve security docs wording, extract workload isolation, recommend DagBundle

- Reword DFP/Triggerer descriptions to clarify software guards vs intentional bypass
- Extract workload isolation section from jwt_token_authentication into workload.rst
- Recommend Dag Bundle mechanism (GitDagBundle) for DAG synchronization
- Fix typo in public-airflow-interface.rst and broken backtick in jwt_token_authentication.rst
- Update cross-references between security docs
(cherry picked from commit 0a03b4e)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
@boring-cyborg boring-cyborg bot added area:ConfigTemplates area:dev-tools backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch kind:documentation labels Apr 7, 2026
@potiuk potiuk marked this pull request as ready for review April 8, 2026 11:23
@potiuk potiuk merged commit 072fda7 into v3-2-test Apr 8, 2026
3 checks passed
@potiuk potiuk deleted the backport-0a03b4e-v3-2-test branch April 8, 2026 11:23
vatsrahul1001 pushed a commit that referenced this pull request Apr 8, 2026
… model (#64760) (#64849)

* Docs: Add JWT authentication docs and strengthen security model

Add comprehensive JWT token authentication documentation covering both
the REST API and Execution API flows, including token structure, timings,
refresh mechanisms, and the DFP/Triggerer in-process bypass.

Update the security model to:
- Document current isolation limitations (DFP/Triggerer DB access,
  shared Execution API resources, multi-team not guaranteeing task-level
  isolation)
- Add deployment hardening guidance (per-component config, asymmetric
  JWT keys, env vars with PR_SET_DUMPABLE protection)
- Add "What is NOT a security vulnerability" section covering all
  categories from the security team's response policies
- Fix contradicting statements across docs that overstated isolation
  guarantees or recommended sharing all config across components

Update AGENTS.md with security model awareness so AI agents performing
security research distinguish intentional design choices from actual
vulnerabilities.

* Fix spelling errors and use 'potentially' for DFP/Triggerer access

- Add dumpable, sandboxing, unsanitized, XSS to spelling wordlist
- Use 'potentially' consistently when describing Dag File Processor
  and Triggerer database access and JWT authentication bypass, since
  these are capabilities that Dag author code could exploit rather
  than guaranteed behaviors of normal operation

* Add prek hook to validate security doc constants against config.yml

New hook `check-security-doc-constants` validates that:
- [section] option references in security RST files match config.yml
- AIRFLOW__X__Y env var references correspond to real config options
- Default values in doc tables match config.yml defaults
- Sensitive config variables are listed (warning, not error, since
  the list is documented as non-exhaustive)

Loads both airflow-core config.yml and provider.yaml files to cover
all config sections (including celery, sentry, workers, etc.).

Runs automatically when config.yml or security RST docs are modified.

* Expand sensitive vars to full list with component mapping and auto-update

Update security_model.rst sensitive config variables section:
- List ALL sensitive vars from config.yml and provider.yaml files
- Core vars organized in a table with "Needed by" column mapping each
  var to the components that require it (API Server, Scheduler, Workers,
  Dag File Processor, Triggerer)
- Provider vars in a separate table noting they should only be set where
  the provider functionality is needed
- Tables are auto-generated between AUTOGENERATED markers

Update prek hook to auto-update the sensitive var tables:
- Reads config.yml and all provider.yaml files
- Generates RST list-table content for core and provider sensitive vars
- Replaces content between markers on each run
- Warns when new sensitive vars need component mapping added to the hook
- Validates [section] option and AIRFLOW__X__Y references against config
- Skips autogenerated sections when checking env var references

* Clarify software guards vs intentional access in DFP/Triggerer

Address issues raised in security discussion about the gap between
Airflow's isolation promises and reality:

- Clearly distinguish software guards (prevent accidental DB access)
  from the inability to prevent intentional malicious access by code
  running as the same Unix user as the parent process
- Document the specific mechanisms: /proc/PID/environ, config files,
  _CMD commands, secrets manager credential reuse
- Clarify that worker isolation is genuine (no DB credentials at all)
  while DFP/Triggerer isolation is software-level only
- Add Unix user impersonation as a deployment hardening measure
- Document strategic (API-based DFP/Triggerer) and tactical (user
  impersonation) planned improvements
- Add warning about sensitive config leakage through task logs
- Add guidance to restrict task log access

* Docs: Improve security docs wording, extract workload isolation, recommend DagBundle

- Reword DFP/Triggerer descriptions to clarify software guards vs intentional bypass
- Extract workload isolation section from jwt_token_authentication into workload.rst
- Recommend Dag Bundle mechanism (GitDagBundle) for DAG synchronization
- Fix typo in public-airflow-interface.rst and broken backtick in jwt_token_authentication.rst
- Update cross-references between security docs
(cherry picked from commit 0a03b4e)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant