[v3-2-test] Docs: Add JWT authentication docs and strengthen security model (#64760)#64849
Merged
[v3-2-test] Docs: Add JWT authentication docs and strengthen security model (#64760)#64849
Conversation
… model (#64760) * Docs: Add JWT authentication docs and strengthen security model Add comprehensive JWT token authentication documentation covering both the REST API and Execution API flows, including token structure, timings, refresh mechanisms, and the DFP/Triggerer in-process bypass. Update the security model to: - Document current isolation limitations (DFP/Triggerer DB access, shared Execution API resources, multi-team not guaranteeing task-level isolation) - Add deployment hardening guidance (per-component config, asymmetric JWT keys, env vars with PR_SET_DUMPABLE protection) - Add "What is NOT a security vulnerability" section covering all categories from the security team's response policies - Fix contradicting statements across docs that overstated isolation guarantees or recommended sharing all config across components Update AGENTS.md with security model awareness so AI agents performing security research distinguish intentional design choices from actual vulnerabilities. * Fix spelling errors and use 'potentially' for DFP/Triggerer access - Add dumpable, sandboxing, unsanitized, XSS to spelling wordlist - Use 'potentially' consistently when describing Dag File Processor and Triggerer database access and JWT authentication bypass, since these are capabilities that Dag author code could exploit rather than guaranteed behaviors of normal operation * Add prek hook to validate security doc constants against config.yml New hook `check-security-doc-constants` validates that: - [section] option references in security RST files match config.yml - AIRFLOW__X__Y env var references correspond to real config options - Default values in doc tables match config.yml defaults - Sensitive config variables are listed (warning, not error, since the list is documented as non-exhaustive) Loads both airflow-core config.yml and provider.yaml files to cover all config sections (including celery, sentry, workers, etc.). Runs automatically when config.yml or security RST docs are modified. * Expand sensitive vars to full list with component mapping and auto-update Update security_model.rst sensitive config variables section: - List ALL sensitive vars from config.yml and provider.yaml files - Core vars organized in a table with "Needed by" column mapping each var to the components that require it (API Server, Scheduler, Workers, Dag File Processor, Triggerer) - Provider vars in a separate table noting they should only be set where the provider functionality is needed - Tables are auto-generated between AUTOGENERATED markers Update prek hook to auto-update the sensitive var tables: - Reads config.yml and all provider.yaml files - Generates RST list-table content for core and provider sensitive vars - Replaces content between markers on each run - Warns when new sensitive vars need component mapping added to the hook - Validates [section] option and AIRFLOW__X__Y references against config - Skips autogenerated sections when checking env var references * Clarify software guards vs intentional access in DFP/Triggerer Address issues raised in security discussion about the gap between Airflow's isolation promises and reality: - Clearly distinguish software guards (prevent accidental DB access) from the inability to prevent intentional malicious access by code running as the same Unix user as the parent process - Document the specific mechanisms: /proc/PID/environ, config files, _CMD commands, secrets manager credential reuse - Clarify that worker isolation is genuine (no DB credentials at all) while DFP/Triggerer isolation is software-level only - Add Unix user impersonation as a deployment hardening measure - Document strategic (API-based DFP/Triggerer) and tactical (user impersonation) planned improvements - Add warning about sensitive config leakage through task logs - Add guidance to restrict task log access * Docs: Improve security docs wording, extract workload isolation, recommend DagBundle - Reword DFP/Triggerer descriptions to clarify software guards vs intentional bypass - Extract workload isolation section from jwt_token_authentication into workload.rst - Recommend Dag Bundle mechanism (GitDagBundle) for DAG synchronization - Fix typo in public-airflow-interface.rst and broken backtick in jwt_token_authentication.rst - Update cross-references between security docs (cherry picked from commit 0a03b4e) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
1 task
vatsrahul1001
pushed a commit
that referenced
this pull request
Apr 8, 2026
… model (#64760) (#64849) * Docs: Add JWT authentication docs and strengthen security model Add comprehensive JWT token authentication documentation covering both the REST API and Execution API flows, including token structure, timings, refresh mechanisms, and the DFP/Triggerer in-process bypass. Update the security model to: - Document current isolation limitations (DFP/Triggerer DB access, shared Execution API resources, multi-team not guaranteeing task-level isolation) - Add deployment hardening guidance (per-component config, asymmetric JWT keys, env vars with PR_SET_DUMPABLE protection) - Add "What is NOT a security vulnerability" section covering all categories from the security team's response policies - Fix contradicting statements across docs that overstated isolation guarantees or recommended sharing all config across components Update AGENTS.md with security model awareness so AI agents performing security research distinguish intentional design choices from actual vulnerabilities. * Fix spelling errors and use 'potentially' for DFP/Triggerer access - Add dumpable, sandboxing, unsanitized, XSS to spelling wordlist - Use 'potentially' consistently when describing Dag File Processor and Triggerer database access and JWT authentication bypass, since these are capabilities that Dag author code could exploit rather than guaranteed behaviors of normal operation * Add prek hook to validate security doc constants against config.yml New hook `check-security-doc-constants` validates that: - [section] option references in security RST files match config.yml - AIRFLOW__X__Y env var references correspond to real config options - Default values in doc tables match config.yml defaults - Sensitive config variables are listed (warning, not error, since the list is documented as non-exhaustive) Loads both airflow-core config.yml and provider.yaml files to cover all config sections (including celery, sentry, workers, etc.). Runs automatically when config.yml or security RST docs are modified. * Expand sensitive vars to full list with component mapping and auto-update Update security_model.rst sensitive config variables section: - List ALL sensitive vars from config.yml and provider.yaml files - Core vars organized in a table with "Needed by" column mapping each var to the components that require it (API Server, Scheduler, Workers, Dag File Processor, Triggerer) - Provider vars in a separate table noting they should only be set where the provider functionality is needed - Tables are auto-generated between AUTOGENERATED markers Update prek hook to auto-update the sensitive var tables: - Reads config.yml and all provider.yaml files - Generates RST list-table content for core and provider sensitive vars - Replaces content between markers on each run - Warns when new sensitive vars need component mapping added to the hook - Validates [section] option and AIRFLOW__X__Y references against config - Skips autogenerated sections when checking env var references * Clarify software guards vs intentional access in DFP/Triggerer Address issues raised in security discussion about the gap between Airflow's isolation promises and reality: - Clearly distinguish software guards (prevent accidental DB access) from the inability to prevent intentional malicious access by code running as the same Unix user as the parent process - Document the specific mechanisms: /proc/PID/environ, config files, _CMD commands, secrets manager credential reuse - Clarify that worker isolation is genuine (no DB credentials at all) while DFP/Triggerer isolation is software-level only - Add Unix user impersonation as a deployment hardening measure - Document strategic (API-based DFP/Triggerer) and tactical (user impersonation) planned improvements - Add warning about sensitive config leakage through task logs - Add guidance to restrict task log access * Docs: Improve security docs wording, extract workload isolation, recommend DagBundle - Reword DFP/Triggerer descriptions to clarify software guards vs intentional bypass - Extract workload isolation section from jwt_token_authentication into workload.rst - Recommend Dag Bundle mechanism (GitDagBundle) for DAG synchronization - Fix typo in public-airflow-interface.rst and broken backtick in jwt_token_authentication.rst - Update cross-references between security docs (cherry picked from commit 0a03b4e) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add comprehensive JWT token authentication documentation covering both
the REST API and Execution API flows, including token structure, timings,
refresh mechanisms, and the DFP/Triggerer in-process bypass.
Update the security model to:
shared Execution API resources, multi-team not guaranteeing task-level
isolation)
JWT keys, env vars with PR_SET_DUMPABLE protection)
categories from the security team's response policies
guarantees or recommended sharing all config across components
Update AGENTS.md with security model awareness so AI agents performing
security research distinguish intentional design choices from actual
vulnerabilities.
and Triggerer database access and JWT authentication bypass, since
these are capabilities that Dag author code could exploit rather
than guaranteed behaviors of normal operation
New hook
check-security-doc-constantsvalidates that:the list is documented as non-exhaustive)
Loads both airflow-core config.yml and provider.yaml files to cover
all config sections (including celery, sentry, workers, etc.).
Runs automatically when config.yml or security RST docs are modified.
Update security_model.rst sensitive config variables section:
var to the components that require it (API Server, Scheduler, Workers,
Dag File Processor, Triggerer)
the provider functionality is needed
Update prek hook to auto-update the sensitive var tables:
Address issues raised in security discussion about the gap between
Airflow's isolation promises and reality:
from the inability to prevent intentional malicious access by code
running as the same Unix user as the parent process
_CMD commands, secrets manager credential reuse
while DFP/Triggerer isolation is software-level only
impersonation) planned improvements
(cherry picked from commit 0a03b4e)
Co-authored-by: Jarek Potiuk jarek@potiuk.com