Skip to content

Fix scheduler refusing to start when [scheduler] only_idle is enabled - #73141

Closed
Eason09053360 wants to merge 0 commit into
apache:mainfrom
Eason09053360:fix-scheduler-only-idle-config-blocks-startup
Closed

Eason09053360 wants to merge 0 commit into
apache:mainfrom
Eason09053360:fix-scheduler-only-idle-config-blocks-startup

Conversation

@Eason09053360

Copy link
Copy Markdown
Contributor

Why

Enabling the documented [scheduler] only_idle option leaves the deployment without a scheduler. ARG_ONLY_IDLE resolved its default from config at import time, so the guard in scheduler_command.scheduler() could not tell an explicitly typed --only-idle from a config-supplied default. Since [scheduler] num_runs defaults to -1, the guard tripped on every invocation:

$ AIRFLOW__SCHEDULER__ONLY_IDLE=True airflow scheduler
The --only-idle flag requires --num-runs to be set to a positive number.   # exit 1

The same import-time default made the option impossible to switch off: action="store_true" only pushes towards True, so once config enabled it, no command line could get back to False — the opposite of the usual command-line-beats-config precedence.

The guard itself is only about usability. SchedulerJobRunner already ignores only_idle when the run limit is not positive (run_count >= self.num_runs > 0), so a non-positive num_runs is inert rather than dangerous.

What

airflow-core/src/airflow/cli/cli_config.py — ARG_ONLY_IDLE becomes tri-state: default=None with argparse.BooleanOptionalAction, so None means "user said nothing", and --no-only-idle can override a config-enabled default.

airflow-core/src/airflow/cli/commands/scheduler_command.py — _resolve_only_idle() reads [scheduler] only_idle when the command runs instead of when the parser is built. An explicit --only-idle without a positive --num-runs still raises SystemExit with the same message; the config-supplied case now logs a warning and starts. The warning names both --num-runs and [scheduler] num_runs, since the non-positive limit can come from either.

airflow-core/src/airflow/config_templates/config.yml — record that only_idle has no effect unless num_runs is positive.

Tests cover the three-way flag, that a config-enabled only_idle starts the scheduler and reaches SchedulerJobRunner, and that --no-only-idle wins over config. test_only_idle_help_survives_rich_markup_rendering asserts against rendered --help output rather than the raw help= kwarg, because the Rich-based help formatter silently swallows [section] text — the reason the help string spells the option scheduler.only_idle.


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

@shahar1 shahar1 left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall, could you please add a newsfragment?

@Eason09053360

Copy link
Copy Markdown
Contributor Author

LGTM overall, could you please add a newsfragment?

Added — airflow-core/newsfragments/73141.bugfix.rst .
Thanks for the review!

@Eason09053360
Eason09053360 force-pushed the fix-scheduler-only-idle-config-blocks-startup branch from 4e47001 to b56b5df Compare September 24, 2026 10:37
This was referenced Sep 25, 2026
@Eason09053360

Copy link
Copy Markdown
Contributor Author

cc @shahar1 friendly ping — I’ve added the newsfragment and rebased onto ‎main. Would you mind taking another look when you get a chance?
All checks are green. Thanks for reviewing!

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — a clean fix: resolving only_idle at run time instead of parser build time stops config-enabled deployments from tripping the explicit-flag guard, and --no-only-idle now overrides config as expected. I rebased onto main and dropped 73141.bugfix.rst: since the earlier review asked for it, AGENTS.md's newsfragment rules were tightened so airflow-core only takes significant entries, and the release notes pick bug fixes up from the PR. Approving with auto-merge on.


Drafted-by: Claude Code (Claude Opus 5); reviewed by @potiuk before posting

@potiuk

potiuk commented Nov 1, 2026

Copy link
Copy Markdown
Member

Apologies, @Eason09053360 — while rebasing this onto main I accidentally force-pushed main itself to your branch, which emptied the PR and GitHub closed it; maintainer edit access lapses on closed PRs, so I can't push the fix back. Your change is carried over unchanged (rebased, with you as the commit author, minus the bugfix newsfragment that AGENTS.md no longer allows) in #76262, with auto-merge on once it gets a committer's approval. If you'd rather keep it here, force-push d3328ae957 back to your branch and reopen this one, and I'll close the other. Sorry for the noise.


Drafted-by: Claude Code (Claude Opus 5); reviewed by @potiuk before posting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants