Binary flags vs `default=None`: semantics and issue with prompt

[janluke]

The current behavior is:

• if default is not provided, the option default is set to False
• if default is set to None, default is set to None.

I think this may be accidental behavior but it's not senseless: settings default=None you can force the prompt to require an explicit value to be provided (it'll be [y/n] rather than [y/N]).

Nonetheless, I noticed that if you provide default=None to a flag not having a secondary option, the prompt message is broken: it won't include "[y/n]" at the end.

Originally posted by @janluke in #1965 (comment)

[janluke]

This issue needs some discussion. Related issue:

• required=True does not work for flags #1978.

I considered two options:

1. forbid default=None for boolean flags; required=True would have no effect, since boolean flags would always have a default value.
2. allow for default=None; fix the already described prompt issue.

Let's explore different cases with default=None:

• prompt=False and required=False -- this means that the option has 3 possible states (None, False, True): in my opinion, such an option shouldn't be a boolean flag in the first place, it should be a ternary click.Choice

• prompt=True -- asking the value of a flag with a prompt is more reasonable but default=None has little to no relevance here: the only difference is that setting default=None the prompt won't have a default value

• required=True -- personally, I'd find weird a CLI with a required boolean flag (and no prompt for it) as I perceive boolean flags as the epitome of something optional; I'm aware this is not a real argument but I'm open to read about some real or hypothetical examples where this makes sense.

All this considered, I currently lean towards option 1 (but not strongly).

Additional note: allowing for default=None means accepting the fact that not providing default versus setting default=None have different semantics for boolean flags. Currently, we can rely on the presence/absence of the keyword default in the kwargs provided to option() but if the intention is that of removing **kwargs from signatures, the default argument will need to be set to a special MISSING value to distinguish it from None.

EDIT: added the additional note.

[davidism]

Didn't expect this to be related, but as part of rewriting the parser in #2205, I've found that the interaction between boolean flags, default, and flag_value is caught up in how the existing parser was constructed. I started untangling it and realized there was this open issue about the same thing.

[davidism]

@kdeldycke was this addressed by #2956?

[kdeldycke]

2. allow for default=None; fix the already described prompt issue.

This is now supported by #3030.

Let's explore different cases with default=None:

• prompt=False and required=False -- this means that the option has 3 possible states (None, False, True): in my opinion, such an option shouldn't be a boolean flag in the first place, it should be a ternary click.Choice
• prompt=True -- asking the value of a flag with a prompt is more reasonable but default=None has little to no relevance here: the only difference is that setting default=None the prompt won't have a default value
• required=True -- personally, I'd find weird a CLI with a required boolean flag (and no prompt for it) as I perceive boolean flags as the epitome of something optional; I'm aware this is not a real argument but I'm open to read about some real or hypothetical examples where this makes sense.

I covered all these cases in #3030.

These unittests should demonstrate the new (fixed) behavior: https://github.com/pallets/click/pull/3030/files#diff-e9b43a3662176468b449eee04c777e29d67212bfc0dc0ca42a03b32d970b0deaR491-R800

Additional note: allowing for default=None means accepting the fact that not providing default versus setting default=None have different semantics for boolean flags. Currently, we can rely on the presence/absence of the keyword default in the kwargs provided to option() but if the intention is that of removing **kwargs from signatures, the default argument will need to be set to a special MISSING value to distinguish it from None.

I also introduced a new UNSET sentinel to differentiate the absence of value from the value of absence: https://github.com/pallets/click/pull/3030/files#diff-0140337741a334b642a5170e3bd7c408eea4bb4394df9226cd033fae8f5c0a19R33-R48

@janluke Unless you spot something weird, I'll consider this issue addressed by #3030.