feat(cli): add structured exit codes for proper error signaling#1670
Closed
frankentini wants to merge 1 commit intoNVIDIA:mainfrom
Closed
feat(cli): add structured exit codes for proper error signaling#1670frankentini wants to merge 1 commit intoNVIDIA:mainfrom
frankentini wants to merge 1 commit intoNVIDIA:mainfrom
Conversation
Closes NVIDIA#1221 Adds an ExitCode enum with well-defined integer codes following Unix conventions: 0 = SUCCESS - normal completion 1 = RUNTIME_ERROR - general / uncaught errors 2 = USAGE_ERROR - invalid CLI arguments or parameter values 3 = CONFIG_ERROR - missing/invalid config files, fixer migrations 4 = PLUGIN_ERROR - missing API keys, bad generator, plugin config 5 = INTERRUPTED - user-initiated Ctrl-C / SIGINT Changes: - garak/exception.py: add ExitCode(IntEnum) with the six codes above - garak/cli.py: main() now returns an ExitCode int instead of None; replaces bare exit(1) calls with typed returns; catches specific exception subclasses (APIKeyMissingError, PluginConfigurationError, ConfigFailure) before the generic GarakException handler - garak/__main__.py: propagates cli.main() return to sys.exit() - tests/cli/test_cli_exit_codes.py: validates enum values, return types, and exit codes for success/failure scenarios
Contributor
|
DCO Assistant Lite bot: I have read the DCO Document and I hereby sign the DCO You can retrigger this bot by commenting recheck in this Pull Request |
Collaborator
|
Closing as duplicate of #1651, this can be reopened if that PR does not land. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #1221
Adds an
ExitCodeenum togarak/exception.pywith well-defined integer codes following common Unix conventions, and updatescli.main()to return them instead of using bareexit(1)calls or returningNone.Exit Codes
SUCCESSRUNTIME_ERRORGarakExceptionUSAGE_ERRORCONFIG_ERRORPLUGIN_ERRORINTERRUPTEDChanges
garak/exception.py: AddedExitCode(IntEnum)with the six codes abovegarak/cli.py:main()now returns anExitCodeint instead ofNone; replaced bareexit(1)calls with typed returns; catches specific exception subclasses (APIKeyMissingError,PluginConfigurationError,ConfigFailure) before the genericGarakExceptionhandler for more precise exit codesgarak/__main__.py: Propagatescli.main()return value tosys.exit()tests/cli/test_cli_exit_codes.py: Tests for enum values, return types, and exit codes for success paths and various failure scenariosMotivation
As noted in #1221, proper exit codes help wrapping entities (CI pipelines, orchestration tools, garak-as-a-service) distinguish between different failure modes without parsing stderr output. The codes follow the common Unix convention where 0 = success, 1 = general error, and 2 = usage error.