[API change] New way to create command line options.

[gabrieldemarmiesse]

This is backward compatible.

Old command line options:

class DebugOption(CommandLineOption):
    """
    Suppress warnings about missing observers and don't filter the stacktrace.

    Also enables usage with ipython --pdb.
    """

    @classmethod
    def apply(cls, args, run):
        """Set this run to debug mode."""
        run.debug = True


class LoglevelOption(CommandLineOption):
    """Adjust the loglevel."""

    arg = 'LEVEL'
    arg_description = 'Loglevel either as 0 - 50 or as string: DEBUG(10), ' \
                      'INFO(20), WARNING(30), ERROR(40), CRITICAL(50)'

    @classmethod
    def apply(cls, args, run):
        """Adjust the loglevel of the root-logger of this run."""

        try:
            lvl = int(args)
        except ValueError:
            lvl = args
        run.root_logger.setLevel(lvl)

New way (borrowed from the click API, intuitive for people who have used it):

@cli_option('-d', '--debug', is_flag=True)
def debug_option(args, run):
    """
    Set this run to debug mode.
    Suppress warnings about missing observers and don't filter the stacktrace.
    Also enables usage with ipython --pdb.
    """
    run.debug = True


@cli_option('-l', '--loglevel')
def loglevel_option(args, run):
    """
    Loglevel either as 0 - 50 or as string: DEBUG(10),
    INFO(20), WARNING(30), ERROR(40), CRITICAL(50)
    """

    try:
        lvl = int(args)
    except ValueError:
        lvl = args
    run.root_logger.setLevel(lvl)

ex = Experiment('My_exp', additional_cli_options=[debug_option, loglevel_option]

[coveralls]

Coverage increased (+0.09%) to 85.393% when pulling 0d5a95f on gabrieldemarmiesse:new_command_line_experience into d3fe102 on IDSIA:master.

[JarnoRFB]

looks good to me together with #569. From my side we could agree on making this the new default api.

[gabrieldemarmiesse]

@JarnoRFB Cool, I'm waiting a green light from @Qwlouse and then I'll update the docs.

[Qwlouse]

Interface looks good to me. The only thing I am unsure about is how to handle default commandline options. Currently you are using gather_commandline_options() to provide the defaults and you have a hardcoded list of options there. That makes it difficult both to extend the default options (with say a plugin to sacred), and to remove options from the user side. How about using a global list of default commandline options instead? (Only for the default options. User options would still be passed to the experiment).

[gabrieldemarmiesse]

@Qwlouse good point. Let me change that.

[gabrieldemarmiesse]

@Qwlouse I added DEFAULT_COMMAND_LINE_OPTIONS as per your recommendation.

[gabrieldemarmiesse]

@Qwlouse Are you ok with the new implementation?

[Qwlouse]

Looks very good, but I would add some validation to the CLI options to catch errors early and provide an informative error message.

[gabrieldemarmiesse]

I added the correctness checks. One last look maybe before we can merge?