Python executable from config file

[fnattino]

Hi there!

I encountered the same issue as #517, also for a similar (the same?) use case.

I am running Dask Jobqueue on a SLURM system where all Python applications have to run inside (Apptainer) containers. Thus, I would like to be able to configure Dask jobqueue to use a containerized Python executable.

Currently, I can specify a custom Python path using the Dask Jobqueue Python API, but I cannot do it via the config file. It would be nice to allow for such feature, especially for users who use the Jupyter dask-lab extension, which relies uniquely on the values from the config files.

Introducing the changes as in this PR allows me to set the following in the Dask config file, so that all workers spawned by the cluster run inside containers:

jobqueue:
  slurm:
    python: 'apptainer exec oras://ghcr.io/fnattino/test-jupyterdask-image-apptainer:latest python'

[guillaumeeb]

Thanks! I think you need to reformat some files using black.

[guillaumeeb]

Thanks, merging!