-
Notifications
You must be signed in to change notification settings - Fork 183
Description
Please describe your wishes and possible alternatives to achieve the desired result.
I think that every datastructure that can be opened as dask can also be opened as not-dask like sparse has https://anndata.readthedocs.io/en/stable/generated/anndata.io.sparse_dataset.html and even categoricals within xarray have https://anndata.readthedocs.io/en/stable/generated/anndata.experimental.backed.CategoricalArray.html.
Sometimes, there is a use-case where the lazy-compute feature of dask is not necessary, and thus lazy-loading can be done without the overhead of dask.
I would propose updating read_lazy to have use_dask kwarg that uses dask in all instances where possible. The main change here to default behavior (i.e., True) would be that categoricals and nullables within Dataset2D would be dask arrays by default.
Other than that I would expect no change to defaults since even things like dense arrays are opened with dask instead of the zarr.Array or h5py.Dataset