interp - Prefer broadcast over reindex when possible

[Illviljan]

When a variable is a scalar it is faster to broadcast instead of using reindex. Use that when doing dataset interpolation.

• Closes reindex is very slow with small chunksizes #10054
• Tests added
• User visible changes (including notable bug fixes) are documented in whats-new.rst

import numpy as np
import dask.array as da

import xarray as xr

ds = xr.Dataset(
    data_vars={
        "variable_name": (
            "time",
            da.from_array(np.array(["test"], dtype=str), chunks=(1,)),
        )
    },
    coords={"time": ("time", np.array([0]))},
)

%timeit ds.interp(time=np.linspace(0, 10, 50))
%timeit ds.interp(time=np.linspace(0, 10, 100))
%timeit ds.interp(time=np.linspace(0, 10, 1000))

Main:

16.8 ms ± 620 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)
27.5 ms ± 2.62 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
214 ms ± 7.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

PR:

5.3 ms ± 138 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)
5.29 ms ± 87 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)
5.29 ms ± 188 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)

[dcherian]

This changes the semantics from copies to views. We'll have to manually deepcopy these vars to avoid confusing downstream errors.

[Illviljan]

broadcast_variables(*to_broadcast)[0].copy(deep=True) should do the trick I think.

[dcherian]

Yes that would work. This could be a good opportunity to look for optimizations in reindex if you have the bandwidth.

[Illviljan]

Using copy(deep=True) now. I couldn't see a noticeable difference with the example above.

Last time I followed the reindex path Dask was the bottleneck. Though I'm not very familiar with those functions.
I recall reindex was about the same speed as interpolation a few years ago.