span(x) range doesn't always include correct maximum value

[JB0NK]

Problem

The span implementation creates ranges with step size 1.0 for floating-point data. That leads to scenarios where:

span([-2.73, 22.75])

returns

-2.73:1.0:22.27 

which includes the minimum but not the correct maximum value.

I know this is the expected result of range(-2.73, 22.75), which span uses, but I'm wondering if it is appropriate for most (statistical) use cases. Before happening upon this and having a look at span(x)'s implementation, I assumed span to refer to the statistical range $T$, as in $T=max(x_1,x_2,...,x_n) - min(x_1,x_2,...,x_n)$.

Even if that is not the case, I think span not returning the correct maximum value is a potential source for confusion and/or trouble for users.

[anurag-mds]

Hey there,
I am happy to take this on. the issue seems to be that span defaults to a step of 1.0, which rarely aligns with the floating point boundaries. I'll see for a fix that ensures that the endpoint is inclusive, likely by calculating the required length or by adjusting the behaviour of step for AbstractFloat inputs...

[anurag-mds]

I looked into the implementation and usages of span. Internally it's defined as a:b using extrema(x) and all internal callers rely on the integer behaviour of this range like counts and proportions both rely on this.

For floats the fact that the upper bound may not be included probably comes directly from base range semantics not from any additional logic in statsbase.

That said, the current behaviour is consistent but it's possible underspecified or surprising for such inputs. Now, I think a documentation clarification or an explicit alternative like (min(x), max(x)) for strict bounds might be safer than changing the behaviour of span itself...

[JB0NK]

Hey, thank you for taking the time. I also had a look at the implementation of span() and I understand it is consistent and expected behaviour of the range() function used in span(). But is there really the need to construct a range? I think for 99% of the uses cases, something like:

span(x) = maximum(x) - minimum(x)

would do the trick.

[anurag-mds]

I agree @JB0NK . Using a default step of 1.0 for AbstractFloat data is mathematically inconsistent with the concept of a 'span,' as it frequently excludes the actual maximum.

If span must return an iterable for internal binning, we should at least ensure the endpoint is inclusive for floats. Otherwise, returning extrema(x) (a Tuple) would be more accurate and avoid the 'missing maximum' issue entirely.