PR: Add `scipy-stubs` as development dependency. by jorenham · Pull Request #1342 · colour-science/colour

jorenham · 2025-04-06T01:23:11Z

Summary

This adds scipy-stubs as a development dependency. By doing so, 58 new pyright errors popped up. There were 18 that were caused by two bugs and two other issues in scipy-stubs, and I've fixed them. See scipy/scipy-stubs#495 for details.

The other 40 errors turned out to be typing issues in colour itself. I have included their fixes in this PR.

I also noticed that an additional error pops up when the latest numpy==2.2.4 release is installed (currently pinned at 2.2.0). I've addressed this in numpy/numpy#28660, and the fix will be included in the upcoming 2.2.5 (and 2.3.0) NumPy release(s). (resolved in numpy 2.2.5)

This additionally improves the static-typing compatibility for different versions of pyright, numpy, scipy, and scipy-stubs.

Preflight

Code Style and Quality

Pyright static checking has been run and passed.
Pre-commit hooks have been run and passed.

jorenham · 2025-04-06T03:12:59Z

I have analyzed the typing errors in scipy/scipy-stubs#495 (comment). Through this, I found three bugs in scipy-stubs that I'll soon fix and publish a release for.

But the majority of the typing errors are true positives, and should probably be fixed in within colour itself. Unless anyone objects, I'd be willing to do include those fixes in this PR.

KelSolaar · 2025-04-06T05:06:47Z

Thanks @jorenham, sounds good with me!

…nitions

for more information, see https://pre-commit.ci

jorenham · 2025-04-06T17:05:33Z

All errors are fixed when the master branch of scipy-stubs is installed. I could either (subtly) work around the remaining errors for now, or publish a scipy-stubs release and upgrade scipy and scipy-stubs to 1.15.2 and 1.15.2.2 here in requirements.txt, respectively (that will take me a bit more time though). What would have your preference @KelSolaar?

edit

I didn't want to force you to upgrade to the latest scipy, and chose to patch the remaining issues in colour itself.

jorenham · 2025-04-06T17:16:47Z


    def __init__(self, x: ArrayLike, y: ArrayLike, *args: Any, **kwargs: Any) -> None:
-        super().__init__(x, y, *args, **kwargs)
+        super().__init__(as_float_array(x), as_float_array(y), *args, **kwargs)


typing.cast would also be an option, but strictly speaking I'd have to cast it to Any for it to be theoretically allowed. Downcasting this correctly would require recreating the ToFloatND or ToFloat1D types from optype.numpy (docs) here, which would be rather messy.

jorenham · 2025-04-06T17:17:47Z

    M = np.matmul(np.matmul(MATRIX_R, row_as_diagonal(LMS_a_L)), MATRIX_XYZ_TO_HPE)
    XYZ_ref = vecmul(M, XYZ)

+    Y_ref: NDArrayFloat


This ensures that the right @overload of spow is selected, so that LR will be inferred as NDArrayFloat

jorenham · 2025-04-06T17:18:59Z

    ) = optimisation_factory()
-    optimisation_settings = {
+
+    optimisation_settings: dict[str, Any] = {


without this, it'd be inferred as dict[str, str], which due to its invariant type-parameters, would lead to it being rejected when passed as **kwargs to minimize below

jorenham · 2025-04-06T17:21:04Z

                        **settings_BT2246,
                    )
-                ),
+                ).item(),


The fmin objective function must return a scalar type, but this returns an array of shape (1,). This sneaky workaround was was the least intrusive I could find.

jorenham · 2025-04-06T17:24:14Z

    def __init__(self, *args: Any, **kwargs: Any) -> None: ...  # pragma: no cover

-    def __call__(self, x: ArrayLike) -> NDArray:  # noqa: D102
+    def __call__(self, x: NDArrayFloat) -> NDArray:  # noqa: D102


input positions are contravariant, so assigning to this protocol, requires that the parameter types of that type are less strict than that of this Protocol. Put differently; narrowing the ArrayLike parameter type to NDArrayFloat results in a broader Protocol type.

jorenham · 2025-04-06T17:30:21Z

-        target_o: ArrayLike, coefficients_0_o: ArrayLike
-    ) -> Tuple[NDArrayFloat, float]:
+        target_o: NDArrayFloat, coefficients_0_o: NDArrayFloat
+    ) -> Tuple[NDArrayFloat, float | np.float64]:


Both result.fun: np.float64 and errors.error: float are assignable to float | np.float64, and on numpy >= 2.2 it's even equivalent to float64 (float64 has been made a proper subtype of builtins.float)

jorenham · 2025-04-06T17:30:53Z

        coefficients = dimensionalise_coefficients(coefficients, cmfs.shape)

-    return coefficients, error
+    return coefficients, float(error)


because error: float | np.float64 at this point

jorenham · 2025-04-06T17:33:12Z

        """Define the objective function."""

-        return cast(NDArrayFloat, np.sum(np.diff(a) ** 2))
+        return np.sum(np.square(np.diff(a)))


In numpy==2.2.0 there was a bug that caused np.diff(a) ** 2 to be inferred as np.signedinteger. This has been resolved in 2.2.4, but I figured that square would be a more stable solution.

jorenham · 2025-04-06T17:36:09Z

Pardon the spam. I wanted to explain some of the non-obvious choices I made here. Feel free to resolve them as you see fit.

…rsions

jorenham · 2025-04-11T12:39:42Z

This is ready for review, in case you missed it :)

KelSolaar

Thanks a lot! LGTM besides the private _typeshed import.

Co-authored-by: Thomas Mansencal <thomas.mansencal@gmail.com>

KelSolaar · 2025-04-20T07:09:17Z

Thanks @jorenham, merged!

add scipy-stubs as dev dependency

d911f02

jorenham marked this pull request as draft April 6, 2025 01:37

jorenham mentioned this pull request Apr 6, 2025

investigate typing errors in colour-science/colour scipy/scipy-stubs#495

Closed

jorenham mentioned this pull request Apr 6, 2025

TYP: Accept objects that write() to str in savetxt numpy/numpy#28660

Merged

jorenham and others added 8 commits April 6, 2025 18:41

fix typing errors related to scipy.optimize.minimize

4854906

narrow ArrayLike types to float arrays for use with scipy-stubs

f8a7785

ensure that scipy.optimize.fmin is optimizing a scalar function

32ee1c0

fix LSP issues in the interpolation and extrapolation Protocol defi…

0ca6b59

…nitions

remove redundant typing.cast

5d60cce

unleash ruff

f15f0f7

overload coefficient_K_Br_Nayatani1997

5068d65

[pre-commit.ci] auto fixes from pre-commit.com hooks

ba1857f

for more information, see https://pre-commit.ci

jorenham commented Apr 6, 2025

View reviewed changes

jorenham marked this pull request as ready for review April 6, 2025 17:56

jorenham mentioned this pull request Apr 6, 2025

✨ optimize: narrowed OptimizeResult.{fun,x} attrs for minimize[_scalar] scipy/scipy-stubs#504

Merged

jorenham added 2 commits April 7, 2025 21:34

fix typing issues when using numpy==2.2.4

5ded0b1

fix typing errors caused by resolved issues in older scipy-stubs ve…

6b5cebe

…rsions

KelSolaar changed the title ~~add scipy-stubs as dev dependency~~ PR: Add scipy-stubs as development dependency. Apr 12, 2025

KelSolaar requested changes Apr 19, 2025

View reviewed changes

Comment thread colour/io/luts/sony_spimtx.py Outdated

prefer str | os.PathLike[str] over _typeshed.StrPath

e25d0b3

Co-authored-by: Thomas Mansencal <thomas.mansencal@gmail.com>

KelSolaar merged commit 12044e4 into colour-science:develop Apr 20, 2025
16 checks passed

Uh oh!

Conversation

jorenham commented Apr 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Preflight

Uh oh!

jorenham commented Apr 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KelSolaar commented Apr 6, 2025

Uh oh!

jorenham commented Apr 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jorenham Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

jorenham Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

jorenham Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

jorenham Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

jorenham Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

jorenham Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

jorenham Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

jorenham Apr 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jorenham commented Apr 6, 2025

Uh oh!

jorenham commented Apr 11, 2025

Uh oh!

KelSolaar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

KelSolaar commented Apr 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jorenham commented Apr 6, 2025 •

edited

Loading

jorenham commented Apr 6, 2025 •

edited

Loading

jorenham commented Apr 6, 2025 •

edited

Loading

jorenham Apr 6, 2025 •

edited

Loading