Skip to content

ENH: Add optimize_pls method#69

Merged
derb12 merged 40 commits intodevelopmentfrom
optimize_lam
Mar 15, 2026
Merged

ENH: Add optimize_pls method#69
derb12 merged 40 commits intodevelopmentfrom
optimize_lam

Conversation

@derb12
Copy link
Copy Markdown
Owner

@derb12 derb12 commented Mar 15, 2026

Description

Adds the new optimizer method optimize_pls in order to estimate the best lam value for most penalized least squares methods in pybaselines (all Whittaker and P-Spline methods, plus beads). The method uses either GCV, BIC, or "U-Curve", which is essentially a simplified L-curve approach, as selection criteria for identifying the optimal lam.

I'm not married to the name "U-Curve"; Park, et al's implementation which it was originally based on was never given a name, and something like "L-curve-simplified" is clunky... U-curve is used in literature for a lot of other L-curve-based methods, so I'm hesitant to use it in the final version.

The current implementation is not complete, but the optimize_lam branch has unintentionally encompassed additional changes and is preventing making other easier, more granular, changes, so best to merge it now and address remaining points later. Missing things for later reference:

  • extensive documentation comparing the different selection criteria; also compare to optimize_extended_range
  • additional selection criteria that I've intentionally omitted for now
  • additional tests for the various Results objects
  • additional tests comparing the computed GCV, BIC, etc. results to some reliable source; maybe look at the R packages survival or jops or WH and the Python package psplines
  • 2D implementation

Also of note in regards to new features: this introduces "Result" objects; currently implemented for Whittaker smoothing and P-splines, but I could see this extending to polynomials as well. The idea is that these Result objects handle any additional calculations that aren't required within the actual algorithm, so things like computing polynomial coefficients could be deferred to these objects such that parameters like return_coef or return_dof can be deprecated and performed by the result objects.

Type of Pull Request

  • Bug Fix
  • New Feature
  • Miscellaneous Changes (refactor, code improvements, etc.)
  • Documentation or Example Programs

Pull Request Checklist

  • New code and/or documentation is valid for use with the BSD 3-clause license.
  • New code is fully documented with docstrings that follow Numpy style,
    if applicable.
  • New code follows PEP 8 standards as closely as possible, if applicable.
  • Added/updated tests and ensured they pass locally, if applicable.
  • Verified that documentation builds locally, if applicable.

derb12 added 30 commits October 4, 2025 15:01
Directly use arange with the input min, max, and step values to simplify behavior for both polynomials and lam-using methods. Max values are no longer included, so breaking behavior.
Allows beads to be incorporated into optimization frameworks. Also updated the beads documentation to add more details.
Includes both true and stochastic calculations.
Versions prior to 1.15.0 did not allow 1d indexing for the sparse arrays.
Will be returned from relevant algorithms to more easily calculate the effective dimensions for optimize_pls.
Also fixed polynomial coefficient shapes so they are not an incorrect cross reference.
Not a deprecation since `tck` was only added to the params dict during v1.3.0 development.
@derb12
Copy link
Copy Markdown
Owner Author

derb12 commented Mar 15, 2026

More flaky test failures in the 3.14 test case that I can't recreate locally...

@derb12
Copy link
Copy Markdown
Owner Author

derb12 commented Mar 15, 2026

CI failure is unrelated, now in 3.12... The aspls sparse comparison needs rewritten given how flaky the test seems to be, but that's outside the scope of this PR.

@derb12 derb12 merged commit b0d49f4 into development Mar 15, 2026
10 of 11 checks passed
@derb12 derb12 deleted the optimize_lam branch March 15, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant