Add WSD scheduler

[visheratin]

What does this PR do?

This PR adds the Warmup-Stable-Decay learning rate scheduler. The scheduler and its benefits are described in detail in the MiniCPM blog post. In short, it allows continuous training and performs better than standard cosine scheduler.

Here is an example of learning rate dynamics with this schedule:

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@muellerzr, @pacman100

[muellerzr]

Nice! Seems handy. Could you rebase from main and see if that fixes the failing tests?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[visheratin]

Looks like it fixed some tests but not all.

[muellerzr]

LG2M, no idea why that pytorch test is failing 😕

cc @amyeroberts for final review (and ideas(?) was seeing that on another PR of mine iirc)

[amyeroberts]

Thanks for adding this!

[muellerzr]

@visheratin CI should be green on rebase, we found the issue :)

[visheratin]

@muellerzr a couple of tests are still failing.

[amyeroberts]

@visheratin Looks like they were failing because of connection issues with the hub. I've re-run - which should hopefully resolve