Streaming Sortformer release PR03: NeMo documentations and tutorial notebook

[tango4j]

What does this PR do ?

This PR is adding NeMo documentations and tutorial notebook for streaming Sortformer.

Collection: [Note which collection this PR will affect]

ASR/speaker_tasks

• Add specific line by line info of high level changes in this PR.

• Adding two images

• Adding descriptions for streaming Sortformer.

• You can potentially add a usage example below

# For evaluation of streaming speaker diarization
python $BASEPATH/neural_diarizer/e2e_diarize_speech.py \
    model_path=/path/to/diar_sortformer_4spk_v1.nemo \
    batch_size=1 \
    dataset_manifest=/path/to/diarization_manifest.json

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

• New Feature
• [] Bugfix
• Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in NeMo ASR

Additional Information

• Related to # (issue)

[Copilot]

Pull Request Overview

This PR adds documentation for the streaming version of the Sortformer diarization model to NeMo's ASR speaker diarization documentation. The streaming Sortformer enables real-time speaker diarization by processing audio in chunks while maintaining speaker identity through an Arrival-Order Speaker Cache (AOSC).

Key changes include:

• Addition of streaming Sortformer architecture explanation and visual aids
• Documentation of training and inference procedures for streaming mode
• Complete configuration reference for streaming Sortformer models

Reviewed Changes

Copilot reviewed 3 out of 5 changed files in this pull request and generated 3 comments.

File	Description
docs/source/asr/speaker_diarization/results.rst	Adds training and inference commands for streaming Sortformer diarizer
docs/source/asr/speaker_diarization/models.rst	Introduces streaming Sortformer concept with architectural diagrams and explanations
docs/source/asr/speaker_diarization/configs.rst	Provides comprehensive configuration documentation for streaming Sortformer training

Comments suppressed due to low confidence (1)

docs/source/asr/speaker_diarization/configs.rst:234

• [nitpick] The model name 'StreamingSortFormerDiarizer' uses inconsistent capitalization compared to 'Sortformer' used elsewhere in the documentation. Consider using 'StreamingSortformerDiarizer' for consistency.

  name: "StreamingSortFormerDiarizer"

[MahmoudAshraf97]

in table 1 in the paper, the bold numbers are not the lowest in the table

is there a mistake here?

[ipmedenn]

in table 1 in the paper, the bold numbers are not the lowest in the table

is there a mistake here?

Not a mistake, bold numbers are the lowest DER for 1s latency setup (this is explained in the table caption)

[ipmedenn]

LGTM

[tango4j]

@ko3n1g Now that streaming Sortformer V2 has been released, https://huggingface.co/nvidia/diar_streaming_sortformer_4spk-v2
the link error is gone.
Running CICD to get this merged.

[tango4j]

@ko3n1g

Hi, I am having link error that I haven't touched upon.
https://github.com/NVIDIA/NeMo/actions/runs/17001337818/job/48203558716?pr=14388

Can we disable it or skip this ..?
This PR needs to be merged for model release for Interspeech 2025 (Aug/17th)
So it is quite urgent.

{
"filename": "nlp/text_normalization/wfst/wfst_text_processing_deployment.rst",
"lineno": 10,
"status": "broken",
"code": 0,
"uri": "https://www.openfst.org/",
"info": "HTTPSConnectionPool(host='www.openfst.org', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1016)')))"
}
{
"filename": "nlp/text_normalization/wfst/wfst_text_processing_deployment.rst",
"lineno": 76,
"status": "broken",
"code": 0,
"uri": "https://www.openfst.org/twiki/bin/view/GRM/Thrax",
"info": "HTTPSConnectionPool(host='www.openfst.org', port=443): Max retries exceeded with url: /twiki/bin/view/GRM/Thrax (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1016)')))"
}