Skip to content

Animate dataset name+version during load/publish#556

Open
hagenw wants to merge 27 commits intomainfrom
animate-name-for-progress
Open

Animate dataset name+version during load/publish#556
hagenw wants to merge 27 commits intomainfrom
animate-name-for-progress

Conversation

@hagenw
Copy link
Copy Markdown
Member

@hagenw hagenw commented Apr 7, 2026

Closes #553
Alternative to #554

Adds a "shimmer" animation (bold text sweeping across the database name) to load(), load_attachment(), load_media(), load_table(), load_to(), and publish(). The animation
runs in a background thread, pauses when tqdm progress bars are active, and resumes in between. A new audb/core/shimmer.py module implements the animation via ANSI escape codes and stream proxying.

Example video (the animation will reappear between progress bars, not shown in this video as we cancel during the first progress bar):

Screencast.from.07.04.2026.13.33.55.webm
Code for video
import audb
from tempfile import TemporaryDirectory

with TemporaryDirectory() as tmpdir:
    db = audb.load("librispeech", version="3.2.0", cache_root=tmpdir)

For audb.load_to() and audb.publish() we also add the following print statements to allow for the animation (before nothing was shown):

Get: emodb v2.0.0
To: /path/to/db_root
Put: emodb v2.0.1
Source: /path/to/db_root

As we now have an animation directly at the start, I removed showing a progress bar for downloading db.parquet as it downloads in reasonable time and most users do not know what db.parquet is.


An alternative implementation would be to always show the animation, also when progress bars are active:

Screencast.from.07.04.2026.13.36.20.webm

@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented Apr 7, 2026

Reviewer's Guide

Introduces a reusable ANSI-based Shimmer animation that highlights database names during long-running operations, wiring it into load/publish APIs so that a bold sweeping effect appears when verbose output is enabled, pauses during tqdm progress bars, and restores terminal streams cleanly after completion.

Sequence diagram for shimmer animation integrated into audb.load with tqdm

sequenceDiagram

actor User
participant Audb as audb
participant Load as load
participant Shimmer as Shimmer
participant Backend as backend_interface
participant Tqdm as tqdm
participant Stdout as stdout
participant Stderr as stderr

User->>Audb: load(name, version, verbose=True)
Audb->>Load: load(name, version, verbose=True)

Note over Load: Setup cache paths
Load->>Shimmer: new Shimmer("Get:   ", "name vversion")
Load->>Shimmer: start()
Shimmer->>Stdout: print initial line and newline
Shimmer->>Stdout: install stdout.write hook
Shimmer->>Stderr: install stderr.write hook
activate Shimmer
Shimmer-->>Shimmer: background _animate() loop

loop Dependency resolution and locking
  Load->>Load: dependencies(...)
  Load->>Backend: load_header_to(...)
end

loop Download with progress bars
  Backend->>Tqdm: update progress
  Tqdm->>Stderr: "\r...progress bar..."
  Stderr->>Shimmer: stderr_write_hook("\r...")
  Shimmer-->>Shimmer: set _paused = True
  Shimmer-->>Stdout: write plain text frame when first paused
end

Tqdm->>Stderr: "\r   ...clear...\r" or "\n"
Stderr->>Shimmer: stderr_write_hook(clear or newline)
Shimmer-->>Shimmer: set _paused = False
Shimmer-->>Stdout: resume shimmering frames

Load-->>Audb: return db
Audb-->>User: db

Load->>Shimmer: stop()
Shimmer-->>Shimmer: set stop event, join thread
Shimmer->>Stdout: restore original write
Shimmer->>Stderr: restore original write
Shimmer->>Stdout: write final static line
deactivate Shimmer
Loading

Class diagram for the new Shimmer animation utility

classDiagram

class Shimmer {
  -str _prefix
  -str _text
  -str _suffix
  -float _interval
  -int _width
  -Event _stop_event
  -Thread _thread
  -int _lines_below
  -bool _paused
  -Lock _lock
  -callable _original_stdout_write
  -callable _original_stderr_write
  +__init__(prefix, text, suffix, interval, width)
  +start()
  +stop()
  +_stdout_write_hook(s)
  +_stderr_write_hook(s)
  +_write_frame(rendered_text)
  +_render_frame(center)
  +_animate()
  +__enter__()
  +__exit__(exc_type, exc_val, exc_tb)
}
Loading

File-Level Changes

Change Details Files
Add a reusable shimmer animation utility that animates text via ANSI escape codes while tracking stdout/stderr output and tqdm-style progress bar activity.
  • Implement Shimmer class with background thread, bold-window rendering, and cursor-movement-based line rewriting using ANSI escape codes.
  • Monkey-patch sys.stdout.write and sys.stderr.write to track newlines, count lines printed below the shimmer line, and detect progress bar activity via carriage returns.
  • Ensure shimmer pauses when progress bars are active and resumes afterward, including safe teardown and context manager support.
  • Expose configuration parameters such as prefix, text, suffix, animation interval, and bright window width.
audb/core/shimmer.py
Integrate shimmer animation into user-facing load APIs to animate the database name instead of printing a static 'Get:' line, and ensure cleanup on all code paths.
  • In load(), load_attachment(), load_media(), and load_table(), replace print('Get: ...') with construction and start() of a Shimmer instance when verbose is true, keeping the 'Cache:' print as a static line.
  • Wrap the main body of each function in a try block and add a finally that stops the shimmer if it was started, ensuring threads and stream hooks are restored even on errors or timeouts.
  • Adjust dependencies() calls where necessary (e.g., remove verbose argument in load()), aligning with new control flow around shimmer startup.
audb/core/load.py
Add shimmer-based animated output and clearer status lines to load_to() while preserving its multi-step download/update logic.
  • Instantiate and start a Shimmer with Get: prefix and name v{version} text when verbose is true, and print a new static To: <db_root> line.
  • Wrap the entire load_to() implementation in a try/finally so shimmer is always stopped and stream hooks restored.
  • Refactor the function body so dependency loading, header loading, attachments/tables/media downloads, dependency saving, and database saving all run inside the try block without changing their underlying logic.
audb/core/load_to.py
Add shimmer-based animated output to publish() and print a new 'Source:' line, wrapping the publishing workflow in a safe shimmer lifecycle.
  • When verbose is true, create and start a Shimmer with prefix Put: and text <db.name> v<version>, and print Source: <db_root> once.
  • Enclose the full publish flow—from creating the backend interface and checking versions through dependency checks, database loading, validation, uploading attachments/tables/media, and header/dependency upload—in a try block and add a finally that stops the shimmer if started.
  • Preserve existing error handling for header upload cleanup while nesting it within the new shimmer-managed control flow.
audb/core/publish.py
Introduce tests for the shimmer utility to validate its hooks, rendering behavior, and interaction with Python’s IO streams.
  • Test that the stdout write hook correctly counts newlines and delegates to the original writer.
  • Test that the stderr write hook detects progress-bar-like carriage-return patterns and sets the internal paused flag appropriately.
  • Verify that start() monkey-patches sys.stdout.write/sys.stderr.write without replacing the stream objects themselves, and that stop() restores the original methods.
  • Test _render_frame() to confirm that characters near the animation center are bolded, fading behavior at the edges, and that no bolding occurs when the center is far outside the text range.
tests/test_shimmer.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@hagenw hagenw marked this pull request as ready for review April 7, 2026 13:08
@hagenw hagenw self-assigned this Apr 7, 2026
@hagenw hagenw changed the title Animate name for progress Animate dataset name+version during load/publish Apr 7, 2026
sourcery-ai[bot]

This comment was marked as resolved.

@hagenw hagenw force-pushed the animate-name-for-progress branch from 2577c8c to ecb2a7d Compare April 13, 2026 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Silent Periods in audb.load(..., verbose=True)

1 participant