Improve HTML reprs by shoyer · Pull Request #10816 · pydata/xarray

shoyer · 2025-10-05T20:30:06Z

This PR adds a number of improvements and revisions to the Xarray's HTML reprs, especially for DataTree:

No line breaks in long headers like "Data variables" and "Inherited Coordinates"
Add ~4px of extra padding at the end of HTML reprs, to make pages like Xarray's docs look a little better
Remove 2px shift on headers when actively clicked on. (I think this was intentional, but it seems to result in weird layout glitches because the :active selector doesn't always go away when focus is moved elsewhere)
Remove the collapsable "Groups" header from DataTree. Instead, each group is separately collapsable, and shows the total number of contained elements.
Truncation for too HTML elements is revised. I've added the options display_max_items and display_max_html_elements for controlling at what point the DataTree HTML repr collapses and truncates nodes, instead of doing this all based on display_max_children.

This needs a few more tests and release notes, but is ready for feedback! @jsignell @TomNicholas @benbovy

Tests added
User visible changes (including notable bug fixes) are documented in whats-new.rst

Code to generate HTML previews:

Details

import xarray as xr
import numpy as np

# Set up coordinates
time = xr.DataArray(data=["2022-01", "2023-01"], dims="time")
stations = xr.DataArray(data=list("abcdef"), dims="station")
lon = [-100, -80, -60]
lat = [10, 20, 30]

# Set up fake data
wind_speed = xr.DataArray(np.ones((2, 6)) * 2, dims=("time", "station"))
pressure = xr.DataArray(np.ones((2, 6)) * 3, dims=("time", "station"))
air_temperature = xr.DataArray(np.ones((2, 6)) * 4, dims=("time", "station"))
dewpoint = xr.DataArray(np.ones((2, 6)) * 5, dims=("time", "station"))
infrared = xr.DataArray(np.ones((2, 3, 3)) * 6, dims=("time", "lon", "lat"))
true_color = xr.DataArray(np.ones((2, 3, 3)) * 7, dims=("time", "lon", "lat"))

dt2 = xr.DataTree.from_dict(
    {
        "/": xr.Dataset(
            coords={"time": time},
        ),
        "/weather": xr.Dataset(
            coords={"station": stations},
            data_vars={
                "wind_speed": wind_speed,
                "pressure": pressure,
            },
        ),
        "/weather/temperature": xr.Dataset(
            data_vars={
                "air_temperature": air_temperature,
                "dewpoint": dewpoint,
            },
        ),
        "/satellite": xr.Dataset(
            coords={"lat": lat, "lon": lon},
            data_vars={
                "infrared": infrared,
                "true_color": true_color,
            },
        ),
    },
)
dt2['/other'] = xr.Dataset({f'x{i}': 0 for i in range(500)})

number_of_files = 20
number_of_groups = 50
tree_dict = {}
for f in range(number_of_files):
    for g in range(number_of_groups):
        tree_dict[f"file_{f}/group_{g}"] = xr.Dataset({"g": f * g})
tree_too_many = xr.DataTree.from_dict(tree_dict)

with open('datatree.html', 'w') as f:
    
    print("<html><body>", file=f)
    print("""
    <style>
    body {
      font-family: sans-serif;
    }
    pre {
      white-space: pre-wrap; /* Preserves whitespace and wraps lines as needed */
      word-wrap: break-word; /* Ensures long words are broken to fit within the container */
    }
    </style>
    """, file=f)
    
    print("<h1>DataTree root</h1>", file=f)
    print(dt2._repr_html_(), file=f)
    
    print("<hr />", file=f)
    print("<h1>Dataset</h1>", file=f)
    
    print(dt2.weather.to_dataset()._repr_html_(), file=f)


    print("<hr />", file=f)
    print("<h1>DataArray</h1>", file=f)
    
    print(dt2.weather.pressure._repr_html_(), file=f)

    print("<hr />", file=f)
    
    print("<h1>DataTree inherited</h1>", file=f)
    print(dt2.weather._repr_html_(), file=f)
    
    print("<hr />", file=f)
    print("<h1>DataTree too many nodes</h1>", file=f)
    print(tree_too_many._repr_html_(), file=f)
    
    print("</body></html>", file=f)

Revised (this PR)

Interactive preview

Baseline

Interactive preview

jsignell · 2025-10-09T21:30:43Z

Ok I took a look at this with this kind of evil DataTree from the truncation work:

import numpy as np
import xarray as xr

number_of_files = 700
number_of_groups = 5
number_of_variables= 10

datasets = {}
for f in range(number_of_files):
    for g in range(number_of_groups):
        # Create random data
        time = np.linspace(0, 50 + f, 1 + 1000 * g)
        y = f * time + g

        # Create dataset:
        ds = xr.Dataset(
            data_vars={
                f"temperature_{g}{i}": ("time", y)
                for i in range(number_of_variables // number_of_groups)
            },
            coords={"time": ("time", time)},
        ).chunk()

        # Prepare for xr.DataTree:
        name = f"file_{f}/group_{g}"
        datasets[name] = ds

dt = xr.DataTree.from_dict(datasets)

I really like the space changes and removing the collapsible "Groups" header and having each group be collapsible on its own.

I wasn't quite sure how to interpret the collapsed count for a group that just has one dataset in it. It seems like it is the n coords + n data_vars. Which seems odd. I think there shouldn't be a count on a group that just contains a single dataset.

The group level count when there are child groups should just be the number of groups.

I like the idea of having a display_max_html_elements and would be happy for it to be a lot lower than 300 by default, but truncation is still necessary for the case where there just are more than display_max_html_elements at the top level.

For instance you still get 700 top-level nodes in the repr when you do:

with xr.set_options(display_max_html_elements=5):
    display(dt)

I think in general it would be nice to be able to drill down into a particular node within the repr even if there are a bunch of items at a particular level.

shoyer · 2025-10-09T22:15:58Z

I wasn't quite sure how to interpret the collapsed count for a group that just has one dataset in it. It seems like it is the n coords + n data_vars. Which seems odd. I think there shouldn't be a count on a group that just contains a single dataset.

The group level count when there are child groups should just be the number of groups.

The strategy I was using is counting the number of hidden items (at any level), with the idea being that it should be obvious if a large amount of data is hidden. Otherwise you could have a collapsed group marked as "(1)" that hides hundreds of data variables, which felt wrong to me.

I like the idea of having a display_max_html_elements and would be happy for it to be a lot lower than 300 by default, but truncation is still necessary for the case where there just are more than display_max_html_elements at the top level.

Do you think this is common? I don't think we do this for the other Xarray HTML reprs. They get collapsed but nodes are not truncated at the top level.

I think in general it would be nice to be able to drill down into a particular node within the repr even if there are a bunch of items at a particular level.

I am currently displaying DataTree elements in priority order, based on showing the top-most levels as completely as possible (breadth-first). We could start by going deep (depth-first), but this would mean that some high-level nodes could be truncated.

Maybe there's some compromise algorithm that could work better?

benbovy · 2025-10-10T08:48:34Z

This looks great @shoyer!

I've always been confused by the collapsible "Groups" header so I think this is a nice improvement.

I haven't tried this branch yet. I only looked at the JSFiddle previews linked above where I noticed some UI/UX regressions in the "new reprs" one (e.g., no pointer cursor for variable attrs / data icons) although the CSS code seem to be not exactly the same as in this PR so it may not be an issue.

Regarding truncation of large objects I'm wondering whether we should use a same set of consistent rules across all objects for both the text and html reprs? It is tricky to find a solution that accommodates everyone so perhaps best would be to come up with a very basic solution (e.g., show the first and last n items for groups at any level, for variables, etc.) and provide a more flexible solution (pagination, etc.) via a 3rd-party widget with client-server communication.

The strategy I was using is counting the number of hidden items (at any level), with the idea being that it should be obvious if a large amount of data is hidden. Otherwise you could have a collapsed group marked as "(1)" that hides hundreds of data variables, which felt wrong to me.

I think we can be more specific for collapsed groups, e.g., show "(2 subgroups)" for groups with no variable, "(5 variables)" for leaf groups, "(3 subgroups, 10 variables)" for hybrid groups, etc.

benbovy · 2025-10-10T12:43:59Z

A few more thoughts:

I get the idea of showing the full path for each group label, but combined with tree rendering (indentation and line symbols) this is unusual and I think potentially confusing. I'd rather expect only the group name than the full path (and keep a leading slash symbol only for the root group).
It would be nice if we could better distinguish the tree structure vs. the content of each node (dimensions, coordinates, data variables, attrs). Perhaps it is enough to just add a bit more of top/bottom padding for the content ? (better done by wrapping it in its own html div).
The revised repr in this PR seems to have increased indentation for each hierarchical level but I think it is too much especially for datatree objects with many nested levels.

shoyer · 2025-10-11T17:21:37Z

If we go from a full path to just node names, we should adjust the text repr, too. I like the full easily copied identification, though.

The increased indentation for child nodes was an attempt to make it easier to identify child content! I'm definitely open to other ideas.

Anyways if anyone wants to try their hand at iterating on this, please fill free to push commits to my branch. These CSS edits are pretty fiddly for me 😄.

for more information, see https://pre-commit.ci

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

shoyer · 2025-12-15T23:31:07Z

I've added unit tests, and restored the truncation for too many child nodes. I'm still open to suggestions, but in the interest of merging in marginal improvements rather than waiting for a perfect solution, this is ready for a final review!

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

benbovy

Thanks @shoyer, that's definitely an improvement!

I left a few CSS change suggestions in order to avoid display regressions.

xarray/static/css/style.css

shoyer · 2025-12-22T22:37:41Z

Thanks @benbovy for the re-review!

I've fixed the styles as you've suggested (including the regression for DataTree!).

I also figured out a way to restore the visual indications of focus without looking weird when using the mouse -- it turns out using the :focus-visble rather than :focus attribute is the way to go. I'm also using outline: auto instead of border to set the style of the selected element, which does not change the layout. This results in a nicer looking focus ring:

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

jsignell

This looks so great! It's a real improvement over the current datatree.

shoyer · 2026-01-14T23:12:54Z

Doc build failure looks entirely unrelated, so merging. Thanks everyone for the reviews!

shoyer added 8 commits September 28, 2025 13:15

Less vertical whitespace in HTML reprs

c2e1384

ensure consistent line-height in google colab

b3bb386

Refactor DataTree HTML repr

ca16214

Merge branch 'main' into html-tree

bade174

Merge branch 'main' into html-tree

2459e64

Collapsable DataTree nodes

1842a39

more formatting

73e46ec

Tweaks

1958bc9

github-actions bot added the topic-html-repr label Oct 5, 2025

shoyer and others added 5 commits December 14, 2025 17:29

Merge remote-tracking branch 'upstream/main' into html-tree

4637d36

Add tests and truncation for many child nodes

96efaee

add missing DataTree.xindexes into repr

0e0b60a

[pre-commit.ci] auto fixes from pre-commit.com hooks

2a98a78

for more information, see https://pre-commit.ci

Add whats-new entry for DataTree HTML repr improvements

180b4d3

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Fix ruff PERF401: use list.extend instead of append in loop

8451eff

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

benbovy reviewed Dec 16, 2025

View reviewed changes

shoyer added 2 commits December 22, 2025 14:29

Adjust style per review

79b79ab

Merge branch 'main' into html-tree

dad31ed

Use browser default focus outline style

03f8469

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

jsignell approved these changes Dec 23, 2025

View reviewed changes

maxrjones mentioned this pull request Jan 8, 2026

Ideal representation for DataTree objects englacial/ismip-indexing#3

Open

Merge upstream/main into html-tree

f855858

shoyer merged commit cbe8cde into pydata:main Jan 14, 2026
39 of 40 checks passed

Uh oh!

Conversation

shoyer commented Oct 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Revised (this PR)

Baseline

Uh oh!

jsignell commented Oct 9, 2025

Uh oh!

shoyer commented Oct 9, 2025

Uh oh!

benbovy commented Oct 10, 2025

Uh oh!

benbovy commented Oct 10, 2025

Uh oh!

shoyer commented Oct 11, 2025

Uh oh!

shoyer commented Dec 15, 2025

Uh oh!

benbovy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shoyer commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jsignell left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

shoyer commented Jan 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shoyer commented Oct 5, 2025 •

edited

Loading

shoyer commented Dec 22, 2025 •

edited

Loading