diff --git a/docs/demos/24-07-11-scipy-2024/images/doc-qr-code.png b/docs/demos/24-07-11-scipy-2024/images/doc-qr-code.png new file mode 100644 index 000000000..d766066ad Binary files /dev/null and b/docs/demos/24-07-11-scipy-2024/images/doc-qr-code.png differ diff --git a/docs/demos/24-07-11-scipy-2024/images/pres-qr-code.png b/docs/demos/24-07-11-scipy-2024/images/pres-qr-code.png new file mode 100644 index 000000000..a958cf008 Binary files /dev/null and b/docs/demos/24-07-11-scipy-2024/images/pres-qr-code.png differ diff --git a/docs/demos/24-07-11-scipy-2024/images/tom.jpg b/docs/demos/24-07-11-scipy-2024/images/tom.jpg index 3b25c83dc..713f03d3f 100644 Binary files a/docs/demos/24-07-11-scipy-2024/images/tom.jpg and b/docs/demos/24-07-11-scipy-2024/images/tom.jpg differ diff --git a/docs/demos/24-07-11-scipy-2024/scipy-2024.ipynb b/docs/demos/24-07-11-scipy-2024/scipy-2024.ipynb index 60f6a02ef..8c180b4ac 100644 --- a/docs/demos/24-07-11-scipy-2024/scipy-2024.ipynb +++ b/docs/demos/24-07-11-scipy-2024/scipy-2024.ipynb @@ -16,13 +16,14 @@ " \"SciPy\n", "\n", "\n", - "# SciPy 2024 (07/11/2024) - xCDAT: A Python Package for Simple and Robust Analysis of Climate Data\n", + "# SciPy 2024 - xCDAT (Xarray Climate Data Analysis Tools)\n", "\n", - "
\n", - "

\n", - " Core Developers: Tom Vo, Stephen Po-Chedley, Jason Boutte, Jill Zhang, Jiwoo Lee\n", - "

\n", - "

With thanks to Peter Gleckler, Paul Durack, Karl Taylor, and Chris Golaz

\n", + "
\n", + "

\"A Python Package for Simple and Robust Analysis of Climate Data\"

\n", + "
\n", + "\n", + "
\n", + "

Presented by Tom Vo

\n", "
\n", "\n", "---\n", @@ -73,6 +74,23 @@ "```\n" ] }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### This presentation is available on xCDAT's Read The Docs Page\n", + "\n", + "
\n", + " \"Presentation\n", + "
\n", + "\n", + "#### https://xcdat.readthedocs.io/en/latest/demos/24-07-11-scipy-2024/scipy-2024.html\n" + ] + }, { "cell_type": "markdown", "metadata": { @@ -87,20 +105,21 @@ { "cell_type": "markdown", "metadata": { - "cell_style": "split", + "cell_style": "center", "slideshow": { "slide_type": "-" } }, "source": [ - "### Who am I?\n", + "### A little about me\n", "\n", - "- My name is Tom Vo\n", - "- **Climate Research Software Engineer** at **Lawrence Livermore National Laboratory (LLNL)**\n", - "- Work on the **Energy Exascale Earth System Model (E3SM)** and **Simplifying ESM Analysis Through Standards (SEATS)** projects\n", + "- Software Engineer at **Lawrence Livermore National Laboratory (LLNL)**\n", + "- **Energy Exascale Earth System Model (E3SM)** and **Simplifying ESM Analysis Through Standards (SEATS)** projects\n", + "- Lead developer of xCDAT\n", + "- Contributor and DevOps engineer for various E3SM tools\n", "\n", "
\n", - " \"Tom\n", + " \"Tom\n", " \"LLNL\n", "
\n" ] @@ -108,22 +127,19 @@ { "cell_type": "markdown", "metadata": { - "cell_style": "split", "slideshow": { - "slide_type": "-" + "slide_type": "slide" } }, "source": [ - "### What do I do?\n", + "#### A big shoutout to the xCDAT core development team including\n", "\n", - "- Lead developer of xCDAT (roadmap planning, end-to-end development)\n", - "- Contributor and DevOps engineer for various E3SM tools\n", + "- Stephen Po-Chedley\n", + "- Jason Boutte\n", + "- Jill Zhang\n", + "- Jiwoo Lee\n", "\n", - "
\n", - " \"xCDAT\n", - " \"E3SM\n", - " \"SEATS\n", - "
\n" + "With thanks to Peter Gleckler, Paul Durack, Karl Taylor, and Chris Golaz.\n" ] }, { @@ -140,12 +156,11 @@ "## An Overview of this Talk\n", "\n", "1. The driving force behind xCDAT\n", - "2. Basic intro to Xarray\n", - "3. History, scope, and mission of xCDAT\n", - "4. Design philosophy and key features of xCDAT\n", - "5. Technical demo: end-to-end analysis workflow\n", - "6. Parallelism with xCDAT and Dask\n", - "7. xCDAT’s community and how to get involved\n" + "2. Scope and mission of xCDAT\n", + "3. Design philosophy and key features of xCDAT\n", + "4. Technical demo: end-to-end analysis workflow\n", + "5. Parallelism with xCDAT and Dask\n", + "6. xCDAT’s community and how to get involved\n" ] }, { @@ -173,13 +188,13 @@ "cell_type": "markdown", "metadata": { "slideshow": { - "slide_type": "-" + "slide_type": "subslide" } }, "source": [ - "### CDAT has been end-of-life as of December 2023\n", + "### The present-day challenge: **CDAT is end-of-life** as of December 2023\n", "\n", - "- This presents an issue for the many users and packages that depend on CDAT\n", + "- A big issue for users and packages that depend on CDAT\n", "- All of these factors sparked a driving need for new analysis software\n" ] }, @@ -193,15 +208,15 @@ "source": [ "### What should this new analysis software (aka xCDAT) offer?\n", "\n", - "- Offer similar core capabilities as CDAT\n", + "- **Offer similar core capabilities** as CDAT\n", " - e.g., geospatial averaging, temporal averaging, regridding\n", - "- Use modern technologies in the library’s stack\n", + "- **Use modern technologies** in the library’s stack\n", " - Capable of handling large datasets (e.g., parallelism, lazy operations)\n", - "- Be maintainable, extensible, and easy-to-use\n", + "- **Maintainable, extensible, and easy-to-use**\n", " - Python Enhancement Proposals (PEPs)\n", - "- Automated DevOps processes (unit testing, code coverage)\n", - " - Actively maintained documentation\n", - "- An open-source community that can sustain the project\n", + " - Software sustainability\n", + " - Reproducible science\n", + "- **Foster open-source community**\n", " - Serve the needs of the climate community in the long-term\n", " - Community engagement efforts (e.g., Pangeo, ESGF)\n" ] @@ -219,9 +234,7 @@ "

\"N-D labeled arrays and datasets in Python\"

\n", "
\n", "\n", - "**Xarray is the core technology of xCDAT**\n", - "\n", - "**Why Xarray?**\n", + "**Why is Xarray the core technology of xCDAT?**\n", "\n", "- Mature, stable, widely adopted\n", "- Introduces labels in the form of dimensions, coordinates, and attributes on top of raw NumPy-like arrays\n", @@ -267,9 +280,21 @@ " \"SEATS\n", "\n", "\n", - "- xCDAT is an extension of Xarray for climate data analysis on structured grids, a modern successor to the Community Data Analysis Tools (CDAT) library\n", - "- Scope is focused on routine climate research analysis operations such as loading, wrangling, averaging, and regridding data\n", - "- Aims to provide features and utilities for simple and robust analysis of climate data\n", + "- xCDAT is an extension of Xarray for climate data analysis on structured grids\n", + "- A modern successor to the Community Data Analysis Tools (CDAT) library\n", + "- The core development team of software engineers and climate scientists are users of the software\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "- **Scope** is focused on routine climate research analysis operations such as loading, wrangling, averaging, and regridding data\n", + "- **Goal** of providing features and utilities for simple and robust analysis of climate data\n", "- Leverages other powerful Xarray-based packages such as xESMF, xgcm, and cf-xarray\n", "\n", "
\n", @@ -298,7 +323,7 @@ "- Geospatial weighted averaging\n", "- Temporal averaging, climatologies, departures\n", "- Horizontal structured regridding (extension of xESMF and Python port of regrid2)\n", - "- Vertical structured regridding (extension of xGCM)\n", + "- Vertical structured regridding (extension of xgcm)\n", "\n", "
\n", " \"Spatial\n", @@ -319,9 +344,9 @@ "

The Software Design Philosophy of xCDAT

\n", "
\n", "\n", - "- Intentionality designed to **encourage software sustainability** and **reproducible science**\n", + "- **Encourage software sustainability** and **reproducible science**\n", "- **Well-documented and configurable features** allow scientists to rapidly develop robust, reusable, less-error prone, more maintainable code\n", - "- xCDAT aims to contribute to **Pangeo's** effort to foster an ecosystem of mutually compatible geoscience Python packages\n", + "- **Contribute to Pangeo's** effort of fostering an ecosystem of mutually compatible geoscience Python packages\n", "\n", "
\n", " \"Pangeo\n", @@ -4981,14 +5006,19 @@ "source": [ "#### 6.2 Departures\n", "\n", - "It can also be useful to show the departures from the climatological average.\n" + "It can also be useful to show the departures (\"anomalies\") from the climatological average.\n", + "\n", + "In climatology, anomalies refer to the difference between the value during a given time\n", + "interval and the long-term average value for that time interval.\n", + "\n", + "- _For example, the difference between the January average surface air temperature and the average surface temperature over the last 30 Januaries._\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { - "slide_type": "-" + "slide_type": "fragment" } }, "source": [ @@ -4996,18 +5026,8 @@ "\n", "In this case, `xcdat` will operate on the global mean time series we calculated above.\n", "\n", - "Note that you can set the climatological reference period (e.g., with `reference_period=(\"2000-01-01\", \"2009-12-31\")` for historical era departures).\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "API Documentation: https://xcdat.readthedocs.io/en/stable/generated/xarray.Dataset.temporal.departures.html\n" + "Note that you can set the climatological reference period for historical era departures.\n", + "We use `reference_period=(\"2000-01-01\", \"2009-12-31\")` below.\n" ] }, { @@ -5034,6 +5054,17 @@ ")" ] }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "API Documentation: https://xcdat.readthedocs.io/en/stable/generated/xarray.Dataset.temporal.departures.html\n" + ] + }, { "cell_type": "markdown", "metadata": { @@ -5091,7 +5122,9 @@ "source": [ "#### 6.3 Group averages\n", "\n", - "`xcdat` also allows you to calculate group averages (e.g., annual or seasonal mean from monthly data or monthly mean from daily data).\n" + "`xcdat` also allows you to calculate group averages.\n", + "\n", + "- _For example, annual or seasonal mean from monthly data or monthly mean from daily data_\n" ] }, { @@ -5188,8 +5221,14 @@ "source": [ "### 7. General Dataset Utilities\n", "\n", - "xCDAT includes various utilities for data manipulation, including\n", - "reorientation of the longitude axis, centering of time coordinates using time bounds, and adding and getting bounds.\n" + "xCDAT includes various utilities for data manipulation including:\n", + "\n", + "- Reorient the longitude axis\n", + "- Add missing bounds\n", + "- Centering of time coordinates using time bounds\\*\n", + "- Add and get bounds\\*\n", + "\n", + "

* not shown

\n" ] }, { @@ -5295,7 +5334,7 @@ "execution_count": 30, "metadata": { "slideshow": { - "slide_type": "-" + "slide_type": "fragment" } }, "outputs": [ @@ -5320,7 +5359,11 @@ { "cell_type": "code", "execution_count": 31, - "metadata": {}, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, "outputs": [ { "data": { @@ -6201,7 +6244,7 @@ "For example, for global spatial averaging across various file sizes:\n", "\n", "
\n", - " \"JOSS\n", + " \"JOSS\n", "

* CDAT only runs functions serially and crashes with 22 GB+ datasets.

\n", "
\n", "\n", @@ -6220,13 +6263,10 @@ "\n", "Visit these pages for more guidance (e.g., when to parallelize):\n", "\n", - "- Parallel computing with Dask (xCDAT, image shown below): https://xcdat.readthedocs.io/en/latest/examples/parallel-computing-with-dask.html\n", + "- Parallel computing with Dask (xCDAT): https://xcdat.readthedocs.io/en/latest/examples/parallel-computing-with-dask.html\n", "- Parallel computing with Dask (Xarray): https://docs.xarray.dev/en/stable/user-guide/dask.html\n", "- Xarray with Dask Arrays: https://examples.dask.org/xarray.html\n", - "\n", - "
\n", - " \"Dask\n", - "
\n" + "\n" ] }, { @@ -6237,15 +6277,12 @@ } }, "source": [ - "
\n", - " \"xCDAT\n", - "

Recap of Key Points

\n", - "
\n", + "### What are some things in the works with xCDAT?\n", "\n", - "- xCDAT is an **extension of Xarray for climate data analysis on structured grids**, a modern successor to the Community Data Analysis Tools (CDAT) library\n", - "- **Focused on routine climate research analysis operations** including loading, wrangling, averaging, such as temporal averaging, spatial averaging, and regridding\n", - "- Designed to encourages **software sustainability and reproducible science**\n", - "- **Parallelizable** through Xarray’s support for Dask, which enables efficient processing of large datasets\n" + "- **Collaborate with UXarray** for interoperation of two tools to support end-to-end and more streamlined operation on unstructured (i.e. E3SM native output) datasets.\n", + "- **Continue assisting DOE funded projects** including PCMDI Metrics Package, E3SM Diags\n", + "- **Explore other DOE funded projects** to integrate xCDAT for analysis capabilities (e.g., ARM Diags and ILAMB)\n", + "- **Explore xarray-datatree** package for analyzing ensembles with xCDAT capabilities\n" ] }, { @@ -6262,13 +6299,37 @@ " Get involved and join the xCDAT community!\n", "
\n", "\n", + "
\n", + " \"QR\n", + "
\n", + "\n", "- **Code contributions** are welcome and appreciated\n", " - GitHub Repository: https://github.com/xCDAT/xcdat\n", " - Contributing Guide: https://xcdat.readthedocs.io/en/latest/contributing.html\n", "- **Submit and/or address tickets** for feature suggestions, bugs, and documentation updates\n", " - GitHub Issues: https://github.com/xCDAT/xcdat/issues\n", "- **Participate in forum discussions** on version releases, architecture, feature suggestions, etc.\n", - " - GitHub Discussions: https://github.com/xCDAT/xcdat/discussions\n" + "\n", + "- GitHub Discussions: https://github.com/xCDAT/xcdat/discussions\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "
\n", + " \"xCDAT\n", + "

Recap of Key Points

\n", + "
\n", + "\n", + "- xCDAT is an **extension of Xarray for climate data analysis on structured grids**, a modern successor to the Community Data Analysis Tools (CDAT) library\n", + "- **Focused on routine climate research analysis operations** including loading, wrangling, averaging, such as temporal averaging, spatial averaging, and regridding\n", + "- Designed to encourages **software sustainability and reproducible science**\n", + "- **Parallelizable** through Xarray’s support for Dask, which enables efficient processing of large datasets\n" ] } ],