Skip to content

Commit 1f36ee5

Browse files
authored
Replace datashader with matplotlib in user guide notebooks (#1002)
* Replace datashader with matplotlib in user guide notebooks (#1001) Remove datashader as the rendering backend from all 12 affected user guide notebooks. Each datashader call is replaced with the appropriate matplotlib or xrspatial equivalent: - shade/stack -> plt.imshow overlays with alpha compositing - dynspread -> ax.scatter for points, ax.plot for lines - Canvas.points/line -> manual DataArray creation or shapely + rasterize - set_background -> ax.set_facecolor - datashader.colors.Elevation -> matplotlib 'terrain' cmap - Images side-by-side -> plt.subplots grid Notebooks converted: 1_Surface, 2_Proximity, 3_Zonal, 4_Focal, 5_Classification, 6_Remote_Sensing, 7_Pathfinding, 8_Emerging_Hotspots, 11_Hydrology, 12_Flood_Analysis, 13_Fire_Analysis, 30_Zonal_Crosstab. * Fix rasterize API usage and notebook cell format (#1001) - Add numeric column to GeoDataFrame in 2_Proximity and 7_Pathfinding so rasterize() has a value to burn - Use like= keyword for template DataArray instead of positional arg - Fix 6_Remote_Sensing cell source format (string → list of strings) * Fix invisible overlay colors in 1_Surface notebook (#1001) Use ListedColormap with explicit colors instead of gradient colormaps (Reds, Greens, cool) for binary mask overlays. When all unmasked values are identical, gradient cmaps normalize vmin=vmax and render as white. * Fix curvature zfactor in 1_Surface so overlay is visible (#1001) zfactor=1 produces curvature values in [-0.05, 0.06], far below the thresholds of 0.5 and 1.0 used for the overlay masks. Bump to zfactor=100 which gives values in [-4.6, 6.1], matching the documented range for hilly terrain. * Reuse single terrain and use xarray filtering in 1_Surface (#1001) - Remove separate terrain generation for curvature; reuse the initial terrain throughout the notebook - Replace numpy .data assignment with xarray .where() for slope, curvature, and aspect filtering - Adjust curvature thresholds to match the default terrain's value range * Fix viewshed visualization in 1_Surface (#1001) - Mask on viewshed == -1 (not visible) instead of NaN, which is what the function actually returns for blocked cells - Lower observer_elev from 100 to 5 so roughly 30% of terrain is occluded, making the viewshed result visually obvious - Use ListedColormap for viewshed overlay consistency * Add composite risk map, colorblind fixes, and viewshed notebook (#1001) - Add composite avalanche risk map combining slope, aspect, and curvature - Fix curvature colors from red/green to orange/blue for colorblind users - Add legends to all overlay plots (slope, curvature, aspect) - Reframe aspect-slope map section as a cartographic bonus - Split viewshed examples into separate 31_Viewshed notebook - Move preview image to images/ directory * Add user-guide-notebook command for creating/refactoring notebooks (#1001) Captures the conventions from the 1_Surface.ipynb work: cell structure, xarray.plot usage, colorblind-safe palettes, legends, humanizer pass, and file organization. * Refactor 2_Proximity notebook and update user-guide-notebook command (#1001) - Rewrite 2_Proximity with xarray.plot, legends, colorblind-safe colormaps, humanized markdown, and coverage-planning narrative - Fix rasterize NaN bug (fillna before astype int) in line proximity - Use twilight cyclic colormap for direction, tab10 for allocation zones - Add distance assumptions warning as alert-warning box - Add direction preview image - Update command: require 1-3 GIS alert boxes per notebook, eye-candy preview image with generation instructions * Refactor 3_Zonal notebook and move alerts to relevant sections (#1001) Rewrote 3_Zonal with xr.DataArray.plot.imshow, legends, colorblind-safe palettes, and side-by-side crop visualization. Moved alert boxes in both 2_Proximity and 3_Zonal to sit next to the section they apply to instead of clustering at the top. Updated user-guide-notebook command to match. * Update alert box placement: evaluate per-section, place at end (#1001) * Refactor 4_Focal notebook and update What you'll build sections (#1001) Rebuild 4_Focal.ipynb to match established notebook structure: proper cell sequence, xr.DataArray.plot.imshow everywhere, legends on overlays, colorblind-safe palettes, GIS alert boxes, and new sections for mean smoothing and focal_stats. Switch hotspots base layer to gray hillshade so overlay colors pop. Add focal_hotspots_preview.png. Update What you'll build cells in 2_Proximity, 3_Zonal, and 4_Focal to use ordered lists summarizing each step. Update user-guide-notebook command to specify the ordered-list format. * Refactor 5_Classification notebook, add title format to all notebooks (#1001) Rebuild 5_Classification.ipynb: equal interval, quantile, natural breaks with side-by-side comparison, custom reclassify with named elevation bands, and connected-component regions. Add preview image. Update notebook command to specify title format: "Xarray-Spatial {module}: {tools covered}" with a real-world subtitle. Apply the new format to all five notebooks (1_Surface through 5_Classification). Convert all What you'll build cells to ordered lists. * Remove Esri/ArcGIS references and refactor notebooks 5, 8, 9 (#1001) Strip ArcGIS links from Hydrology, Surface, Corridor, Proximity, Viewshed, and Cost Distance notebooks. Refactor Emerging Hotspots and Cost Distance notebooks to match established structure. * Refactor 6_Remote_Sensing and 7_Pathfinding notebooks (#1001) Remote Sensing: self-contained with synthetic spectral bands from terrain elevation. Covers true color, NDVI, SAVI, ARVI, EVI, NBR, and NDMI. Pathfinding: redesigned with a 20x20 open grid so connectivity differences are immediately visible. 8-conn gives a clean diagonal, 4-conn gives a staircase, friction-weighted curves around expensive cells.
1 parent 2ebab21 commit 1f36ee5

25 files changed

+3457
-3289
lines changed
Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
# User Guide Notebook: Create or Refactor
2+
3+
Create a new xarray-spatial user guide notebook, or refactor an existing one into
4+
the established structure. The prompt is: $ARGUMENTS
5+
6+
If a notebook path is given, refactor it. Otherwise create a new one.
7+
8+
---
9+
10+
## Notebook structure
11+
12+
Every user guide notebook follows this cell sequence:
13+
14+
```
15+
0 [markdown] # Title + subtitle (see title format below)
16+
1 [markdown] ### What you'll build (summary + eye-candy preview image + nav links)
17+
2 [markdown] One-liner about the imports
18+
3 [code ] Imports
19+
4 [markdown] ## Data section header
20+
5 [code ] Generate or load data (ONE call, reused everywhere)
21+
6 [markdown] Brief description of the raw data
22+
7 [code ] Show the data with a different colormap
23+
... Individual analysis sections (repeat pattern below)
24+
... Composite / combined section if multiple factors
25+
... Bonus visualization section (optional, for fun)
26+
N [markdown] ### References (with real URLs)
27+
```
28+
29+
### Individual analysis section pattern
30+
31+
Each analysis gets exactly this:
32+
33+
1. **Markdown intro**: `## Section name`, 2-4 sentences of context with a link to
34+
a real reference if one exists, then a note on what the plot shows.
35+
2. **Code cell**: compute the result, plot it overlaid on hillshade (or base layer),
36+
include a legend.
37+
3. **Markdown result description** (optional, 1-2 sentences): only if the output
38+
needs explanation.
39+
4. **Alert box** (optional): a GIS caveat relevant to the tool just shown, if
40+
there is one worth flagging that the section didn't already cover.
41+
42+
---
43+
44+
## Code conventions
45+
46+
### Plotting
47+
48+
- Use `xr.DataArray.plot.imshow()` for everything. No raw `ax.imshow(data.values)`.
49+
- Overlay pattern:
50+
```python
51+
fig, ax = plt.subplots(figsize=(10, 7.5))
52+
base.plot.imshow(ax=ax, cmap='gray', add_colorbar=False)
53+
overlay.plot.imshow(ax=ax, cmap=cmap, alpha=200/255, add_colorbar=False)
54+
ax.set_axis_off()
55+
```
56+
- Every overlay plot gets a legend via `matplotlib.patches.Patch`:
57+
```python
58+
from matplotlib.patches import Patch
59+
ax.legend(handles=[Patch(facecolor='red', alpha=0.78, label='Label')],
60+
loc='lower right', fontsize=11, framealpha=0.9)
61+
```
62+
- Use `add_colorbar=True` with `cbar_kwargs` only for quantitative maps (risk
63+
scores, continuous values). Use `add_colorbar=False` for categorical overlays.
64+
- Standard figure size: `figsize=(10, 7.5)`. Standalone plots: `size=7.5, aspect=W/H`.
65+
66+
### Colormaps and colorblind safety
67+
68+
- Never pair red and green. Use orange/blue, orange/purple, or red/blue instead.
69+
- For risk/heat maps: `inferno` (perceptually uniform, all CVD types).
70+
- For single-color categorical overlays: `ListedColormap(['color'])`.
71+
- RGB images: `dims=['y', 'x', 'band']` with float values in [0, 1].
72+
73+
### Data handling
74+
75+
- Generate or load data exactly once. Reuse the same array for all sections.
76+
- Use `xarray.where()` for filtering/masking, not manual numpy boolean indexing.
77+
- Handle NaN edges: `fillna(0)` before integer casting, explicit NaN masks for
78+
RGB arrays.
79+
- For hillshade: xrspatial returns values in [0, 1], not [0, 255].
80+
81+
### Imports
82+
83+
Standard import block:
84+
```python
85+
import numpy as np
86+
import pandas as pd
87+
import xarray as xr
88+
89+
import matplotlib.pyplot as plt
90+
from matplotlib.colors import ListedColormap
91+
from matplotlib.patches import Patch
92+
93+
import xrspatial
94+
```
95+
96+
Add extras (e.g. `hsv_to_rgb`) only when needed.
97+
98+
---
99+
100+
## Writing rules
101+
102+
1. **Run all markdown cells and code comments through `/humanizer`.**
103+
2. Never use em dashes (`--`, `---`, or the unicode character).
104+
3. Short and direct. Technical but not sterile.
105+
4. Opening cell has a title and subtitle:
106+
- **Title** (h1): `Xarray-Spatial {parent module}: {list a few tools covered}`.
107+
Examples: `Xarray-Spatial Surface: Slope, aspect, and curvature`,
108+
`Xarray-Spatial Proximity: Distance, allocation, and direction`,
109+
`Xarray-Spatial Focal: Mean, TPI, focal stats, and hotspots`.
110+
- **Subtitle** (plain text below the title): 2-3 sentences tying the tools to a
111+
real-world use case. Keep it grounded, not dramatic. Mention the topic and why
112+
it matters, skip intensity.
113+
5. "What you'll build" cell: an ordered list summarizing the steps/sections the
114+
reader will work through, an eye-candy preview image (`images/filename.png`),
115+
and anchor links to each `##` section. The preview should be the most visually
116+
striking output from the notebook. Generate it by running the relevant code
117+
with `matplotlib.use('Agg')` and
118+
`fig.savefig('examples/user_guide/images/name.png', bbox_inches='tight', dpi=120)`.
119+
6. Use lists for readability when there are 3+ parallel items.
120+
7. Section intros: 2-4 sentences max. Link to a real external reference if one
121+
exists. End with a short note on what the upcoming plot shows.
122+
8. Bonus/fun sections: frame them as "just for fun" or "extra credit", separate
123+
from the main narrative.
124+
9. References section at the end with real URLs, no filler.
125+
126+
---
127+
128+
## GIS alert boxes
129+
130+
After writing each section, evaluate whether it needs a GIS caveat the reader
131+
should know *now that they've seen the tool in action*. If so, add an alert box
132+
as the last cell of that section (after the code output and any result
133+
description). Not every section needs one. Skip the alert if the section's
134+
prose or code already covers the point. The goal is to catch gotchas the reader
135+
might hit when applying the tool to their own data, not to repeat what was just
136+
demonstrated.
137+
138+
Use Jupyter's built-in alert styling:
139+
140+
```html
141+
<div class="alert alert-block alert-warning">
142+
<b>Short label.</b> Concise explanation of the caveat. Keep it practical,
143+
not a legal disclaimer.
144+
</div>
145+
```
146+
147+
Alert types:
148+
- `alert-warning` (yellow): caveats, gotchas, assumptions that can bite you
149+
- `alert-info` (blue): tips, suggestions, "you might also want to look at X"
150+
- `alert-danger` (red): things that will silently give wrong results
151+
152+
Common GIS topics worth flagging (only when relevant and not already covered):
153+
154+
- **Map projection**: Euclidean tools on lat/lon coords give results in degrees.
155+
Mention `GREAT_CIRCLE` or recommend reprojecting to meters.
156+
- **2D vs 3D distance**: raster proximity ignores terrain relief.
157+
Point to `xrspatial.surface_distance` for terrain-following distance.
158+
- **Resolution and units**: cell size affects results. Slope depends on the
159+
ratio of elevation units to cell-spacing units.
160+
- **Edge effects**: convolution-based tools lose data at raster edges.
161+
Mention `boundary="nearest"` or similar padding.
162+
- **Coordinate order**: xrspatial expects `dims=['y', 'x']` with y as rows.
163+
Transposed data silently produces wrong results.
164+
165+
Write the alert text in the same direct, non-AI style as the rest of the
166+
notebook. Run it through `/humanizer` like everything else.
167+
168+
---
169+
170+
## File organization
171+
172+
- Preview images go in `examples/user_guide/images/`.
173+
- One notebook per topic. If a notebook covers too many things, split it.
174+
- Notebooks are self-contained: own imports, own data generation.
175+
176+
---
177+
178+
## Refactoring checklist
179+
180+
When refactoring an existing notebook:
181+
182+
1. Read the entire notebook first.
183+
2. Replace any `ax.imshow(data.values, ...)` with `data.plot.imshow(ax=ax, ...)`.
184+
3. Consolidate data generation to a single call.
185+
4. Add legends to all overlay plots.
186+
5. Fix any red/green color pairings.
187+
6. Add GIS alert boxes for relevant caveats (projection, units, edge effects).
188+
7. Restructure cells to match the section pattern above.
189+
8. Run all markdown through `/humanizer`.
190+
9. Verify the notebook executes: `jupyter nbconvert --execute`.
191+
192+
---
193+
194+
## New notebook checklist
195+
196+
When creating from scratch:
197+
198+
1. Pick a topic and a real-world angle for the opening.
199+
2. Write the full cell sequence following the structure above.
200+
3. Generate a preview image and save to `images/`.
201+
4. Add GIS alert boxes for relevant caveats (projection, units, edge effects).
202+
5. Run all markdown through `/humanizer`.
203+
6. Verify the notebook executes: `jupyter nbconvert --execute`.

examples/user_guide/11_Hydrology.ipynb

Lines changed: 45 additions & 654 deletions
Large diffs are not rendered by default.

examples/user_guide/12_Flood_Analysis.ipynb

Lines changed: 24 additions & 383 deletions
Large diffs are not rendered by default.

examples/user_guide/13_Fire_Analysis.ipynb

Lines changed: 30 additions & 495 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)