Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/scripts/notebook_to_mdx.py
Original file line number Diff line number Diff line change
Expand Up @@ -401,8 +401,8 @@ def convert_notebook_to_mdx(
mdx_parts.append("```")
mdx_parts.append("")

# Generate output filename (underscores to dashes)
output_name = notebook_path.stem.replace("_", "-") + ".mdx"
# Generate output filename (underscores to dashes, lowercase)
output_name = notebook_path.stem.replace("_", "-").lower() + ".mdx"
output_path = output_dir / output_name

# Ensure output directory exists
Expand Down
66 changes: 39 additions & 27 deletions .github/workflows/scripts/update_docs_navigation.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,26 +11,44 @@
from pathlib import Path


# Mapping of MDX filename prefixes/patterns to category groups
CATEGORY_MAPPINGS = {
"Part-": "Getting Started",
"RasterFlow-": "RasterFlow",
"Clustering-": "Analyzing Data",
"Getis-": "Analyzing Data",
"GPS-": "Analyzing Data",
"Isochrones": "Analyzing Data",
"K-Nearest-": "Analyzing Data",
"Local-Outlier-": "Analyzing Data",
"PMTiles-": "Analyzing Data",
"Zonal-": "Analyzing Data",
"Loading-Common-": "Reading and Writing Data",
"Map-Tile-": "Reading and Writing Data",
"STAC-": "Reading and Writing Data",
"Unity-": "Reading and Writing Data",
"ESA-": "Open Data Connections",
"Foursquare-": "Open Data Connections",
"NOAA-": "Open Data Connections",
"Overture-": "Open Data Connections",
# Full mapping of MDX filenames (without extension) to category groups
# Filenames are lowercase with hyphens (converted from notebook names)
FILENAME_TO_CATEGORY = {
# Getting Started
"part-1-loading-data": "Getting Started",
"part-2-reading-spatial-files": "Getting Started",
"part-3-accelerating-geospatial-datasets": "Getting Started",
"part-4-spatial-joins": "Getting Started",
# Analyzing Data
"clustering-dbscan": "Analyzing Data",
"getis-ord-gi*": "Analyzing Data",
"gps-map-matching": "Analyzing Data",
"isochrones": "Analyzing Data",
"k-nearest-neighbor-join": "Analyzing Data",
"local-outlier-factor": "Analyzing Data",
"pmtiles-railroad": "Analyzing Data",
"zonal-stats-esaworldcover-texas": "Analyzing Data",
# RasterFlow
"rasterflow-bring-your-own-model": "RasterFlow",
"rasterflow-bring-your-own-rasters-naip": "RasterFlow",
"rasterflow-changedetection": "RasterFlow",
"rasterflow-chesapeake": "RasterFlow",
"rasterflow-chm": "RasterFlow",
"rasterflow-ftw": "RasterFlow",
"rasterflow-s2-mosaic": "RasterFlow",
"rasterflow-tile2net": "RasterFlow",
# Reading and Writing Data
"loading-common-spatial-file-types": "Reading and Writing Data",
"map-tile-generation": "Reading and Writing Data",
"stac-reader": "Reading and Writing Data",
"unity-catalog-delta-tables": "Reading and Writing Data",
# Open Data Connections
"esa-worldcover": "Open Data Connections",
"foursquare-places": "Open Data Connections",
"noaa-swdi": "Open Data Connections",
"overture-maps": "Open Data Connections",
# Scala
"getting-started": "Scala",
}

# Order of categories in navigation
Expand All @@ -46,13 +64,7 @@

def get_category(filename: str) -> str:
"""Determine category for a notebook based on filename."""
for prefix, category in CATEGORY_MAPPINGS.items():
if filename.startswith(prefix):
return category
# Default to Scala for Getting-Started (scala version)
if filename == "Getting-Started":
return "Scala"
return "Other"
return FILENAME_TO_CATEGORY.get(filename, "Other")


def build_notebook_navigation(notebooks_dir: Path) -> list:
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ help:
@echo "Configuration:"
@echo " DOCS_DIR Path to docs repo (default: ../docs)"

convert:
convert: clean
@echo "Converting notebooks to MDX..."
python3 .github/workflows/scripts/notebook_to_mdx.py \
$(NOTEBOOK_DIRS) \
Expand Down
38 changes: 38 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,44 @@ pre-commit run --all-files
Sometimes this will fail and update your notebooks or the `README` file. Generally, you can re-run the
command and it will pass as the pre-commit hooks will fix the issues it finds.

## Documentation publishing

Notebooks in this repository are automatically converted to MDX format and published to the [Wherobots documentation](https://docs.wherobots.com) site. This happens via a GitHub Actions workflow (`.github/workflows/convert-notebooks.yml`) that runs when notebooks are modified on the `main` branch.

### How it works

1. **Conversion**: The `notebook_to_mdx.py` script converts `.ipynb` files to `.mdx` format, extracting markdown and code cells while generating appropriate frontmatter.
2. **Navigation**: The `update_docs_navigation.py` script updates the docs navigation structure to include the converted notebooks under "Spatial Analytics Tutorials" > "Example Notebooks".
3. **Publishing**: A PR is automatically created against the `wherobots/docs` repository with the converted files.

### Adding a new notebook

When you add a new notebook to this repository, you **must** update the navigation mapping so it appears in the correct category in the documentation:

1. Edit `.github/workflows/scripts/update_docs_navigation.py`
2. Add your notebook to the `FILENAME_TO_CATEGORY` dictionary with the appropriate category
3. The filename key should be lowercase with hyphens (e.g., `My_New_Notebook.ipynb` becomes `"my-new-notebook"`)

Example:
```python
FILENAME_TO_CATEGORY = {
# ... existing entries ...
"my-new-notebook": "Analyzing Data", # Add your notebook here
}
```

Available categories:
- `"Getting Started"`
- `"Analyzing Data"`
- `"RasterFlow"`
- `"Reading and Writing Data"`
- `"Open Data Connections"`
- `"Scala"`

If you don't add your notebook to the mapping, it will appear under an "Other" category in the documentation.

**Note**: Notebooks with the `Raster_Inference_` prefix are excluded from documentation publishing.

## Repository structure

```
Expand Down