Skip to content

fix(context): title Markdown and HTML documents by their h1 - #206

Merged
human0-bot[bot] merged 1 commit into
neuledge:mainfrom
JayOfTheKeyboard:fix/markdown-h1-doc-title
Oct 27, 2026
Merged

human0-bot[bot] merged 1 commit into
neuledge:mainfrom
JayOfTheKeyboard:fix/markdown-h1-doc-title

Conversation

@JayOfTheKeyboard

Copy link
Copy Markdown
Contributor

Problem

parseMarkdown takes a document's title from frontmatter, and otherwise from the file name:

const docTitle =
  frontmatter.title ||
  filePath.split("/").pop()?.replace(/\.(md|mdx|mdoc|qmd|rmd|adoc|rst)$/, "") ||
  "Untitled";

Many doc sets have no frontmatter title and put the page title in an # h1 instead. That h1 is then lost twice: it is not the title, and the h1 before the first ## is skipped, so it is not in the content either. doc_title has a BM25 weight of 5 and is the first half of every result title (docTitle > sectionTitle), so results read jobs > Creating jobs or class-apirequestcontext > ... instead of the page's name.

HTML is hit on every page. parseHtml goes through the same code, .html is not in the extension list, and turndown's # h1 is skipped like any other. Every Python 3.14 page is titled after its file, extension included (logging-cookbook.html, 3.3.html). The existing test "extracts h1 as doc title and h2 as section boundaries" never checks the doc title, which is api.html.

reStructuredText and AsciiDoc already use the document's top heading as the title (= Title, or the first underlined heading).

I built packages with context add on main and with this change and counted pages titled by their file name:

Docs Pages Titled by file name (main → fix)
Python 3.14 HTML 526 526 → 22
Drizzle ORM 517 419 → 49
Playwright 181 81 → 1
Vitest 229 50 → 7
pg-boss 24 24 → 2
Fastify 41 41 → 31
Pydantic 95 94 → 82
Starlette 27 27 → 24
Requests (reStructuredText) 15 0 → 0

Section titles and section content are identical to main in all nine packages. Only doc_title changes. Pages that stay on the file name have no Markdown h1 or several (for example Fastify's Reference/*.md, which open with an HTML <h1 align="center">Fastify</h1> banner and then ##, and Python's c-api/tuple.html, which has two h1s).

Some titles before and after:

Page main fix
Playwright api/class-apirequestcontext.md class-apirequestcontext class: APIRequestContext
Drizzle get-started/gel-existing.mdx gel-existing Get Started with Drizzle and Gel in existing project
Fastify Guides/Write-Plugin.md Write-Plugin How to write a good plugin
Python howto/logging-cookbook.html logging-cookbook.html Logging Cookbook
Python library/lzma.html lzma.html lzma — Compression using the LZMA algorithm

End to end, the top three search results for send jobs in pg-boss are introduction > Quick start, pubsub > Introduction, jobs > Creating jobs (part 3) on main, and Introduction > Quick start, Jobs > Creating jobs (part 3), Jobs > Cancelling, resuming, and retrying jobs with the fix.

Fix

  1. A document with no frontmatter title takes its title from its h1, when it has exactly one top-level h1. Several h1s are a concatenation such as an llms-full.txt, where the first page's name would mislabel the rest, so those keep the file name. Frontmatter still wins.
  2. The title text drops what is not title: an in-page link whose text is only a permalink symbol (the same symbols the existing sectionPermalink rule handles, through a shared isPermalinkText), a trailing attribute block ({#id} in Docusaurus and VitePress, {.unnumbered} or {-} in Pandoc), and doubled spaces left by removed inline HTML. Attribute blocks are only taken from plain text, so # Blocks: `{#each}` keeps its code.
  3. The file-name fallback also drops .html and .htm, in one fileTitle helper that AsciiDoc and reStructuredText now share.
  4. A file over 1 MB is parsed in chunks, and only the first chunk sees the h1. When a split file's chunks end up with different titles, all of its sections take the file name, so a file never carries two titles. A frontmatter title reaches every chunk already, so it is kept.

The permalink is dropped from the title only, not from the content. Sphinx puts navigation before the h1, so the h1 is often inside the Introduction section. Dropping its permalink there would change that section's link ratio for the table-of-contents filter, the reason #175 left anchors outside <h2> alone. I tried extending the turndown rule to <h1>: it changed the content of 651 Python sections and dropped 4. With this approach, section content is unchanged.

Not changed

  • Section titles. A ## Title {#id} keeps the {#id} in its section title, as before.
  • A site's logo in an <h1> that is not inside one of the removed elements counts as an h1. With the page's own h1 that makes two, so the page keeps its file name, as on main.
  • An h1 nested in a blockquote or list is not counted.
  • ParsedDoc.frontmatter.title is still only the frontmatter's title.

Test

build.test.ts:

  • A single h1 with an in-page link, an inline component and {#jobs-api} gives The Jobs API. Two h1s keep llms-full, and frontmatter beats the h1. # Preface {.unnumbered} gives Preface, and # Blocks: `{#each}` gives Blocks: {#each}.
  • HTML: the existing h1 test now checks docTitle (API Reference). Sphinx and Docusaurus zero-width permalinks are dropped from the title and left in the Introduction content. A page with no h1 is titled dict, not dict.html.

package-builder.test.ts: a split concatenation with one h1 per chunk keeps llms-full. The existing split test's HTML expectation changes from flat.html to flat.

search.retrieval.test.ts: the two HTML fixtures' expected titles change from systemctl.html > User services to systemctl > User services (and the same for systemd.service).

On the parent commit 8 tests fail, for example expected 'jobs' to be 'Jobs' and expected 'api.html' to be 'API Reference'. With the fix they pass.

I broke each part of the fix in turn, and each time a test went red:

Mutation Test that fails
No h1 title h1 title, HTML h1, both permalink tests
First h1 of several h1 title (llms-full)
Permalink links kept Sphinx and Docusaurus permalink tests
Every in-page link dropped h1 title (The Jobs API)
Zero-width space not stripped Docusaurus permalink test
No space collapse h1 title
No attribute-block strip h1 title
Only {#id} stripped h1 title (Preface)
Attribute block stripped from inline code too h1 title (Blocks: {#each})
.html not stripped file-name fallback, split test
Split files keep each chunk's own title split test, split concatenation

Validation

  • vitest run in packages/context: 18 files, 366 tests pass.
  • biome ci --error-on-warnings: clean.
  • tsc --noEmit -p tsconfig.build.json: clean.
  • Real docs: the tables above, from context add builds of nine doc sets on main and on this branch.
  • Changeset: .changeset/brave-lamps-glow.md (patch).

A document without a frontmatter title was named after its file, and its h1 was skipped, so neither the title nor the index had it. HTML pages kept their .html extension in the fallback. A document's only top-level h1 now names it (several h1s are a concatenation and keep the file name), without the permalink anchor HTML generators add, and the title is carried to every chunk of a split file.
@changeset-bot

changeset-bot Bot commented Oct 26, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: f65324e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@neuledge/context Patch
@neuledge/registry Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@human0-reviewer human0-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved — Search results now use a page's single top-level heading as its title when frontmatter is absent. HTML fallback titles no longer include the file extension.

Changes behavior: no — This corrects the parser's existing h1-title intent and aligns with its test contract; frontmatter precedence and section content stay unchanged. No earlier review on this pull request; this is the first look.


This codebase is run by Human0.

@human0-bot
human0-bot Bot merged commit 8310703 into neuledge:main Oct 27, 2026
4 checks passed
@github-actions github-actions Bot mentioned this pull request Oct 27, 2026
JayOfTheKeyboard added a commit to JayOfTheKeyboard/JayOfTheKeyboard that referenced this pull request Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant