Skip to content

✨ feat: linkify() — URL/email auto-linking, a bleach successor - #29

Merged
gaborbernat merged 2 commits into
tox-dev:mainfrom
gaborbernat:feat/linkify
Jun 17, 2026
Merged

gaborbernat merged 2 commits into
tox-dev:mainfrom
gaborbernat:feat/linkify

Conversation

@gaborbernat

@gaborbernat gaborbernat commented Jun 17, 2026 •

Copy link
Copy Markdown
Member

bleach.linkify had no successor. nh3 declined to add one, and linkify-it-py finds links in plain text but is not HTML-aware, so projects that ran user content through bleach.linkify (comment systems, forums, markdown pipelines) have had nowhere to go since bleach reached end of life. 🔗 This adds turbohtml.linkify to serve them, closing the second half of bleach's scope after the sanitizer companion.

The new module keeps bleach's surface — linkify(text, callbacks=..., skip_tags=..., parse_email=...), a reusable Linker, and the nofollow/target_blank defaults under bleach's own names, so the common case is just an import change. Custom callbacks get a cleaner contract than bleach's tuple-keyed (attrs, new): each receives a Link with plain url, text, and attrs and returns it to keep the link or None to drop it. It is HTML-aware because it parses with turbohtml's WHATWG tree builder before linking, so it never rewrites a URL that already sits inside an <a>, a <script>/<style>, or a caller's skip_tags, which a regex over the raw string cannot guarantee. The scan for link candidates stays in C as the trigger-then-expand model the Rust linkify crate uses: it looks for the few bytes that can begin a link (:, @, .) and expands outward from each rather than backtracking a regex. A bare domain like example.com links only when its last label is a current TLD, matched against an IANA table regenerated through tox -e regen the same way the tag and entity tables are, now including the xn-- punycode entries so a real xn--p1ai matches while an invented one does not.

Two behaviors diverge from bleach, both deliberate: turbohtml leaves an existing <a> untouched instead of re-running callbacks over it, so linkifying is idempotent, and the TLD set is regenerable rather than frozen. ⚡ The C candidate scan and turbohtml's own tree carry it past bleach's html5lib pass by five to twenty times, and past linkify-it-py's pure-Python scan as well, with the figures on the performance page. Closes #9.

@gaborbernat gaborbernat added the enhancement New feature or request label Jun 17, 2026
bleach is end of life and nothing replaced its linkifier: nh3 declined to add
one, and linkify-it-py finds links in plain text but is not HTML-aware. Projects
that ran user content through bleach.linkify have had no migration target.

turbohtml.linkify takes its place, keeping linkify(), Linker, and the
nofollow/target_blank defaults under bleach's names so the common case is just
an import change. Custom callbacks get a cleaner contract than bleach's
tuple-keyed (attrs, new): each receives a Link with plain url, text, and attrs
and returns it to keep the link or None to drop it. It parses with the WHATWG
tree builder, so it never links inside an existing <a>, a script or style, or a
caller's skip_tags, which a regex over the raw string cannot do.

The scan for link candidates stays in C as the trigger-then-expand model the
Rust linkify crate uses: it looks for the few bytes that can start a link and
expands outward from each, rather than backtracking a regex. A bare domain links
only when its last label is a real TLD, matched against a regenerable IANA table
that now carries the xn-- punycode entries so a genuine xn--p1ai matches and an
invented one does not. The C scan and turbohtml's own tree carry it past
bleach's html5lib pass by five to twenty times and past linkify-it-py too.
The linkify benchmark compares against linkify-it-py but the migration guide
covered only bleach, so a linkify-it-py user had no porting path. Add a "From
linkify-it-py" section, completing migration coverage for every library the
benchmarks compare against.

Also link the first mention of each third-party library on every documentation
page to its project, so a reader meeting bleach, markupsafe, lxml, html5lib, and
the rest can follow through to the source instead of guessing the name.
@gaborbernat
gaborbernat merged commit 47ee79c into tox-dev:main Jun 17, 2026
35 checks passed
@gaborbernat
gaborbernat deleted the feat/linkify branch June 18, 2026 18:25
gaborbernat added a commit that referenced this pull request Oct 27, 2026
Moves every
[`tox-dev/github-workflows`](https://github.com/tox-dev/github-workflows)
pin to
[6.2.0](https://github.com/tox-dev/github-workflows/releases/tag/6.2.0).
👋 The GitHub Release now credits first-time contributors: the publish
step takes the New Contributors list from GitHub's generated notes for
the range since the previous tag and places it above the package and
comparison links
([#34](tox-dev/github-workflows#34)). The
changelog file no longer carries that list.

The pins also pick up the 6.0.2 to 6.1.0 changes: release asset names
match the names GitHub stores
([#29](tox-dev/github-workflows#29)), and the
rendered notes list several packages on one row with an ecosystem emoji
([#32](tox-dev/github-workflows#32)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

✨ feat: linkify() — URL/email auto-linking as a tokenizer filter

1 participant