Repository navigation
extract_text: embedded CFF Type1 font is re-parsed with fontTools for every font resource on every page (no caching) #4156
Description
Description
Since #4032 (parse the encoding of embedded CFF Type1 fonts), when fontTools is installed extract_text() parses an embedded /FontFile3 (/Type1C) font program with fontTools each time it builds the character map for a font resource. The result isn't cached, so the same indirect font object is parsed again:
- for every resource name that points at it (
/F0,/F1, … all referencing one font dictionary), and - on every page that uses those resources.
A large CFF font therefore costs one full fontTools parse per (page × resource name). A small crafted file makes text extraction take orders of magnitude longer with fontTools installed than without it, which matters for anyone running extract_text() on untrusted PDFs, such as mail or document indexers.
Environment
- pypdf 6.19.0
- fonttools 4.66.1
- Python 3.14.7, macOS (also seen in a Debian-based Linux container)
Measurements
One 60,000-glyph CFF font (about 1.2 MB), referenced under 100 resource names, 20 pages, each page showing one short string with /F0. The same file, the same pypdf, in two environments:
extract_text() over all pages |
|
|---|---|
| without fontTools | 0.03 s |
| with fontTools | 38.4 s |
The cost grows linearly with pages × resource names. At 500 pages and 1,000 resource names it extrapolates to hours for a file of about the same size.
Minimal reproduction
repro.py: generates the PDF (needs fontTools) and times extraction
import io, sys, time, logging
from importlib.util import find_spec
GLYPHS, REFS, PAGES = 60000, 100, 20
def build() -> bytes:
from fontTools.fontBuilder import FontBuilder
from fontTools.pens.t2CharStringPen import T2CharStringPen
from pypdf import PdfWriter
from pypdf.generic import (ArrayObject, ContentStream, DecodedStreamObject,
DictionaryObject, NameObject, NumberObject)
glyphs = [".notdef"] + [f"g{i}" for i in range(GLYPHS - 1)]
fb = FontBuilder(1000, isTTF=False)
fb.setupGlyphOrder(glyphs)
pen = T2CharStringPen(600, None)
pen.moveTo((0, 0)); pen.lineTo((1, 1)); pen.closePath()
cs = pen.getCharString()
fb.setupCFF("W", {"FullName": "W"}, {g: cs for g in glyphs}, {})
fb.setupHorizontalMetrics({g: (600, 0) for g in glyphs})
fb.setupHorizontalHeader(ascent=800, descent=-200)
fb.setupCharacterMap({}); fb.setupOS2(); fb.setupPost()
cff = fb.font["CFF "].cff
enc = [".notdef"] * 256
for i in range(1, 200):
enc[i] = glyphs[i]
cff.topDictIndex[0].Encoding = enc
buf = io.BytesIO(); cff.compile(buf, fb.font)
w = PdfWriter()
ff = DecodedStreamObject(); ff.set_data(buf.getvalue())
ff[NameObject("/Subtype")] = NameObject("/Type1C")
desc = w._add_object(DictionaryObject({
NameObject("/Type"): NameObject("/FontDescriptor"),
NameObject("/FontName"): NameObject("/W"),
NameObject("/Flags"): NumberObject(4),
NameObject("/FontBBox"): ArrayObject([NumberObject(0)] * 4),
NameObject("/FontFile3"): w._add_object(ff)}))
font = w._add_object(DictionaryObject({
NameObject("/Type"): NameObject("/Font"),
NameObject("/Subtype"): NameObject("/Type1"),
NameObject("/BaseFont"): NameObject("/W"),
NameObject("/FontDescriptor"): desc}))
res = w._add_object(DictionaryObject({NameObject("/Font"): DictionaryObject(
{NameObject(f"/F{i}"): font for i in range(REFS)})}))
for _ in range(PAGES):
page = w.add_blank_page(width=612, height=792)
s = ContentStream(None, w)
s._data = b"BT /F0 12 Tf 72 720 Td (" + b"A" * 60 + b") Tj ET"
page[NameObject("/Contents")] = w._add_object(s)
page[NameObject("/Resources")] = res
out = io.BytesIO(); w.write(out)
return out.getvalue()
if sys.argv[1:] == ["build"]:
open("cff_refs.pdf", "wb").write(build())
else:
logging.disable(logging.WARNING)
import pypdf
t = time.perf_counter()
for p in pypdf.PdfReader("cff_refs.pdf").pages:
p.extract_text()
print(f"fontTools={bool(find_spec('fontTools'))} {time.perf_counter() - t:.2f}s")python repro.py build # in an env with fontTools
python repro.py # with fontTools -> ~38 s
python repro.py # in an env without fontTools -> ~0.03 s
Suggested direction
Cache the parsed CFF result, or the derived encoding and widths, per font program for the life of the PdfReader, keyed by the indirect reference of the /FontFile3 stream (or the font dictionary). Then each distinct embedded font is parsed at most once per document, however many resource names or pages refer to it. A size or time bound on the fontTools parse would also help callers handling untrusted input, but caching alone removes the multiplication.
Happy to test a patch against this file.
Activity
stefan6419846 commentedon Oct 4, 2026
Thanks for the report.
extract_textis a method of the page, thus both the reader and the writer could be affected - and improving the reader might not be sufficient. On the other hand, only the reader is usually considered read-only from our side, with the writer being able to change such resources at any time.Under the given circumstances, I am not sure if caching for the life of the reader is sufficient or whether we have better ways to do this.
PJBrs commentedon Oct 5, 2026
Also, _character_map_from_cff_type1_font_file is only called when we encounter a simple font with an embedded Type 1C font file. My impression is that these are not very common.
stefan6419846 commentedon Oct 5, 2026
While this might be true, knowing that some service uses pypdf in the background for extracting the PDF text could lead to explicit abuse of this behavior from the user side. Services are expected to deploy suitable limits to avoid larger side effects, but if it is possible for us to optimize this in a suitable fashion without too much overhead, we should look into it from the pypdf side.
marshalltech81 commentedon Oct 5, 2026
For a data point on frequency: on a first full index of one personal mailbox, the "fontTools is required to fully parse the encoding of a CFF Type1 font" warning fired about 430 times across roughly 6,000 messages, from several font families (Arial, Times, Courier and Helvetica Neue subsets). They came from ordinary senders.
On the reader/writer concern: I tried keying a small LRU cache on a hash of the decoded
/FontFile3bytes, applied inside_character_map_from_cff_type1_font_file(the result depends only on those bytes). With the reproduction above (pypdf 6.19.0, fontTools 4.66.1):PdfWriter-owned pagesText output was identical. Because the key is the content, a writer that modifies the font program gets a cache miss and a fresh parse, so there's nothing to invalidate, and it works the same for readers and writers. Keying on the font dictionary's indirect reference would miss the second case. The worst case becomes one parse per distinct font program, so linear in file size.
PJBrs commentedon Oct 5, 2026
Ah, I stand corrected!
marshalltech81 commentedon Oct 5, 2026
The use case I have is that I am building a RAG pipeline around my email and I am using pypdf for processing PDF attachments. So I am processing years worth of them from my inbox and when I initially indexed them all I got the errors from pypdf and submitted them upstream. I hope that helps in understanding what I am attempting to do.
PJBrs commentedon Oct 5, 2026
I'm beginning to wonder whether the correctness in _cmap.py achieved with 1d92adf was worth it...
josuesto commentedon Oct 6, 2026
I tested a private cache prototype with Codex assistance.
The cache is stored on the embedded font stream. Before reusing a result, it compares the current decoded font bytes with the cached bytes. If they change, it parses again. This supports both reader and writer pages, including multiple font dictionaries sharing one stream.
In eight synthetic test cases, parsing dropped from 12 calls to 1, with identical extracted text. All 29 prototype test cases passed with each of fontTools 4.61.0 and 4.66.1. This is a private experiment, not a finished library change.
@marshalltech81, are you planning to submit a PR for your prototype? I would like to avoid duplicating your work.
Would maintainers be interested in this approach? One memory tradeoff is that the previous font bytes can remain cached after an edit until the next extraction or until the stream is released.
marshalltech81 commentedon Oct 6, 2026
@josuesto I would be open to submitting a PR however I was waiting on @PJBrs or @stefan6419846 as I did not want to step on any toes regarding and approach or what should be done.
stefan6419846 commentedon Oct 6, 2026
I currently have no preferred approach, as long as it looks reasonable and safe and does not have unexpected side effects.
We should make sure that this does not have any negative side effects on regular streams. As for detecting data changes, I would prefer the amount of data to store for comparison to be minimal. This means that we should most likely prefer hashing with a suitable (non-optional) hashing algorithm over storing yet another full stream copy.
marshalltech81 commentedon Oct 6, 2026
@josuesto @stefan6419846 PR opened: #4164. It caches the character map on the font file stream with a SHA-256 digest of the decoded data, so a replaced font program triggers a fresh parse and no copy of the previous data is kept. On the file from the report it goes from 36.75 s and 2,000 parses to 0.74 s and 1 parse.
test(extractors): pin the CFF-font PDF extraction gap with a strict x…
PI: Parse an embedded CFF font program once per stream (py-pdf#4164)
marshalltech81 commentedon Oct 9, 2026
@stefan6419846 , @PJBrs & @josuesto I want to personally say I appreciate the partnership!