[compile] Support for beyond-64k fonts by behdad · Pull Request #987 · googlefonts/ufo2ft

behdad · 2026-06-08T11:40:47Z

After postprocessing a compiled TTFont, convert supported tables to their beyond-64k uppercase companions when the glyph order exceeds 64k glyphs. This keeps the behavior in ufo2ft so fontmake and other callers receive a ready-to-save font.

The fontTools beyond64k helper is imported lazily, only when conversion is actually needed, so ordinary small-font builds do not require it at import time.

This builds upon fonttools/fonttools#4097

Assisted-by: OpenAI Codex

After postprocessing a compiled TTFont, convert supported tables to their beyond-64k uppercase companions when the glyph order exceeds 64k glyphs. This keeps the behavior in ufo2ft so fontmake and other callers receive a ready-to-save font. The fontTools beyond64k helper is imported lazily, only when conversion is actually needed, so ordinary small-font builds do not require it at import time. Tested with: - /Users/behdad/ft/venv/bin/python -m pytest -q tests/baseCompiler_test.py - /Users/behdad/ft/venv/bin/python -m pytest -q 'tests/integration_test.py::IntegrationTest::test_compile_filters[ufoLib2-compileTTF]' - /Users/behdad/ft/venv/bin/python -m pytest -q 'tests/integration_test.py::IntegrationTest::test_compileVariableTTF[ufoLib2-None]' - black --check Lib/ufo2ft/_compilers/baseCompiler.py tests/baseCompiler_test.py Assisted-by: OpenAI Codex

Replace the mocked beyond-64k helper tests with integration coverage that compiles generated UFOs. The small-font case verifies that compact tables remain in use, while the large-font case verifies that the compiler emits uppercase beyond-64k companion tables after crossing the glyph-count limit. Tested with: - /Users/behdad/ft/venv/bin/python -m pytest -q tests/baseCompiler_test.py - black --check tests/baseCompiler_test.py Assisted-by: OpenAI Codex

behdad · 2026-06-10T20:49:18Z

Probably also worth adding an option to opt into beyond64k spec or otherwise GLYF, to use the cubic feature. But that's separate from what this PR does, which can stand alone.

khaledhosny · 2026-06-11T21:02:02Z

Probably also worth adding an option to opt into beyond64k spec or otherwise GLYF, to use the cubic feature. But that's separate from what this PR does, which can stand alone.

There is support for the old glyf1 proposal (the allQuadratic option and head.glyphDataFormat = 1), I guess this is obsolete now? Maybe drop glyphDataFormat=1 and re-use the allQuadratic option?

anthrotype · 2026-06-16T12:21:06Z

I found an issue with sparse masters (a UFO layer or source with only a subset of glyphs). ufo2ft uppercases each master to the GLYF/LOCA/MAXP companion tables based on that master's own glyph count, and a sparse master may well stay below 64k, so it keeps the lowercase tables while the full masters go uppercase. varLib then rejects
the mismatched family with VarLibValidationError: Masters must all use the same glyf/GLYF table family.

import ufoLib2
from fontTools.designspaceLib import DesignSpaceDocument, AxisDescriptor, SourceDescriptor
import ufo2ft

order = ["g%05d" % i for i in range(0x10001)]  # 65537 glyphs -> full master is beyond-64k

full = ufoLib2.Font()
full.info.familyName, full.info.styleName = "T", "Regular"
full.info.unitsPerEm, full.info.ascender, full.info.descender = 1000, 800, -200
for n in order:
    full.newGlyph(n).width = 500
full.lib["public.glyphOrder"] = order
# sparse layer with fewer than 64k glyphs
sparse = full.newLayer("sparse")
sparse.newGlyph("g00001").width = 600

doc = DesignSpaceDocument()
doc.addAxis(AxisDescriptor(tag="wght", name="Weight", minimum=0, default=0, maximum=1000))

s0 = SourceDescriptor()
s0.font = full
s0.location = {"Weight": 0}
doc.addSource(s0)

s1 = SourceDescriptor()
s1.font = full
s1.layerName = "sparse"               # sparse master
s1.location = {"Weight": 1000}
doc.addSource(s1)

ufo2ft.compileVariableTTFs(doc)  # VarLibValidationError: Masters must all use the same glyf/GLYF table family

These exercise the top-level compile* functions, not the BaseCompiler class, so they belong with the other integration tests rather than in a separate module.

_maybe_uppercase_beyond64k ran on every postprocessed font, including the per-master fonts compiled while building a variable font. A sparse master (a layer with only a subset of glyphs) can stay below 64k while the full masters cross it, so it kept the lowercase glyf/loca/maxp tables while the others were upgraded to GLYF/LOCA/MAXP, and varLib then rejected the mismatched table family. Move the uppercasing inside the postProcessorClass guard, which is already nulled for VF interpolation masters (see _compileNeededSources), so the masters keep their lowercase tables and varLib gets a consistent family; the merged VF and standalone outputs still uppercase via their own postprocess. Add a regression test building a beyond-64k VF with a sparse master.

anthrotype · 2026-06-16T15:49:09Z

I just added a fix for the sparse master issue (including a test). It's safe to move/delay the uppercasing inside the postProcessorClass guard, which is already None-ed for VF masters, so they stay consistently lowercase (the VF itself is uppercased at the end).

_maybe_uppercase_beyond64k guarded on `len(glyphOrder) <= 0x10000`, so a font with exactly 65536 glyphs stayed on the lowercase glyf/maxp tables and overflowed maxp.numGlyphs (uint16) on save. 65536 is a count limit, not a gid-width one: gids 0..0xFFFF still fit, but the count 65536 does not. Guard on `<= 0xFFFF` instead. 65535 was already fine, 65537 already handled, so the existing tests at 65537 could not catch it; add one at exactly 65536.

anthrotype · 2026-06-18T13:29:12Z

I just fixed an off-by-one in _maybe_uppercase_beyond64k: it was leaving a font with exactly 65536 glyphs with lowercase tabes and maxp.numGlyphs (uint16) overflowed on save

behdad · 2026-06-19T12:08:18Z

Probably also worth adding an option to opt into beyond64k spec or otherwise GLYF, to use the cubic feature. But that's separate from what this PR does, which can stand alone.

There is support for the old glyf1 proposal (the allQuadratic option and head.glyphDataFormat = 1), I guess this is obsolete now? Maybe drop glyphDataFormat=1 and re-use the allQuadratic option?

PTAL.

Stop using the obsolete glyphDataFormat=1 output path for TrueType outlines that preserve cubic curves. The allQuadratic option still controls whether preprocessing is allowed to leave cubic curves in place, but final fonts that actually contain cubic glyf data are now converted to the beyond-64k companion table family. Keep the conversion in the final-output postprocess step so variable-font interpolation masters still use a consistent lowercase table family before varLib merges them. Tested with: - PYTHONPATH=Lib /Users/behdad/ft/venv/bin/python -m pytest tests/integration_test.py -q -k 'GLYF_not_allQuadratic or beyond64k' - black --check Lib/ufo2ft/_compilers/baseCompiler.py Lib/ufo2ft/_compilers/ttfCompiler.py Lib/ufo2ft/_compilers/interpolatableTTFCompiler.py Lib/ufo2ft/outlineCompiler.py tests/integration_test.py - git diff --check Assisted-by: OpenAI Codex

Also read cubic flags via glyf[name] in the existing GLYF tests.

Lazily-packed glyphs after a postprocessor reload (useProductionNames / name dropping) have no .flags, so cubics went undetected and wrongly stayed in the lowercase glyf table.

glyf v1 is gone: cubics now go in GLYF and head.glyphDataFormat stays 0. Fix the allQuadratic=False docstrings and hardcode the head field instead of reading a self.glyphDataFormat attribute that no longer exists.

behdad · 2026-06-19T16:23:18Z

Should we always generate upper tables if allQuadratic is not True?

anthrotype · 2026-06-22T09:36:12Z

Should we always generate upper tables if allQuadratic is not True?

Why? We'd be forcing GLYF onto what is otherwise a perfectly normal sub-64k quadratic glyf that every shipping renderer reads. If you're concerned about a compile-time cost, remember that recalcBBoxes=True by default and ufo2ft doesn't override that so every glyph gets expanded on save() anyway. That glyf[name] scan just pulls the same unpacking forward, it's not extra work.

behdad · 2026-06-22T12:49:49Z

Should we always generate upper tables if allQuadratic is not True?

Why? We'd be forcing GLYF onto what is otherwise a perfectly normal sub-64k quadratic glyf that every shipping renderer reads. If you're concerned about a compile-time cost, remember that recalcBBoxes=True by default and ufo2ft doesn't override that so every glyph gets expanded on save() anyway. That glyf[name] scan just pulls the same unpacking forward, it's not extra work.

Performance was one concern. But my other concern is that by setting allQuadratic to false, one is opting into possibly making fonts that don't work on most platforms currently. Always using GLYF makes it dead obvious of that. Only using GLYF if font has cubics, hides the problem if this allQuadratic value was set by mistake / ignorance.

Quoting Behdad #987 (comment) > by setting allQuadratic to false, one is opting into possibly making > fonts that don't work on most platforms currently. Always using GLYF > makes it dead obvious of that. Only using GLYF if font has cubics, hides > the problem if this allQuadratic value was set by mistake / ignorance.

anthrotype · 2026-06-22T14:17:58Z

@behdad yeah I can see the footgun. I changed to do it unconditionally when allQuadratic=False

behdad requested review from anthrotype and simoncozens June 8, 2026 11:40

davelab6 mentioned this pull request Jun 8, 2026

Support all 4 boring expansion spec changes fontra/fontra#2615

Open

simoncozens approved these changes Jun 9, 2026

View reviewed changes

khaledhosny reviewed Jun 9, 2026

View reviewed changes

Comment thread Lib/ufo2ft/_compilers/baseCompiler.py Outdated

Comment thread tests/baseCompiler_test.py Outdated

anthrotype added 2 commits June 16, 2026 15:16

[tests] Move beyond-64k compile tests to integration_test.py

c33c604

These exercise the top-level compile* functions, not the BaseCompiler class, so they belong with the other integration tests rather than in a separate module.

This was referenced Jun 17, 2026

[compile] Populate glyf.glyphs directly to avoid O(n^2) build #991

Merged

[glyf] Use reverse glyph map for O(1) __setitem__ membership fonttools/fonttools#4103

Merged

anthrotype added 2 commits June 17, 2026 12:24

Merge remote-tracking branch 'origin/main' into beyond-64k

690f6f2

khaledhosny reviewed Jun 19, 2026

View reviewed changes

Comment thread Lib/ufo2ft/_compilers/baseCompiler.py Outdated

behdad force-pushed the beyond-64k branch from 6e4c14d to 023de7c Compare June 19, 2026 12:57

khaledhosny reviewed Jun 19, 2026

View reviewed changes

Comment thread Lib/ufo2ft/_compilers/interpolatableTTFCompiler.py

anthrotype reviewed Jun 19, 2026

View reviewed changes

Comment thread Lib/ufo2ft/_compilers/baseCompiler.py Outdated

anthrotype added 3 commits June 19, 2026 17:01

[compile] Test cubic beyond-64k detection after glyph-name reload

e681bf2

Also read cubic flags via glyf[name] in the existing GLYF tests.

[compile] Detect cubic glyf via glyf[name], not the raw glyphs dict

98c64e6

Lazily-packed glyphs after a postprocessor reload (useProductionNames / name dropping) have no .flags, so cubics went undetected and wrongly stayed in the lowercase glyf table.

[compile] Drop stale glyphDataFormat=1 references

82a4cfa

glyf v1 is gone: cubics now go in GLYF and head.glyphDataFormat stays 0. Fix the allQuadratic=False docstrings and hardcode the head field instead of reading a self.glyphDataFormat attribute that no longer exists.

Uh oh!

Conversation

behdad commented Jun 8, 2026

Uh oh!

Uh oh!

Uh oh!

behdad commented Jun 10, 2026

Uh oh!

khaledhosny commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anthrotype commented Jun 16, 2026

Uh oh!

anthrotype commented Jun 16, 2026

Uh oh!

anthrotype commented Jun 18, 2026

Uh oh!

behdad commented Jun 19, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

behdad commented Jun 19, 2026

Uh oh!

anthrotype commented Jun 22, 2026

Uh oh!

behdad commented Jun 22, 2026

Uh oh!

anthrotype commented Jun 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

khaledhosny commented Jun 11, 2026 •

edited

Loading