Skip to content

Introduce displacement extension to lib3mf - #469

Draft
vijaiaeroastro wants to merge 4 commits into
developfrom
feature/displacement
Draft

vijaiaeroastro wants to merge 4 commits into
developfrom
feature/displacement

Conversation

@vijaiaeroastro

@vijaiaeroastro vijaiaeroastro commented Sep 9, 2026 •

Copy link
Copy Markdown
Collaborator

Displacement extension support

lib3mf supports version 1.0.0 of the 3MF Displacement extension at
http://schemas.3mf.io/3dmanufacturing/displacement/2023/10.

The public component definition exposes four resource types:

  • Displacement2D references a PNG texture attachment and stores its channel,
    U/V tile styles, and filter.
  • NormVectorGroup stores normalized displacement direction vectors.
  • Disp2DGroup links a texture and normal-vector group with height and offset,
    and stores indexed UV/vector/factor coordinates.
  • DisplacementMeshObject extends MeshObject with per-triangle displacement
    group and coordinate indices.

Models provide add, lookup, and iterator methods for each type. Generic object
and mesh lookup preserves the displacement subtype. MergeFromModel copies the
three non-object displacement resources and remaps their dependency graph.

The 3MF reader accepts displacement resources and meshes in root and production
model parts. It applies d1 defaults to d2 and d3, accepts a did default on
the triangles element, keeps core material properties on displacement
triangles, reads triangle sets, and requires the displacement namespace in
requiredextensions when a part has a displacement mesh.
The writer emits dependency-sorted resources, displacement-prefixed mesh data,
triangle sets, the namespace declaration, and the required-extension token.

Validation covers required attributes and resource order, PNG attachments,
enumerations, coordinate and vertex indices, material-resource indices,
nonempty vector and coordinate groups, at least four triangles, manifold and
oriented positive-volume meshes, and strictly positive scalar products between
referenced vectors and face normals (as the spec says). Build item transforms
are not limited, so mirrored displacement meshes are allowed.
Vectors supplied through the API or reader are normalized before storage.

Setting new geometry or replacing a triangle on a displacement mesh clears the
displacement of the replaced triangles. A texture, normal vector group or
coordinate group can be removed while it is still in use, as for other
resource types; writing the model then fails with "Resource not found".

The library preserves displacement data but does not rasterize a displacement
texture into new mesh geometry. Consequently, STL export and MergeToModel
reject meshes with active displacement mappings instead of silently exporting
the undisplaced base surface.

Open questions

These are things we need to agree on before merging. Some need a decision from
us, and some need a clarification in the spec.

For each question below, the code already does the safest thing we could
think of. The "For now" line says what it does. We can still change it.

Decisions for this PR

  1. Unknown attributes on displacement elements. What should the reader do
    when it finds an attribute it does not know on a displacement element? The
    example files in the spec repository have such an attribute (contenttype).

    For now: the reader gives a warning and continues, like the rest of
    lib3mf. (Before, it stopped loading the file.)

  2. STL export. STL cannot hold displacement. Exporting the base mesh would
    give the wrong shape without telling the user.

    • Do we want, later, a step that applies the displacement to the mesh and
      creates real triangles? Then STL, or a plain 3MF mesh, could be written
      with the correct shape. This would be a separate feature, not part of
      this PR.

    For now: STL export and MergeToModel refuse a mesh with
    displacements. The error says: "Displacement is not applied to the mesh
    geometry. Write a 3MF file, or clear the triangle displacements first."

  3. Triangle sets, beam lattice and volume data on a displacement mesh.
    Should these be supported?

    For now: triangle sets are read and written. Beam lattice and volume
    data are refused on read and on write with a clear error, because it is
    not clear what they mean on a displaced mesh. (Before, all of them were
    dropped without a warning.)

  4. requiredextensions check. When must d be listed in
    requiredextensions?

    For now: the reader requires it only when a model part has a
    displacement mesh. A part with only displacement resources (a texture,
    normal vectors or coordinate groups) does not need it. The writer still
    lists d in every part that has any displacement content.

  5. Triangle p1 without pid on normal meshes. The core spec says a
    triangle with p1 but no pid should use the pid of the object. lib3mf
    ignores the p1 in this case. Should we fix this for normal meshes, in a
    separate PR?

    For now: normal meshes work as before this PR (the p1 is ignored).
    Only displacement meshes use the pid of the object, as the spec says.

  6. Displacement mesh in one model part, groups in another. Should a
    displacement mesh in a non-root model part be allowed to use a coordinate
    group from another part? Our reader only looks in the same part.

    For now: the writer refuses this with a clear error, so lib3mf never
    writes a file that it cannot read back.

Questions for the 3MF Consortium

  • The example files use the old namespace
    http://schemas.microsoft.com/3dmanufacturing/displacement/2023/10. The spec
    says http://schemas.3mf.io/3dmanufacturing/displacement/2023/10.
  • In the example files, the displaced triangles use did="8". That id is the
    normvectorgroup. It should be 7, the disp2dgroup.
  • The example files put a contenttype attribute on displacement2d. The
    schema does not have this attribute.
  • The spec text calls the element disp2dcoords, but the schema calls it
    disp2dcoord. We follow the schema.
  • The table for the channel attribute lists R, G and B. The text and the
    schema also allow A. We accept A.

@codecov

codecov Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.63636% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.16%. Comparing base (a8e077d) to head (ce8bfb0).

Files with missing lines Patch % Lines
Autogenerated/Bindings/Cpp/lib3mf_implicit.hpp 86.66% 34 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #469      +/-   ##
===========================================
+ Coverage    60.05%   60.16%   +0.10%     
===========================================
  Files           67       71       +4     
  Lines        25306    26800    +1494     
===========================================
+ Hits         15198    16123     +925     
- Misses       10108    10677     +569     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vijaiaeroastro
vijaiaeroastro requested review from 3dJan and gangatp and removed request for gangatp October 7, 2026 12:02
@vijaiaeroastro vijaiaeroastro self-assigned this Oct 7, 2026
- Normal vectors must point strictly outward: a scalar product of
  exactly 0 with the triangle normal is now rejected, as the spec says.
- Allow mirrored build item transforms for displacement meshes. The
  spec has no such limit.
- Reader: d2/d3 without d1, and p2/p3 without p1, no longer fail. As the
  spec says, the triangle then has no displacement / uses the object
  property.
- SetGeometry and SetTriangle on a displacement mesh now clear the
  displacements of the replaced triangles, so old data does not stay on
  the wrong triangle.
- MergeFromModel: count source resources once, so merging a model into
  itself does not loop forever.
- Writing a model after removing a texture, normal vector group or
  coordinate group that is still in use now fails with "Resource not
  found" instead of a generic error.
- Core <triangles> attributes are ignored again, as before this branch,
  so normal files do not get new warnings.
- Mark the displacement extension as required only in model parts that
  contain displacement content.
- Add test files for valid and invalid displacement files, based on the
  example in the spec repository with its errors corrected.
Each open question in the PR now has a safe behaviour. The questions
stay open in the PR description, so the behaviour can still change.

- Unknown attributes on displacement elements give a warning instead of
  stopping the load, as in the rest of lib3mf.
- STL export and MergeToModel still refuse a mesh with displacements,
  but the error now says what to do (write 3MF, or clear the
  displacements first).
- Triangle sets on a displacement mesh are now read and written.
  Beam lattice and volume data on a displacement mesh are refused on
  read and write, with a clear error, instead of being dropped.
- The displacement extension must be in requiredextensions only when a
  model part has a displacement mesh. Displacement resources alone no
  longer need it.
- Normal (core) meshes read p1 without a triangle pid as before this
  branch (the p1 is ignored). Only displacement meshes use the object
  pid in this case.
- The writer refuses a displacement mesh that uses a displacement group
  from another model part, because our reader cannot read that back.
@vijaiaeroastro

Copy link
Copy Markdown
Collaborator Author

Proposal: apply displacement to the mesh (follow-up PR)

This is a proposal for a follow-up PR, not a change to this PR. It is here so
we can discuss it.

The problem

STL, and a plain 3MF mesh, cannot hold displacement. Today lib3mf refuses to
export a displaced mesh to STL (and refuses MergeToModel), with the message
"Write a 3MF file, or clear the triangle displacements first". This is safe,
but it means users cannot get the real displaced shape as plain triangles.

The idea

Add a step that applies the displacement and creates a normal mesh with many
small triangles. STL export, MergeToModel and users could then use that mesh.

For each displaced triangle:

  1. Split it into small triangles.
  2. For each new vertex, find its texture position (u, v), its direction
    (interpolated from the normal vectors, then normalized) and its factor f.
  3. Read the texture value at (u, v), using the channel, tilestyleu,
    tilestylev and filter of the displacement2d.
  4. Move the vertex: new position = position + (texture × height + offset) × f × direction.

The exact formulas are in
Formulaes.md
in the spec repository.

Points that need care:

  • Shared edges must be split the same way on both sides, or the result has holes.
  • When two triangles move a shared edge differently, section 5.2 of the spec
    says how to close the gap.
  • A strong displacement can make the surface cross itself. The spec solves this
    with the fill rule, which STL cannot express. Most slicers accept this, but we
    should document it.

Proposed API

DisplacementMeshObject::CreateDisplacedMesh(MaxEdgeLength) -> MeshObject

The caller gives a maximum edge length in model units. Triangles are split
until every edge is shorter than this. This is easy to understand, and shared
edges are split the same way on both sides.

Other options: one triangle per texture pixel (no parameter, but can create
very large meshes), or a fixed number of splits (the same number gives very
different detail on big and small triangles).

Reading the PNG

lib3mf cannot read PNG pixels today. The volumetric image stack only stores the
PNG files; it does not read them. So we need a PNG decoder.

The decoder must read every kind of valid PNG (grey, RGB, RGBA, palette;
1, 2, 4, 8 and 16 bits; interlaced), give the raw values without gamma or color
correction, and be safe with broken or malicious files, because a 3MF file can
come from anywhere.

What we checked (October 2026):

lodepng libpng libspng stb_image
Tested by Google OSS-Fuzz Yes Yes Yes Yes
Commits in the last 12 months 8 100+ 2 20
Last release no releases (commit based) 1.6.59, Sep 2026 v0.7.4, May 2023 no releases
Known security problems 2 CVEs (2019, 2022) about 12 CVEs since Nov 2025 1 advisory (2021) 2025 CVEs in other stb files still open
Size 2 files full library 2 files 1 file
License zlib libpng BSD-2 MIT / public domain
  • stb_image: not a good fit. Its README says security problems may take a
    long time to fix, and not to use it if that is a risk.
  • libspng: good design, but almost no activity (no release since 2023).
    Risky for code that reads untrusted files.
  • libpng: the most checked and fixed. It has many CVEs because many people
    look at it, so we would need to update it often. It is also a bigger
    dependency.
  • lodepng (suggested): few security problems, still fixed actively, fuzz
    tested, and only two files, so updates are easy. The risk is that it has one
    maintainer and no formal security advisories, so we need to watch it
    ourselves.

To keep the risk low, whichever decoder we choose:

  1. Read the PNG only when the user asks for the displaced mesh. Reading or
    writing a 3MF file never decodes the PNG.
  2. Check the image size (width, height, memory) before decoding.
  3. Put the decoder behind one small lib3mf function, so we can change it later.
    The volumetric extension could use the same function in the future.
  4. Write down the version we use, and check for updates before each lib3mf
    release.

Questions for the team

  1. Do we want this feature?
  2. Is it OK to add a PNG decoder to Libraries/? Which one?
  3. Is CreateDisplacedMesh(MaxEdgeLength) a good API?
  4. Should STL export use it automatically (with a default edge length), or
    should the user call it first?

Until this is done, this PR keeps the current behaviour: STL export of a
displaced mesh is refused with a clear message.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant