Skip to content

Feature/bfme tool integration - #280

Open
LutBox wants to merge 35 commits into
OpenSAGE:masterfrom
LutBox:feature/bfme-tool-integration
Open

LutBox wants to merge 35 commits into
OpenSAGE:masterfrom
LutBox:feature/bfme-tool-integration

Conversation

@LutBox

@LutBox LutBox commented Aug 3, 2026

Copy link
Copy Markdown
Contributor
image

Asset Search Paths:
Texture Finder: Automatically locate and fix missing textures from BfME .big archives
BIG Archive Support: Extract textures and models from BfME2 and RotWK .big files
Smart Caching: Multi-threaded extraction with intelligent caching system -> might take a while for the first time
Progress Tracking: Real-time progress updates during texture scanning

W3D Model Browser:
Model Discovery: Browse all .w3d models from configured asset paths
Live Preview: Auto-generated 3D previews with lighting
One-Click Import: Direct import from browser to main scene

W3D Tools:
UV Mapping Fix: Automatically fix UV island splits and apply smooth shading (my most favorite feature)
Structure Creation: Generate W3D armature & bones for each mesh
Collision Geometry: Generates Geometry objects automatically (works semi good at the moment)
Bone Creation: Auto-generate effect bones (FIRE, SMOKE, ARROW) with even distribution in the upper area of ​​the model

Export Settings:
Auto-Detection: Automatically configure export settings from scene data (works semi good at the moment)
Conflict Resolution: Auto-fix mesh/bone naming conflicts
Texture Processing: Automatic .dds to .tga conversion in exported files and deletes all .001, .002, etc. suffixes

Animation Tools:
Build-Up Animation: Generate structure build-up animation
Destroy Animation: Generate structure destroy-animation
Existing Animations: List up existing animation which can be imported

LutBox added 2 commits August 3, 2026 20:19
Verified against Blender 5.2.0 LTS (Python 3.13): the full test suite,
an install as extension and as legacy add-on, and a W3D/W3X export and
re-import roundtrip.

Replace all deprecated API usage, so the add-on no longer emits a single
deprecation warning on 5.2:

* Material.blend_method -> Material.surface_render_method. The old
  property has no effect in EEVEE Next, so transparency of imported
  materials was silently lost since Blender 4.2.
* Material.show_transparent_back -> Material.use_transparency_overlap
* Material.use_nodes, which gets removed in Blender 6.0. The node tree
  is only requested when the material does not have one yet.
* Mesh.vertex_colors -> Mesh.color_attributes
* MeshUVLoopLayer.data -> MeshUVLoopLayer.uv

All of them go through version guarded helpers, the add-on keeps working
on the older Blender versions the CI covers.

Make the add-on installable as an extension:

* add blender_manifest.toml, the released archive now works both as an
  extension (Blender 4.2+) and as a legacy add-on archive.
* use relative imports inside the package. As an extension the package
  is named bl_ext.<repo>.io_mesh_w3d, so the absolute io_mesh_w3d.*
  imports failed to resolve and enabling the add-on errored out.
* skip the bundled updater when running as an extension, extensions are
  updated through their repository instead.

Bugfixes found along the way:

* import crashed on Blender 4.1: Mesh.use_auto_smooth was removed in
  4.1, not in 4.2 as the version guard assumed.
* w3x writing relied on the truth value of an xml element, which is
  deprecated in Python and will always be True in a future version.
  Reading it as a child count keeps the intended meaning.
The BfMe Tools were a separate add-on built on top of this one. They now live in
io_mesh_w3d/bfme and are registered by the W3D add-on, so there is a single
add-on to install. pyBIG is vendored under bfme/vendor, which replaces the
sys.path manipulation the old add-on did at import time.

Being in the same package removes the guesswork: three modules used to iterate
all of bpy.ops looking for something whose name contained "w3d" or "opensage",
try it with three different call conventions and hope. They now call
import_mesh.westwood_w3d and export_mesh.westwood_w3d directly.

The tools did not run on Blender 4.2+ in several places:

* Action.fcurves was removed with the slotted actions of 4.4, which broke both
  animation generators and the preview renderer. They go through the channelbags
  now, like the rest of the add-on already did.
* Material.shadow_method and the EEVEE use_bloom/use_ssr settings were removed
  in 4.2, every preview render raised on them.
* Material.blend_method, Material.use_nodes and Mesh.vertex_colors are
  deprecated, they now go through the add-on's version guarded helpers.

Correctness fixes:

* the texture and model scans mutated bpy data from worker threads. The Blender
  API is not thread safe, so the scene is snapshotted on the main thread, the
  worker only touches the file system, and the results are applied back in the
  modal handler.
* preview generation raised NameError when every imported object was hidden, and
  on any failure after switching the window's scene it left the window pointing
  at a scene it then deleted. It no longer switches the window at all.
* the build-up generator removed f-curves while iterating over them, so it only
  cleared every second curve.
* the default bone entries handler was a closure created inside register(), so
  every re-register leaked another copy into load_post.
* the bone UIList's draw_item was missing the index and flt_flag parameters.
* an animation search read every candidate .w3d fully into memory; it is
  streamed now, and the chunk overlap is carried correctly so a skeleton name
  spanning a chunk boundary is still found.

Performance:

* collision geometry analysis, bone placement and the scene height lookup ran
  Python loops over every vertex, several times over, building a Vector per
  vertex. They pull the coordinates out once with foreach_get and reduce with
  numpy, which the old code already imported but never used.
* the UV island flood fill used list.pop(0) and rebuilt a loop map per edge.
* per-bone animation settings and f-curve clearing were linear scans per bone.
* material fixups recursed over Object.children, which walks all objects on
  every access; children_recursive resolves the subtree once.
* the model list re-sliced its pending list on every timer tick.
* .dds to .tga rewriting rescanned the whole file per occurrence.
* the cache directory listing uses scandir, which carries the size along.

The bfme tests are ported into the Blender runner. They previously installed a
MagicMock as sys.modules['bpy'], which would have replaced bpy for every other
test in the shared runner process.

437 tests pass on Blender 5.2, the tools emit no deprecation warnings, and the
add-on still builds and installs as an extension.
Comment thread io_mesh_w3d/bfme/vendor/__init__.py Outdated
LutBox and others added 22 commits August 8, 2026 20:21
The panels lived under Properties > Scene, alongside every other scene
setting. Moved them to their own 'BfMe' tab in the N-panel sidebar
(VIEW_3D/UI) instead, where the modding tools are easier to find and
don't compete for space with unrelated scene properties.

Only the root panel needs bl_category, child panels inherit it from
their bl_parent_id; bl_context is Properties-editor specific and does
not apply to VIEW_3D panels, so it is dropped everywhere.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
bpy.ops.wm.read_homefile(app_template='') loads whatever startup.blend
is saved in the developer's own Blender profile, not Blender's bundled
default scene. On a profile with a customized startup file (no default
'Collection'), test_roundtrip_hlod_only_import and
test_roundtrip_single_mesh_imports failed with '2 != 1' on
len(bpy.data.collections), since the assumed default collection was
missing.

Added TestCase.resetToDefaultScene(), which passes
use_factory_startup=True to pin this to Blender's shipped default
scene regardless of the local profile, and pointed every call site in
both W3D and W3X roundtrip tests at it instead of duplicating the flag.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
create_material_from_vertex_material/create_material_from_shader_material key
their material lookup on '<mesh name>.<material name>', so every mesh gets its
own material datablocks even when several meshes reference an identical
definition (common for kitbashed props built from many meshes sharing one
texture) - one duplicate material per mesh, all pointing at the same texture.

Add a post-import pass that merges materials which are equivalent in every
property this add-on writes onto them, once the whole file has been imported.
The shader chunk that also affects a material's appearance is only applied
after the material itself is created, so comparing the fully built Blender
materials afterwards is simpler and more robust than trying to deduplicate at
creation time.

The signature is built by introspecting every 'is_runtime' (i.e. custom)
property on the material and its nested shader properties instead of hardcoding
a field list, so it stays correct as more custom properties get added, plus the
handful of Blender builtins (diffuse/specular color, Principled BSDF inputs,
referenced textures) this add-on writes directly. Merging goes through
Material.user_remap(), which redirects mesh material slots without touching
slot count or per-face material_index values, so it can't corrupt face
assignments even when a duplicate is one of several materials on the same mesh.

Verified end to end: 20 meshes sharing one material, exported and reimported,
now come back sharing a single material datablock instead of 20 duplicates.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The previous change gave the BfMe tools their own 'BfMe' tab, but the base
add-on already had an unrelated 'W3D Tools' panel (geometry/bone-volume
export) sitting in Blender's fallback 'Misc' tab, since it never set a
bl_category. That wasn't what was wanted: one tab, not two.

TOOLS_PANEL_PT_w3d (io_mesh_w3d/__init__.py) now explicitly declares
bl_category = 'W3D Tools' instead of falling into 'Misc'. All 7 former BfMe
panels move into that same tab as flat top-level panels instead of nesting
under the now-removed SCENE_PT_bfme wrapper, which only ever existed to
anchor their bl_parent_id and never drew anything itself.

This surfaced a second collision the previous change didn't have to deal
with: the base add-on's export panel and BfMe's own w3d_tools.py panel
(UV mapping/structure/collision geometry/bone creation) were both labelled
'W3D Tools', which would show as two identically headed panels in one tab.
Renamed the base add-on's to 'Geometry Export', matching what it actually
does, and left BfMe's panel as 'W3D Tools' since it now doubles as the tab's
namesake.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
TOOLS_PANEL_PT_w3d was a standalone panel with just two export buttons
(Export Geometry Data, Export Bone Volume Data), sitting alongside BfMe's
own more comprehensive 'W3D Tools' panel in the same tab. Both cover the
same subject (collision geometry data for the game's .ini files), so fold
the standalone panel into the existing 'Collision Geometry' box of BfMe's
W3D Tools panel instead of keeping it separate.

The ExportGeometryData/ExportBoneVolumeData operators are unchanged, only
the panel that exposed them is gone.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reported: importing an animation via 'Existing Animations' onto a skeleton
that already has one corrupts it instead of replacing it.

Root cause: the W3D animation importer keyframes the target skeleton
directly (bone.keyframe_insert(...) for every channel) rather than building
a standalone action and assigning it afterwards. keyframe_insert() adds to
whatever action is already assigned to the object if there is one; it only
auto-creates a fresh action when animation_data.action is None. So
re-importing onto an already-animated rig wrote the new animation's
keyframes into the *same* action as the old one, on the same fcurves,
silently overwriting overlapping keyframes and leaving the rest as a mix
of both animations.

Verified with a minimal repro: keyframing a bone at frame 0 twice without
detaching the action in between reuses the same Action object and the
second keyframe_insert() overwrites the first (1.0 -> 5.0 at frame 0), only
3 fcurves total for both "animations" combined.

Fix: BFME_OT_import_animation now detaches the target's existing action(s)
- both the object-level one driving pose bones and the armature data-level
one driving bone visibility channels - before calling the importer, so
keyframe_insert() creates a clean new action instead of writing into the
old one. The detached action is only actually removed once the import has
succeeded; a failed import restores it so the previous animation isn't
lost either.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The model browser used to fill a cache directory before it could show
anything: every .w3d, .dds and .tga in every selected .big was extracted
into %TEMP%, and every loose file in a search path was copied there too.
Measured against a full BfMe II + RotWK install (223 archives, 10 GB) that
is 35262 entries and 6267 MB written before the first model appears, and
any change to the archive selection invalidated the signature and sent it
round again.

Replace the copy-everything cache with an index of references:

    ['file', path]                            a loose file, used directly
    ['big', archive, entry, position, size]   an entry inside a .big

Loose files are now never copied at all - the index simply points at them.
Archive entries carry the entry's byte range, so materialising one later is
a plain seek and read with no archive re-parsing. Nothing is written to
disk until something actually needs to open a file, which only Blender
does, since it cannot read from inside an archive.

Building the index is cheap because opening an archive only parses its
entry table, not its contents. Same install, measured end to end: 0.40 s
to index all 32808 assets (19485 models) with zero files written and the
cache directory not even created; resolving 5 models for import takes 8 ms
and writes 957 KB.

Consequences per tool:

* the model list is built from the index, so it no longer requires anything
  to have been extracted first
* preview validity is checked against the reference (archive stamp plus
  byte range), so a cached preview costs no extraction
* the texture finder resolves only the textures actually missing from the
  scene, rather than materialising every texture in every archive
* the animation search streams each candidate's bytes straight out of its
  archive, so scanning for a skeleton name still touches no disk

The per-search-path 'Load to cache' switch is removed: its only purpose was
to opt into the copying that no longer happens.

Also made which archive wins a duplicate name deterministic. The previous
merge consumed thread results in completion order, so with several archives
providing the same name the winner depended on thread scheduling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Regression from the previous commit. The W3D importer resolves a model's
dependencies by name *relative to the file it is importing*:

    <directory>/<hierarchy name>.w3d      the skeleton
    <directory>/<texture name>.<ext>      each texture

The old copy-everything cache satisfied that by accident: every .w3d, .dds
and .tga from every archive was extracted into one flat directory, so
whatever a model asked for happened to be sitting next to it. Switching to
materialising a single file on demand removed that, so skinned models
imported without their armature and everything imported without textures.
I called this constraint out when analysing the change and then did not
implement it.

Read the names a model refers to out of the file itself and place those
next to it. The scanner walks the W3D chunk tree, taking texture names from
W3D_CHUNK_TEXTURE_NAME and the skeleton from the HLOD and animation
headers, skipping payloads, so it costs a single read of the model.

Staging keeps the point of the previous commit intact:

* a model from an archive is written to the cache directory together with
  its skeleton and textures, and nothing else from that archive
* a loose model whose dependencies already sit beside it is imported where
  it lies, still with nothing copied
* a loose model is only staged when a dependency lives in an archive or in
  a different folder, since we have no business writing into the user's
  own asset directories

Verified against the real BfMe II install: importing a skinned model now
yields its armature with all bones and no missing textures, cold cache and
warm, having written 3 files rather than 6267 MB.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported: ARICECLFFTLL03 imports without its texture, which lives at
art/compiledtextures/ar/ARiceclifftall.dds while the model asks for
ARiceclifftall.tga.

The dependency scanner only looked at W3D_CHUNK_TEXTURE_NAME, the classic
place a texture is named. Dumping the model's chunk tree shows it is a
shader material model (NormalMapped.fx), where the textures are string
valued shader material properties instead:

    0x50 > 0x51 > 0x53   type=STRING, "DiffuseTexture" -> "ARiceclifftall.tga"
                         type=STRING, "NormalMap"      -> "ARiceclifftall_NM.tga"

That is what most BfMe II era models use, so the scanner was missing
textures for a large share of them and only ever found the skeleton.

Read string valued properties out of 0x53 as well. Any string property is
taken as a candidate name; the ones that are not assets simply do not
appear in the index and drop out, so there is no need to keep a list of
which property names hold textures.

The .tga/.dds mismatch needed no special handling: the index is keyed by
name without extension, and the importer already tries every extension it
knows when looking beside the model.

Verified against the reported model: both textures now resolve out of
compiledtextures/ar and end up wired into the material. Across 400 random
models from the BfMe II, RotWK and Edain assets, 95% now have every
reference they name resolvable; the rest point at assets that are not
present in the indexed sources at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… own path

Reported: importing via File > Import > Westwood W3D looked shinier than
importing the same file through the BfMe model browser.

The model browser called the core import operator and then, in its own
wrapper code, zeroed out every imported material's Principled BSDF
specular input - W3D materials do not carry one, and the shininess value
create_material_from_vertex_material() maps onto that socket (typically
0.5 in test fixtures, and non-zero in real exported models) does not
correspond to it. The core operator itself never did this, so it only
ever happened for imports that went through BfMe's wrapper.

Moved the fixup into ImportW3D.execute() in io_mesh_w3d/__init__.py, the
one operator both paths already call, instead of teaching the File > Import
path to duplicate BfMe's post-processing. flatten_materials()/
zero_specular() move to common/utils/material_import.py as shared,
reusable functions. The BfMe model browser's own copies are removed:
its import operator no longer needs any post-processing at all, and its
preview renderer keeps only the forced alpha-blend material override that
is specific to rendering a thumbnail, since the specular fix already
happened inside the shared import call it makes.

Verified with both paths against the same real model
(ARICECLFFTLL03.w3d): bpy.ops.import_mesh.westwood_w3d directly and
bpy.ops.w3d.import_model now both leave Specular IOR Level at 0.0.

Added a regression test that goes through the actual
bpy.ops.import_mesh.westwood_w3d operator rather than the lower level
load()/create_data() the other roundtrip tests use, since the fixup lives
in the operator and none of the existing tests exercised it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…rowser

The sub-panels under W3D Tools all expanded on every install; they now default
closed like the outer panel. The model browser no longer needs a manual "Scan
W3D Models" click either: a persistent background timer indexes the configured
search paths and .big archives shortly after Blender starts and periodically
afterwards, merging in only what changed so the current selection survives.
Search paths are now indexed in the same thread pool as .big archives instead
of sequentially after them.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A model whose texture happened to share a base name with an unrelated .w3d
file (e.g. RBFARM_SKN.w3d's 'pfence01' texture vs. an unrelated
'pfence01.w3d' prop model, both real BfMe II assets) could silently lose that
texture on import or preview. The asset index only kept one reference per
name, so whichever asset's archive got scanned first won regardless of kind.

dependencies.py now tells texture references apart from hierarchy/skeleton
references instead of returning one flat set of names. cache.py's asset index
keeps a same-name texture and .w3d file in separate 'textures'/'models'
buckets alongside the existing flat map, and dependency_keys()/
stage_for_import() resolve through those buckets so a model's own textures
and skeleton can never be shadowed by an unrelated file that happens to share
their name.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A first-time scan on a real install can find tens of thousands of models;
applying that as one big diff synchronously inside a single bpy.app.timers
callback stalled Blender for the frame it took to insert them all, felt as a
hitch right around startup. The diff is now queued and applied 500 items at a
time across successive ticks, the same batch size and reasoning
W3D_OT_scan_models already uses for its own modal insertion. A manual scan
now also drops any in-flight auto-refresh batch so the two can't fight over
the same collection.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Blender calls a UIList's filter_items() for each redraw of the list, including
every frame of a scroll. The alphabetical sort added for the auto-refresh ran
there unconditionally, costing ~15 ms for a real install's 21k models - more
than a 60 fps frame budget on its own - so the list could not be scrolled
smoothly no matter how long the scan had been finished. The sort and filter
result is now cached and recomputed only when the collection or the filter
text actually changes, taking the scroll path from ~15 ms to ~0.0003 ms.

Three other things that were competing with the UI:

- a refresh tagged every area for redraw after each batch, forcing a full 3D
  viewport redraw per batch; it now tags only the sidebar region the list is in
- the first scan of a session started one second in, reading every archive
  header with a cold file cache (~3 s of disk I/O on a full install) while
  Blender was still opening its own files; it now waits until startup settles,
  and periodic rescans went from every 20 s to every 120 s
- cached_asset_index() took the index lock, so importing or previewing while a
  background rescan was running blocked the main thread for the length of the
  rebuild; the index is swapped by one atomic assignment, so it now reads
  without the lock

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The list still stuttered when scrolled. Caching the sort in filter_items had
removed the Python cost, but Blender was still handed a 21k entry reorder
array on every redraw and redid that mapping itself each time.

There is now no reordering at all. The periodic background rescan is gone, so
the list is only ever filled in two places - once per Blender session by a
self-unregistering timer, and by 'Scan W3D Models' - and both insert the
models already sorted, which is what makes an empty reorder array correct.
Matching against the filter text is still cached. A redraw with a full
install's 21512 models went from ~15 ms to ~0.01 ms.

Filling the list also redraws the sidebar once at the end instead of once per
batch, and the scan is sorted case insensitively, since that order is now
what actually reaches the UI.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s and Animation

'Existing Animations' becomes 'Bindings and Animation' with two sub-tabs:

- Existing Animations: the previous animation search, its results list can now
  be collapsed.
- Bindings (new): an 'Auto-Bind' button that adds an Armature modifier to every
  mesh in the scene without one yet and weights each vertex to its nearest one
  or two deforming bones (inverse-distance to the bone's segment, normalised to
  sum to exactly 100%). A 'Show Weights' toggle bakes the result into a vertex
  color layer and switches the viewport to solid shading with vertex colors,
  the wireframe overlay, and the armature in Pose Mode - bone colors only
  render in Edit/Pose Mode, plain Object Mode always shows the default bone
  shape regardless of Bone.color, which only showed up by taking a screenshot
  of the actual result rather than assuming the property alone was enough.
  Anything not bound to exactly 1-2 bones summing to ~100% shows as magenta.

The binding math is pure numpy (closest_point_distances, nearest_bone_weights)
so it is unit tested without needing bpy. Verified end to end against a live
Blender session: a two-bone chain produces a smooth 100%-summing weight
gradient, and the visualisation renders exactly as designed once switched to
Pose Mode.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reported against a real asset (kudirewolf_skn.w3d): a mesh imported and
re-exported unmodified had vertices bound to two bones end up with an
extreme offset in-game, while single-bone vertices were fine.

Root cause: Blender stores vertex group weights as float32, so an intended
weight like 42% is actually stored as 0.41999998. VertexInfluence.write()
converted that to a percentage with int(weight * 100), which truncates
rather than rounds, silently losing up to a whole percentage point - on
both the primary and secondary weight independently. Verified against the
real file's 262 multi-bone vertices: several pairs summed to 98-99% instead
of 100%, and 7 vertices lost their entire 1% secondary weight, becoming
rigidly single-bone.

Confirmed against OpenSAGE's own skinning shader (Mesh.h) that the engine
blends a vertex as weight0*position0 + weight1*position1, with each
position stored in its own bone's local space - harmless at bind pose, but
once the affected bone is posed away from it during animation, an
under-scaled weight sum becomes a visibly large offset. Single-bone
vertices (100%/0%) hit no rounding boundary and were never affected, which
matches what was reported.

The second weight is now written as the exact complement of the first
(100 - bone_pct), which always sums to exactly 100 by construction, except
when both weights are genuinely zero (an unset placeholder influence, not
an actual 100%/0% bone binding).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…rkers

Three changes to Bindings, based on comparing Auto-Bind's output against
several real exported models (GUMAARMS_SKL armor, rutheoden, wudrogoth,
kudirewolf):

- Auto-Bind no longer always blends every vertex across its two nearest
  bones. A real hard-surface asset's vertices are almost always rigidly
  bound to exactly one bone (100%); a real organic asset blends roughly a
  third of its vertices, favouring whichever bone is closer rather than an
  even split. nearest_bone_weights() now only lets a second bone contribute
  if it is within BLEND_RATIO_THRESHOLD times the nearest bone's own
  distance, otherwise its weight is zeroed and the vertex ends up rigid.
  No single geometric threshold reproduces every real model's exact choice
  - hand weight-painting depends on body topology a distance measure can't
  see - so this is tuned to avoid the more visibly wrong mistake (blending
  a hard-surface part that should stay rigid) at the cost of under-blending
  some genuine joints, verified against the reference assets above.

- The 'problem' visualisation color is now white instead of magenta, and a
  vertex is only flagged if its total weight does not sum to ~100% -
  bone count on its own is no longer treated as a problem, since a real
  model can legitimately use more than two.

- Show Weights now gives every bone a small colored marker sphere (shared
  custom pose bone shape, sized as a fraction of the armature's own extent)
  and shows bone names, with the armature drawn in front of solid geometry
  - a W3D hierarchy's pivot bones are otherwise near zero length and easy
  to miss entirely.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Materials with identical W3D properties were never merged when addons
like BlenderKit registered their own custom property on Material,
since the signature comparison included that property too and its
value is never equal across two different material instances. The
signature now only compares this addon's own Material properties,
using a snapshot-diff of the RNA property set taken before/after
custom_properties.py registers them, so it stays correct regardless
of other addons' load order.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@Tarcontar

Copy link
Copy Markdown
Collaborator

@LutBox please check for the merge issues after the other PR is now merged

LutBox and others added 4 commits September 13, 2026 19:36
read_string() decoded every null-terminated string field strictly as
UTF-8. Older Renegade/BFME-era 3ds Max exporters wrote these strings
in the Windows ANSI codepage, so any file containing a non-ASCII
character (mesh user text, texture name, shader/vertex material name
or argument) failed the entire import with a UnicodeDecodeError. It
now falls back to cp1252, which covers every byte value and matches
what those tools actually wrote.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Every checkbox/number-field click in Destroy Animation's Splitting
Objects list stalled Blender for several seconds. This is a Blender
5.2.1 engine issue, not addon-specific: committing a property widget
drawn directly on a CollectionProperty item stalls the whole UI,
reproduced with an isolated test panel sharing no code with this
addon. Writing the same properties through a small operator instead
of a direct property widget avoids the stall, so the checkbox and
piece-count controls now go through bfme.toggle_split_object and
bfme.adjust_split_count.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolves the conflicts the maintainer asked about after OpenSAGE#279 was merged.
Both sides had independently added Blender 5.2 support and their own
blender_manifest.toml, so every conflict was the same work appearing
twice; the merged tree is identical to this branch's content.

Resolutions:
- version stays 0.9.0 (this branch), not master's 0.7.4
- TOOLS_PANEL_PT_w3d is not reinstated: this branch replaced it with
  W3D_TOOLS_PT_panel, which offers the same two operators, and the class
  no longer exists here, so keeping master's CLASSES entry would raise
  NameError on register
- CHANGELOG keeps the v0.9.0 section above master's v0.7.4 section
- README keeps the BfMe Tools section

Full test suite passes headless (529 tests).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Addresses the review comment on vendor/__init__.py. Follows the same
pattern the add-on already uses for blender_addon_updater, and both CI
and the publish workflow already check out with submodules enabled.

pyBIG has no release tags, so the submodule is pinned to 3f6ef37, the
commit whose sources are byte-identical to the copy that was checked in
(verified per file) - this change carries no code difference. The
package sits in a 'pyBIG' directory inside the repository, so the import
in cache.py gains one level.

The publish workflow now also keeps the new submodule's .git out of the
release archive, as it already does for blender_addon_updater.

Note for review: upstream is ClementJ18/pyBIG directly, since there is no
OpenSAGE fork of it to point at the way blender_addon_updater does. Happy
to re-point the URL if one gets created.

Full test suite passes headless (529 tests).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A model and its texture often share a name in mods (art/w3d/hu_r_treb.w3d
uses art/compiledtextures/hu_r_treb.dds), and several places treated that
name as one asset:

- the model list and the preview/import lookup read the flat index, which
  keeps one asset per name, so a model whose texture won that slot was not
  listed (157 models on a full Edain + BfMe II setup). They now use the
  models bucket; the texture finder likewise uses the textures bucket
- dependency_keys() skipped a texture named like its own model, so it was
  never staged; with no other textures the model was imported in place,
  away from all of them
- Existing Animations rebuilt the single cached index with .w3d files only,
  leaving the model browser without any textures until the next scan. All
  tools now index the same extensions, which adds .jpg/.png/.bmp textures

Previews carry a version now, bumped so the ones rendered without those
textures are rendered again once.

Verified against a 400-model sample of the live setup: textures that exist
but were not staged went from 9 models to 0, and the affected models import
with no placeholder textures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 47.43122% with 1586 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.30%. Comparing base (808db62) to head (31212f0).
⚠️ Report is 113 commits behind head on master.

Files with missing lines Patch % Lines
io_mesh_w3d/bfme/tools/model_browser.py 29.06% 360 Missing and 6 partials ⚠️
io_mesh_w3d/bfme/tools/w3d_tools.py 41.04% 295 Missing and 11 partials ⚠️
io_mesh_w3d/bfme/tools/destroy_animation.py 21.06% 264 Missing and 2 partials ⚠️
io_mesh_w3d/bfme/tools/texture_finder.py 30.23% 150 Missing ⚠️
io_mesh_w3d/bfme/tools/existing_animations.py 34.19% 96 Missing and 6 partials ⚠️
io_mesh_w3d/bfme/tools/export_settings.py 40.00% 92 Missing and 1 partial ⚠️
io_mesh_w3d/bfme/utils.py 34.35% 82 Missing and 4 partials ⚠️
io_mesh_w3d/bfme/tools/build_up_animation.py 35.45% 69 Missing and 2 partials ⚠️
io_mesh_w3d/bfme/tools/bindings.py 83.89% 30 Missing and 23 partials ⚠️
io_mesh_w3d/bfme/cache.py 82.16% 34 Missing and 17 partials ⚠️
... and 4 more

❗ There is a different number of reports uploaded between BASE (808db62) and HEAD (31212f0). Click for more details.

HEAD has 2 uploads less than BASE
Flag BASE (808db62) HEAD (31212f0)
4 2
Additional details and impacted files
@@             Coverage Diff             @@
##           master     #280       +/-   ##
===========================================
- Coverage   95.16%   76.30%   -18.87%     
===========================================
  Files          52       67       +15     
  Lines        4651     8110     +3459     
  Branches      811     1500      +689     
===========================================
+ Hits         4426     6188     +1762     
- Misses        222     1795     +1573     
- Partials        3      127      +124     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

LutBox and others added 6 commits September 16, 2026 18:01
Several sources (search paths, game archives) often ship a model under
the same name, and the browser only listed whichever won overall. The
list is now grouped under a header per source - each search path is its
own, labelled after its mod folder, and a game's selected archives form
one - with a model listed under every source that has it. Within a
source only the copy it uses is listed (patch archive over base game).
The name filter keeps a header when a model below it matches.

Listing both is only useful if picking one gets that one, so:
- the index also keeps its references per origin (archive/search path)
- previews and imports take the source's copy of the model, and resolve
  its skeleton and textures from that source first, then the game
  archives, then other search paths, like the game does
- models are staged into a cache subdirectory per source, and previews
  are keyed per source, so same-named models cannot overwrite each other

Verified live on a three-mod setup: 1302 additional entries listed, and
Edain's and AOTR's different 'duoin_skn' stage and preview separately
with their own skeleton and textures. Full suite passes (540 tests).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… the scene

Editing a value on a scene collection item - Build-Up and Destroy
Animation per-bone timings, Destroy Animation's splitting objects -
froze Blender for about 3.4 s per change. The UI looks up the RNA path
of the edited property by walking the scene's registered collections,
and the model browser kept its ~29,000-row list there. The lookup grows
with the square of the list size (145 ms at 6,000 rows, 587 ms at
12,000); with the list on the window manager it is ~0.01 ms.

This replaces the earlier workaround for Splitting Objects, which blamed
a Blender engine bug and routed the widgets through operators; they are
native property widgets again, with dragging, typing and undo.

The list also added ~25 MB to every saved .blend. A load_post handler
and the startup scan unset what earlier versions left on the scene.
Unregistered, that data does not slow the lookup, but it still weighs
on the file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…and destroy split counts

1. A build-up/destroy animation exported as HAM under its own file name
   exported without errors but never played in-game - not an encoding
   issue. HAM always renamed the embedded hierarchy to the output file's
   name, so e.g. 'HB_W_STALLS' became 'hb_w_walls_a' inside the file and
   the game no longer recognised it as belonging to the original
   building. 'Use Existing Skeleton' now also applies to HAM mode: it
   still embeds the hierarchy (HAM needs its own geometry regardless),
   but keeps the original model's name instead of renaming it.

2. Create Build-up/Destroy Animation renamed the action to 'name.001',
   'name.002', ... on every repeated click instead of reusing the
   configured name, since a new action was always created rather than
   replacing one of the same name. utils.replace_action() now removes
   an existing action of that name first, so repeated clicks always
   leave exactly the requested name.

3. Rewrote Export Settings' Auto-Detect. It looked at whichever object
   happened to be active, which on a multi-object model depended on
   what was last clicked and could enable or skip 'Use Existing
   Skeleton' seemingly at random - almost certainly why builds like
   hb_w_stalls hit bug 1 in the first place. It now looks at the
   scene's armature directly (the same assumption every other BfMe
   tool makes) and covers four cases by comparing the armature's
   collection to its own name and checking for an animation: matching
   with no animation exports the whole model (HM); matching with an
   animation exports just that (HAM, Use Existing Skeleton, name =
   the animation); a differing collection with no animation assumes a
   mesh-only part of a model exported elsewhere (HM, Use Existing
   Skeleton); anything else falls back to a plain model export. The
   detected path now also prefers the current .blend's own save
   location over where a model was imported from, and importing a
   model through the W3D Model Browser now actually records that path
   for the fallback to use - it was previously never written anywhere.

4. Halved Destroy Animation's automatic piece-count estimate per size
   bucket: real assets were ending up fractured into far more
   sub-objects than a destruction animation needs.

Full test suite passes (559 tests). Every new/changed test was checked
against the pre-fix code and confirmed to fail there.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Every length-prefixed string in the W3D format (mesh User Text, texture
names, vertex material names/args, dazzle names) is preceded by a byte
count so a reader knows where it ends. text_size() computed that count
from len(text) - Python's character count - while write_string() writes
the text as UTF-8 bytes. Any character outside ASCII takes more than one
byte in UTF-8, so whenever such a character appeared the declared count
came out smaller than what was actually written, and everything read
after that string started a few bytes early: garbage from there on.

write_fixed_string()/write_long_fixed_string() (16/32 byte pivot and
material names) had the same character-vs-byte mistake in their null
padding, which could write past the fixed field's own size for the
opposite reason - too many padding bytes for a string measured short
in characters, shifting every subsequent chunk instead.

Root-caused against a real corrupted export: two of three meshes wrote
a plain-ASCII User Text and parsed fine, the third's User Text was
'Boite693' with an 'i' with a circumflex, and everything after that
mesh's User Text chunk was unreadable garbage - chunk_dump confirmed
the exact byte offset where the file diverges.

Verified live: exporting a mesh with this same User Text through the
fixed addon and dumping the chunk structure shows no overrun, and
re-importing the file returns the text unchanged.

Full test suite passes (563 tests). Every new test was checked against
the pre-fix code and confirmed to fail there.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1. The path recorded for a model imported through the W3D Model Browser
   was read off the (possibly staged-into-cache) import filepath rather
   than the model's own reference. A loose model whose textures are not
   right next to it - the common layout, a mod's 'art/w3d' and
   'art/compiledtextures' as separate folders - gets staged into the
   cache directory so the importer finds everything together, and the
   recorded path then pointed there instead of at the model's real
   source folder. It is now read from the reference itself, which for a
   loose file is always its true location regardless of staging.

2. Auto-Detect enabled 'Use Existing Skeleton' for a build-up/destroy
   animation whose collection matches its armature's name, deliberately
   added last session to keep the exported hierarchy bound to the base
   model (see the HAM hierarchy-naming bugfix). Per explicit instruction
   this is reverted to match the original rule exactly: that case now
   leaves it unchecked, same as a plain model export. Confirmed and
   accepted consequence: exported this way, the animation's hierarchy
   is renamed to the animation's own name and the game no longer
   recognises it as the base model's - 'Use Existing Skeleton' needs to
   be checked by hand when that matters.

Full test suite passes (564 tests). The path fix was checked against
the pre-fix code and confirmed to fail there.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CI was failing under Blender 2.93 with AttributeError/TypeError from several
BfMe-tools code paths that (unlike the rest of the codebase) had no version
guard for APIs added after 2.93: Bone.color (4.0+), Mesh.color_attributes
(3.2+), Object.children_recursive (3.1+), Context.temp_override (3.2+),
bmesh.ops.create_uvsphere's 'radius' kwarg (3.0+, was 'diameter'), and
PoseBone.custom_shape_scale_xyz (3.0+, was the scalar custom_shape_scale).
Added version-guarded fallbacks following the existing hasattr/bpy.app.version
convention (see common/utils/helpers.py), so Show Weights degrades gracefully
(skips per-bone tinting and the color-coded mesh overlay, but still shows bone
markers and enters Pose Mode) on Blender < 4.0 instead of crashing silently
inside a property-update callback. Also fixed a genuinely Blender-4.4+-only
test assumption (Action.slots) in a slotted-actions test.

Verified locally against the exact Blender versions/command CI uses (GitHub's
own log UI required authentication this environment doesn't have): full suite
passes on Blender 2.93.13 (564 tests, 6 correctly skipped for APIs that don't
exist that far back) and Blender 5.1.2 (564 tests, 0 skipped, no regressions).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@LutBox
LutBox requested a review from Tarcontar September 18, 2026 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants