Skip to content

Feature/kfxs save format - #5424

Open
cerwym wants to merge 5 commits into
dkfans:masterfrom
cerwym:feature/kfxs-save-format
Open

cerwym wants to merge 5 commits into
dkfans:masterfrom
cerwym:feature/kfxs-save-format

Conversation

@cerwym

@cerwym cerwym commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Adds in the new KFXS save format, including checks in CI that prove / warn if a save has deliberately been broken.
Realistically this should be hard because this is no longer a mem-dump, we now use fields and save them in a structured manner.

This should result in saves that never break.

Save CI checks may be squashed at a later date

cerwym added 2 commits October 8, 2026 11:36
…ma-driven codec

Savegames, continue and campaign progress files, high scores, the network config,
replay headers and network resyncs are now written as KFXS files: a checked header,
then chunks with a CRC and optional zlib. Each file carries a schema of the structs
and fields it holds, and loading matches fields by name, so adding, removing or
widening a field no longer breaks old saves. Changes of meaning are handled by
migrations, and the config a level changed is stored as an overlay on the config
files rather than as a copy of it.
@cerwym

cerwym commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

@Loobinex please see https://github.com/cerwym/keeperfx/blob/7110e498729781d757c0a8d0f18cd777e87cf207/docs/lua_saves_for_mappers.md for guidance regarding lua best practices that should occur. I've made a best effort to preserve old maps working.

But this is effectively also the security fix for loading arbritrary byte code from saves as lua was stored there

@nstbayless

Copy link
Copy Markdown
Contributor

I would suggest expanding the magic header from 4 to 8 bytes, with a \0 and invalid utf8 in the upper portion. This guarantees the OS understands it as a binary file, and also guarantees we can distinguish this from the legacy formats. (bytes 4-8 in the old code are guaranteed printable; and in legacy dk saves had \0 in byte 3 i believe.)

Comment thread src/kfx/save/FORMAT.txt
Comment thread src/kfx/save/FORMAT.txt Outdated
Comment thread src/kfx/save/FORMAT.txt
--------------------------------------------------------------------------------
Describes every struct and field of the records in the file.
The writer lists the structs reachable from the roots of the file.
Structs are sorted by name (strcmp). No two structs share a name.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps sort topographically, i.e. no struct has another struct in it unless that struct is defined earlier. Makes parsers easier to write if struct length can be decided without backtracking.

OTOH, if you don't expect parsers to validate, maybe offer no promises about struct order at all, not even strcmp.

Comment thread src/kfx/save/FORMAT.txt
Comment thread src/kfx/save/FORMAT.txt
Comment thread src/kfx/save/FORMAT.txt Outdated
Comment thread src/kfx/save/FORMAT.txt Outdated
Comment thread src/kfx/save/FORMAT.txt Outdated
Comment thread src/kfx/save/FORMAT.txt Outdated
Comment thread src/kfx/save/FORMAT.txt Outdated
Comment thread src/kfx/save/core/save_schema.h
Comment thread src/kfx/save/core/save_schema.h
Comment thread src/kfx/save/core/save_schema.h Outdated
Comment thread src/kfx/save/core/save_schema.h
Comment thread src/kfx/save/core/save_decode.cpp
Comment thread src/kfx/save/core/save_schema.h Outdated

This branch was successfully deployed

2 active (outdated) deployments
Windows Prototype — 7110e498 Deployed Oct 8, 2026 by Loobinex via Build Prototype Windows x86 #1253
Linux Prototype — 7110e498 Deployed Oct 8, 2026 by Loobinex via Build Prototype Linux x86_64 #1253
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants