diff --git a/AGENTS.md b/AGENTS.md index 44d25ab29..4806ade6b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -160,6 +160,19 @@ restyling anything large: - a jump table - Ghidra usually renders the dispatch as a call through an unnamed pointer array and drops the arms, so rebuild the `switch` from the byte and jump tables instead of restyling the decompiler output. +`quick_diff.py` is a triage tool and it misranks two things. It prints the **wrong immediates for the +original side** (`0x51eb851f` where the binary holds `0x66666667` or `0x10624dd3`), so never read a constant +out of its output - use `orig_asm.py` or Ghidra. And its difflib ratio can invert the reccmp ranking even well +above the 0.6 the README warns about, because a changed base anchor makes every displacement differ textually +while the opcodes still line up: on `updateCrowding` it preferred a variant that measured 25.5% over one that +measured 33.3%. Rank with it, then settle small functions with `try_styles.py`, which narrows the build list to +one file and therefore costs about a minute per variant. + +reccmp symbolizes an immediate that happens to coincide with a data address, and it does so per binary. The +thirty `push 0xffffff` of `renderDebugDataCurrentPlayerData` come out as +`push I::instance+424671` on the original side and as the plain immediate on ours, which caps +that function at ~96% with no source change possible. Check for that before chasing a stable few percent. + `diff_triage.py` gives one line per function in the build list and says whether more source work can pay at all: it flags a `sub esp` frame-size mismatch (`FRAME`), byte-sized stack slots we have and the original does not (`BYTE`), a cold return block we inline where the original outlines it (`RET`), and the case where only register and stack-slot @@ -176,6 +189,40 @@ dropped because every hit it produced was a loop-induction stride rather than a which is the only lever left once a function is `alloc-only`; it pays about one move in ten, and mostly where the statements sit between two calls or on a loop back-edge. +### Switching the reimplementation on before measuring + +There are **two** switches, and both are deliberately uncommitted, so a rebase, a fresh clone or a +`git checkout -- src/OpenSHC` turns both off at once: + +- `tools/reimplementation-control/enable_reimplemented_data.py` - the `MACRO_STRUCT_RESOLVER` flags in + `src/OpenSHC/Globals/*.hpp`. +- `tools/reimplementation-helper/batch/enable_local_functions.py` - the `MACRO_FUNCTION_RESOLVER` flags for + exactly the files in `cmake/openshc-sources.txt.local`. The repository-wide + `tools/reimplementation-control/Enable-Reimplemented-Functions.ps1` flips every resolver that has a .cpp + anywhere in the tree, which does not link while the build list is a subset: each enabled resolver whose + .cpp is not compiled becomes an unresolved external. + +With the function resolvers off, every call leaves the module for the original game address, so **`/GL` +buys nothing**: LTCG cannot see a callee it does not compile. `getSalesPrice` sat at 68.4% for as long as +`getSellResourceAmount`'s resolver was `false`, and reached 100% the moment it was `true` - the original +keeps `this` in the volatile EDX across that call, which is exactly what LTCG provides. `clearEnemyRelatedStructures` +went 97.7% -> 100% the same way. So to chase a "volatile register survives a call" diff: put the callee's .cpp in +the build list, enable its resolver, and add **both** files to `cmake/compiler-flags-gl.txt`. + +**`/GL` can also make a function worse, and not only through the LTCG unit.** LTCG lets MSVC see that a +callee does not touch a given struct, so it starts hoisting loads the original's compiler had to repeat: +`spawnPoisonCloudsAroundBuilding` fell 92.4% -> 69.6% because LTCG hoisted +`buildings[buildingID].x`/`.y` out of the retry loop across `nextRandomNumber2`, which the original +re-reads every iteration. So add `/GL` for a pair only when the *original* shows the cross-call knowledge - +a volatile register still live after a `call`, or a missing `mov ecx, ` reload. A reimplemented +callee is not a reason on its own. + +`/GL` is whole-program, so the list is not a free-for-all. Adding files that are not part of the pair under +test re-codegens everything in the LTCG unit: one such round cost `processFoodConsumption` 20 points and +`spawnPoisonCloudsAroundBuilding` 23 while gaining 1.3 elsewhere. Add one caller/callee pair at a time and +keep only what measures better. A callee that is not itself at 100% is worse than useless here, because +LTCG then propagates *our* register use rather than the original's. + ### Binary Comparison (`reccmp`) Compares generated binaries against the original executable. diff --git a/IMPLEMENTATION_CHEAT_SHEET.md b/IMPLEMENTATION_CHEAT_SHEET.md index 12991e55d..1c9e0313a 100644 --- a/IMPLEMENTATION_CHEAT_SHEET.md +++ b/IMPLEMENTATION_CHEAT_SHEET.md @@ -642,3 +642,407 @@ Naturally, the stack reserved for the function indicated by `sub esp, ` At best, not only the stack fits, but variables are also at the correct positions and sizes within. If the stack usage shows a proper order in the function (i.e. the initial array occupies the first part in the allocated function stack, which means it requires the highest `[esp + ]` to reach), it might be sometimes good to fit one "side" of the stack first and then experiment with the rest of the function that has a non-fitting stack usage. + +### A constant that reaches a field through a register was a local in the source + +Where several arms of a chain each store the *same* literal both into a field and into an +arithmetic expression, the original loads the literal into a register first: + +```asm +mov eax, -0x96 ; the constant, in a register +add ebp, eax ; popularity = popularityAfterFear + penalty +mov dword ptr [ebx], ebp +mov word ptr [esi + 0x32ff8], ax ; someCount54 = penalty, from the SAME register +``` + +Writing the constants out per arm (`popularity = snapshot - 150; someCount54 = -150;`) gives +`mov word ptr [...], -0x96` with an *immediate* instead, because the two uses then share nothing. +The source assigned one local in each arm and used it after the chain; MSVC then duplicated the +two statements into every arm, which is why the stores look per-arm in the asm: + +```cpp +int siegePenalty; +if (c <= 0) { siegePenalty = -150; } else if (c <= 5) { siegePenalty = -125; } /* ... */ +this->...popularity = popularityAfterFear + siegePenalty; +this->...someCount54 = (short)siegePenalty; +``` + +This single change took `updatePopularity` from 68.3% to 93.2%; it also fixed the register +allocation of the whole surrounding function, which had looked like an unrelated allocator gap. +The tell is an *immediate* on our side where the original uses a byte/word register (`ax`, `cx`) +that was just loaded with the same constant. + +### The struct/loop base anchor moves when a field loses its stores + +MSVC picks one address as the base for a run of accesses into the same object +(`lea esi, [ecx + 0x367e4]`, then `[esi - 0x10]`, `[esi + 0xe0]`, ...). Which field lands at +offset 0 is not the most-used one and is not controllable from the source, but it *does* change +when the source changes: introducing a local for a field that was previously stored in several +arms moved the anchor by 0xe0 in `processFoodConsumption` and by 0x10 in +`processPeasantSpawnAndDespawnCycle`, and every displacement in the function then differs. + +Both of those locals are provably present in the original (a value survives a call in a +callee-saved register), yet the anchor shift cost more than the local gained: +81.7% -> 61.7% and 85.3% -> 69.0%. So when a `FRAME`-free function regresses by a large, +suspiciously uniform amount after a one-line change, compare the `lea`/`add` that sets up the +base before looking anywhere else, and keep the form with the matching anchor. + +### A function that looks huge may be a tail call + +`nof_fpoints_locationFinder` had been reimplemented with a 40-line `do/while` body; the original +is eleven instructions ending in `mov ecx, ; jmp 0x4a6940`, a tail call to +`PathFindingState::searchCertainPathLayer`, whose body Ghidra had folded into the caller's +decompilation. Check the original's last instruction for a `jmp` to another function's entry +before reproducing a long body: 35.6% -> normalized 100% here. + +### The join block of a long if/else-if chain moves with the size of the first arm + +In `computeBuildingCategoryEntryPointAndDestroyEarlierBuilding` the chain order, every arm's body and the +shared tail all match the original, yet the original ends +`[prologue][first arm][else chain][tail][epilogue]` and this toolchain emits +`[prologue][first arm][tail][epilogue][else chain]` - the `ret` lands at 51% of the function instead of 99%, +and because reccmp aligns instruction streams, almost the whole function then counts as differing (33.5%). + +The flip is a threshold on the **first arm**, not a source shape. Thirteen source forms (switch, fully +nested `else`, combined guard, empty trailing `else`, nested early returns, `__forceinline` helpers for the +tail and for the chain, parameter copies, casts, declaration order) all produced byte-identical code. +Shrinking the first arm does move the tail to the end, and three unrelated shrinks did it: dropping one arm +of its inner `orientation` chain, dropping its three early returns, or dropping its nested +`currentMenuViewType` block. The threshold sits between 168 and 175 instructions for that function, and the +original is 178, so it is not reachable by trimming anything the original actually contains. + +`#pragma optimize("s", on)` also produces the original's layout, which is a useful diagnosis - but it is not +the explanation: measured over the thirteen functions still short of 100% in `Game::GameStateStructures`, +`/Os` lost ground on **every single one** (e.g. `showPopAndGoldPopup` 0.93 -> 0.20, +`renderDebugDataCurrentPlayerData` 0.96 -> 0.17). The original was built for speed; only its block layout +differs, which points at a different compiler build rather than a different flag. `/G5`, `/G6` and `/G7` +change nothing in this toolchain (they are deprecated no-ops in VS2005). + +### The struct base anchor is coupled to the transformation you are trying to make + +MSVC bases a run of accesses into one object on a single address (`lea esi, [ecx + 0x367e4]`, then +`[esi - 0x10]`, `[esi + 0xe0]`, ...). Which field lands at offset 0 is not a size or frequency decision - +with the right anchor `processFoodConsumption` gets 30 byte-sized displacements and with the wrong one 3, +and MSVC still picks the wrong one - and it cannot be steered from the source: the global vs `this->` +spelling of a field, an extra read of it, the index's cast, the statement order and the number of stores to +other fields all leave it unchanged (probed with a `/FA` compile per variant, about ten seconds each). + +What *does* move it is the very change you want: + +- `processFoodConsumption`: the original keeps `currentPopulation * 3` in a callee-saved register across the + `isAIPlayer` call, so a local is provably right. Writing that local - or getting the same CSE for free by + enabling LTCG over the callee - moves the anchor by 0xe0 and costs 20 points, more than the local gains. +- `processPeasantSpawnAndDespawnCycle`: the original stores `vclockIncrement` once after the chain rather + than in each arm. Hoisting the store moves the anchor by 0x10 and costs 15 points. + +Both originals have the local *and* the matching anchor, so the combination is simply not reachable here. +When a one-line change costs a large and suspiciously uniform amount, read the `lea`/`add` that sets up the +base before looking anywhere else. + +One narrow rule did hold, for a condition that reads two arrays of the same struct: MSVC anchors the +induction variable on the **second** array mentioned in the source, while the comparison order follows the +source. `showPopAndGoldPopup` and `createStatsPopUpEntities` need `currentPlayerFullIDArray` tested first +*and* used as the anchor, so the two requirements conflict and the faithful form (the one the asm's +comparison order shows) is the one that loses the anchor. + +### Check whether the original shares a tail before duplicating a call + +Ghidra renders a cross-jumped tail as a `goto` into the middle of another branch, and the shared part can be +a single instruction. In `processPeasantSpawnAndDespawnCycle` the two `playSpeechSFX` sites are one call +reached from both paths with the speech ID in a register (`speechID = SEID_POP_EMIGRATE; goto LAB_0045b684;`), +and in `updatePopularity` the six siege arms share `popularity = snapshot + penalty; someCount54 = penalty;`. + +The two need opposite treatment, so read the asm rather than the decompiler's shape: + +- `updatePopularity` had to *gain* the variable: MSVC then duplicated the two statements into each arm by + itself, which is what the original does, and the function went 68.3% -> 93.2%. +- `processPeasantSpawnAndDespawnCycle` already emits one call site - MSVC cross-jumped the two identical + tails on its own - so routing the ID through a `SpeechEffectID` local changed nothing (MSVC propagates the + constant straight back into the push). Count the call sites in a `/FA` compile before restructuring. + +### Declaration order decides stack slots, and two swapped locals cost a whole function + +MSVC assigns stack slots in declaration order. Two locals declared the wrong way round therefore sit +in each other's slots and, because the slot drives the spill and reload, usually end up in each +other's registers as well. The damage spreads over every instruction that touches either of them, +while `diff_triage.py` still calls the function register-naming only - `assignSelectionToKey` read +95.9% with a `norm` of 1.000. + +`diff_slots.py` is the detector: the same number of slots on both sides with a different use count +per slot. For `assignSelectionToKey` the original spills +`mov [esp+0x18], edi; mov [esp+0x1c], ebx; mov [esp+0x20], esi` and reloads in the same order, so the +local in the lowest slot is declared first. Our source had + +```cpp +int slotID = 0; // -> [esp+0x20] +int tribeIndex = 0; // -> [esp+0x18] +``` + +and swapping the two declarations put both the slots and the registers back: **95.9% -> 100%**, one +line moved. `perm_decls.py` automates the search over a short run of declarations. It also found a +field store that belonged after the declarations in `updatePopularity` (93.2% -> 93.9%), and nothing +on seven other functions - so run it on anything `diff_slots.py` flags, but do not expect a hit +everywhere. + +### `lea` where the original has `add` means a snapshot local the original did not keep + +`field = snapshotLocal + delta` has to preserve the snapshot, so MSVC emits +`lea ecx, [eax + ebp]; mov [field], ecx`. `field = field + delta` lets it update the register that +already caches the field read: `add ebp, eax; mov [field], ebp`. The two differ by one byte per site +and by the register the result lands in, which looks exactly like allocator noise in the diff. + +`updatePopularity` spent the whole pass at 93% on six such sites: + +```cpp +int popularityAfterFear = this->playerDataArray[playerID].popularity; // the original has no such local +... +this->playerDataArray[playerID].popularity = popularityAfterFear + siegePenalty; // -> lea +this->playerDataArray[playerID].popularity = this->playerDataArray[playerID].popularity + siegePenalty; // -> add +``` + +Dropping the local took it to **100%** (and the six `lea`s were also why every jump offset after them +was 6 bytes off). The snapshot was redundant anyway: nothing between the read and the use writes +`popularity`, which is the condition to check before removing one. The original still loads the field +into a register early - that is MSVC hoisting the read, not evidence of a local. + +Note that the sibling line in the same function, `popularity = popularityAfterTax + popularityChange`, +*does* match with its local, because there the original really does keep a separate snapshot. Decide +per site from the `lea`/`add` in the original, not from the shape of the neighbouring code. + +### Two arrays of one struct in a condition: the original reaches the second off a pointer to the first + +`showPopAndGoldPopup` and `createStatsPopUpEntities` both guard their player loop with + +```cpp +if ((currentPlayerFullIDArray[playerID] == -1) && (currentAIArray[playerID] == 0)) { continue; } +``` + +and both sat at 87-89% because MSVC based the loop's induction variable on `currentAIArray` while the +original based it on `currentPlayerFullIDArray` - four instructions, but they are the pointer set-up, +both guard reads and the loop's end compare, so they carry the whole loop with them. No spelling of +the condition moves it: swapping the operands fixes the base and breaks the comparison order, and +nine other forms (nested `if`, De Morgan, casts, `ptr->`, a local for either read) compile identically. + +The decompilation says why. The original walks one pointer and reaches the second array off it: + +```c +_currentPlayer = DAT_GameSynchronyState.currentPlayerFullIDArray + 1; +... +if (*_currentPlayer != -1 || _currentPlayer[0x1b] != 0) { ... } +_currentPlayer = _currentPlayer + 1; +``` + +`0x1b` is the distance between the two arrays in ints. Writing that - a pointer local for the first +array with the second read as `currentPlayerFullID[0x1b]` - took **both functions to 100%**. A pointer +local for the first read *alone*, with the second still by name, changes nothing: both reads have to +go through the one pointer. Keep the magic index, with a comment; it is the only form that matches. + +### Three shared-tail rewrites that the decompilation justifies and that still measure worse + +The decompilation shows a shared tail for all three of these, and reproducing it without `goto` lost +every time. Read them before spending a day on the same idea: + +- `processFoodConsumption` has **one** `processResourceLoss` call with the type in a variable where we + write four calls with literal types. Setting a `ResourceType` and calling once after the loop is + exactly the original's shape and measured **83.7% -> 59.7%**: the extra local swaps the zero + constant and the loop counter between `ebp` and `ebx`, renaming twenty lines. +- `computeBuildingCategoryEntryPointAndDestroyEarlierBuilding` has one `destroyBuilding` call shared by + five arms and one `buildingEntry` read shared by nine. Hoisting both (an `existingBuildingID` + variable plus one `category != CAMPGROUND` re-test) lands within one instruction of the original's + length, 177 against 178, and still loses: the join block stays in the middle, so the alignment does + not recover. +- `resetVariousCountsAndStatisticsAndStartGoodsAndResources` has one `startResources[0xf] = gold * + multiplier` store shared by the human and AI arms. Writing the table value into a local in each arm + so the tails are identical text does not get MSVC to cross-jump them (92.4% -> 91.0%). + +A pointer local is not a general fix for the base anchor either: `PlayerData*` locals leave +`processPeasantSpawnAndDespawnCycle`'s anchor to move with the shared store exactly as before, and in +`spawnMotherOrChild` they remove the three `global - this` slots the original has, taking the frame +further from `sub esp, 0x18` rather than closer. + +### Read the decompilation, not only the disassembly, before calling a function allocator-bound + +Four of the finds above came from `decompile_function`, not from the interleaved reccmp diff, and two +of them were worth 100%. The decompiler names the original's locals and pointers, which is exactly the +information the asm hides: `processFoodConsumption`'s AI branch is +`(currentPopulation * 180) / 100` with `currentPopulation` as the local, not +`(foodClockSpeed * 60) / 100` - the same value, but MSVC reaches the original's instruction sequence +only from the first spelling. `updateCrowding`'s pointer is anchored at `currentPopulation`, which is +why its base is 0x2100 away from the field it stores to. + +## reccmp can score a byte-identical function below 100% + +`renderDebugDataCurrentPlayerData` scores 96.0% and every single one of the 41 diff lines is this: + + -push I::instance+424671 (OFFSET) + +push 0xffffff + +Both sides are the same four bytes. `0xffffff` is opaque white, a colour argument to the text calls, +and reccmp renders it as an address because in the *original* image that value lands inside +BuildingsState; in ours it does not. Verified with `disassemble_function 0x0045A060`, which shows +`PUSH 0xffffff` throughout. The function is byte-identical to the original and no source change can +raise its score. `tools/reimplementation-helper/batch/diff_falsesym.py` finds this class; it is the +only function in GameStateStructures affected, so nothing else below 100% is a tooling artifact. + +The reverse mistake looks the same in the diff and *is* a real bug: an operand that should be an +address, written as a literal in our source. Always confirm against the original disassembly. + +## spawnMotherOrChild: ebx goes to the anchor instead of the loop counter (54.6%, closed) + +The whole gap is one allocator decision. The original keeps `spawnBudget` in `ebx` and the +`playerDataArray` anchor in scratch `edx`, reloading it after calls. Ours does the opposite: the +anchor takes `ebx` (callee-saved, so it survives `spawnUnit` without a reload - arguably better +code) and `spawnBudget` spills, which is the 7th stack slot and the `sub esp, 0x1c` against the +original's `0x18`. Use that frame size as a free oracle: `0x18` means the counter stayed in a +register. + +Measured and ineffective - all four produce a *byte-identical* instruction stream to the current +form, so the shape is canonicalised before allocation: + +* `int buildingID = 0;` before the budget, as the decompilation has it, with an empty `for` init +* `spawnBudget` declared outside the `playerID` loop (a longer live range does not buy a register) +* both of the above +* both, with `buildingID` also at function scope + +Spelling the inner-loop `popularity` read as `this->` instead of `DAT_GameState::instance` *does* +reach the original's `sub esp, 0x18`, because it drops the `0xfffcf0ec` base to one register instead +of two - but that base exists in the original, so the frame matches by compensation and reccmp +measured 54.5%, a slight regression. quick_diff ranked it 0.751 over 0.739; trust reccmp. + +## spawnPoisonCloudsAroundBuilding: the memory decrement (92.4%, closed) + +The only gap is the loop tail. The original folds it to a read-modify-write plus a fresh re-read: + + sub dword ptr [esp + 0x10], 1 + cmp dword ptr [esp + 0x10], 0 + +while we keep the value in a register (`mov eax, mem; sub eax, 1; test eax, eax; mov mem, eax`). The +peephole needs the load/sub/store contiguous, and our scheduler puts the `test` between the sub and +the store. The two `continue`s force the decrement into the `for` header, so the obvious forms were +already known to canonicalise; rewriting the `continue`s as nested `if`s to put the decrement at the +bottom of the body - the decompilation's own shape, and the one the peephole wants - measured +*worse* (0.905 against 0.950) and still did not produce `sub dword ptr`. + +## processPeasantSpawnAndDespawnCycle: do not hoist the popularity read (84.0%, closed) + +The original computes `popularity / 500` twice, once per arm. Hoisting it into one local and taking +the shared `vclockIncrement` store - the `processFoodConsumption` fix, applied here - moves the base +anchor from `0x347a4` to `0x36978` and drops the score from 0.92 to 0.68. The duplicated division is +load-bearing: it is what keeps the anchor on `vclockIncrement`. + +## A per-file /Os does not reach the block layout (disproven) + +`cmake/compiler-flags-gl.txt` sets `/GL` per file, so the same mechanism can carry any flag: + + file_dependent_read_list(".../compiler-flags-os.txt" COMPILER_FLAG_OS_FILES) + set_source_files_properties(${COMPILER_FLAG_OS_FILES} PROPERTIES COMPILE_OPTIONS /Os) + +That looked like the way to give `computeBuildingCategoryEntryPointAndDestroyEarlierBuilding` the +original's basic-block layout without the cost a global `/Os` was measured to have. It does not +work. With `/O2 /Os`, and then with `/O2 /O1`, the reccmp diff for that function is identical to +`/O2` alone - all 297 lines, 33.5% / 32.8% every time. The `.obj` hash does change, which is +misleading; it is the command line recorded in the debug section, not the code. MSVC resolves the +trailing size flag to nothing more than `/Os` over `/O2`'s `/Ot`, and that bit does not move this +function's layout. + +So the earlier "`/Os` reproduces the layout" note was a whole-build effect (different inlining +elsewhere), not something a per-file flag can reach. The layout difference - the common tail of the +if/else-if chain laid out after the first arm, which turns the leading `jne` into +0x136 where the +original has +0xb4 and so changes every displacement after it - remains unreachable from the source +or from per-file flags. + +## /GL does nothing for a function that makes no reimplemented member call + +`computeBuildingCategoryEntryPointAndDestroyEarlierBuilding`, +`processPeasantSpawnAndDespawnCycle`, `spawnMotherOrChild`, `spawnPoisonCloudsAroundBuilding` and +`updateCrowding` were never in `cmake/compiler-flags-gl.txt`. Adding all five changes nothing: every +score in the build list is identical to the digit and the average stays at 97.76%. `/GL` earns its +place by removing the redundant `mov ecx, offset` before a call into another reimplemented method, +so it can only pay where such a call exists and the callee's resolver is on. `updateCrowding` has no +calls at all, and the others call out to methods that are not in the build list. It does not +re-roll register allocation inside a function, which is what these five actually need. Do not add +entries to that list speculatively - measure, and leave it out when the number does not move. + +## resetVariousCountsAndStatisticsAndStartGoodsAndResources: a one-byte encoding (92.2%, closed) + +The original loads the current player's slot into `edx`, we use `eax`: + + -mov edx, dword ptr [GameSynchronyState::instance+1089140] imul edx, edx, 0x39f4 + +mov eax, dword ptr [GameSynchronyState::instance+1089140] imul eax, eax, 0x39f4 + +`mov eax, [abs]` is the one-byte-shorter `A1` form, so our version of that line is a byte shorter, +which moves the alignment padding before the `startResources` copy loop (the original pads with a +three-byte `lea esp, [esp]`, we emit `jmp 0x3` over a `lea ecx, [ecx]`) and renames a register in +the `_memset` call set-up. One encoding choice, three diff hunks. Spelling the index as its own +local, taking a `const PlayerData*` to the source player, and reading through `this->` instead of +the global all produce a byte-identical function (the `this->` form measures worse, 0.819 against +0.924). MSVC picks the destination register for a load it is about to overwrite, and the source +cannot name it. + +## The /GS security cookie is a permanent ~4% floor + +`playWAVSFX` (96.2%) and `playSpeechSFX` (96.4%) are finished. Their entire remaining diff is the +`/GS` stack-cookie code that MSVC adds around a function with a large local buffer: + + -mov eax, dword ptr [StructResolver::Instance::instance (DATA)] + +mov eax, dword ptr [__security_cookie (DATA)] + -call + +call __security_check_cookie (FUNCTION) + +Each module gets its own cookie, so the original reads the game's at 0x00B94220 and we read our +DLL's. The `__security_check_cookie` call normalizes away; the cookie *load* does not, which is why +both sit at normalized 96.0% and cannot go higher. Any reimplemented function with a stack buffer +big enough to trigger `/GS` carries the same floor. + +The project already declares the game's cookie - `Globals/MSVC_SecurityCookie.hpp`, address +0x00B94220 - and `src/symbols/data.asm` can publish absolute symbols (`SYM EQU 00B94220h`), so +redirecting the compiler's reference looks like it should work. It does not: + + PUBLIC ___security_cookie + ___security_cookie EQU 00B94220h + + LIBCMT.lib(gs_cookie.obj) : error LNK2005: ___security_cookie already defined in data.asm.obj + +The CRT owns that symbol. Getting past it needs `/FORCE:MULTIPLE` on the whole link, with undefined +symbol selection, and the `OpenSHC.exe` target would then read a cookie at an address that only +exists inside the game's process. Not worth 4% on two functions - but note `/GS-` is *not* the +answer either, because the original has the cookie code and dropping it loses more than it gains. + +`playSFXAtLocation` is likewise done at 98.1% / normalized 100%: its only diff is the call targets +of the CRT helpers `_CIsqrt` and `_ftol2_sse`. + +## updateCrowding: the anchor and the store count cannot both be had (37.7%) + +Correcting an earlier note that said the anchor was wrong here - in the current one-store form the +anchor is **right**. The original (`disassemble_function 0x00458E60`) is: + + add ecx, 0x368b4 ; anchor on currentPopulation, displacement 0 + mov ebx, 8 ; a countdown, no playerID + xor edi, edi ; a zero register, before the loop + mov esi, dword ptr [ecx - 0x210c] ; populationCap + mov eax, dword ptr [ecx] ; currentPopulation + ... + mov dword ptr [ecx - 0x2100], edi ; crowding = 0, four stores, one per arm + ... + add ecx, 0x39f4 + sub ebx, 1 + jnz + +We match the anchor, the countdown, the pointer stride and both reads. The gap is that the original +assigns the *field* in each of the four arms, where we compute into a local and store once. The two +forms measure: + +* one store, value in `eax` -> **37.7%**, anchor `0x368b4` correct +* four field stores -> 33.3%, anchor moves to `0x347b4`, i.e. onto `crowding` itself + +The four stores drag the anchor because our zeros are immediates (`mov dword ptr [ecx], 0`), and an +immediate store is shortest at displacement 0, so MSVC bases on the field being stored. The original +pays nothing for a displacement because its zero is in `edi`, which is also why it can spell +`currentPopulation <= 0` as `cmp eax, edi` instead of our `test eax, eax`. Giving the zero a name to +force it into a register does not work - MSVC constant-propagates it straight back to an immediate: + +* `int noCrowding = 0;` inside the loop, used as the store source -> 33.3% +* the same declared before the loop, live range spanning it -> 33.3% + +So the choice is the correct anchor or the correct store count, and the anchor is worth 4.4 points +more. Keep the one-store form. diff --git a/cmake/compiler-flags-gl.txt b/cmake/compiler-flags-gl.txt index d0c81b8d5..d646b7f7d 100644 --- a/cmake/compiler-flags-gl.txt +++ b/cmake/compiler-flags-gl.txt @@ -16,4 +16,7 @@ src/OpenSHC/Audio/SFX/SFXState/playUnitSpeech.cpp src/OpenSHC/Game/GameStateStructures/getSellResourceAmount.cpp src/OpenSHC/Map/TileMapState/determineBuildingPlacementRotation.cpp src/OpenSHC/Map/Navigation/DirectionAlgorithmState/getMouseVectorLengthBasedOnDirection.cpp -src/OpenSHC/Input/MouseState/updateRightDragCameraControl.cpp \ No newline at end of file +src/OpenSHC/Input/MouseState/updateRightDragCameraControl.cpp +src/OpenSHC/Game/GameStateStructures/processFoodConsumption.cpp +src/OpenSHC/Game/GameStateStructures/computePopulationStatistics.cpp +src/OpenSHC/Game/GameStateStructures/resetVariousCountsAndStatisticsAndStartGoodsAndResources.cpp diff --git a/src/OpenSHC/Game/GameStateStructures/computeBuildingCategoryEntryPointAndDestroyEarlierBuilding.cpp b/src/OpenSHC/Game/GameStateStructures/computeBuildingCategoryEntryPointAndDestroyEarlierBuilding.cpp new file mode 100644 index 000000000..291cc7847 --- /dev/null +++ b/src/OpenSHC/Game/GameStateStructures/computeBuildingCategoryEntryPointAndDestroyEarlierBuilding.cpp @@ -0,0 +1,155 @@ +#include "../GameStateStructures.func.hpp" + +#include "OpenSHC/Map/Buildings/BuildingsState.func.hpp" +#include "OpenSHC/Game/Player/PlayerDataBuildingCategoryEnum.hpp" +#include "OpenSHC/UI/Enums/MenuViewType.hpp" + +#include "OpenSHC/Globals/DAT_BuildingsState.hpp" +#include "OpenSHC/Globals/DAT_GameCore.hpp" +#include "OpenSHC/Globals/DAT_TileMapState.hpp" +#include "OpenSHC/Globals/DAT_ViewportRenderState.hpp" + +namespace OpenSHC { +namespace Game { + + using Player::PlayerDataBuildingCategoryEnum; + using UI::Enums::MenuViewType; + + // FUNCTION: STRONGHOLDCRUSADER 0x00456FD0 + void GameStateStructures::computeBuildingCategoryEntryPointAndDestroyEarlierBuilding( + uint buildingID, int playerID, PlayerDataBuildingCategoryEnum category) + { + int entryX = 0; + int entryY = 0; + DAT_TileMapState::instance.showNoRubbleWhenDestroyingBuilding = 1; + if (category == Player::PDBCE_KEEP) { + if ((int)this->playerDataArray[playerID].keep.id > 0 + && this->playerDataArray[playerID].keep.id != buildingID) { + MACRO_CALL_MEMBER(Map::Buildings::BuildingsState_Func::destroyBuilding, DAT_BuildingsState::ptr)( + this->playerDataArray[playerID].keep.id); + } + short orientation = DAT_BuildingsState::instance.buildings[buildingID].orientation; + entryX = (short)DAT_BuildingsState::instance.buildings[buildingID].x; + entryY = (short)DAT_BuildingsState::instance.buildings[buildingID].y; + if (orientation == 4) { + entryX = entryX + 3; + --entryY; + } else if (orientation == 2) { + --entryX; + entryY = entryY + 3; + } else if (orientation == 6) { + entryX = entryX + 7; + entryY = entryY + 3; + } else { + entryX = entryX + 3; + entryY = entryY + 7; + } + if (DAT_GameCore::instance.currentMenuViewType == UI::Enums::MVT_MAP_EDITOR_LANDSCAPING) { + DAT_TileMapState::instance.showNoRubbleWhenDestroyingBuilding = 1; + if (this->playerDataArray[playerID].stockpile.id != 0) { + MACRO_CALL_MEMBER(Map::Buildings::BuildingsState_Func::destroyBuilding, DAT_BuildingsState::ptr)( + this->playerDataArray[playerID].stockpile.id); + } + } + } else if (category == Player::PDBCE_MARKETPLACE) { + if (this->playerDataArray[playerID].marketplace.id > 0) { + MACRO_CALL_MEMBER(Map::Buildings::BuildingsState_Func::destroyBuilding, DAT_BuildingsState::ptr)( + this->playerDataArray[playerID].marketplace.id); + } + entryX = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryX; + entryY = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryY; + } else if (category == Player::PDBCE_ENGINEERSGUILD) { + if (this->playerDataArray[playerID].engineersGuild.id > 0) { + MACRO_CALL_MEMBER(Map::Buildings::BuildingsState_Func::destroyBuilding, DAT_BuildingsState::ptr)( + this->playerDataArray[playerID].engineersGuild.id); + } + entryX = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryX; + entryY = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryY; + } else if (category == Player::PDBCE_TUNNELERSGUILD) { + if (this->playerDataArray[playerID].tunnelersGuild.id > 0) { + MACRO_CALL_MEMBER(Map::Buildings::BuildingsState_Func::destroyBuilding, DAT_BuildingsState::ptr)( + this->playerDataArray[playerID].tunnelersGuild.id); + } + entryX = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryX; + entryY = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryY; + } else if (category == Player::PDBCE_STOCKPILE) { + // a storage building of this kind already exists, keep the old entry + if (this->playerDataArray[playerID].stockpile.id > 0) { + return; + } + entryX = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryX; + entryY = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryY; + } else if (category == Player::PDBCE_ARMORY) { + // a storage building of this kind already exists, keep the old entry + if (this->playerDataArray[playerID].armory.id > 0) { + return; + } + entryX = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryX; + entryY = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryY; + } else if (category == Player::PDBCE_GRANARY) { + // a storage building of this kind already exists, keep the old entry + if (this->playerDataArray[playerID].granary.id > 0) { + return; + } + entryX = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryX; + entryY = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryY; + } else if (category == Player::PDBCE_BARRACKS) { + if (this->playerDataArray[playerID].barracks.id > 0) { + MACRO_CALL_MEMBER(Map::Buildings::BuildingsState_Func::destroyBuilding, DAT_BuildingsState::ptr)( + this->playerDataArray[playerID].barracks.id); + } + entryX = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryX; + entryY = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryY; + } else if (category == Player::PDBCE_MERCENARYPOST) { + if (this->playerDataArray[playerID].mercenaryPost.id > 0) { + MACRO_CALL_MEMBER(Map::Buildings::BuildingsState_Func::destroyBuilding, DAT_BuildingsState::ptr)( + this->playerDataArray[playerID].mercenaryPost.id); + } + entryX = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryX; + entryY = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryY; + } else if (category == Player::PDBCE_OILSMELTER) { + entryX = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryX; + entryY = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryY; + } else if (category == Player::PDBCE_CAMPGROUND) { + short orientation = DAT_BuildingsState::instance.buildings[buildingID].orientation; + entryX = (short)DAT_BuildingsState::instance.buildings[buildingID].x; + entryY = (short)DAT_BuildingsState::instance.buildings[buildingID].y; + if (orientation == 4) { + entryX = entryX + 3; + entryY = entryY + 6; + } else if (orientation == 2) { + entryY = entryY + 3; + } else if (orientation == 6) { + entryX = entryX + 6; + entryY = entryY + 3; + } else { + entryX = entryX + 3; + } + } + if (entryX < 0) { + return; + } + if (entryX >= 400) { + return; + } + if (entryY < 0) { + return; + } + if (entryY >= 400) { + return; + } + if (DAT_ViewportRenderState::instance.DAT_BinaryTileMap400x400[entryY * 400 + entryX] != 0) { + (&this->playerDataArray[playerID].keep)[category].id = buildingID; + (&this->playerDataArray[playerID].keep)[category].yEntry = entryY; + (&this->playerDataArray[playerID].keep)[category].xEntry = entryX; + (&this->playerDataArray[playerID].keep)[category].tileEntry + = DAT_ViewportRenderState::instance.translationMatrix[entryY].addXgetTile + entryX; + (&this->playerDataArray[playerID].keep)[category].areaEntry + = (short)DAT_TileMapState::instance + .PathConnectionLayer[(&this->playerDataArray[playerID].keep)[category].tileEntry]; + DAT_TileMapState::instance.showNoRubbleWhenDestroyingBuilding = 0; + } + } + +} +} diff --git a/src/OpenSHC/Game/GameStateStructures/computePopulationStatistics.cpp b/src/OpenSHC/Game/GameStateStructures/computePopulationStatistics.cpp new file mode 100644 index 000000000..f9df446ad --- /dev/null +++ b/src/OpenSHC/Game/GameStateStructures/computePopulationStatistics.cpp @@ -0,0 +1,125 @@ +#include "../GameStateStructures.func.hpp" + +#include "OpenSHC/Game/GameStateStructures.func.hpp" +#include "OpenSHC/Game/GameMode2.hpp" +#include "OpenSHC/WindowsHelper/Enums/BOOLEnum.hpp" + +#include "OpenSHC/Globals/DAT_GameCore.hpp" +#include "OpenSHC/Globals/DAT_GameSynchronyState.hpp" +#include "OpenSHC/Globals/DAT_UIDragDropDefinedData.hpp" +#include "OpenSHC/Globals/DAT_UnitsState.hpp" +#include "OpenSHC/Globals/DAT_WindowAndDirectDraw.hpp" + +namespace OpenSHC { +namespace Game { + + using WindowsHelper::Enums::BOOLEnum; + + // FUNCTION: STRONGHOLDCRUSADER 0x0045AF30 + void GameStateStructures::computePopulationStatistics() + { + for (int playerID = 1; playerID < 9; ++playerID) { + this->playerDataArray[playerID].currentPopulation + = ((this->playerDataArray[playerID].countEntities + - this->playerDataArray[playerID].nonInteractiveCitizenCountUnk) + - this->playerDataArray[playerID].field639_0x2174) + - this->playerDataArray[playerID].armySize; + if (this->playerDataArray[playerID].lordID != 0 + && DAT_UnitsState::instance.units[this->playerDataArray[playerID].lordID].uid + == this->playerDataArray[playerID].lordUID) { + this->playerDataArray[playerID].currentPopulation + = this->playerDataArray[playerID].currentPopulation - 1; + } + if (this->playerDataArray[playerID].ladyIDUnk != 0 + && DAT_UnitsState::instance.units[this->playerDataArray[playerID].ladyIDUnk].uid + == this->playerDataArray[playerID].someUnitIDSelfRef) { + this->playerDataArray[playerID].currentPopulation + = this->playerDataArray[playerID].currentPopulation - 1; + } + if (this->playerDataArray[playerID].jesterIDUnk != 0 + && DAT_UnitsState::instance.units[this->playerDataArray[playerID].jesterIDUnk].uid + == this->playerDataArray[playerID].someUnitIDSelfRef_2) { + this->playerDataArray[playerID].currentPopulation + = this->playerDataArray[playerID].currentPopulation - 1; + } + this->playerDataArray[playerID].populationRelatedCrowdingCountUnk + = this->playerDataArray[playerID].currentPopulation_2 + - this->playerDataArray[playerID].availablePeasantsOrHousedPeasants; + if (this->playerDataArray[playerID].blessedPeopleCountUnk <= 0) { + this->playerDataArray[playerID].blessedPeoplePercentage = 0; + } else { + int blessedPercentage; + if (this->playerDataArray[playerID].unblessedPeopleCountUnk + + this->playerDataArray[playerID].blessedPeopleCountUnk + == 0) { + blessedPercentage = 100; + } else { + blessedPercentage = (this->playerDataArray[playerID].blessedPeopleCountUnk * 100) + / (this->playerDataArray[playerID].unblessedPeopleCountUnk + + this->playerDataArray[playerID].blessedPeopleCountUnk); + } + this->playerDataArray[playerID].blessedPeoplePercentage = blessedPercentage; + } + if (this->mapAndTime.startOfDay != FALSE) { + this->playerDataArray[playerID] + .populationGrowth[this->playerDataArray[playerID].populationGrowthStatisticCounter] + = this->playerDataArray[playerID].availablePeasantsOrHousedPeasants; + this->playerDataArray[playerID].populationGrowthStatisticCounter + = this->playerDataArray[playerID].populationGrowthStatisticCounter + 1; + if (this->playerDataArray[playerID].populationGrowthStatisticCounter >= 8) { + this->playerDataArray[playerID].populationGrowthStatisticCounter = 0; + } + int growthSum = 0; + for (int sample = 0; sample < 8; ++sample) { + growthSum += this->playerDataArray[playerID].populationGrowth[sample]; + } + this->playerDataArray[playerID].averagePopulationGrowthUnk = growthSum / 8; + } + if (this->mapAndTime.monthChanged != 0) { + if (this->mapAndTime.populationIndex < 300) { + this->mapAndTime.playerPopulationStatistics[playerID][this->mapAndTime.populationIndex] + = (short)this->playerDataArray[playerID].currentPopulation; + } else { + // Running average move values + for (int sample = 0; sample < 299; ++sample) { + this->mapAndTime.playerPopulationStatistics[playerID][sample] + = this->mapAndTime.playerPopulationStatistics[playerID][sample + 1]; + } + this->mapAndTime.playerPopulationStatistics[playerID][299] + = (short)this->playerDataArray[playerID].currentPopulation; + } + } + if (this->playerDataArray[playerID].currentPopulation + > DAT_GameSynchronyState::instance.finalResults.finalMaxPopulation[playerID]) { + DAT_GameSynchronyState::instance.finalResults.finalMaxPopulation[playerID] + = (short)this->playerDataArray[playerID].currentPopulation; + } + if (DAT_GameCore::instance.gameMode_2 != GM_EDITOR) { + if (this->playerDataArray[playerID].tacticalPowersBarLevel < 7000) { + this->playerDataArray[playerID].tacticalPowersBarLevel + = this->playerDataArray[playerID].tacticalPowersBarLevel + 1; + } + if (this->playerDataArray[playerID].goldDonation > 0) { + int goldIncrement = this->playerDataArray[playerID].goldDonation; + if (goldIncrement > 10) { + goldIncrement = 10; + } + this->playerDataArray[playerID].currentResources[15] + = this->playerDataArray[playerID].currentResources[15] + goldIncrement; + this->playerDataArray[playerID].goldDonation + = this->playerDataArray[playerID].goldDonation - goldIncrement; + if (playerID == DAT_GameSynchronyState::instance.currentPlayerSlotID) { + DAT_UIDragDropDefinedData::instance.MenuView_TriggerInitial = TRUE; + DAT_WindowAndDirectDraw::instance.unk_resetViewportRelated = 1; + } + } + } + } + if (this->mapAndTime.monthChanged != 0) { + ++this->mapAndTime.populationIndex; + } + MACRO_CALL_MEMBER(GameStateStructures_Func::updateFoodTypesInStockForAllPlayers, this)(); + } + +} +} diff --git a/src/OpenSHC/Game/GameStateStructures/processFoodConsumption.cpp b/src/OpenSHC/Game/GameStateStructures/processFoodConsumption.cpp new file mode 100644 index 000000000..3e16c8e44 --- /dev/null +++ b/src/OpenSHC/Game/GameStateStructures/processFoodConsumption.cpp @@ -0,0 +1,165 @@ +#include "../GameStateStructures.func.hpp" + +#include "OpenSHC/Audio/MSS/SoundSystem.func.hpp" +#include "OpenSHC/Audio/SFX/SFXState.func.hpp" +#include "OpenSHC/Game/GameStateStructures.func.hpp" +#include "OpenSHC/Map/Buildings/BuildingsState.func.hpp" +#include "OpenSHC/Synchrony/GameSynchronyState.func.hpp" +#include "OpenSHC/Game/GameMode.hpp" +#include "OpenSHC/Game/GameMode2.hpp" +#include "OpenSHC/Game/Resources/ResourceType.hpp" +#include "OpenSHC/Map/MapType2.hpp" +#include "OpenSHC/WindowsHelper/Enums/BOOLEnum.hpp" + +#include "OpenSHC/Globals/DAT_BuildingsState.hpp" +#include "OpenSHC/Globals/DAT_CounterFoodWarningInterval.hpp" +#include "OpenSHC/Globals/DAT_GameCore.hpp" +#include "OpenSHC/Globals/DAT_GameState.hpp" +#include "OpenSHC/Globals/DAT_GameSynchronyState.hpp" +#include "OpenSHC/Globals/DAT_MapPropertiesState.hpp" +#include "OpenSHC/Globals/DAT_SFXState.hpp" +#include "OpenSHC/Globals/DAT_SoundSystemState.hpp" + +namespace OpenSHC { +namespace Game { + + using Map::MapType2; + using Resources::ResourceType; + using WindowsHelper::Enums::BOOLEnum; + + // FUNCTION: STRONGHOLDCRUSADER 0x00458AD0 + void GameStateStructures::processFoodConsumption() + { + if (DAT_GameCore::instance.gameMode_2 == GM_EDITOR) { + return; + } + if (DAT_GameCore::instance.gameMode_2 == GM_SIEGE_THAT) { + return; + } + if (DAT_GameCore::instance.gameMode_2 == GM_BUILDERUnk + && DAT_MapPropertiesState::instance.SEC_U3_MapType2_1 == Map::MT_SIEGE) { + return; + } + MACRO_CALL_MEMBER(GameStateStructures_Func::updateFoodTypesInStockForAllPlayers, this)(); + for (int playerID = 1; playerID < 9; ++playerID) { + if (this->playerDataArray[playerID].keep.id <= 0 || this->playerDataArray[playerID].campground.id <= 0) { + continue; + } + this->playerDataArray[playerID].foodStorageLevel = -1; + if (this->playerDataArray[playerID].foodTypesInStock <= 0) { + this->playerDataArray[playerID].foodTypesCurrentlyEaten = 0; + continue; + } + int foodClockSpeed = this->playerDataArray[playerID].currentPopulation * 3; + this->playerDataArray[playerID].foodClockSpeed = foodClockSpeed; + if (DAT_GameSynchronyState::instance.currentGameMode != GM_SOLITARY + && MACRO_CALL_MEMBER(Synchrony::GameSynchronyState_Func::isAIPlayer, DAT_GameSynchronyState::ptr)( + playerID) + != FALSE) { + this->playerDataArray[playerID].foodClockSpeed = (foodClockSpeed * 60) / 100; + } + switch (this->playerDataArray[playerID].rationsSetting) { + case 0: + this->playerDataArray[playerID].foodClockSpeed = 0; + this->playerDataArray[playerID].foodTypesCurrentlyEaten = 0; + break; + case 1: + this->playerDataArray[playerID].foodClockSpeed = this->playerDataArray[playerID].foodClockSpeed / 2; + break; + case 3: + this->playerDataArray[playerID].foodClockSpeed + = (this->playerDataArray[playerID].foodClockSpeed * 3) / 2; + break; + case 4: + this->playerDataArray[playerID].foodClockSpeed = this->playerDataArray[playerID].foodClockSpeed * 2; + } + int storageLevel = -1; + if (this->playerDataArray[playerID].foodClockSpeed != 0) { + storageLevel = ((15000 / this->playerDataArray[playerID].foodClockSpeed) + * this->playerDataArray[playerID].totalFood) + / 800; + this->playerDataArray[playerID].foodStorageLevel = storageLevel; + } + if (DAT_GameState::instance.mapAndTime.monthChanged != 0) { + short leftover = (short)storageLevel; + short lastMonthLeftover = this->playerDataArray[playerID].foodStorageLevelLastMonth; + this->playerDataArray[playerID].foodStorageLevelLastLastMonth = lastMonthLeftover; + this->playerDataArray[playerID].foodStorageLevelLastMonth = leftover; + if (playerID == DAT_GameSynchronyState::instance.currentPlayerSlotID + && DAT_GameState::instance.mapAndTime.singlePlayerHasKeepAndGranary != FALSE) { + if (this->playerDataArray[playerID].totalFood == 0) { + if (DAT_CounterFoodWarningInterval::instance == 0 + && MACRO_CALL_MEMBER( + Audio::MSS::SoundSystem_Func::shouldSoundXNotBePlaying, DAT_SoundSystemState::ptr)() + == FALSE) { + // "No food distributed this month" + MACRO_CALL_MEMBER(Audio::SFX::SFXState_Func::playWAVSFX, DAT_SFXState::ptr)( + "food_warning4.wav"); + } + ++DAT_CounterFoodWarningInterval::instance; + if ((int)DAT_CounterFoodWarningInterval::instance > 2) { + DAT_CounterFoodWarningInterval::instance = 0; + } + } else if (leftover < 2 && leftover != lastMonthLeftover + && this->playerDataArray[playerID].rationsSetting != 0) { + if (leftover == 1) { + if (MACRO_CALL_MEMBER( + Audio::MSS::SoundSystem_Func::shouldSoundXNotBePlaying, DAT_SoundSystemState::ptr)() + == FALSE) { + // "Granary stocks are very low sire" + MACRO_CALL_MEMBER(Audio::SFX::SFXState_Func::playWAVSFX, DAT_SFXState::ptr)( + "food_warning2.wav"); + } + } else if (MACRO_CALL_MEMBER(Audio::MSS::SoundSystem_Func::shouldSoundXNotBePlaying, + DAT_SoundSystemState::ptr)() + == FALSE) { + // "We are almost out of food sire" + MACRO_CALL_MEMBER(Audio::SFX::SFXState_Func::playWAVSFX, DAT_SFXState::ptr)( + "food_warning3.wav"); + } + } + } + } + this->playerDataArray[playerID].foodClock + = this->playerDataArray[playerID].foodClock + this->playerDataArray[playerID].foodClockSpeed; + if (this->playerDataArray[playerID].foodClock > 15000) { + this->playerDataArray[playerID].foodClock = 0; + this->playerDataArray[playerID].foodTypesCurrentlyEaten + = this->playerDataArray[playerID].foodTypesInStock; + while (true) { + this->playerDataArray[playerID].foodTypeToBeEatenNext + = this->playerDataArray[playerID].foodTypeToBeEatenNext + 1; + if (this->playerDataArray[playerID].foodTypeToBeEatenNext > 3) { + this->playerDataArray[playerID].foodTypeToBeEatenNext = 0; + } + if (this->playerDataArray[playerID].foodTypeToBeEatenNext == 0 + && this->playerDataArray[playerID].breadCount > 0) { + MACRO_CALL_MEMBER(Map::Buildings::BuildingsState_Func::processResourceLoss, + DAT_BuildingsState::ptr)(playerID, Resources::RT_BREAD, 1, 0); + break; + } + if (this->playerDataArray[playerID].foodTypeToBeEatenNext == 1 + && this->playerDataArray[playerID].cheeseCount > 0) { + MACRO_CALL_MEMBER(Map::Buildings::BuildingsState_Func::processResourceLoss, + DAT_BuildingsState::ptr)(playerID, Resources::RT_CHEESE, 1, 0); + break; + } + if (this->playerDataArray[playerID].foodTypeToBeEatenNext == 2 + && this->playerDataArray[playerID].meatCount > 0) { + MACRO_CALL_MEMBER(Map::Buildings::BuildingsState_Func::processResourceLoss, + DAT_BuildingsState::ptr)(playerID, Resources::RT_MEAT, 1, 0); + break; + } + if (this->playerDataArray[playerID].foodTypeToBeEatenNext == 3 + && this->playerDataArray[playerID].appleCount > 0) { + MACRO_CALL_MEMBER(Map::Buildings::BuildingsState_Func::processResourceLoss, + DAT_BuildingsState::ptr)(playerID, Resources::RT_APPLE, 1, 0); + break; + } + } + } + } + } + +} +} diff --git a/src/OpenSHC/Game/GameStateStructures/processPeasantSpawnAndDespawnCycle.cpp b/src/OpenSHC/Game/GameStateStructures/processPeasantSpawnAndDespawnCycle.cpp new file mode 100644 index 000000000..2dfd626ec --- /dev/null +++ b/src/OpenSHC/Game/GameStateStructures/processPeasantSpawnAndDespawnCycle.cpp @@ -0,0 +1,137 @@ +#include "../GameStateStructures.func.hpp" + +#include "OpenSHC/Audio/MSS/SoundSystem.func.hpp" +#include "OpenSHC/Audio/SFX/SFXState.func.hpp" +#include "OpenSHC/Game/GameStateStructures.func.hpp" +#include "OpenSHC/Map/Units/UnitsState.func.hpp" +#include "OpenSHC/UI/Helpers.func.hpp" +#include "OpenSHC/Audio/SFX/SpeechEffectID.hpp" +#include "OpenSHC/Game/GameMode.hpp" +#include "OpenSHC/Game/GameMode2.hpp" +#include "OpenSHC/Map/Buildings/BuildingType.hpp" +#include "OpenSHC/Map/MapType2.hpp" +#include "OpenSHC/Map/Units/UnitType.hpp" +#include "OpenSHC/WindowsHelper/Enums/BOOLEnum.hpp" + +#include "OpenSHC/Globals/DAT_BuildingsState.hpp" +#include "OpenSHC/Globals/DAT_GameCore.hpp" +#include "OpenSHC/Globals/DAT_GameSynchronyState.hpp" +#include "OpenSHC/Globals/DAT_MapPropertiesState.hpp" +#include "OpenSHC/Globals/DAT_SFXState.hpp" +#include "OpenSHC/Globals/DAT_SoundSystemState.hpp" +#include "OpenSHC/Globals/DAT_TroopDefinedData.hpp" +#include "OpenSHC/Globals/DAT_UnitsState.hpp" + +namespace OpenSHC { +namespace Game { + + using Audio::SFX::SpeechEffectID; + using Map::MapType2; + using Map::Buildings::BuildingType; + using Map::Units::UnitType; + using WindowsHelper::Enums::BOOLEnum; + + // FUNCTION: STRONGHOLDCRUSADER 0x0045B4A0 + void GameStateStructures::processPeasantSpawnAndDespawnCycle() + { + // handle peasant (de)spawning + if (DAT_GameCore::instance.gameMode_2 == GM_BUILDERUnk + && DAT_MapPropertiesState::instance.SEC_U3_MapType2_1 == Map::MT_SIEGE) { + return; + } + for (int playerID = 1; playerID < 9; ++playerID) { + if (this->playerDataArray[playerID].lordKilledByPlayerID != 0) { + continue; + } + if (this->playerDataArray[playerID].playerDeathRelated != 0 + && DAT_GameSynchronyState::instance.currentGameMode == GM_SOLITARY) { + continue; + } + if (DAT_GameSynchronyState::instance.currentGameMode != GM_SOLITARY) { + this->playerDataArray[playerID].vclockIncrement + = DAT_TroopDefinedData::instance + .PeasantSpawnClockIncrementSkirmish[this->playerDataArray[playerID].popularity / 500]; + } else { + int popularityIndex = this->playerDataArray[playerID].popularity / 500; + if (this->playerDataArray[playerID].currentPopulation > 100) { + this->playerDataArray[playerID].vclockIncrement + = DAT_TroopDefinedData::instance + .PeasantSpawnClockIncrementSolitaryPopMoreThan100[popularityIndex]; + } else { + this->playerDataArray[playerID].vclockIncrement + = DAT_TroopDefinedData::instance + .PeasantSpawnClockIncrementSolitaryPopLessThan101[popularityIndex]; + } + } + this->playerDataArray[playerID].vclock + = this->playerDataArray[playerID].vclockIncrement + this->playerDataArray[playerID].vclock; + if (this->playerDataArray[playerID].vclock >= 4000) { + this->playerDataArray[playerID].vclock = 4000; + } else if (this->playerDataArray[playerID].vclock < 0) { + this->playerDataArray[playerID].vclock = 0; + } + if (this->playerDataArray[playerID].vclockIncrement == 0) { + this->playerDataArray[playerID].vclock = 2000; + } + if (this->playerDataArray[playerID].keep.id <= 0) { + continue; + } + int campgroundID = this->playerDataArray[playerID].campground.id; + if (campgroundID <= 0) { + continue; + } + if (this->playerDataArray[playerID].vclock >= 4000) { + if (this->playerDataArray[playerID].currentPopulation < this->playerDataArray[playerID].populationCap + && this->playerDataArray[playerID].availablePeasantsOrHousedPeasants < 24) { + // Havent reached peasant limit. + int x = this->playerDataArray[playerID].campground.xEntry; + int y = this->playerDataArray[playerID].campground.yEntry; + this->playerDataArray[playerID].vclock = 2000; + int peasantUnitID = MACRO_CALL_MEMBER(Map::Units::UnitsState_Func::spawnUnit, DAT_UnitsState::ptr)( + playerID, playerID, x * 8, y * 8, + DAT_BuildingsState::instance.buildings[campgroundID].terrainHeightUnk, Map::Units::UT_PEASANT); + if (peasantUnitID != 0) { + // Successfully spawned peasant. + DAT_UnitsState::instance.units[peasantUnitID].animationCycleNumber + = DAT_UnitsState::instance.units[peasantUnitID].fixedRng & 0xf; + DAT_UnitsState::instance.units[peasantUnitID].isDisappearingUnk = 1; + if (DAT_GameCore::instance.gameMode_2 == GM_CRUSADER_TUTORIAL) { + MACRO_CALL(UI::Helpers_Func::SetTutorialBuildingActionState)(9, Map::Buildings::BT_HOVEL); + } + } + if (this->playerDataArray[playerID].field26_0x42 == 1) { + this->playerDataArray[playerID].field26_0x42 = 0; + if (playerID == DAT_GameSynchronyState::instance.currentPlayerSlotID + && MACRO_CALL_MEMBER( + Audio::MSS::SoundSystem_Func::shouldSoundXNotBePlaying, DAT_SoundSystemState::ptr)() + == FALSE) { + // people are coming to the castle + MACRO_CALL_MEMBER(Audio::SFX::SFXState_Func::playSpeechSFX, DAT_SFXState::ptr)( + Audio::SFX::SEID_POP_IMMIGRATE); + } + } + } + } else if (this->playerDataArray[playerID].vclock <= 0 + && this->playerDataArray[playerID].availablePeasantsOrHousedPeasants + + this->playerDataArray[playerID].populationRelatedCrowdingCountUnk + > 4) { + this->playerDataArray[playerID].vclock = 2000; + // let a peasant disappear? + MACRO_CALL_MEMBER(GameStateStructures_Func::despawnPeasantOrWorker, this)(playerID); + if (this->playerDataArray[playerID].field26_0x42 == 0) { + this->playerDataArray[playerID].field26_0x42 = 1; + if (playerID == DAT_GameSynchronyState::instance.currentPlayerSlotID + && MACRO_CALL_MEMBER( + Audio::MSS::SoundSystem_Func::shouldSoundXNotBePlaying, DAT_SoundSystemState::ptr)() + == FALSE) { + // sound is probably about peasants leaving the castle + MACRO_CALL_MEMBER(Audio::SFX::SFXState_Func::playSpeechSFX, DAT_SFXState::ptr)( + Audio::SFX::SEID_POP_EMIGRATE); + } + } + } + } + } + +} +} diff --git a/src/OpenSHC/Game/GameStateStructures/resetVariousCountsAndStatisticsAndStartGoodsAndResources.cpp b/src/OpenSHC/Game/GameStateStructures/resetVariousCountsAndStatisticsAndStartGoodsAndResources.cpp new file mode 100644 index 000000000..097136396 --- /dev/null +++ b/src/OpenSHC/Game/GameStateStructures/resetVariousCountsAndStatisticsAndStartGoodsAndResources.cpp @@ -0,0 +1,127 @@ +#include "../GameStateStructures.func.hpp" + +#include "OpenSHC/AI/AICState.func.hpp" +#include "OpenSHC/OS.func.hpp" +#include "OpenSHC/Game/GameMode.hpp" +#include "OpenSHC/Game/GameMode2.hpp" +#include "OpenSHC/Game/TrailType.hpp" +#include "OpenSHC/WindowsHelper/Enums/BOOLEnum.hpp" + +#include "OpenSHC/Globals/DAT_AICState.hpp" +#include "OpenSHC/Globals/DAT_GameCore.hpp" +#include "OpenSHC/Globals/DAT_GameState.hpp" +#include "OpenSHC/Globals/DAT_GameSynchronyState.hpp" +#include "OpenSHC/Globals/DAT_RenderingDefinedData.hpp" + +namespace OpenSHC { +namespace Game { + + using WindowsHelper::Enums::BOOLEnum; + + // FUNCTION: STRONGHOLDCRUSADER 0x00455E10 + void GameStateStructures::resetVariousCountsAndStatisticsAndStartGoodsAndResources() + { + for (int playerID = 1; playerID < 9; ++playerID) { + this->playerDataArray[playerID].weeksWithoutFood = 0; + this->playerDataArray[playerID].foodTypesCurrentlyEaten = 0; + this->playerDataArray[playerID].someCount40 = 0; + this->playerDataArray[playerID].foodStorageLevelLastLastMonth = 0; + this->playerDataArray[playerID].foodStorageLevelLastMonth = 0; + this->playerDataArray[playerID].vclock = 2000; + this->playerDataArray[playerID].someCount43 = 0; + int startingPopularity = this->mapAndTime.startingPopularity * 10; + this->playerDataArray[playerID].popularity = startingPopularity; + this->playerDataArray[playerID].storedPopularityPercent = startingPopularity; + this->playerDataArray[playerID].someCount44 = (short)startingPopularity; + this->playerDataArray[playerID].taxesSetting = 3; + this->playerDataArray[playerID].rationsSetting = 2; + int currentPlayerRations + = DAT_GameState::instance.playerDataArray[DAT_GameSynchronyState::instance.currentPlayerSlotID] + .rationsSetting; + this->playerDataArray[playerID].rationsSetting2 = currentPlayerRations; + this->playerDataArray[playerID].rationsSetting3 = currentPlayerRations; + this->playerDataArray[playerID].someCount45 = 1; + this->playerDataArray[playerID].someCount46 = 1; + this->playerDataArray[playerID].someCount47 = 0; + this->playerDataArray[playerID].textYOffset = 0x3c; + this->playerDataArray[playerID].someCountdown01 = 0; + this->playerDataArray[playerID].playerDeathRelated = 0; + this->playerDataArray[playerID].previousBlessedPeoplePercentage = 0; + this->playerDataArray[playerID].popularityReligionBasedDiv25 = 0; + this->playerDataArray[playerID].someCount48 = 0; + this->playerDataArray[playerID].someCount49 = 0; + this->playerDataArray[playerID].someCount50 = 0; + this->playerDataArray[playerID].someCount51 = 0; + this->playerDataArray[playerID].someCount52 = 0; + this->playerDataArray[playerID].someCount53 = 0; + this->playerDataArray[playerID].someCount54 = 0; + this->playerDataArray[playerID].someCount55 = 0; + this->playerDataArray[playerID].someCount56 = 0; + this->playerDataArray[playerID].someCount57 = 0; + this->playerDataArray[playerID].someCount58 = 0; + this->playerDataArray[playerID].someCount59 = 0; + this->playerDataArray[playerID].someCount60 = 0; + this->playerDataArray[playerID].keepEnclosementRelatedCountdown = 0; + this->playerDataArray[playerID].defensesDamagedByPlayer = 0; + for (int resourceType = 0; resourceType < 25; ++resourceType) { + this->playerDataArray[playerID].startResources[resourceType] + = this->mapAndTime.startGoods[resourceType]; + } + MACRO_CALL(OS_Func::_memset)(this->playerDataArray[playerID].snoozedBuildings, 0, 100); + this->playerDataArray[playerID].troopsKilled = 0; + this->playerDataArray[playerID].troopsLost = 0; + this->playerDataArray[playerID].weightedLosses = 0; + this->playerDataArray[playerID].field25_0x40 = 0; + this->playerDataArray[playerID].field26_0x42 = 0; + if (this->mapAndTime.editScenarioExtraOptions != 0) { + this->playerDataArray[playerID].taxesSetting = this->mapAndTime.scenarioTaxesSetting; + int scenarioRations = this->mapAndTime.scenarioRationsSetting; + this->playerDataArray[playerID].rationsSetting2 = scenarioRations; + this->playerDataArray[playerID].rationsSetting3 = scenarioRations; + this->playerDataArray[playerID].rationsSetting = scenarioRations; + this->playerDataArray[playerID].currentResources[0xf] = this->mapAndTime.scenarioGold; + } + if (DAT_GameSynchronyState::instance.currentGameMode != GM_SOLITARY) { + int gameTypeMultiplier = 1; + if (DAT_GameCore::instance.gameMode_2 == GM_SKIRMISH_AND_MULTIPLAYER + && DAT_GameCore::instance.isSkirmishTrail == TRUE + && DAT_GameCore::instance.currentTrailType == TT_EXTREME) { + gameTypeMultiplier = 3; + } + if (DAT_GameSynchronyState::instance.currentPlayerFullIDArray[playerID] != -1) { + this->playerDataArray[playerID].startResources[0xf] + = DAT_RenderingDefinedData::instance + .field451_0x53bb4[(DAT_GameSynchronyState::instance.skirmishCurrentAdvantageBalance + + DAT_GameSynchronyState::instance.skirmishGameIntensityType * 5) + * 2 + + 0xc] + * gameTypeMultiplier; + } else if (DAT_GameSynchronyState::instance.currentAIArray[playerID] != 0) { + this->playerDataArray[playerID].startResources[0xf] + = DAT_RenderingDefinedData::instance + .field451_0x53bb4[(DAT_GameSynchronyState::instance.skirmishCurrentAdvantageBalance + + DAT_GameSynchronyState::instance.skirmishGameIntensityType * 5) + * 2 + + 0xd] + * gameTypeMultiplier; + } + } + DAT_GameState::instance.mapAndTime.populationIndex = 0; + this->playerDataArray[playerID].beforeLastMonthsGold + = (short)this->playerDataArray[playerID].currentResources[0xf]; + this->playerDataArray[playerID].lastMonthsGold + = (short)this->playerDataArray[playerID].currentResources[0xf]; + this->playerDataArray[playerID].aiControlStatusRelated = -1000; + for (int foodType = 0; foodType < 4; ++foodType) { + this->playerDataArray[playerID].isFoodTypeBanned[foodType] = 0; + } + MACRO_CALL_MEMBER(AI::AICState_Func::setCurrentAttackStrength, DAT_AICState::ptr)(playerID); + MACRO_CALL_MEMBER(AI::AICState_Func::setCurrentAttackRaidParameter, DAT_AICState::ptr)(playerID); + DAT_GameState::instance.playerDataArray[playerID].nervousBikCountdown = 0x30; + this->playerDataArray[playerID].tacticalPowersBarLevel = 0; + this->playerDataArray[playerID].goldDonation = 0; + } + } + +} +} diff --git a/src/OpenSHC/Game/GameStateStructures/spawnMotherOrChild.cpp b/src/OpenSHC/Game/GameStateStructures/spawnMotherOrChild.cpp new file mode 100644 index 000000000..d523af8c1 --- /dev/null +++ b/src/OpenSHC/Game/GameStateStructures/spawnMotherOrChild.cpp @@ -0,0 +1,145 @@ +#include "../GameStateStructures.func.hpp" + +#include "OpenSHC/Map/Units/UnitsState.func.hpp" +#include "OpenSHC/Game/GameMode.hpp" +#include "OpenSHC/Game/GameMode2.hpp" +#include "OpenSHC/Map/Buildings/BuildingLogicalState.hpp" +#include "OpenSHC/Map/Buildings/BuildingType.hpp" +#include "OpenSHC/Map/Units/UnitType.hpp" + +#include "OpenSHC/Globals/DAT_BuildingsState.hpp" +#include "OpenSHC/Globals/DAT_GameCore.hpp" +#include "OpenSHC/Globals/DAT_GameState.hpp" +#include "OpenSHC/Globals/DAT_GameSynchronyState.hpp" +#include "OpenSHC/Globals/DAT_TileMapState.hpp" +#include "OpenSHC/Globals/DAT_UnitsState.hpp" +#include "OpenSHC/Globals/DAT_ViewportRenderState.hpp" + +namespace OpenSHC { +namespace Game { + + using Map::Buildings::BuildingLogicalState; + using Map::Buildings::BuildingType; + using Map::Units::UnitType; + + // FUNCTION: STRONGHOLDCRUSADER 0x00458280 + void GameStateStructures::spawnMotherOrChild() + { + if (DAT_GameCore::instance.gameMode_2 == GM_EDITOR) { + return; + } + if (DAT_GameCore::instance.gameMode_2 == GM_SIEGE_THAT) { + return; + } + for (int playerID = 1; playerID < 9; ++playerID) { + // if has keep and campground + if (this->playerDataArray[playerID].keep.id <= 0 || this->playerDataArray[playerID].campground.id <= 0) { + continue; + } + // Only spawn in non-multiplayer + if (DAT_GameSynchronyState::instance.currentGameMode == GM_SKIRMISH_SINGLE_PLAYER) { + if (DAT_GameState::instance.playerDataArray[playerID].someCount46 != 0) { + // if total enemy troop value < 101 + if (DAT_GameState::instance.playerDataArray[playerID].totalEnemyTroopValue > 100) { + DAT_GameState::instance.playerDataArray[playerID].someCount46 = 0; + continue; + } + } else { + // if total enemy troop value < 41 + if (DAT_GameState::instance.playerDataArray[playerID].totalEnemyTroopValue > 40) { + continue; + } + DAT_GameState::instance.playerDataArray[playerID].someCount46 = 1; + } + } else if (this->playerDataArray[playerID].sumOfTotalEnemyUnitsCount > 0) { + continue; + } + int spawnBudget + = (75 - this->playerDataArray[playerID].someCount03) - this->playerDataArray[playerID].someCount02; + for (int buildingID = 0; buildingID < DAT_BuildingsState::instance.maxBuildingsCount; ++buildingID) { + if (DAT_BuildingsState::instance.buildings[buildingID].logicalState == (BuildingLogicalState)0 + || DAT_BuildingsState::instance.buildings[buildingID].logicalState == Map::Buildings::BLS_REMOVE + || DAT_BuildingsState::instance.buildings[buildingID].owner != playerID + || DAT_BuildingsState::instance.buildings[buildingID].fireDuration != 0 + || (DAT_BuildingsState::instance.buildings[buildingID].buildingType != Map::Buildings::BT_HOVEL + && DAT_BuildingsState::instance.buildings[buildingID].buildingType + != Map::Buildings::BT_HOUSE)) { + continue; + } + if (spawnBudget <= 0) { + break; + } + // if building has no unit + if (DAT_BuildingsState::instance.buildings[buildingID].unitRefID == 0) { + short motherEntryX = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryX; + if (motherEntryX > 0) { + short motherEntryY = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryY; + if (motherEntryY > 0) { + int motherTile + = DAT_ViewportRenderState::instance.translationMatrix[motherEntryY].addXgetTile + + motherEntryX; + int motherUnitID = MACRO_CALL_MEMBER(Map::Units::UnitsState_Func::spawnUnit, + DAT_UnitsState::ptr)(playerID, playerID, motherEntryX * 8, motherEntryY * 8, + DAT_TileMapState::instance.HeightLayer[motherTile], Map::Units::UT_MOTHER); + if (motherUnitID != 0) { + // store in unitRef + DAT_BuildingsState::instance.buildings[buildingID].unitRefID = (short)motherUnitID; + DAT_BuildingsState::instance.buildings[buildingID].unitRefUID + = DAT_UnitsState::instance.units[motherUnitID].uid; + DAT_UnitsState::instance.units[motherUnitID].workplaceBuildingUID + = DAT_BuildingsState::instance.buildings[buildingID].uid; + DAT_UnitsState::instance.units[motherUnitID].workplaceBuildingID_1 = (short)buildingID; + --spawnBudget; + DAT_BuildingsState::instance.buildings[buildingID].childSpawnCountdown + = (char)((DAT_BuildingsState::instance.buildings[buildingID].fireRelatedRNG1 >> 10) + % 3) + + 1; + } + } + } + } else if (DAT_GameState::instance.playerDataArray[playerID].popularity / 100 >= 50) { + // if popularity is higher than 49 + DAT_BuildingsState::instance.buildings[buildingID].childSpawnCountdown + = DAT_BuildingsState::instance.buildings[buildingID].childSpawnCountdown - 1; + if ((char)DAT_BuildingsState::instance.buildings[buildingID].childSpawnCountdown <= 0 + && (short)(char)DAT_BuildingsState::instance.buildings[buildingID].maxOccupants + != DAT_BuildingsState::instance.buildings[buildingID].currentEmployeeCount) { + short childEntryX = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryX; + DAT_BuildingsState::instance.buildings[buildingID].childSpawnCountdown + = (char)((DAT_BuildingsState::instance.buildings[buildingID].fireRelatedRNG1 >> 10) % 3) + + 1; + if (childEntryX > 0) { + short childEntryY = DAT_BuildingsState::instance.buildings[buildingID].buildingEntryY; + if (childEntryY > 0) { + int childTile + = DAT_ViewportRenderState::instance.translationMatrix[childEntryY].addXgetTile + + childEntryX; + int childUnitID = MACRO_CALL_MEMBER(Map::Units::UnitsState_Func::spawnUnit, + DAT_UnitsState::ptr)(playerID, playerID, childEntryX * 8, childEntryY * 8, + DAT_TileMapState::instance.HeightLayer[childTile], Map::Units::UT_CHILD); + if (childUnitID != 0) { + --spawnBudget; + DAT_BuildingsState::instance.buildings[buildingID] + .workerID[DAT_BuildingsState::instance.buildings[buildingID] + .currentEmployeeCount] = (short)childUnitID; + DAT_BuildingsState::instance.buildings[buildingID].workerUID + [DAT_BuildingsState::instance.buildings[buildingID].currentEmployeeCount] + = DAT_UnitsState::instance.units[childUnitID].uid; + int workplaceUID = DAT_BuildingsState::instance.buildings[buildingID].uid; + // increment currentEmployeeCount of building + DAT_BuildingsState::instance.buildings[buildingID].currentEmployeeCount + = DAT_BuildingsState::instance.buildings[buildingID].currentEmployeeCount + 1; + DAT_UnitsState::instance.units[childUnitID].workplaceBuildingID_1 + = (short)buildingID; + DAT_UnitsState::instance.units[childUnitID].workplaceBuildingUID = workplaceUID; + } + } + } + } + } + } + } + } + +} +} diff --git a/src/OpenSHC/Game/GameStateStructures/spawnPoisonCloudsAroundBuilding.cpp b/src/OpenSHC/Game/GameStateStructures/spawnPoisonCloudsAroundBuilding.cpp new file mode 100644 index 000000000..9e31db7ce --- /dev/null +++ b/src/OpenSHC/Game/GameStateStructures/spawnPoisonCloudsAroundBuilding.cpp @@ -0,0 +1,55 @@ +#include "../GameStateStructures.func.hpp" + +#include "OpenSHC/Audio/SFX/SFXState.func.hpp" +#include "OpenSHC/Map/Entities/EntityState.func.hpp" +#include "OpenSHC/Random/RNG.func.hpp" +#include "OpenSHC/UI/MinimapViewState.func.hpp" +#include "OpenSHC/DE/SHCDE/eSFX.hpp" +#include "OpenSHC/Map/Entities/EntityType.hpp" + +#include "OpenSHC/Globals/DAT_BuildingsState.hpp" +#include "OpenSHC/Globals/DAT_EntityState.hpp" +#include "OpenSHC/Globals/DAT_MinimapViewState.hpp" +#include "OpenSHC/Globals/DAT_SFXState.hpp" +#include "OpenSHC/Globals/DAT_TileMapState.hpp" +#include "OpenSHC/Globals/DAT_ViewportRenderState.hpp" +#include "OpenSHC/Globals/SEC_RNG.hpp" + +namespace OpenSHC { +namespace Game { + + using DE::SHCDE::eSFX; + using Map::Entities::EntityType; + + // FUNCTION: STRONGHOLDCRUSADER 0x00459A20 + void GameStateStructures::spawnPoisonCloudsAroundBuilding(int buildingID) + { + for (int attemptsLeft = 20; attemptsLeft > 0; attemptsLeft--) { + uint xPosition = SEC_RNG::instance.currentNumber2 % 30 - 15 + + (short)DAT_BuildingsState::instance.buildings[buildingID].x; + uint yPosition = (SEC_RNG::instance.currentNumber2 >> 8) % 30 - 15 + + (short)DAT_BuildingsState::instance.buildings[buildingID].y; + MACRO_CALL_MEMBER(Random::RNG_Func::nextRandomNumber2, SEC_RNG::ptr)(); + if (xPosition > 399 || yPosition > 399 + || DAT_ViewportRenderState::instance.DAT_BinaryTileMap400x400[yPosition * 400 + xPosition] == 0) { + continue; + } + int tile = DAT_ViewportRenderState::instance.translationMatrix[yPosition].addXgetTile + xPosition; + if ((DAT_TileMapState::instance.LogicLayer[tile] & 0x4a5014b1U) != 0) { + continue; + } + for (int cloudIndex = SEC_RNG::instance.currentNumber2 % 5 - 2; cloudIndex < 8; ++cloudIndex) { + MACRO_CALL_MEMBER(Map::Entities::EntityState_Func::spawnProjectileEntity, DAT_EntityState::ptr)(0, 0, 0, + xPosition * 8, yPosition * 8, (byte)DAT_TileMapState::instance.EntityLayerLT25[tile + 0x13a10], 0, + 0, 0, Map::Entities::ET_COW_POISON_CLOUD, 0); + } + MACRO_CALL_MEMBER(Audio::SFX::SFXState_Func::playSFXAtLocation, DAT_SFXState::ptr)( + xPosition, yPosition, DE::SHCDE::FX_FLIES); + MACRO_CALL_MEMBER(UI::MinimapViewState_Func::setSpawnMoment, DAT_MinimapViewState::ptr)( + xPosition, yPosition); + return; + } + } + +} +} diff --git a/src/OpenSHC/Game/GameStateStructures/updateCrowding.cpp b/src/OpenSHC/Game/GameStateStructures/updateCrowding.cpp new file mode 100644 index 000000000..367580c3e --- /dev/null +++ b/src/OpenSHC/Game/GameStateStructures/updateCrowding.cpp @@ -0,0 +1,27 @@ +#include "../GameStateStructures.func.hpp" + +namespace OpenSHC { +namespace Game { + + // FUNCTION: STRONGHOLDCRUSADER 0x00458E60 + void GameStateStructures::updateCrowding() + { + for (int playerID = 1; playerID != 9; ++playerID) { + int populationCap = this->playerDataArray[playerID].populationCap; + int currentPopulation = this->playerDataArray[playerID].currentPopulation; + int crowding; + if (currentPopulation <= populationCap) { + crowding = 0; + } else if (currentPopulation <= 0) { + crowding = 0; + } else if (populationCap <= 0) { + crowding = 150; + } else { + crowding = (currentPopulation * 100) / populationCap; + } + this->playerDataArray[playerID].crowding = crowding; + } + } + +} +} diff --git a/status/addresses-SHC-3BB0A8C1.txt b/status/addresses-SHC-3BB0A8C1.txt index c51eb419d..bbf516a34 100644 --- a/status/addresses-SHC-3BB0A8C1.txt +++ b/status/addresses-SHC-3BB0A8C1.txt @@ -17039,7 +17039,7 @@ SHC_3BB0A8C1_0x00455D40 | 0.0% | Pending SHC_3BB0A8C1_0x00455D90 | 0.0% | Pending -SHC_3BB0A8C1_0x00455E10 | 0.0% | Pending +SHC_3BB0A8C1_0x00455E10 | 92.2% | Volatile reg naming differs SHC_3BB0A8C1_0x00455E40 | 0.0% | Pending @@ -17155,7 +17155,7 @@ SHC_3BB0A8C1_0x00456C50 | 0.0% | Pending SHC_3BB0A8C1_0x00456EF0 | 0.0% | Pending -SHC_3BB0A8C1_0x00456FD0 | 0.0% | Pending +SHC_3BB0A8C1_0x00456FD0 | 33.5% | Join block placed after arm 1 SHC_3BB0A8C1_0x004572A0 | 0.0% | Pending @@ -17685,7 +17685,7 @@ SHC_3BB0A8C1_0x004580C0 | 0.0% | Pending SHC_3BB0A8C1_0x004580F0 | 0.0% | Pending -SHC_3BB0A8C1_0x00458280 | 0.0% | Pending +SHC_3BB0A8C1_0x00458280 | 54.6% | Budget spilled, base in EBX SHC_3BB0A8C1_0x00458360 | 0.0% | Pending @@ -17733,7 +17733,7 @@ SHC_3BB0A8C1_0x00458A70 | 0.0% | Pending SHC_3BB0A8C1_0x00458A80 | 0.0% | Pending -SHC_3BB0A8C1_0x00458AD0 | 0.0% | Pending +SHC_3BB0A8C1_0x00458AD0 | 96.8% | Field read would move the anch SHC_3BB0A8C1_0x00458D38 | 0.0% | Pending @@ -17749,7 +17749,7 @@ SHC_3BB0A8C1_0x00458D50 | 0.0% | Pending SHC_3BB0A8C1_0x00458E20 | 0.0% | Pending -SHC_3BB0A8C1_0x00458E60 | 0.0% | Pending +SHC_3BB0A8C1_0x00458E60 | 37.7% | 4 stores anchor on crowding SHC_3BB0A8C1_0x00458EC0 | 0.0% | Pending @@ -17901,7 +17901,7 @@ SHC_3BB0A8C1_0x004598B0 | 0.0% | Pending SHC_3BB0A8C1_0x00459930 | 0.0% | Pending -SHC_3BB0A8C1_0x00459A20 | 0.0% | Pending +SHC_3BB0A8C1_0x00459A20 | 92.4% | loop counter kept in a reg SHC_3BB0A8C1_0x00459A40 | 0.0% | Pending @@ -18027,7 +18027,7 @@ SHC_3BB0A8C1_0x0045AEB0 | 0.0% | Pending SHC_3BB0A8C1_0x0045AF00 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045AF30 | 0.0% | Pending +SHC_3BB0A8C1_0x0045AF30 | 98.0% | Reg alloc differs SHC_3BB0A8C1_0x0045B1C0 | 0.0% | Pending @@ -18037,7 +18037,7 @@ SHC_3BB0A8C1_0x0045B3E0 | 0.0% | Pending SHC_3BB0A8C1_0x0045B460 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045B4A0 | 0.0% | Pending +SHC_3BB0A8C1_0x0045B4A0 | 84.0% | Shared store moves the anchor SHC_3BB0A8C1_0x0045B6B0 | 0.0% | Pending @@ -18147,17 +18147,17 @@ SHC_3BB0A8C1_0x0045CD10 | 0.0% | Pending SHC_3BB0A8C1_0x0045D060 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D080 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D080 | 100.0% | Reimplemented SHC_3BB0A8C1_0x0045D0B0 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D0C0 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D0C0 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D140 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D140 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D1A0 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D1A0 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D200 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D200 | 100.0% | Reimplemented SHC_3BB0A8C1_0x0045D300 | 0.0% | Pending @@ -18327,41 +18327,41 @@ SHC_3BB0A8C1_0x0045D364 | 0.0% | Pending SHC_3BB0A8C1_0x0045D365 | 0.0% | Pending -SHC_3BB0A8C1_0x0045D370 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D370 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D3C0 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D3C0 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D430 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D430 | 100.0% | Reimplemented SHC_3BB0A8C1_0x0045D460 | 0.0% | Pending -SHC_3BB0A8C1_0x0045D690 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D690 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D6C0 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D6C0 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D740 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D740 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D890 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D890 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D8F0 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D8F0 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D950 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D950 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D990 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D990 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045D9E0 | 0.0% | Pending +SHC_3BB0A8C1_0x0045D9E0 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045DA20 | 0.0% | Pending +SHC_3BB0A8C1_0x0045DA20 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045DA40 | 0.0% | Pending +SHC_3BB0A8C1_0x0045DA40 | 100.0% | Reimplemented SHC_3BB0A8C1_0x0045DA60 | 0.0% | Pending -SHC_3BB0A8C1_0x0045DA80 | 0.0% | Pending +SHC_3BB0A8C1_0x0045DA80 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045DAE0 | 0.0% | Pending +SHC_3BB0A8C1_0x0045DAE0 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045DB40 | 0.0% | Pending +SHC_3BB0A8C1_0x0045DB40 | 100.0% | Reimplemented SHC_3BB0A8C1_0x0045DDF4 | 0.0% | Pending @@ -18709,7 +18709,7 @@ SHC_3BB0A8C1_0x0045EF84 | 0.0% | Pending SHC_3BB0A8C1_0x0045EF90 | 0.0% | Pending -SHC_3BB0A8C1_0x0045F080 | 0.0% | Pending +SHC_3BB0A8C1_0x0045F080 | 100.0% | Reimplemented SHC_3BB0A8C1_0x0045F0B4 | 0.0% | Pending @@ -18745,21 +18745,21 @@ SHC_3BB0A8C1_0x0045F0CC | 0.0% | Pending SHC_3BB0A8C1_0x0045F0CD | 0.0% | Pending -SHC_3BB0A8C1_0x0045F0D0 | 0.0% | Pending +SHC_3BB0A8C1_0x0045F0D0 | 100.0% | Reimplemented, requires SHC_3BB0A8C1_0x0045F130 active SHC_3BB0A8C1_0x0045F120 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045F130 | 0.0% | Pending +SHC_3BB0A8C1_0x0045F130 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045F240 | 0.0% | Pending +SHC_3BB0A8C1_0x0045F240 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045F470 | 0.0% | Pending +SHC_3BB0A8C1_0x0045F470 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045F580 | 0.0% | Pending +SHC_3BB0A8C1_0x0045F580 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x0045F5B0 | 0.0% | Pending +SHC_3BB0A8C1_0x0045F5B0 | 99.0% | Reimplemented, two cases where an "edx + ecx" is flipped to "ecx + edx" -SHC_3BB0A8C1_0x0045FDC0 | 0.0% | Pending +SHC_3BB0A8C1_0x0045FDC0 | 70.1% | Reimplemented, but heavy structural mismatch, seems functional SHC_3BB0A8C1_0x0045FEC0 | 0.0% | Pending @@ -18825,15 +18825,15 @@ SHC_3BB0A8C1_0x004613D0 | 0.0% | Pending SHC_3BB0A8C1_0x004613D4 | 0.0% | Pending -SHC_3BB0A8C1_0x004613E0 | 0.0% | Pending +SHC_3BB0A8C1_0x004613E0 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x004614D0 | 0.0% | Pending +SHC_3BB0A8C1_0x004614D0 | 100.0% | Reimplemented SHC_3BB0A8C1_0x004614F0 | 0.0% | Pending SHC_3BB0A8C1_0x00461520 | 0.0% | Pending -SHC_3BB0A8C1_0x00461550 | 0.0% | Pending +SHC_3BB0A8C1_0x00461550 | 100.0% | Reimplemented SHC_3BB0A8C1_0x00461570 | 0.0% | Pending @@ -18879,23 +18879,23 @@ SHC_3BB0A8C1_0x004619C4 | 0.0% | Pending SHC_3BB0A8C1_0x004619C8 | 0.0% | Pending -SHC_3BB0A8C1_0x004619D0 | 0.0% | Pending +SHC_3BB0A8C1_0x004619D0 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x00461A20 | 0.0% | Pending +SHC_3BB0A8C1_0x00461A20 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x00461B90 | 0.0% | Pending +SHC_3BB0A8C1_0x00461B90 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x00461CF0 | 0.0% | Pending +SHC_3BB0A8C1_0x00461CF0 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x00461E50 | 0.0% | Pending +SHC_3BB0A8C1_0x00461E50 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x00461F90 | 0.0% | Pending +SHC_3BB0A8C1_0x00461F90 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x004620F0 | 0.0% | Pending +SHC_3BB0A8C1_0x004620F0 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x00462150 | 0.0% | Pending +SHC_3BB0A8C1_0x00462150 | 100.0% | Reimplemented -SHC_3BB0A8C1_0x00462190 | 0.0% | Pending +SHC_3BB0A8C1_0x00462190 | 100.0% | Reimplemented SHC_3BB0A8C1_0x00462340 | 0.0% | Pending diff --git a/tools/reimplementation-helper/batch/README.md b/tools/reimplementation-helper/batch/README.md index be86d7414..08f5ad1e0 100644 --- a/tools/reimplementation-helper/batch/README.md +++ b/tools/reimplementation-helper/batch/README.md @@ -31,6 +31,7 @@ Analysis helpers, for deciding *what* to change when a function is stuck below 1 | `diff_slots.py NAME` | Every `[esp + N]` slot with its access count on each side, and the slots only we use. Run after `diff_triage.py` reports `FRAME`. | | `diff_jcc.py [NAME]` | Comparisons whose operator is one strictness step off the original (`jl`/`jle`, `jg`/`jge`, `ja`/`jae`) or whose `cmp` operands are the other way round, with context. | | `diff_types.py [--run] [NAME]` | The non-comparison counterpart of `diff_jcc.py`: `movsx` against `movzx` for the same field (its signedness is wrong in the generated header), `add` against `sub` with the same magnitude (a sign error in a formula - how `2400/n + 40` was caught being `- 40`). It pairs only equal-length runs inside one reccmp block, so it stays quiet once the streams drift: it reports nothing across the 101 functions of `Game::GameStateStructures` and fires on both bugs when they are reintroduced. | +| `diff_anchor.py [--run] [NAME]` | Functions whose struct/loop base anchor differs from the original's (`lea esi, [ecx + 0x367e4]` against a different constant). Every displacement in the function then differs while the opcodes still line up, which reads as allocator noise - it is the whole gap between 83.7% and 61.7% on `processFoodConsumption`. The delta is the distance between the two fields MSVC chose; the decompilation names the original's as the pointer it starts the loop with. | | `diff_lea.py [--run] [NAME]` | `lea reg, [a + b]` on our side where the original has the destructive `add a, b`: `field = snapshotLocal + delta` has to keep the snapshot, `field = field + delta` updates the cached read in place. Drop the local when nothing between the read and the use writes the field - that was `updatePopularity`'s last 6.1%. `RLEA` is the reverse, a local the original keeps and we do not. | | `diff_signed.py [NAME-SUBSTRING]` | Comparisons the original makes signed and we make unsigned (or the reverse): `jg/jge/jl/jle` against our `ja/jae/jb/jbe` over the same `cmp`. Fix it by comparing through `(int)` in the source, not by editing the generated header, and re-run the tool to confirm - one instruction barely moves the percentage. | | `quick_diff.py FILE.cpp... [-q]` | Compile one file with `/FA` and diff it against the original function disassembled straight from `_original/` - about 6 seconds, no link and no reccmp, so a dozen source variants can be tried in the time of one build. Shows the whole original function, which reccmp's diff does not when ours is shorter. A triage loop: confirm a 1.0 with `reccmp_report.py --run`, and do not trust a *rise* of its ratio below about 0.6 - on a badly matching function difflib rewards a different alignment, and one such "gain" was a 10-point loss in reccmp. | diff --git a/tools/reimplementation-helper/batch/diff_anchor.py b/tools/reimplementation-helper/batch/diff_anchor.py new file mode 100644 index 000000000..7ce6bcc8e --- /dev/null +++ b/tools/reimplementation-helper/batch/diff_anchor.py @@ -0,0 +1,120 @@ +"""Find functions whose struct/loop base anchor differs from the original's. + +usage: diff_anchor.py [--run] [NAME-SUBSTRING] + +MSVC bases a run of accesses into one object on a single address and expresses every field as a +displacement off it: + + lea esi, [ecx + 0x367e4] ; the anchor + cmp dword ptr [esi - 0x201c], ebp + mov dword ptr [esi + 0xe0], -1 + +Which field lands at offset 0 is MSVC's choice, and when it picks a different one than the original +did, *every* displacement in the function differs while the opcodes still line up. That reads as +allocator noise in the diff and is the single most expensive mismatch in this codebase: the anchor +alone is the whole gap between 83.7% and 61.7% on `processFoodConsumption`. + +ANCHOR reports the two base constants and their difference, which is the distance between the two +fields MSVC chose. Look that offset up in the struct header to see which field the original anchored +on; the decompilation names it directly, as the pointer it starts the loop with +(`_playerData = &DAT_GameState.playerDataArray[1].foodClockSpeed`). + +What to do about a hit, in the order that has paid: + +1. Check whether a *recent source change* caused it. The anchor moves when a field loses stores - a + local for a repeatedly stored field, or an LTCG-enabled CSE, both do it. Such a change can be + correct and still cost 20 points; keep the form with the matching anchor and record why. +2. Otherwise the anchor is probably not reachable. Tested and ineffective on `updateCrowding`, + `processFoodConsumption` and `processPeasantSpawnAndDespawnCycle`: the global vs `this->` spelling + of any field, an extra read of the anchor field, casts on the index, declaration and statement + order, `PlayerData*` pointer locals, and even an `int*` local pointing straight at the field the + original anchors on - MSVC re-bases to the store target regardless. + +An exception worth knowing: where a condition reads two arrays of the *same* struct, reaching the +second off a pointer to the first does move the base, and took `showPopAndGoldPopup` and +`createStatsPopUpEntities` from 87-89% to 100%. See the cheat sheet. + +--run re-runs reccmp first (otherwise reccmp/dll/diff.json from the last run is used). +""" + +import re +import sys + +import common + +# `lea esi, [ecx + 0x367e4]`, `add esi, 0x367e4`, `lea ebx, [eax + ebp]` is not an anchor +ANCHOR_RE = re.compile(r"^(?:lea (e[a-z]{2}), \[(e[a-z]{2}) \+ (0x[0-9a-f]{4,}|\d{4,})\]" + r"|add (e[a-z]{2}), (0x[0-9a-f]{4,}|\d{4,}))$") +MIN_ANCHOR = 0x1000 + + +def op(line): + return line.split("\t")[0].strip() + + +def anchors(lines): + """[(register, constant)] for every base set-up in order.""" + out = [] + for line in lines: + m = ANCHOR_RE.match(line) + if not m: + continue + reg = m.group(1) or m.group(4) + value = int(m.group(3) or m.group(5), 0) + if value >= MIN_ANCHOR: + out.append((reg, value)) + return out + + +def blocks(entry): + pairs = [] + for hunk in entry.get("diff") or []: + for block in hunk[1]: + o = [op(r[1]) for r in block.get("orig", [])] + r = [op(r[1]) for r in block.get("recomp", [])] + if o and r: + pairs.append((o, r)) + return pairs + + +def hints_for(entry): + hits = [] + for o, r in blocks(entry): + a, b = anchors(o), anchors(r) + if not a or len(a) != len(b): + continue + for (_, x), (_, y) in zip(a, b): + if x != y and (x, y) not in [(h[0], h[1]) for h in hits]: + hits.append((x, y, y - x)) + return hits + + +def main(): + args = sys.argv[1:] + if args and args[0] == "--run": + common.run_reccmp() + args = args[1:] + want = args[0] if args else "" + byaddr = common.load_diff() + shown = total = 0 + for f, e, a in common.entries(byaddr): + if e is None or float(e["matching"]) >= 1 or want not in e["name"]: + continue + total += 1 + hits = hints_for(e) + if not hits: + continue + shown += 1 + print("### %s %.1f%% norm %.1f%%" + % (e["name"], float(e["matching"]) * 100, common.normalized_ratio(e) * 100)) + for x, y, delta in hits: + print(" ANCHOR orig: 0x%-8x ours: 0x%-8x ours is 0x%x %s - look that offset up in the " + "struct header" % (x, y, abs(delta), "higher" if delta > 0 else "lower")) + if not total: + print("nothing below 100% in the current list") + else: + print("\n%d of %d function(s) below 100%% anchor differently" % (shown, total)) + + +if __name__ == "__main__": + main() diff --git a/tools/reimplementation-helper/batch/diff_falsesym.py b/tools/reimplementation-helper/batch/diff_falsesym.py new file mode 100644 index 000000000..590611a38 --- /dev/null +++ b/tools/reimplementation-helper/batch/diff_falsesym.py @@ -0,0 +1,99 @@ +"""Find diff lines that are only a reccmp *symbolization* difference, not a codegen difference. + +usage: diff_falsesym.py [--run] [NAME-SUBSTRING] + +reccmp compares the symbolized disassembly text, so it renders an operand as `SYMBOL+N (OFFSET)` +whenever the value happens to land inside a known object. A plain integer constant that is large +enough can land inside one by coincidence - and because the two images lay their data out +differently, it lands inside an object in the original and not in ours. The same instruction then +reads as a mismatch: + + -push I::instance+424671 (OFFSET) + +push 0xffffff + +Those two lines are the same four bytes. `0xffffff` is opaque white, a colour argument to a text +call, and it sits inside BuildingsState in the original image only. This was the *entire* remaining +gap of `renderDebugDataCurrentPlayerData`: 41 lines, all of them that one constant, scoring 96.0% +for code that is byte-identical to the original. + +FALSESYM pairs an orig line carrying a symbol with a recomp line carrying a bare immediate when the +mnemonic and the rest of the operands are identical. It cannot prove the bytes match - it does not +know the object's base address - so confirm each hit by disassembling the original: + + ghidra-mcp disassemble_function # or tools/.../orig_asm.py + +and checking that the original really is a plain immediate. If it is, the function is finished and +no source change can raise its score; record it rather than chasing it. Beware the reverse mistake +too: a constant that *should* be an address, spelled as a literal in our source, produces the same +diff and is a real bug. + +--run re-runs reccmp first (otherwise reccmp/dll/diff.json from the last run is used). +""" + +import re +import sys + +import common + +IMM_RE = re.compile(r"^(0x[0-9a-f]+|\d+)$") +SYM_RE = re.compile(r"[A-Za-z_<>:]") + + +def op(line): + return line.split("\t")[0].strip() + + +def split(line): + parts = line.split(None, 1) + return (parts[0], parts[1].strip() if len(parts) > 1 else "") + + +def hints_for(entry): + hits = {} + for hunk in entry.get("diff") or []: + for block in hunk[1]: + o = [op(r[1]) for r in block.get("orig", [])] + r = [op(r[1]) for r in block.get("recomp", [])] + if not o or len(o) != len(r): + continue + for a, b in zip(o, r): + if a == b: + continue + ma, oa = split(a) + mb, ob = split(b) + # same mnemonic, ours a bare immediate, theirs carrying a symbol + if ma == mb and IMM_RE.match(ob) and SYM_RE.search(oa) and int(ob, 0) >= 0x10000: + hits[(a, b)] = hits.get((a, b), 0) + 1 + return sorted(hits.items(), key=lambda kv: -kv[1]) + + +def main(): + args = sys.argv[1:] + if args and args[0] == "--run": + common.run_reccmp() + args = args[1:] + want = args[0] if args else "" + byaddr = common.load_diff() + shown = total = 0 + for f, e, a in common.entries(byaddr): + if e is None or float(e["matching"]) >= 1 or want not in e["name"]: + continue + total += 1 + hits = hints_for(e) + if not hits: + continue + shown += 1 + lines = sum(n for _, n in hits) + print("### %s %.1f%% norm %.1f%% (%d line(s) look like symbolization only)" + % (e["name"], float(e["matching"]) * 100, common.normalized_ratio(e) * 100, lines)) + for (a, b), n in hits[:10]: + print(" FALSESYM x%-4d orig: %s" % (n, a)) + print(" %14s ours: %s - confirm the original is a plain immediate" % ("", b)) + if not total: + print("nothing below 100% in the current list") + else: + print("\n%d of %d function(s) below 100%% have a symbolization-only hint" % (shown, total)) + + +if __name__ == "__main__": + main()