Repository navigation
Fixes from a third triage pass over PE samples - #361
Open
williballenthin wants to merge 260 commits into
Open
williballenthin wants to merge 260 commits into
williballenthin wants to merge 260 commits into
Conversation
SHGetSpecialFolderLocation, SHGetFolderLocation and SHGetPathFromIDList had no handlers. The signature fallback wrote a NULL PIDL and then returned TRUE with an empty path, so a program that builds a path from CSIDL_COMMON_APPDATA got "\Synaptics" instead of "C:\ProgramData\Synaptics". On Windows, the folder location functions allocate an ITEMIDLIST for the folder, write it to *ppidl and return S_OK, or return E_INVALIDARG for an unknown CSIDL. SHGetPathFromIDList writes the file system path of the PIDL and returns TRUE, or returns FALSE for a NULL or unknown PIDL. The handlers allocate a one-item ITEMIDLIST with a zero terminator and map it to the folder path. The CSIDL paths move from SHGetFolderPath into a CSIDL_PATHS table, which all of these handlers use. The table adds CSIDL_PERSONAL and corrects the CSIDL_COOKIES and CSIDL_INTERNET_CACHE paths. SHGetKnownFolderIDList and SHGetFolderPathAndSubDir use the same tables. Sample: 03485105f3ef3ac8856eca25127fc4b07707aed2773fc953cd3f4bd0fd5fbde1
The default registry had no Software\Microsoft\Windows\CurrentVersion\Run or RunOnce key under HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER. RegOpenKeyEx on them returned ERROR_FILE_NOT_FOUND, so a program that persists through a Run value failed before it set the value, and the report had no registry write. These keys exist on every Windows system. The default configuration now has them as empty keys, so the open succeeds and RegSetValueEx records the value. Sample: 03485105f3ef3ac8856eca25127fc4b07707aed2773fc953cd3f4bd0fd5fbde1 Sample: 0fc506bfe29835d3e966ddaef3f187a98dcf0d63e861403ae1d74dc4afc802b6
CreateFile, CreateNamedPipe and FindFirstFile/FindFirstFileEx returned the 32-bit value 0xFFFFFFFF on failure. On Windows INVALID_HANDLE_VALUE is (HANDLE)-1, so on x64 it is 0xFFFFFFFFFFFFFFFF. Programs that test the result with a 64-bit "cmp rax, -1" took the success path. The handlers now return the pointer-sized -1. CreateFileMapping now accepts both forms of INVALID_HANDLE_VALUE for a mapping that the paging file backs. DeleteFile with a NULL path set the last error to INVALID_HANDLE_VALUE, which is not an error code. It now sets ERROR_PATH_NOT_FOUND. Sample: 0384e5e2588556c68e1ae6db9be71264edb0e9825d7568e848847fc2e04f4425 Sample: 03a5391b46f058e8ee755e1fa1298b893b73a8aa9440500b22ac2b324742d6dd
_crt_atexit, _controlfp_s, _initialize_narrow_environment, _configure_narrow_argv, _set_fmode, _set_new_mode, _configthreadlocale, __current_exception and __current_exception_context returned None. Dispatch does not write the return register for None, so the caller saw a stale value. A MinGW UCRT program called abort() because rax still held the function pointer passed to _crt_atexit. On Windows _crt_atexit, _controlfp_s, _initialize_narrow_environment and _configure_narrow_argv return 0 on success. _set_fmode writes the _fmode global and returns 0, or EINVAL for a mode other than _O_TEXT or _O_BINARY. _set_new_mode returns the previous mode (0 at start). _configthreadlocale returns the previous setting, which is _DISABLE_PER_THREAD_LOCALE at start. __current_exception and __current_exception_context return the address of a per-thread slot that holds NULL when no exception is in flight. Sample: 03c6d22745e9e04ccae6416705cebba5193c35b961ad7d0d2006f948ce8139b7
Both handlers ignored the table, so C++ static constructors and CRT initializers did not run. They now call each non-null entry in order as a guest callback. _initterm_e stops at the first initializer that returns nonzero and returns that value, as the CRT does. Sample: 00c03cc6c9ffb34eb7cfe5fb13c4d44cfc1d0f35520b4d8c4613ab6982bb3171 Sample: 097884735b068c09adaf1bdce293b274a430aa9f4bd3eca9a05d86077bf85850
GetSystemWindowsDirectoryA/W had no handler. The signature fallback
returned 0 and zero-filled the buffer, so a dropper that built its
install path from the result wrote to "\{GUID}.exe" and not to
"C:\Windows\{GUID}.exe".
On a single-user system Windows returns the same directory as
GetWindowsDirectory. The new handler shares the GetWindowsDirectory
code, which writes "C:\Windows" and returns its length.
Sample: 03a251492d904c2a815e3452cbf1d20abf38a661a54f8dfbd593827ad4f4d257
The default environment had no PATH. Every Windows process has PATH, so a launcher that searched PATH for git.exe found nothing and took its error path. The default environment now also has TMP, PROGRAMDATA, COMMONPROGRAMFILES, PUBLIC, APPDATA, LOCALAPPDATA, HOMEDRIVE, HOMEPATH, USERNAME, COMPUTERNAME, PATHEXT, OS and NUMBER_OF_PROCESSORS. The values agree with the default user name, host name, TEMP and the processor count from GetSystemInfo. PROCESSOR_ARCHITECTURE depends on the bitness of the process, so the emulator sets it when it loads the first image: "AMD64" for an x64 process and "x86" for an x86 process (IsWow64Process reports a native 32-bit system). A value from the configuration takes precedence. Sample: 04b9666929dd6d5ef2943dff481f7169387da475e2586726dd6b361ddc5b0cd0
MessageBox returned IDCANCEL for every box. A box with only an OK button cannot return IDCANCEL on Windows; Esc and the close button give IDOK. MB_ABORTRETRYIGNORE and MB_YESNO boxes have no Cancel button and cannot return IDCANCEL either. MessageBox now returns IDOK for MB_OK, IDABORT for MB_ABORTRETRYIGNORE and IDYES for MB_YESNO, which is the first button of each box. Box types that have a Cancel button (MB_OKCANCEL, MB_YESNOCANCEL, MB_RETRYCANCEL, MB_CANCELTRYCONTINUE) still return IDCANCEL, as if the user dismissed the box. Sample: 03a251492d904c2a815e3452cbf1d20abf38a661a54f8dfbd593827ad4f4d257 Sample: 04b9666929dd6d5ef2943dff481f7169387da475e2586726dd6b361ddc5b0cd0
LoadCursor, LoadIcon, LoadBitmap, LoadAccelerators and DialogBoxParam take a resource name that can also be an integer ID made with MAKEINTRESOURCE (a value of 0xFFFF or less). The report rendered the argument as a string pointer, so LoadCursorW(0, IDC_ARROW) showed the bytes at address 0x7f00 as text. FindResource and FindResourceEx had the same problem for integer IDs. normalize_res_identifier moves from kernel32 to the API handler base class, and the new read_res_identifier sets the display of the argument: the name for a string, or the hex value for an integer ID or a "#123" name. The handlers above use it. Sample: 04b9666929dd6d5ef2943dff481f7169387da475e2586726dd6b361ddc5b0cd0
The default registry had no SOFTWARE\Microsoft\Windows NT\CurrentVersion\Svchost key. Service DLLs that install themselves into a svchost group open this key and read a group such as netsvcs. The open failed, so they took their error path. This key exists on every Windows system. The default configuration now has it, with the netsvcs and LocalService groups. Sample: 5eced7367ed63354b4ed5c556e2363514293f614c2c2eb187273381b2ef5f0f9 Sample: eb1079bdd96bc9cc19c38b76342113a09666aad47518ff1a7536eebff8aadb4a
StartServiceCtrlDispatcher walked the table until an entry had both a NULL name and a NULL ServiceMain. A sample passed a table with one entry and no terminator, so the walk read the name of a garbage entry, faulted inside the handler, and the run ended with an access violation. Windows ends the table at the first entry with a NULL name. The walk now does the same, and it also stops at a name that is not in mapped memory. Sample: 093edce50cd8c254f3680c9b52bc9d51051082bb3c0ec3c0a94c801f7a4a491f Sample: 055dcd2036b6931c46d20f158edb83da85e6984c3e3565edd3cfbdd4205ea96d
Imports from api-ms-win-service-core and api-ms-win-service-winsvc did not resolve to a module, so calls to OpenSCManager and the other service functions through these API sets did not reach their handlers. Sample: 0b02e88b79b2aaedf6c9e083388deed7e4bf7eed6d316eaf2acd396d4afa1b8b
Binaries built with WPP software tracing call EtwRegisterTraceGuidsW in ntdll at start-up. It had no handler and no signature, so the run stopped at the first trace call. The handlers act as Windows does when no trace session is active: registration succeeds, the logger handle is not valid, the enable flags and level are 0, and messages are dropped. Sample: 0b02e88b79b2aaedf6c9e083388deed7e4bf7eed6d316eaf2acd396d4afa1b8b Sample: 0a7cfc4adfdc89b1499982977e8d328c8c86716f511f83b583b3fe5e584f14e4
The .NET apphost prints its start-up errors to stderr with fputws, so a host without its application DLL stopped at that call and did not reach its exit. The wide stream functions write to the stream as fwprintf does. Sample: 0b0b0ceba28cad32d3509ef40183c38f519bf8cf4e4be39f018e8deea0c96b39
…trftime Programs built for the multibyte character set call _mbsstr, _mbsrchr, _mbscmp and the other _mbs functions where other programs call the str functions. None had a handler, so such a program stopped at its first string search. The emulated code page has one byte for each character, so the _mbs functions use the str handlers, _mbsinc moves one byte and _mbclen is 1. strftime formats in the C locale and the emulated time zone (UTC). Sample: 02517f55db31d455bdaec70343c2cdce7c256ce57cbc2a6af890f1d6e818508a
When the client closed the connection while the target ran, the reader thread queued a detach, but the run went on until the next stop. The stop reply to the closed socket then raised BrokenPipeError, which ended the run with an error. A disconnect while the target runs now stops it, and a stop reply that cannot be sent detaches the session. Emulation then continues without the debugger, the same as after a D packet. Sample: 0562c00ac18c8489cfa0ddb94c69212b40dfc6e992b147194a9e2117529db311 Sample: 00bcc4699e127402529bf12f5dc48b0f6c05904442c37f13d34434c3e598c1f5
When every guest handler returned ExceptionContinueSearch, the dispatch ended the run with a fault stop for the debugger. The engine loop did not look for that stop after the handler's return, so it ran the faulting instruction again. That dispatched the exception a second time, and the debugger stopped at the callback return address or in the handler. The loop now returns to the debugger as soon as the declined dispatch records the fault stop. The stop is at the faulting instruction, and the exception is dispatched once, as on Windows and as in a run without the debugger. Sample: 00864a3c69dd8c14e1ba04f74ccdab9b562c666686a6ad6477ab981af97821e1
begin_run skipped the breakpoint at the current pc once, so that a resume from a breakpoint stop does not stop again at once. It did this for every run, and each new run (TLS callback, entry point, export) starts with its start address in pc. A breakpoint there never hit. The server now records the pc where the client resumes, and only a run that starts at that pc skips its breakpoint. Sample: 0999d3689c544bd0d803f8e05efcf07f977a7111987911ce828a1e724a35bc24 Sample: 0340f7b7a6ce430d2d517902392dd65cad888b1799ffc69b05d25019e148179f Sample: 00c03cc6c9ffb34eb7cfe5fb13c4d44cfc1d0f35520b4d8c4613ab6982bb3171
An interrupt with no handler, such as the int 0x29 of __fastfail, ended the run from inside the interrupt hook. With a debugger attached, the client got no stop: a continue or step went on to the next run. With a debugger attached, the run now ends through end_run_on_fault, as for a memory fault. The client gets a SIGSEGV stop after the interrupt instruction, and the run ends when the client resumes. Runs without a debugger do not change. Sample: 022f140f77c62c45880def6c3bd7a4d5c796ad02e77bd1046ab87730f7956160
After a fault stop in one run, the run loop kept the fault signal for the end of emulation. Later runs that completed then ended with X0b and no final T05 stop, so the client could not inspect the final state. The signal now resets when the next run starts. The end of emulation reflects the last run: a final T05 stop and W00 when it completes, X0b when it faults. Sample: 022f140f77c62c45880def6c3bd7a4d5c796ad02e77bd1046ab87730f7956160 Sample: 0a6f474a73c9c1157ead39e7e8d510b67350961b229d9877c73cbfb5276a180c Sample: 0c6d02f1fb5d116d1c94c20e76c4e6c4daaaf905f976506b2c2b5560afe02ad9
A step always gave a step stop, also when the step ended emulation. The run loop then stopped again at the same pc for each later step, so the client could not get past the last instruction. A step that ends emulation now gets the final stop, the same as a continue. The next resume gives the exit reply. Sample: 0999d3689c544bd0d803f8e05efcf07f977a7111987911ce828a1e724a35bc24 Sample: 0340f7b7a6ce430d2d517902392dd65cad888b1799ffc69b05d25019e148179f Sample: 00c03cc6c9ffb34eb7cfe5fb13c4d44cfc1d0f35520b4d8c4613ab6982bb3171
The watch, rwatch and awatch stop replies gave the start address of the memory access. An access that begins before a watchpoint and overlaps it gave an address outside the watched range, so the client could not find the watchpoint (lldb showed "watchpoint -1"). The reply now gives the first address of the access that is in the watched range. Sample: 0340f7b7a6ce430d2d517902392dd65cad888b1799ffc69b05d25019e148179f
Both had no handler, so a program that copies a bounded string with them stopped at the first call. They copy at most count characters and return 0 when the text fits, return STRUNCATE with _TRUNCATE, and clear the first character and return ERANGE when it does not fit, as the CRT does. Sample: 013f16ba78c6fd565673bac2f61b3bb7c7b1ef4c2d56c6e0b8d431cd3110f399
On x64, HKEY_LOCAL_MACHINE and the other predefined keys are (HKEY)(ULONG_PTR)(LONG)0x8000000N, so programs pass 0xFFFFFFFF8000000N. The registry manager and get_hkey_type matched only the 32-bit value, so every registry call on a root key from x64 code returned ERROR_INVALID_HANDLE. Both now map the sign-extended value to the 32-bit key in one helper, normalize_hkey. RegGetValue also returned ERROR_SUCCESS with no data when the handle was not a known key. Windows returns ERROR_INVALID_HANDLE for that. Sample: 01344424b21e21a8c9feb1d5c9b9cea01b2d3dada7b25db8fda75cca7bf7a50c Sample: 00949240dee005c34d694b7b4bbdccb01fbf433c147aeda7808a3320fb0b8f8e Sample: 0dd722334ad379399827d30efe0d9bb3952445d07c9250cdc5302f6cc4aa3fa0
pefile stores the parsed strings of a string table block on the directory of the block entry, keyed by string ID. The loader looked for them one level lower, on the language entries, found nothing, and the string table stayed empty. LoadStringA/W then returned 0 for every ID. Windows returns the string from block ID / 16 + 1, index ID % 16. Sample: 0a6f474a73c9c1157ead39e7e8d510b67350961b229d9877c73cbfb5276a180c
The handlers read the string at a NULL pointer, and the emulation stopped with an invalid read. On Windows these functions catch the access violation: lstrlen returns 0, and lstrcpy, lstrcpyn and lstrcat return NULL. Sample: 084586109ad2b693c951d463eb9edb2e1e2606521089e8621c7b3b040b4a00f9
The handler was missing. StringFromGUID2 writes the GUID as a wide
"{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}" string with a NUL into lpsz
and returns 39, the count of characters with the NUL. If cchMax is less
than 39 it writes nothing and returns 0.
Sample: 010f0f68eabd4514d4b16bd53ccac8ee900cf33cc84b1c921316342f344b2892
The handler was missing, so the signature fallback returned STATUS_SUCCESS and left the UNICODE_STRING empty. Callers then read the NULL Buffer. Windows allocates "\REGISTRY\USER\<SID of the current user>" into the UNICODE_STRING and returns STATUS_SUCCESS. The SID is the user SID from the configuration, the same one that GetTokenInformation(TokenUser) gives. Sample: 0666577c844cc494f02d9c19545d952ad5022cf6a3637bf9835e2a6b3be5432e
The handlers were missing, and a program that calls freopen_s(&f, "CONOUT$", "w", stdout) after AllocConsole stopped as unsupported_api. On Windows, freopen reuses the FILE that the caller passes: freopen_s writes that stream to *pFile and returns 0, and freopen returns it. For CONOUT$ and CONIN$ the stream stays on the console. For other paths the stream now writes to the file that fopen would open. If the file cannot be opened, freopen returns NULL and freopen_s writes NULL and returns ENOENT. Sample: 05fc254e7cce155a35bdff201bebd87f08f34c7e06e9d4b8b1dff8a677449aa6
msvcrt.dll exports __CxxFrameHandler2 next to __CxxFrameHandler3, but the handler was missing, so a frame that names it as its handler ended the run as unsupported_api when an exception reached it. It now uses the same handler as __CxxFrameHandler, __CxxFrameHandler3 and __CxxFrameHandler4, which returns ExceptionContinueSearch because Speakeasy does not run catch blocks. Sample: 0104d051badaa463afca21f4e89d457c5318b086025d491bf71a2b3517663b46
PathFileExistsA and PathFileExistsW had no handler, and the signature fallback returned TRUE for every path. A program that checks for a file it has not created yet took the wrong branch. On Windows the function returns TRUE when the file or directory exists, and FALSE with ERROR_FILE_NOT_FOUND when it does not. The handler uses the same test as GetFileAttributes. Sample: 0699677d2b4eec200a3d9d05fa0adeecae314372bfd5097f2e0847fa59b82ed5
_CIfmod had no handler, so the run stopped as unsupported_api. MSVC calls this x87 intrinsic for fmod in 32-bit code. It takes x in ST(1) and y in ST(0), pops both, and leaves fmod(x, y) in ST(0). The new pop_fpu_double reads and pops the x87 stack top, the inverse of push_fpu_double. Sample: 0699677d2b4eec200a3d9d05fa0adeecae314372bfd5097f2e0847fa59b82ed5
GetFileInformationByHandle returned FALSE for every handle and set no last error, even for a handle that CreateFile had just opened. Rust std::fs::metadata uses this call, so is_file() was false for files that exist in the emulated file system. On Windows the function fills the structure for an open file and returns TRUE. The handler now writes the attributes (archive for a file, directory for a directory), the file times as GetFileTime gives them, the volume serial number that GetVolumeInformation reports, the size, one link and a file index that is the same for each handle to one path. An unknown handle gives ERROR_INVALID_HANDLE. Sample: 092e17cdc629e122bbf597eb9a81d555fe7502e49f31a97a5280959798499845
SystemTimeToTzSpecificLocalTime returned TRUE and did not write lpLocalTime, so the caller read uninitialized memory as the local time. On Windows the function converts the UTC time with the bias of the given time zone, or of the current time zone when the pointer is NULL, and writes the result. The handler now subtracts the Bias field in minutes; the current time zone has a bias of 0, as GetTimeZoneInformation reports. A date that is not valid gives ERROR_INVALID_PARAMETER. TzSpecificLocalTimeToSystemTime, the inverse, had no handler and now adds the bias. Sample: 092e17cdc629e122bbf597eb9a81d555fe7502e49f31a97a5280959798499845
GetProcessId had no handler, and the signature fallback returned 0 for every handle, so a program that created a child process could not get its pid. On Windows the function returns the id of the process that the handle refers to, the current process for the pseudo handle -1, and 0 with ERROR_INVALID_HANDLE for a handle that is not a process. GetProcessIdOfThread had no handler either; it now returns the id of the process that owns the thread. Sample: 092e17cdc629e122bbf597eb9a81d555fe7502e49f31a97a5280959798499845
CryptStringToBinaryA decoded its input as UTF-8 and raised UnicodeDecodeError on binary data, which stopped the emulation. Base64 decoding with the default b64decode also dropped invalid characters without an error. On Windows the A function reads bytes and the W function reads UTF-16 code units. For CRYPT_STRING_BASE64 it skips spaces, tabs and line breaks, and a character outside the base64 alphabet makes the call return FALSE with ERROR_INVALID_DATA. The handler now reads the input as raw code units and gives that result for invalid input. Sample: 0d65239438fb18bdd4a80e1a2b071eed3215ac0615d4f62854aeeefde98e69ba
The x86 basic_ios<wchar_t> default constructor had no handler, so DllMain of a sample stopped in its static initializers. The narrow x86 pair was also missing. On Windows these are thiscall members. The protected default constructor leaves the object for basic_ios::init and returns this, and the destructor returns nothing. The handlers take this from ECX like the other x86 iostream constructors. Sample: 027464e67edc703363dc91ba77e7e7ee2826127e840212300daa94d7bad5f602
WdfVersionBind always wrote a pointer to the function table into BindInfo->FuncTable, and only on the first call. A KMDF 1.11 driver calls each WDF function from an array in its own image, so it read the pointer as WdfDriverCreate and other entries as zero, and called address 0. The KMDF loader (LibraryCommonRegisterClient) copies the full function table into FuncTable when FuncCount is at most 438 (KMDF 1.13 and older), and writes a pointer to the table for KMDF 1.15 and later. The handler now does the same on every call: it copies the known entries, sets the rest up to FuncCount to zero, and also writes the component globals each time. Sample: 0ebbe6089f00cd7dc8c065608a1cdd3bfe95989547d84e823eebcc20bc69a312
CoGetObject had no handler. The signature fallback returned S_OK and left
*ppv unchanged, so a caller such as the AutoHotkey ComObjGet("winmgmts:")
used an uninitialized interface pointer.
On Windows CoGetObject sets *ppv to NULL when it fails, and returns
MK_E_SYNTAX when it cannot parse the display name into a moniker. No
monikers are modeled, so the handler now gives that result for all names.
Sample: 0cfa3248392b14bb51b4dfd08c3445b05710e2ac8c1f1d89caa1cdaaf882886e
GetActiveProcessorGroupCount and GetActiveProcessorCount had no handler, so the signature fallback returned 0. GetProcessAffinityMask wrote the mask 0x1 while GetSystemInfo reported 2 processors with the mask 0x3. On Windows a machine has at least one processor group. The processor count functions return the number of processors for group 0 or ALL_PROCESSOR_GROUPS, and 0 with ERROR_INVALID_PARAMETER for a group that does not exist. GetProcessAffinityMask gives the same mask as GetSystemInfo when all processors are available to the process. All of these handlers now use one processor count. Sample: 0d6f5e46f4e25e46cf4a8738aec0fa42b81b69421ae9ae144ab01f180d37e1b8
normalize_dll_name did not map any api-ms-win-mm-* API set, so imports such as api-ms-win-mm-misc-l1-1-0!DefDriverProc or api-ms-win-mm-time-l1-1-0!timeGetTime did not reach the winmm handlers. On Windows these sets resolve to winmmbase.dll, winmm.dll or kernel32.dll, and winmm.dll exports all of their functions, so they now use the winmm handlers. winmm had no DefDriverProc. On Windows it returns 1 for DRV_LOAD, DRV_ENABLE, DRV_DISABLE, DRV_FREE, DRV_INSTALL and DRV_REMOVE, and 0 for all other messages. Sample: 0acf76667b5448e0e6d30ee1f90984900c075c9cbdb1de3e0971f6e6ef314cc2
SendMessage returned 0 for every message and kept no list-view state, so LVM_GETCOLUMN did not give back the width that LVM_INSERTCOLUMN set. On Windows a SysListView32 control keeps its columns. LVM_INSERTCOLUMN inserts at the index, or at the end if the index is past the last column, and returns the new index. LVM_GETCOLUMN fills the LVCOLUMN fields that the mask asks for. LVM_GETCOLUMNWIDTH, LVM_SETCOLUMNWIDTH and LVM_DELETECOLUMN read, change and remove one column. Each window now has a column list, and SendMessage handles these messages for SysListView32 windows. The sample computes the address of its loader code from the column width. Sample: 0c09d533c473f285dd260d9f7b3539a855527d7d3c68937231cf8581b4aef08b
The x86 user-mode GDT put the code descriptor at index 17, the data descriptor at index 16 and the TEB descriptor at index 19, so guest code read CS 0x8B, DS 0x83, FS 0x9B and ES 0. On native 32-bit Windows CS is 0x1B, DS and ES are 0x23 and FS is 0x3B. Malware compares the low byte of CS with 0x1B to tell native x86 from WoW64; with 0x8B it took the WoW64 path, although IsWow64Process returns FALSE, and made a far call to selector 0x33. The x86 user-mode GDT now has the ring 3 code descriptor at index 3, the ring 3 data descriptor at index 4 and the TEB descriptor at index 7. SS stays at the ring 0 selector 0x90: Unicorn runs the guest at CPL 0 and rejects an RPL 3 SS. The kernel-mode GDT and the x64 GDT do not change. Sample: 0be93fa8c17e75b9a3c01bd2d3ed153638c2e26d3724dafdd33eb6ba23389af1
MSVCP140 names route to the msvcrt module, which had the _Mtx functions but no _Cnd functions. A DLL whose static initializer constructs a std::condition_variable stopped in DllMain with an unsupported_api error on _Cnd_init_in_situ. Add _Cnd_init_in_situ, _Cnd_destroy_in_situ, _Cnd_init, _Cnd_destroy, _Cnd_signal, _Cnd_broadcast, _Cnd_wait and _Cnd_timedwait with the argument counts of xthreads.h. _Cnd_init allocates the condition variable and writes its pointer. The functions that return _Thrd_result give _Thrd_success. Threads run one after another, so _Cnd_wait returns at once. _Cnd_timedwait succeeds while a queued run is left that could signal; otherwise it moves the clock to the target time and returns _Thrd_timedout, as the event waits of kernel32 do. Sample: 006ba021a53715b8c05b37729caec622a31a72442e89f07be9ff5e23d9cc36ba
The module had no _makepath, so a program that builds a file name from its parts stopped with an unsupported_api error. Add _makepath, _wmakepath, _makepath_s, _wmakepath_s, _splitpath and _wsplitpath. As in the CRT, _makepath writes the first character of the drive and a colon, adds a backslash to a directory that does not end in a backslash or a slash, and adds a dot to an extension that has none. NULL and empty parts are left out. The _s forms return EINVAL for a NULL buffer or a zero size, and clear the buffer and return ERANGE when the path does not fit. _splitpath takes the drive when the second character is a colon, the directory up to the last separator, and the extension from the last dot of the file name. It does not write NULL outputs. Sample: 0222d52f4e3391293706b771d37a53c81cea1952fa50bb42993f976b64e6515b
The module had no handler for _endthreadex or _endthread. A thread that ended itself with one of them stopped its run with an unsupported_api error, not a normal thread exit. On Windows these functions free the CRT data of the thread and end it, as ExitThread does. Add cdecl handlers that end the current run the same way as kernel32 ExitThread, so the next queued run starts. Sample: 024e5e537c02b2a5b9a41be151cc36807498f5e338b8c7ee592305af0398283e Sample: 0255ba864944be52d31a38fd7e604a82f044ebcd07ff18cee37778f6461069a8
CreateFile looked for "NUL" as a file in the emulated file system. It did not find it, so OPEN_EXISTING failed with INVALID_HANDLE_VALUE, and the other dispositions created a dropped file. On Windows "NUL" and "\\.\NUL" (and "\??\NUL" for the native API) name the null device in any case. CreateFile opens it for every creation disposition. ReadFile gives 0 bytes and returns TRUE, WriteFile discards the data, gives the full count and returns TRUE, and GetFileType returns FILE_TYPE_CHAR. The file manager now opens a null device object for these names, and the data does not go to the dropped files. Sample: 0068795fba64614a716c96d144350b6068f89d85ac987414e87ac4705cbca885
create_process made the Process object with an empty path, which set the image name to "", and then set only the path. kernel32 writes the image name into szExeFile, so a Toolhelp Process32First/Next walk gave an empty name for every process that the sample started. On Windows szExeFile is the file name of the image of the process. The image name is now the base name of the file path. Sample: 010087654f9ac290e38aa11c14e084728d49fe551e5cbf1f75141205baed556d
SHGetSpecialFolderPathA/W had no handler. The signature fallback returned TRUE and wrote no path, so the sample combined an empty folder with a file name and used a relative path for the copy of itself. On Windows the function writes the path of the folder that the CSIDL names into pszPath and returns TRUE, or returns FALSE for a CSIDL that it does not know. The handler uses the same folder paths as SHGetFolderPath. Sample: 021480276eac06bcb5ff34c19643b7c5ab0e6ee0a013a481a26042ad2fc4860b
WinHttpQueryHeaders gave data only for WINHTTP_QUERY_STATUS_CODE. For every other header it returned FALSE without the needed size when the buffer was NULL, and TRUE with no data when there was a buffer. The samples query WINHTTP_QUERY_RAW_HEADERS_CRLF, allocate the size that the first call gives, and parse an empty string. On Windows WINHTTP_QUERY_RAW_HEADERS_CRLF gives the status line and each header line, each ending in CRLF, and a final CRLF, as UTF-16. WINHTTP_QUERY_RAW_HEADERS gives the same lines separated by NUL, with a final extra NUL. A NULL or small buffer gives FALSE, ERROR_INSUFFICIENT_BUFFER and the needed size in bytes. A header that the response does not have gives FALSE and ERROR_WINHTTP_HEADER_NOT_FOUND. The emulated response has the status line "HTTP/1.1 200 OK" and a Content-Length header for the configured response body. Sample: 024e5e537c02b2a5b9a41be151cc36807498f5e338b8c7ee592305af0398283e Sample: 0255ba864944be52d31a38fd7e604a82f044ebcd07ff18cee37778f6461069a8
HttpQueryInfo gave data only for HTTP_QUERY_STATUS_CODE. For every other header it returned FALSE, and with a NULL buffer it did not write the needed size. On Windows HTTP_QUERY_RAW_HEADERS_CRLF gives the status line and each header line, each ending in CRLF, and a final CRLF. HTTP_QUERY_RAW_HEADERS gives the same lines separated by NUL, with a final extra NUL. A NULL or small buffer gives FALSE, ERROR_INSUFFICIENT_BUFFER and the needed size in bytes. A header that the response does not have gives FALSE and ERROR_HTTP_HEADER_NOT_FOUND. HttpQueryInfo now uses the same response headers as WinHttpQueryHeaders. Sample: 024e5e537c02b2a5b9a41be151cc36807498f5e338b8c7ee592305af0398283e Sample: 0255ba864944be52d31a38fd7e604a82f044ebcd07ff18cee37778f6461069a8
SHGetMalloc returns an IMalloc object, but ComApi had no handler for its methods, so a call to IMalloc::Free stopped the run as an unsupported API. IMalloc uses the same allocator as CoTaskMemAlloc. Alloc, Realloc and Free behave as CoTaskMemAlloc, CoTaskMemRealloc and CoTaskMemFree. GetSize returns the size of a block, or (SIZE_T)-1 for NULL or a block that the allocator did not give. DidAlloc returns 1 for a block that the allocator gave, 0 for other pointers, and -1 for NULL. HeapMinimize does nothing. Sample: 042a14d94dd1d5e338b6a919b7276f856b54a8a73d718c430e4c4fceb6988bdf
CoInitializeEx, CoInitialize and OleInitialize always returned S_OK, so a program that calls CoInitializeEx twice to detect an emulator saw S_OK on the second call. Windows keeps a COM initialization count and an apartment model for each thread. The first call returns S_OK. A later call with the same model returns S_FALSE and increments the count. A later call with a different model returns RPC_E_CHANGED_MODE and does not change the count. CoInitialize and OleInitialize use the single-threaded apartment model. CoUninitialize and OleUninitialize decrement the count, and at zero the thread has no model. Sample: 04de5f5e76d43e935c47373f0ad2302eb461e23f0b0488bbf2438b806b4cb9df
GetModuleHandle compared only base names, with the path kept in the name, so a name with a path never matched a module and the call returned NULL. On Windows, a name that has a path matches the module with that full path, without regard to case. A name with no extension gets ".dll", and a trailing dot means no extension. GetModuleHandle now compares a name that has a path separator to the emulated full path of each module. Sample: 04de5f5e76d43e935c47373f0ad2302eb461e23f0b0488bbf2438b806b4cb9df
The PEB was built with NumberOfProcessors 0, but GetSystemInfo reports two processors. On Windows the PEB field holds the same processor count that GetSystemInfo gives in dwNumberOfProcessors. alloc_peb now writes the kernel32 processor count into the PEB, for x86 and x64. Sample: 0d6f5e46f4e25e46cf4a8738aec0fa42b81b69421ae9ae144ab01f180d37e1b8
WdfDriverCreate did not write the WDFDRIVER out parameter, so the driver passed handle 0 to the next WDF call. On Windows, WdfDriverCreate writes the framework driver handle to *Driver when the pointer is not NULL, and stores the same handle in WDF_DRIVER_GLOBALS.Driver, where WdfGetDriver reads it. The handler now allocates a handle, keeps the driver for it, and writes it to both places. Sample: 0ebbe6089f00cd7dc8c065608a1cdd3bfe95989547d84e823eebcc20bc69a312
GetExitCodeThread wrote 0 for every handle, because no code recorded the exit code of a thread. The thread object now keeps its exit code. A run that returns to the exit address sets it from EAX/RAX. ExitThread, FreeLibraryAndExitThread and msvcrt _endthreadex set it from their argument, and msvcrt _endthread sets 0. A run on the thread clears it when the run starts. On Windows, GetExitCodeThread writes the return value of the thread function or the code given to ExitThread. For a thread that has not ended, it writes STILL_ACTIVE (259). For a handle that is not a thread, it returns FALSE with ERROR_INVALID_HANDLE. The emulator runs threads one after another, so only the current thread gives STILL_ACTIVE. A queued thread that has not run gives 0, as a wait on it completes. Otherwise a run that polls for STILL_ACTIVE would never end, because the queued thread cannot run during the poll. Sample: 024e5e537c02b2a5b9a41be151cc36807498f5e338b8c7ee592305af0398283e Sample: 0255ba864944be52d31a38fd7e604a82f044ebcd07ff18cee37778f6461069a8
The first CPU exception for a mode ran get_exception_state_offset from the interrupt hook, and the probe created a Unicorn instance while the engine ran with its timeout. Unicorn initializes its clock source again for each new instance and sets it to the wall clock for a short time. The timer thread of the running start can read the wall clock against its monotonic start time in that period, so it stopped the start as timed out at once. The run loop then ended the run with a timeout error at the entry of the exception handler, after less than a second. Run both probes when the engine is initialized, before any start.
williballenthin
force-pushed
the
pe-filter-fixes-3
branch
from
October 25, 2026 15:35
e169d55 to
a7a945b
Compare
williballenthin
commented
Oct 25, 2026
| self.enabled = False | ||
|
|
||
|
|
||
| class Watchdog: |
Collaborator
Author
There was a problem hiding this comment.
i don't really like this concept. we should address the underlying issue if possible
| self.stop() | ||
|
|
||
|
|
||
| class EventStop: |
Collaborator
Author
There was a problem hiding this comment.
yeah also seems too complicated
| def __init__(self): | ||
| self.name = "unicorn" | ||
| self.emu = None | ||
| self.mode = None |
Collaborator
Author
There was a problem hiding this comment.
we need type hints on all these things
| offset = get_exception_state_offset(self.mode) | ||
| if offset is None: | ||
| return | ||
| context = self.emu.context_save() # type: ignore[union-attr] |
Collaborator
Author
There was a problem hiding this comment.
avoid type ignore. assert
| self._resume_from_breakpoint = pc if at_breakpoint else None | ||
| # Only the stop the client resumes from skips its breakpoint. A | ||
| # later run that starts on a breakpoint hits it. | ||
| resuming, self._resume_pc = self._resume_pc == pc, None |
Collaborator
Author
There was a problem hiding this comment.
this region of code is too tricky
|
|
||
| # Each process has its own address space, so the image of a process | ||
| # that already ran gives way to a child image at the same base. | ||
| pe = _PeParser(data=child.pe_data, fast_load=True) |
Collaborator
Author
There was a problem hiding this comment.
i'm not sure about this. we're reusing the same address space as the parent, with just the parent unmapped?
crazy idea: could we relocate the child image and just keep everything mapped everywhere?
but doesn't this screw up the global objman state? i'm confused here
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This branch has the fixes from a third triage pass over VT feed samples. We studied 1004 PE samples (not .NET) one at a time. For each sample, we compared the speakeasy report with the code in IDA. When the report was wrong or the run stopped early, we found the cause, wrote a test that failed, and fixed it. Each commit has one fix and its tests, and a
Sample:line for each sample that showed the defect. We also ran samples and test programs under the GDB stub with gdb and lldb to find more stub defects.The full test suite passes (the two signature generator test files were not run).
Engine and memory: allocations fit in the user address space, do not use the first 64 KB and skip the return sentinel range. A handled fault maps only the page that faulted. Repeated contributory faults do not become a double fault. The x64 stack is aligned at the start of a run and has room for stack arguments. A run stops at an address that guest code cannot reach. Runs start with the x87 and SSE state of a new Windows thread. On x86, user mode uses the selectors of 32-bit Windows (CS 0x1B, DS and ES 0x23, FS 0x3B), and the float CRT functions take doubles on the stack and return them in ST(0). After a guest callback, the caller's nonvolatile registers are restored. An SEH handler that declines a fault gives one stop. An engine stop with no cause gives the error
engine_stopped.Timeouts: the run loop asks Unicorn whether the timeout was hit, and a watchdog stops a run when Unicorn loses a stop request. The exit event stops a native run within 0.1 s. The engine runs its CPU context probes when it starts. Before, a probe during a timed run created a Unicorn instance, and Unicorn's clock setup could then end the run as timed out.
Loaders: the TLS directory is read from the mapped image. A TLS callback array that cannot be read stops the walk. Exports stay when the export directory size goes past the image. A section whose raw data goes past the end of the file keeps the bytes that the file has. LoadString reads RT_STRING tables from the mapped image. LoadLibrary loads a DLL that the program wrote. GetProcAddress reads forwarders from the mapped export directory. GetModuleHandle accepts a full path. Version and resource language data are read from loaded modules.
GDB stub: the
CandSpackets work. Exec breakpoints hook only their own addresses, so they do not slow the run. Breakpoints that are set after code runs are hit, and breakpoints at the start of later runs are hit. Z0 and Z1 at one address are separate entries. The stub detectshlt. A step completes a store into its own code page. A watch stop gives an address in the watched range. The x87 control registers can be read and written.vKillreplies OK andkreplies X09. The stub detaches when the client disconnects, and a timeout is logged once. Negative register numbers get an error.Processes, threads and time: Sleep advances the tick count, the system time, QueryPerformanceCounter and rdtsc. QueryPerformanceCounter also moves 1 ms on each clock query. The msvcrt time functions use the emulated clock. TerminateProcess on the current process ends the run. GetCurrentThread returns the pseudo-handle. Semaphores are real objects, and events keep their signaled state. A wait on an unsignaled event with a finite timeout gives WAIT_TIMEOUT when no queued run is left. GetExitCodeThread gives the thread return value or the ExitThread code. The main EXE and new processes are in the Toolhelp lists, with every field filled. The emulated machine has two processors.
Files: relative paths resolve against the current directory. FILE_APPEND_DATA writes at the end. Deleted files stay deleted. Known directories have the directory attribute. CreateFile opens the NUL device. GetFileInformationByHandle and GetFinalPathNameByHandle are new. The msvcrt low-level file functions (
_open,_read,_write,_lseek,_fstatand others) work, and FILE objects have a file descriptor.Registry and services: a failed registry open writes NULL. A closed key handle is invalid. RegQueryValueEx rejects unknown keys. Sign-extended predefined keys work on x64. CreateService creates the service key. The default registry has the Run, RunOnce, Svchost, ProgramFilesDir and Windows NT\CurrentVersion values. A key path with a trailing backslash opens the key.
CRT:
_inittermcalls the initializers._seh_filter_execalls signal handlers.randuses the MSVC generator.systemreturns the exit status. There are new handlers for theswprintfand_sprintf families, the C locale ctype functions and tables,fmod,_CIfmod,ceil,freopen,strftime,wcsftime, the_mbsfunctions,_makepathand_splitpath,_endthreadex, the_itoa_sfamily,_strtoi64, the MSVCR80/90 internal allocators, the lconv initializers and__std_exception_copy. MSVCP140 has new iostream constructors,_Cnd_*,_Xtime_get_ticksand_Random_device. The C++ frame handlers return ExceptionContinueSearch.free,HeapFreeandLocalFreefree only heap blocks.COM: methods without a handler stop as
unsupported_apiwith the interface and method name. CoInitialize and OleInitialize count calls per thread. IMalloc, CoTaskMemAlloc and CoCreateGuid are new or changed, and CoTaskMemAlloc uses the emulated heap.Other APIs: new or fixed handlers in user32 (windows, dialogs, window properties, EnumWindows, CallWindowProc, list-view columns), gdi32, shell32, shlwapi, ntdll (ETW, RtlRandom, NtCreateThreadEx, ZwQueryInformationProcess), bcrypt (BCryptGenRandom and AES), crypt32 (CryptProtectData, CryptStringToBinary), WinHTTP and WinINet headers, winmm, msvfw32, rpcrt4, imagehlp, dxgi and d3d11, and WDF (WdfVersionBind, WdfDriverCreate). IsProcessorFeaturePresent does not report AVX, because the emulated CPU does not support it.
These behaviors change for existing users. x86 user mode selectors are 0x1B, 0x23 and 0x3B. GetProcessAffinityMask gives 0x3. QueryPerformanceCounter advances on each call, and
time()reads the emulated clock. A COM method without a handler stops the run. A second CoInitializeEx gives S_FALSE. With-k, a child image replaces finished images, and a child with no image is skipped. x86 thiscall exports get a placeholder buffer in ECX.These defects are known and not fixed here.
_CxxThrowExceptionreturns to its caller, so C++ catch blocks do not run. ANSI strings are decoded as UTF-8, so bytes above 0x7F can be lost. This needs a design decision.Missing APIs: OpenFile, Control_RunDLL, EnumServicesStatusEx, gdiplus and the MSVCP140 x86 wide basic_ostream constructor.
PostMessage does not queue messages, so BM_CLICK does not give WM_COMMAND. WM_TIMER ignores the interval. LoadIcon does not find module icons. WH_CBT hooks do not run.
Processes started with CreateProcess are not in Toolhelp snapshots. GetExitCodeProcess always gives 0. TerminateThread does not stop the thread. WaitForSingleObject on a thread that has not run gives WAIT_OBJECT_0.
The global atom table is a stub. Token objects are not in the object manager. FindFirstFile does not find created files and treats
[as a wildcard. VirtualFree does not unmap memory.All runs share one stack, and it is cleared for each run. The thread ID in the report is different from the CreateThread ID. The import descriptor walk does not stop at a zero Name. The signature fallback returns success for NULL handles and for Nt functions without a handler. On x86, the EXE entry has EBX 0.
GDB stub:
vContdoes not check thread IDs. fiseg, foseg and fooff read 0, because Unicorn does not expose them.🤖 Generated with Claude Code