Previously, we’ve run into a function which produces a cryptic error if you try to decompile it:
In such situations, you need to go back to disassembly to see what could be wrong. More specifically, check the stack frame layout by double-clicking a stack variable or pressing Ctrl–K.
On the first glance it looks normal:
However, if you compare with another function which decompiles fine, you may notice some notable differences:
This frame has two members which are mentioned in the top comment:
Two special fields " r" and " s" represent return address and saved registers.
They’re absent in the “bad” function, so the whole layout is probably wrong and the function can’t be decompiled reliably. On closer inspection, we can discover that the structure sigset_t (type of the variable oset in sub_8058FF0) is 0x80 bytes, so applying it to the frame overwrote the special members. You can also see that the variable crossed over from the local variable area (negative offsets) to the argument area (positive offsets), which normally should not happen.
Although you can try to fix the frame layout by rearranging or editing the local variables, this won’t bring back the special variables, so usually the best solution is to recreate the function (and thus its stack frame). This can be done by undefining (U) the first instruction, then creating the function (P). A quicker and less destructive way is to delete just the function (Ctrl–P, Del), then recreate it (P). Normally this should recreate the default frame then add local variables and stack arguments based on the instructions accessing the stack:
And now the function decompiles fine:
Some code is wrong because the function prototype still uses wrongly detected sigset_t argument. This is easy to fix – just delete the prototype (Y, Del) to let the decompiler guess the arguments:
See also:
Igor’s Tip of the Week #148: Fixing “call analysis failed”
Igor’s tip of the week #65: stack frame view
Decompiler Manual: Failures and troubleshooting