Igor’s tip of the week #27: Fixing the stack pointer – Hex Rays

As explained in Simplex method in IDA Pro, having correct stack change information is essential for correct analysis. This is especially important for good and correct decompilation. While IDA tries its best to give good and correct results (and we’ve made even more improvements since 2006), sometimes it can still fail (often due to wrong or conflicting information). In this post we’ll show you how to detect and fix problems such as:

“sp-analysis failed”

“positive sp value has been detected”

Both examples are from the 32-bit build of notepad.exe from Windows 10 (version 10.0.17763.475) with PDB symbols from Microsoft’s public symbol server applied.

Note: in many cases the decompiler will try to recover and still produce reasonable decompilation but if you need to be 100% sure of the result it may be best to fix them.

Detecting the source of the problem

The first steps to resolve them are usually:

1. Switch to the disassembly view (if you were in the decompiler);
2. Enable “Stack pointer” under “Disassembly, Disassembly line parts” in Options > General…;
   
3. Look for unusual or unexpected changes in the SP value (actually it’s the SP delta value) now added before each instruction.

To detect “unusual changes” we first need to know what is “usual”. Here are some examples:

• push instructions should increase the SP delta by the number of pushed bytes (e.g. push eax by 4 and push rbp by 8)
• conversely, pop instructions decrease it by the same amount
• call instructions usually either decrease SP to account for the pushed arguments (__stdcall or __thiscall functions on x86), or leave it unchanged to be decreased later by a separate instruction
• the values on both  ends of a jump (conditional or unconditional) should be the same
• the value at the function entry and return instructions should be 0
• between prolog and epilog the SP delta should remain the same with the exception of small areas around calls where it can increase by pushing arguments but then should return back to “neutral” before the end of the basic block.

In the first example, we can see that loc_406F9D has the SP delta of 00C and the first jump to it is also 00C, however the second one is 008. So the problem is likely in that second block. Here it is separately:

00C mov     ecx, offset dword_41D180
00C call    _TraceLoggingRegister@4 ; TraceLoggingRegister(x)
008 push    offset _TraceLogger__GetInstance____2____dynamic_atexit_destructor_for__s_instance__ ; void (__cdecl *)()
00C call    _atexit
00C pop     ecx
008 push    ebx
00C call    __Init_thread_footer
00C pop     ecx
008 jmp     short loc_406F9D

We can see that 00C changes to 008 after the call to _TraceLoggingRegister@4. On the first glance it makes sense because the @4 suffix denotes __stdcall function with 4 bytes of arguments (which means it removes 4 bytes from the stack). However, if you actually go inside and analyze it, you’ll see that it does not use stack arguments but the register ecx. Probably the file has been compiled with Link-time Code Generation which converted __stdcall to __fastcall to speed up the code.

In the second case the disassembly looks like following:

Here, the problem is immediately obvious: the delta becomes negative after the call. It seems IDA decided that the function is subtracting 0x14 bytes from the stack while there are only three pushes (3*4 = 12 or 0xC). You can also go inside StringCopyWorkerW and observe that it ends with retn 0Ch – a certain indicator that this is the correct number.

Fixing wrong stack deltas

How to actually fix the wrong delta depends on the specific situation but generally there are two approaches:

1. Fix just the place(s) where things go wrong. For this, press Alt–K (Edit > Functions > Change stack pointer…) and enter the correct amount of the SP change. In the first example it should be 0 (since the function is not using any stack arguments) and in the second 12 or 0xc. Often this is the only option for indirect calls.
2. If the same function called from multiple places causes stack unbalance issues, edit the function’s properties (Alt–P or Edit > Functions > Edit function… ) and change the “Purged bytes” value.
   

This simple example shows that even having debug symbols does not guarantee 100% correct results and why giving override options to the user is important.