Igor’s tip of the week #106: Outlined functions – Hex Rays

Written by Igor Skochinsky | Sep 15, 2022

The release notes for IDA 8.0 mention outlined functions. What are those and how to deal with them in IDA?

Function outlining is an optimization that saves code size by identifying recurring sequences of machine code and replacing each instance of the sequence with a call to a new function that contains the identified sequence of operations. It can be considered an extension of the shared function tail optimization by sharing not only tails but arbitrary common parts of functions.

Function outlining example

For example, here’s a function from iOS’s debugserver with some calls to outlined fragments:

The first fragment contains only two instructions besides the return instruction so it may not sound like we’re saving much, but by looking at the cross-references you’ll see that it is used in many places:

So the savings accumulated across the whole program can be quite substantial.

Handling outlined functions in decompiler

If we decompile the function, the calls to outlined fragments are shown as is, and the registers used or set by them show up as potentially undefined (orange color):

To tell the decompiler that the calls should be inlined into the function’s body, all the OUTLINED_FUNCTION_NN should be marked as outlined code. This can be done manually, via the Edit Function (Alt–P) dialog:

The added attribute is also displayed in the listing:

Once all outlined functions are marked up, the decompiler inlines them and there are no more possibly undefined variables:

Automating outlined function processing

If you have a big binary with hundreds or thousands of functions, it may become pretty tedious to mark up outlined functions manually. In such case, making a small script may speed things up. For example, if you have symbols and outlined functions have a known naming pattern, the following Python snippet should work:

import idautils
import ida_name
import ida_funcs
for f in idautils.Functions():
    nm = ida_name.get_name(f)
    if nm.startswith("_OUTLINED_FUNCTION") or nm.find(".cold.") != -1:
        print ("%08X: %s"% (f, nm))
        pfn = ida_funcs.get_func(f) 
        pfn.flags |= idaapi.FUNC_OUTLINE 
        ida_funcs.update_func(pfn)

It can be executed using File > Script command… (Shift+F2)