Igor’s Tip of the Week #179: Bitmask enums – Hex Rays

Written by Igor Skochinsky | Mar 26, 2024

We’ve covered simple enums previously, but there is a different kind of enum that you may sometimes encounter or need to create manually. They are used to represent various bits (or flags) which may be set in an integer value. For example, the file mode on Unix filesystems contains Access Permission bits (you can see them in the output of ls as string like -rwxr-xr-x), and each bit has a corresponding constant:

#define S_IRWXU 00700
#define S_IRUSR 00400
#define S_IWUSR 00200
#define S_IXUSR 00100

#define S_IRWXG 00070
#define S_IRGRP 00040
#define S_IWGRP 00020
#define S_IXGRP 00010

#define S_IRWXO 00007
#define S_IROTH 00004
#define S_IWOTH 00002
#define S_IXOTH 00001

Whenever you have a value which can be represented as a combination of bit values, you can use bitmask enums (used to be called bitfields before 8.4 but renamed to reduce confusion with bitfields in structures).

To create a bitmask enum, check “Bitmask” on the Enum tab in the “Add type” dialog:

The new enum gets the IDA-specific __bitmask attribute:

FFFFFFFF enum __bitmask __oct FILE_PERMS // 4 bytes
FFFFFFFF {
FFFFFFFF };

And when you add new members (N shortcut), IDA automatically offers the next free bit as the value:

You can also use the C syntax tab editor but you’ll have to ensure that there are no overlapping bits yourself.

To apply the enum to an integer value in disassembly or pseudocode, you may need to explicitly invoke Edit > Operand type > Enum member… action (or use M shortcut) as bitmask enums do not always appear in the context menu.

IDA will then display the value as a combination of enum’s members:

.data:00001000 dd S_IWGRP or S_IXGRP or S_IXUSR