Generally speaking most plugins for IDA can be written by using only the provided SDK. The API environment provided by IDA is vast and gives the plugin writer the capability to display graphical elements such as colored text views, graphs, forms and choosers.
However, there are cases when this is not enough. In idag the developer could use the Windows/.NET environment to go beyond the limits of the IDA SDK. While this is still possible in idaq, it is not advised, as it binds the code of the plugin to Windows and forces idaq to switch from alien widgets to system windows (more about that later).
Since accessing Qt from C++ requires setting up a development environment on every platform the developer wishes to deploy his plugin, one might take into consideration using PySide to access the Qt environment. The advantages of this approach are many. The first one is that the code once written will work on every platform without additional work. Moreover, there’s no need to recompile a plugin for every major Qt release deployed with idaq.
That being said, there might be cases where the developer/company needs or prefers to access the Qt framework directly from C++ and that is what is going to be covered in this article.
Setting up the Qt environment
It is necessary to build the Qt environment, because IDA is shipped with a custom version of Qt which wraps its classes inside the QT namespace (we’ll see later why that is so).
On Windows the same runtime has to be used. IDA 6.0, for instance, has been compiled with Visual C++ 2008. The express edition will do just fine.
Open the Visual C++ x86 prompt, go to the Qt source directory and type:
configure -debug-and-release -platform win32-msvc2008 -no-qt3support -qtnamespace QT
The same line on Linux will be:
./configure -debug-and-release -platform linux-g++ -no-qt3support -qtnamespace QT -prefix /home/daniel/Qt/4.6.3
And on OSX:
./configure -debug-and-release -cocoa -arch x86 -qtnamespace QT -prefix /Users/Shared/Qt/4.6.3
It should be noted that the Qt framework deployed with IDA 6.0 was linked against carbon (the configuration defaults to it). In the next release the framework will be linked against cocoa:
./configure -debug-and-release -cocoa -arch x86 -qtnamespace QT -prefix /Users/Shared/Qt/4.7.0
After the configuration process is over, the build process can begin. Type “nmake” on Windows and “make” on all other platforms. If the prefix parameter has been specified, the libraries have to be installed: type “sudo make -j1 install”.
Optional: if you want to use Qt Creator as IDE, download & install the stand-alone version and add the path of the freshly compiled Qt framework in the settings. Additionally, to set up the CDB debugger, the debugging tools for x86 are required. You can find them inside the WDK: Debuggers\setup_x86.exe. It is also necessary to specify the path for the symbols in the settings.
General guidelines
The QT namespace is necessary because of an unfortunate coincidence: the default prefix for many functions in the IDA SDK is “q”. Generally this is not a problem, since Qt is all about classes which begin with “Q”, there are some plain C lower-case functions in the Qt SDK which do, in fact, collide against some IDA SDK functions. The colliding functions are string functions such as: qstrlen, qstrdup, qstrcmp, etc.
Thus, in the beginning of the development, it was decided to wrap the Qt classes inside the QT namespace. Fortunately, the Qt environment has foreseen this case and allows itself to be wrapped with a simple configuration parameter.
In this scenario, the developer has to pay attention to mainly four things.
1) The QT namespace has to be specified in the qmake project file (or as build parameter). In a qmake project it is sufficient to add the line:
QT_NAMESPACE = QT
2) Forward declarations to Qt classes have to be wrapped by the namespace. Which means that:
namespace Ui {
class MyDialog;
}
class QTreeWidget;
class QTableWidget;
becomes:
QT_BEGIN_NAMESPACE
namespace Ui {
class MyDialog;
}
class QTreeWidget;
class QTableWidget;
QT_END_NAMESPACE
Pay attention that Qt Creator doesn’t add these delimiters when adding a QWidget/QDialog designer class to the project.
3) From inside a Qt class it is possible to call a colliding function this way:
len = ::qstrlen(str);
4) When calling open_tform the FORM_QWIDGET flag has to be specified. This flag has been introduced for backwards compatibility on Windows, in order to allow plugins to still obtain a native window handle when the flag is not specified. All new cross-platform plugins have to specify this flag.
What happens behind the scenes is the following.
The windows opened by the IDA process (more specifically the main window) should look like this:
There are no children, because by default Qt uses alien widgets. These widgets exist in the context of Qt, but are not real system windows.
However, when a plugin doesn’t specify FORM_QWIDGET, the winId() method is called on the created widget. This method returns a valid system handle and forces all alien widgets to become native widgets.
The Qt project sample
Although the SDK shipped with IDA 6.0 already provided a sample of how to access Qt from C++, it is minimalistic. That’s why a new sample plugin called “qproject” has been added. This plugin features a fully commented qmake project (which can also be used to obtain compiler specific project files if the developer wishes). It’s based on the classic Qt SDK example “elastic nodes”.
To use this project for your own plugins, just replace the TARGET name inside “qproject.pro”, remove all source/header files apart of “qproject.cpp”, edit the configuration settings contained in “qproject.cpp” like for all other IDA plugins and finally replace this line with the class name of your own widget:
GraphWidget *userWidget = new GraphWidget();
Build with Qt Creator or type:
qmake theprojectname.pro
nmake/make
OSX adjustments
While on Linux it is sufficient to specify the rpath to load the correct dependencies, there’s additional work to do on OSX.
To list the dependencies for your plugin, type:
otool -L pluginname.pmc
The output will be something like:
/Users/Shared/Qt/4.6.3/lib/QtGui.framework/Versions/4/QtGui (compatibility version 4.6.0, current version 4.6.3)
/Users/Shared/Qt/4.6.3/lib/QtCore.framework/Versions/4/QtCore (compatibility version 4.6.0, current version 4.6.3)
/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 103.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.1)
It is necessary to change the path of the dependencies through the install_name_tool utility. For instance:
install_name_tool -change /Users/Shared/Qt/4.6.3/lib/QtGui.framework/Versions/4/QtGui @executable_path/../Frameworks/QtGui.framework/Versions/4/QtGui pluginname.pmc
The executable_path is a variable specifying the initial search path.
You can invoke the install_name_tool utility as a post link action:
mac:QMAKE_POST_LINK += install_name_tool -change /idapathsample/libida$${SUFF64}.dylib @executable_path/libida$${SUFF64}.dylib $(TARGET)
IDA SDK controls
While almost the entire IDA SDK is agnostic in regards to Qt, there are a few exceptions:
1) the CLI (command line interpreter)
2) the custom viewer
These controls offer now a Qt awareness option: the CLIF_QT_AWARE flag for the CLI and the set_custom_viewer_qt_aware function for the custom viewer.
When these controls are set as Qt aware, they will return the actual Qt key and modifier codes to their key press notification callbacks. Vice versa, when they’re not Qt aware, the Win32/VCL key and shift codes will be passed, even on non-Win32 platforms. Check out “kernwin.hpp” for the Win32-compatible defined key codes (they all start with “IK_” instead of “VK_”). This basic emulation allows non-Qt graphical plugins to be recompiled for other platforms by just changing the key code prefix.
Conclusions
This should be all. In case I’ve forgotten something, let me know and it will be added.
