Qt4 GUI Testing with Hooq :: Blog
Qt4 GUI Testing with Hooq
(posted at 2010-01-18 21:30:50 UTC)
One of the things I've had the pleasure of working with for Mendeley is Hooq; Hooq is a GPL/LGPL-licensed system for testing Qt4 GUI applications, on multiple platforms (Linux and Windows being where most of the focus has been at the moment). It's mostly GPL, but an LGPL library is loaded into applications being tested, at runtime.
With apologies in advance for the incredibly unfriendly UI at the moment (patches welcome :p), here's a lots-of-pictures demonstration on using it to test a trivial application.
After building the demo application and Hooq, and running hooq, you'll get a screen like the following:
For organisation, Hooq uses the concept of a 'test set' - this can be one per application, tests for different areas of the same application, or whatever you choose. First, you need to create one - select "New Test Set..." from the file menu, and you'll get another ugly dialog:
After filling it in and clicking OK, you'll be back to the main screen, except that 'Test set' will be filled in (screenshot); type in 'Foo' in the box at the bottom, then click 'Record' (screenshot). The demo application will then start - if you type in "Hello, World" you should see something like this:
Click the close button in the window decorations, and the Hooq window will now have a test entry, with run and edit buttons:
After clicking on the edit button highlighted above, you'll get a script editor (Hooq uses QtScript for test scripts):
If you then click in the margin on the line after "QTextEdit_0.sendText(string0)", there'll then be a breakpoint marker on that line:
If you then click the run button in the toolbar, the script will stop running at that line:
At this point, if you click 'pick property' in the toolbar (new icons welcome), you can then click on any widget in the application being tested, and get a list of its Q_PROPERTYs:
For this example, scroll down to "plainText", and click compare; this will insert a check into the JavaScript code in the editor:
Remove the breakpoint, and run it a few times; it should repeat your input and quit each time, without an error; however, if you change the "string0" variable at the top from "Hello, World" to something else, like "Hello, World!", then click run, the test will error, if you don't also change the compare statement:
This is more useful when the test hasn't changed, but when the application itself has (i.e. a regression). Hooq also has a "Run All" feature, to run all the tests in a set unattended, and show a summary at the end.
As some more detailed information:
Hooq loads a shared library into the application being tested (via gdb under Linux, and a very ugly hack under Windows) which recieves every QEvent in the application, and sends user input events, encoded in XML, to a hooq client (the GUI client is shown above, but there's also a CLI one that just records and plays the XML data). In the GUI client, the XML is converted into a tree of sorts, and then a few modifications are optionally made (selectable via the "Options" menu):
These options are:
- Ignore Mouse Movements
- Don't create JavaScript code for mouse events, except where they end up creating drag and drop events.
- Simplify Mouse Movements
- Only available if the above is off; instead of creating code for every single 1-pixel mouse movement made, create simplified code, like "move in a straight line from (x1, y1) to (x2, y2) over 300 milliseconds"
- Simplify Strings
- Repeated keyPress/keyRelease events are composed into "sendText" events, which take a string parameter. Like the two above options, this greatly decreases the length, and increases the readability of the generated code.
- Use Variables For Strings
- Any string used by sendText is stored in a JavaScript variable, instead of being hardcoded every time it is used; this makes it easier to modify the script in the future.
- Use Variables For Object References
- This doesn't change the meaning of the code at all, but generally makes the code shorter.
- Include Raw XML as Comment
- Mainly useful for debugging; includes the raw XML code received over the socket as a JavaScript comment
- Ignore Qt Internal Widgets
- Qt in some circumstances (such as drag and drop under X11) automatically creates magic widgets; this option makes Hooq ignore them.
Comments
Re: Qt4 GUI Testing with Hooq
Posted at 2010-01-18 22:02:12 UTC by "Fred"
One other thing; the code is based on qtscriptgenerator - so, for example, you can easily access QSqlDatabase from javascript, to check that actions in the UI have given effects on the data store.
Re: Qt4 GUI Testing with Hooq
Posted at 2010-01-18 22:49:44 UTC by "Peter"
Great and important project! Ui testing is definitely missing in KDE's development process. Do you know the QtUiTest from canceled QtExtended? http://qtopia.net/doc/qtextended4.4/qtuitestmodule.html
Re: Qt4 GUI Testing with Hooq
Posted at 2010-01-19 00:03:35 UTC by "ruurd"
I know there is a commercial tool called Squish to do automated Qt UI testing, but this suits me better. BTW, thanks Fred for helping me out getting this to run on OpenSUSE 11.2.
Re: Qt4 GUI Testing with Hooq
Posted at 2010-01-19 08:50:30 UTC by "Aurélien Gâteau"
Looks very promising! Going to try it.
Re: Qt4 GUI Testing with Hooq
Posted at 2010-01-19 19:38:07 UTC by "Mihai Adrian"
Need a little help....The button Record doesn't enable if I write a test name. The Script is loaded: Loaded qt.core QtScript plugin Loaded qt.gui QtScript plugin When I first started the application, I didn't have QtScript installed, but now I have and I still can add new test name(The button Record doesn't enable) Do you have any idee ? Thank you
Re: Qt4 GUI Testing with Hooq
Posted at 2010-01-19 19:41:20 UTC by "Fred"
Have you added a test set, with a valid executable path? Hooq will enable the button if a test set is selected, a test name is entered, and a valid application path is entered for the test set.
Re: Qt4 GUI Testing with Hooq
Posted at 2010-01-20 09:53:08 UTC by "Mihai Adrian"
Thank you, the command for running aplication was wrong, I fixed, and it works now.
Re: Qt4 GUI Testing with Hooq
Posted at 2010-01-20 15:37:49 UTC by "Mihai Adrian"
Do you have any ideea how to click on a QToolButton ? What method do I call ? Where do I found any documentation ? Thank you
Re: Qt4 GUI Testing with Hooq
Posted at 2010-01-21 19:49:14 UTC by "Mihai Adrian"
Now I succed. All you have to do is record you movements and then play. Very greate application. Congratulations All the better
Re: Qt4 GUI Testing with Hooq
Posted at 2010-01-31 13:39:56 UTC by "skuda"
Hello, i am trying to make work the demo application, i have followed the steps written in the article but when i press Record the demo application it is not launched, the test it is directly added to the list, in the console where i have launched hooq a line it is added with "". If i use the edit button of the recently added test i get this "requireHooqScriptVersion(2);", if i use the run button inside the editor i get this from the console "void Interpreter::run()" i am using in Linux and i have installed qtscriptgenerator, what can be the problem? Apart from that question do i have any way to use hooq with pyqt applications? Thanks for the project.
Re: Qt4 GUI Testing with Hooq
Posted at 2010-02-04 00:13:25 UTC by "Fred"
I have no idea about pyqt, I'll look into it; could you email me what gets shown after running hooq with --spam?
Re: Qt4 GUI Testing with Hooq
Posted at 2010-02-18 18:40:04 UTC by "Mihai Adrian"
When testing arora with hooq, if the events from the aplication are to fast, it doesn't record them all. For example, if you type something in adress bar very fast, only a part of characters are captured by hooq. Any ideea how to fix this ?
Re: Qt4 GUI Testing with Hooq
Posted at 2010-02-20 11:10:31 UTC by "Fred"
What platform are you using, and are you using any virtualisation? I'm aware of an issue like that in windows under VirtualBox. As a temporary solution (which may have side effects), you can comment out "QCoreApplication::postEvent(this, new CanaryEvent(&m_ignoreEvents))" in injectedHooq/Logger.cpp
Re: Qt4 GUI Testing with Hooq
Posted at 2010-02-23 14:17:44 UTC by "Mihai Adrian"
Thank you, that works
Re: Qt4 GUI Testing with Hooq
Posted at 2010-03-13 01:06:17 UTC by "rowinggolfer"
Hi Fred, wonderful application, I got it working great on ubuntu 9.10 with your demo app. With regards to pyQT applications, however, the applications do not start, I think your shared library never gets inserted. below is the output with -spam I would love to get hooq working with pyqt, but have a VERY limited understanding how the runtimes work. ========================================================= neil@slim-maroon:~$ hooq --spam Loaded qt.core QtScript plugin Loaded qt.gui QtScript plugin void Interpreter::run() void MainWindow::startApplication() "~"GNU gdb (GDB) 7.0-ubuntu\n" ~"Copyright (C) 2009 Free Software Foundation, Inc.\n" ~"License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>\n" " "~"This is free software: you are free to change and redistribute it.\n" ~"There is NO WARRANTY, to the extent permitted by law. Type \"show copying\"\n" ~"and \"show warranty\" for details.\n" ~"This GDB was configured as \"x86_64-linux-gnu\".\n" ~"For bug reporting instructions, please see:\n" ~"<http://www.gnu.org/software/gdb/bugs/>...\n" " "&"\"/usr/bin/openmolar\": not in executable format: File format not recognised\n" (gdb) " "&"set auto-solib-add off\n" " "^done (gdb) &"break _start\n" " "&"No symbol table is loaded. Use the \"file\" command.\n" " "~"Make breakpoint pending on future shared library load? (y or [n]) [answered N; input not from terminal]\n" ^done (gdb) " "&"set args \"\" \"\" \"\" \"\"\n" ^done (gdb) &"run\n" " "~"Starting program: \"\" \"\" \"\" \"\"\n" &"No executable file specified.\n" " "&"Use the \"file\" or \"exec-file\" command.\n" ^error,msg="No executable file specified.\nUse the \"file\" or \"exec-file\" command." (gdb) " "&"sharedlibrary libdl\n" " "~"No loaded shared libraries match the pattern `libdl'.\n" ^done (gdb) &"sharedlibrary libc\n" " "~"No loaded shared libraries match the pattern `libc'.\n" ^done (gdb) " "&"sharedlibrary libQtCore\n" ~"No loaded shared libraries match the pattern `libQtCore'.\n" ^done (gdb) " "&"break QCoreApplication::exec()\n" " "&"No symbol table is loaded. Use the \"file\" command.\n" " "~"Make breakpoint pending on future shared library load? (y or [n]) [answered N; input not from terminal]\n" ^done (gdb) &"continue\n" " "&"The program is not being run.\n" ^error,msg="The program is not being run." (gdb) " "&"call dlopen(\"/usr/bin/../lib/libinjectedHooq.so.1.0.0\", 2)\n" " "&"No symbol table is loaded. Use the \"file\" command.\n" " "^error,msg="No symbol table is loaded. Use the \"file\" command." (gdb) " "&"call __dlopen(\"/usr/bin/../lib/libinjectedHooq.so.1.0.0\", 2)\n" &"No symbol table is loaded. Use the \"file\" command.\n" " "^error,msg="No symbol table is loaded. Use the \"file\" command." (gdb) &"sharedlibrary injectedHooq\n" " "~"No loaded shared libraries match the pattern `injectedHooq'.\n" ^done (gdb) " "&"call startHooq()\n" &"No symbol table is loaded. Use the \"file\" command.\n" " "^error,msg="No symbol table is loaded. Use the \"file\" command." (gdb) &"continue\n" &"The program is not being run.\n" " "^error,msg="The program is not being run." (gdb) &"backtrace\n" &"No stack.\n" ^done (gdb) &"quit\n"
Re: Qt4 GUI Testing with Hooq
Posted at 2010-03-14 16:53:34 UTC by "rowinggolfer"
Fred, Not sure if it helps, but the python equivalent of your demo app is this... http://paste.debian.net/64234/
Testing KDE desktop applets
Posted at 2010-03-19 12:16:34 UTC by "comfylizard"
Hello, I would love to use Hooq to test my KDE desktop applet. I use the KDE plasmoidviewer to launch my applet for testing. Recording an action works fine, but replaying not so much, unfortunately. The applet is launched correctly but any mouse actions seem to have no effect. I tried clicking links and buttons and accessing context menu, all of which has no effect when replaying. If by chance you are a KDE user maybe you could try this? Basically you could try any KDE applet. For instance use "/usr/bin/plasmoidviewer" with "folderview" as parameter. Here is an example of such a recording, which clicks on a button in the widget: requireHooqScriptVersion(2); msleep(2199); var QWidget_3 = objectFromPath("FullView-0::QWidget-3"); QWidget_3.pressMouseButton({"button": Qt.LeftButton, "buttons": Qt.LeftButton, "modifiers": Qt.NoModifier, "x": 73, "y": 67}); msleep(137); QWidget_3.releaseMouseButton({"button": Qt.LeftButton, "buttons": Qt.NoButton, "modifiers": Qt.NoModifier, "x": 73, "y": 67}); msleep(863); var FullView_0 = objectFromPath("FullView-0"); FullView_0.close();
Hooq with a renamed library and a custom namespace
Posted at 2010-03-25 15:01:49 UTC by "fla_2"
Will Hooq work if Qt is built using these Qt configuration options which rename the Qt libraries and use a custom namespace? -qtnamespace AcmeQt -qtlibinfix Acme
Re: Qt4 GUI Testing with Hooq
Posted at 2010-04-15 05:59:39 UTC by "Arnstein"
Regarding PyQt4. I got this to work, but it required a modification to GdbInjector.cpp. You will need to add ' m_gdbStream << QString("call dlopen(\"/usr/lib/libQtCore.so.4\", %1)").arg(QString::number(RTLD_NOW)) << endl;' before the 'm_gdbStream << "sharedlibrary libQtCore" << endl;' line. The path to the libQtCore.so.4 library should be exactly the same as PyQt4/QtCore.so links to (use ldd to find out). Recompile and install hooq. In the GUI use /usr/bin/python as the application and the absolute path to your python script as an argument.
Re: Qt4 GUI Testing with Hooq
Posted at 2010-04-27 14:54:53 UTC by "Julien"
Hi, I have download the file hooq-hooq-master.tar.gz (http://gitorious.org/hooq/hooq/trees/master) but I don't see the file msi.nmake when I open the tar.gz. I can see the file on the same page where I have download the tar.gz. I must use this file? Thanks.
Re: Qt4 GUI Testing with Hooq
Posted at 2010-05-24 23:57:47 UTC by "Neeraj"
Hi There, I have been trying to build hooq on MacOS, however I haven't had much luck building it. Did anybody have any success building it on MacOS or Windows? If yes, could you please provide the steps that you went through to get hooq up and running. Much appreciate your help. Thanks Neeraj
Re: Qt4 GUI Testing with Hooq
Posted at 2010-05-30 09:04:11 UTC by "Peter"
For the Qt Windows SDK 2010.02.1 (Qt 4.6.2 based on MinGW) I've uploaded precompiled binaries of Hooq at <a href="http://sourceforge.net/projects/kst/files/Snapshots/"> download Hook-for-Qt-SDK-for Windows-2010.02.1-7z.exe.
Problems recording test with Hooq.
Posted at 2010-06-01 20:52:59 UTC by "Neeraj"
I am trying to record a new test with Hooq and once I record the test and look at the generated test script it just shows one line: requireHooqScriptVersion(2); I am not sure what does that mean. I was wondering if anybody else ran into the same issue and what do I need to do to fix it.
Re: Qt4 GUI Testing with Hooq
Posted at 2010-06-16 06:56:37 UTC by "Mike G"
I have the same problem as Neeraj. I used the binaries linked to by Peter. I believe the 'requireHooqScriptVersion(2);' is just a standard header statement to say what version of 'HooqScript' a particular test script uses. My guess is that if that is the only line, nothing is being recorded as one clicks on things. A sort of empty script.
Re: Qt4 GUI Testing with Hooq
Posted at 2010-07-13 21:03:47 UTC by "TT"
I am new to Hooq. In the MSI_README.txt, it says: - hooq-release from "nmake install" of hooq - hooq-debug from "nmake install" of hooq what does it mean?
Where's the script hooq stores? hooqcli can run the script which generated by hooqGUI?
Posted at 2010-08-31 15:52:13 UTC by "Tony"
When create a test suite and a test case, after finishing the recording, where does the hooq store the script and how can find the script it generated? hooqcli can run the script which generated by hooqGUI? Because I want to write a shell script to run the script which generated by hooqGUI, how can I do it?
New Comment
My Blog ▶ 2010 ▶ January ▶ 18 ▶ Qt4 GUI Testing with Hooq