Development/Tutorials/Unittests: Difference between revisions

From KDE TechBase
Line 361: Line 361:
********* Finished testing of testDate *********
********* Finished testing of testDate *********
</pre>
</pre>
==Tutorial 2: Testing Graphical User Interfaces==
In the previous two tutorials, we've tested a date management class. This is an pretty typical use of unit testing. However Qt and KDE applications will make use graphical classes that take user input (typically from a keyboard and mouse). QtTestLib offers support for testing these classes, which we'll see in this tutorial.
Again, we'll use an existing class as our test environment, and again it will be date related - the standard Qt {{qt|QDateEdit}} class. For those not familiar with this class, it is a simple date entry widget (although with some powerful back end capabilities). A picture of the widget is shown below.
[[Image:Qdateedit_dlg.png|thumb|'''Figure 1. QDateEdit widget screenshot''']]
The way QtTestLib provides GUI testing is by injecting {{qt|QInputEvent}} events. To the application, these input events appear the same as normal key press/release and mouse clicks/drags. However the mouse and keyboard are unaffected, so that you can continue to use the machine normally while tests are being run.
An example of how you can use the GUI functionality of QtTestLib is shown below.
'''Example 13. QDateEdit test code'''
<pre>
#include <QtTest>
#include <QtCore>
#include <QtGui>
Q_DECLARE_METATYPE(QDate)
class testDateEdit: public QObject
{
    Q_OBJECT
private slots:
    void testChanges();
    void testValidator_data();
    void testValidator();
};
void testDateEdit::testChanges()
{
    // 11 March 1967
    QDate date( 1967, 3, 11 );
    QDateEdit dateEdit( date );
    // up-arrow should increase day by one
    QTest::keyClick( &dateEdit, Qt::Key_Up );
    QCOMPARE( dateEdit.date(), date.addDays(1) );
    // we click twice on the "reduce" arrow at the bottom RH corner
    // first we need the widget size to know where to click
    QSize editWidgetSize = dateEdit.size();
    QPoint clickPoint(editWidgetSize.rwidth()-2, editWidgetSize.rheight()-2);
    // issue two clicks
    QTest::mouseClick( &dateEdit, Qt::LeftButton, 0, clickPoint);
    QTest::mouseClick( &dateEdit, Qt::LeftButton, 0, clickPoint);
    // and we should have decreased day by two (one less than original)
    QCOMPARE( dateEdit.date(), date.addDays(-1) );
    QTest::keyClicks( &dateEdit, "25122005" );
    QCOMPARE( dateEdit.date(), QDate( 2005, 12, 25 ) );
    QTest::keyClick( &dateEdit, Qt::Key_Tab, Qt::ShiftModifier );
    QTest::keyClicks( &dateEdit, "08" );
    QCOMPARE( dateEdit.date(), QDate( 2005, 8, 25 ) );
}
void testDateEdit::testValidator_data()
{
    qRegisterMetaType<QDate>("QDate");
    QTest::addColumn<QDate>( "initialDate" );
    QTest::addColumn<QString>( "keyclicks" );
    QTest::addColumn<QDate>( "finalDate" );
    QTest::newRow( "1968/4/12" ) << QDate( 1967, 3, 11 )
                                << QString( "12041968" )
                                << QDate( 1968, 4, 12 );
    QTest::newRow( "1967/3/14" ) << QDate( 1967, 3, 11 )
                                << QString( "140abcdef[" )
                                << QDate( 1967, 3, 14 );
    // more rows can go in here
}
void testDateEdit::testValidator()
{
    QFETCH( QDate, initialDate );
    QFETCH( QString, keyclicks );
    QFETCH( QDate, finalDate );
    QDateEdit dateEdit( initialDate );
    // this next line is just to start editing
    QTest::keyClick( &dateEdit, Qt::Key_Enter );
    QTest::keyClicks( &dateEdit, keyclicks );
    QCOMPARE( dateEdit.date(), finalDate );
}
QTEST_MAIN(testDateEdit)
#include "tutorial3.moc"
</pre>
Much of this code is common with previous examples, so I'll focus on the new elements and the more important changes as we work through the code line-by-line.
Lines 1 to 3 import the various Qt declarations, as before.
Line 4 is a macro that is required for the data-driven part of this test, which I'll come to soon.
Lines 5 to 12 declare the test class - while the names have changed, it is pretty similar to the previous example. Note the testValidator and testValidator_data functions - we will be using data driven testing again in this example.
Our first real test starts in line 13. Line 16 creates a QDate, and line 17 uses that date as the initial value for a QDateEdit widget.
Lines 19 and 20 show how we can test what happens when we press the up-arrow key. The QTest::keyClick function takes a pointer to a widget, and a symbolic key name (a char or a Qt::Key). At line 20, we check that the effect of that event was to increment the date by a day. The QTest:keyClick function also takes an optional keyboard modifier (such as Qt::ShiftModifier for the shift key) and an optional delay value (in milliseconds). As an alternative to using QTest::keyClick, you can use QTest::keyPress and QTest::keyRelease to construct more complex keyboard sequences.
Lines 23 to 29 show a similar test to the previous one, but in this case we are simulating a mouse click. We need to click in the lower right hand part of the widget (to hit the decrement arrow - see Figure 1), and that requires knowing how large the widget is. So lines 23 and 24 calculate the correct point based off the size of the widget. Line 26 (and the identical line 27) simulates clicking with the left-hand mouse button at the calculated point. The arguments to Qt::mouseClick are:
*a pointer to the widget that the click event should be sent to.
the mouse button that is being clicked.
*an optional keyboard modifier (such a Qt::ShiftModifier), or 0 for no modifiers.
*an optional click point - this defaults to the middle of the widget if not specified.
*an optional mouse delay.
In addition to QTest::mouseClick, there is also QTest::mousePress, QTest::mouseRelease, QTest::mouseDClick (providing double-click) and QTest::mouseMove. The first three are used in the same way as QTest::mouseClick. The last takes a point to move the mouse to. You can use these functions in combination to simulate dragging with the mouse.
Lines 30 and 31 show another approach to keyboard entry, using the QTest::keyClicks. Where QTest::keyClick sends a single key press, QTest::keyClicks takes a QString (or something equivalent, in line 30 a character array) that represents a sequence of key clicks to send. The other arguments are the same.
Lines 32 to 34 show how you may need to use a combination of functions. After we've entered a new date in line 30, the cursor is at the end of the widget. At line 32, we use a Shift-Tab combination to move the cursor back to the month value. Then at line 33 we enter a new month value. Of course we could have used individual calls to QTest::keyClick, however that wouldn't have been as clear, and would also have required more code.
== Data-driven GUI testing ==
Lines 50 to 60 show a data-driven test - in this case we are checking that the validator on QDateEdit is performing as expected. This is a case where data-driven testing can really help to ensure that things are working the way they should.
At lines 52 to 54, we fetch in an initial value, a series of key-clicks, and an expected result. These are the columns that are set up in lines 39 to 41. However note that we are now pulling in a QDate, where in previous examples we used three integers and then build the QDate from those. However QDate isn't a registered type for {{qt|QMetaType}, and so we need to register it before we can use it in our data-driven testing. This is done using the Q_DECLARE_METATYPE macro in line 4 and the qRegisterMetaType function in line 38.
Lines 42 to 47 add in a couple of sample rows. Lines 42 to 44 represent a case where the input is valid, and lines 45 to 47 are a case where the input is only partly valid (the day part). A real test will obviously contain far more combinations than this.
Those test rows are actually tested in lines 55 to 59. We construct the QDateEdit widget in line 55, using the initial value. We then send an Enter key click in line 57, which is required to get the widget into edit mode. At line 58 we simulate the data entry, and at line 59 we check whether the results are what was expected.
Lines 61 and 62 are the same as we've seen in previous examples.
=== Re-using test elements ===
If you are re-using a set of events a number of times, then it may be an advantage to build a list of events, and then just replay them. This can improve maintainability and clarity of a set of tests, especially for mouse movements.
The key class for building a list of test events is imaginatively known as QTestEventList. It is a QList of QTestEvents. The normal approach is to create the list, and then use various member functions to add key and mouse events. The normal functions that you'll need are addKeyClick and addMouseClick, which are very similar to the QTest::keyClick and QTest::mouseClick functions we used earlier in this tutorial. For finer grained operations, you can also use addKeyPress, addKeyRelease, addKeyEvent, addMousePress, addMouseRelease, addMouseDClick and addMouseMove to build up more complex event lists. You can also use addDelay to add a specified delay between events. When the list has been built up, you just call simulate on each widget.
You can see how this works in the example below, which is the QDateEdit example (from above) converted to use QTestEventList.
'''Example 14. QDateEdit test code, using QTestEventList'''
<pre>
     
#include <QtTest>
#include <QtCore>
#include <QtGui>
Q_DECLARE_METATYPE(QDate)
class testDateEdit: public QObject
{
    Q_OBJECT
private slots:
    void testChanges();
    void testValidator_data();
    void testValidator();
};
void testDateEdit::testChanges()
{
    // 11 March 1967
    QDate date( 1967, 3, 11 );
    QDateEdit dateEdit( date );
    // up-arrow should increase day by one
    QTest::keyClick( &dateEdit, Qt::Key_Up );
    QCOMPARE( dateEdit.date(), date.addDays(1) );
    // we click twice on the "reduce" arrow at the bottom RH corner
    // first we need the widget size to know where to click
    QSize editWidgetSize = dateEdit.size();
    QPoint clickPoint(editWidgetSize.rwidth()-2, editWidgetSize.rheight()-2);
    // build a list that contains two clicks
    QTestEventList list1;
    list1.addMouseClick( Qt::LeftButton, 0, clickPoint);
    list1.addMouseClick( Qt::LeftButton, 0, clickPoint);
    // call that list on the widget
    list1.simulate( &dateEdit );
    // and we should have decreased day by two (one less than original)
    QCOMPARE( dateEdit.date(), date.addDays(-1) );
    QTest::keyClicks( &dateEdit, "25122005" );
    QCOMPARE( dateEdit.date(), QDate( 2005, 12, 25 ) );
    QTestEventList list2;
    list2.addKeyClick( Qt::Key_Tab, Qt::ShiftModifier );
    list2.addKeyClicks( "08" );
    list2.simulate( &dateEdit );
    QCOMPARE( dateEdit.date(), QDate( 2005, 8, 25 ) );
}
void testDateEdit::testValidator_data()
{
    qRegisterMetaType<QDate>("QDate");
    QTest::addColumn<QDate>( "initialDate" );
    QTest::addColumn<QTestEventList>( "events" );
    QTest::addColumn<QDate>( "finalDate" );
    QTestEventList eventsList1;
    // this next line is just to start editing
    eventsList1.addKeyClick( Qt::Key_Enter );
    eventsList1.addKeyClicks( "12041968" );
    QTest::newRow( "1968/4/12" ) << QDate( 1967, 3, 11 )
                                << eventsList1
                                << QDate( 1968, 4, 12 );
    QTestEventList eventsList2;
    eventsList2.addKeyClick( Qt::Key_Enter );
    eventsList2.addKeyClicks( "140abcdef[" );
    QTest::newRow( "1967/3/14" ) << QDate( 1967, 3, 11 )
                                << eventsList2
                                << QDate( 1967, 3, 14 );
    // more rows can go in here
}
void testDateEdit::testValidator()
{
    QFETCH( QDate, initialDate );
    QFETCH( QTestEventList, events );
    QFETCH( QDate, finalDate );
    QDateEdit dateEdit( initialDate );
    events.simulate( &dateEdit);
    QCOMPARE( dateEdit.date(), finalDate );
}
QTEST_MAIN(testDateEdit)
#include "tutorial3a.moc"
</pre>
This example is pretty much the same as the previous version, up to line 25. In line 26, we create a QTestEventList. We add events to the list in lines 27 and 28 - note that we don't specify the widget we are calling them on at this stage. In line 30, we simulate each event on the widget. If we had multiple widgets, we could call simulate using the same set of events.
Lines 31 to 34 are as per the previous example.
We create another list in lines 35 to 37, although this time we are using addKeyClick and addKeyClicks instead of adding mouse events. Note that an event list can contain combinations of mouse and keyboard events - it just didn't make sense in this test to have such a combination. We use the second list at line 38, and check the results in line 39.
You can also build lists of events in data driven testing as well, as shown in lines 41 to 70. The key difference is that instead of fetching a QString in each row, we are fetching a QTestEventList. This requires that we add a column of QTestEventList, rather than QString (see line 45). At lines 47 to 50, we create a list of events. At line 52 we add those events to the applicable row. We create a second list at lines 54 to 56, and add that second list to the applicable row in line 58.
We fetch the events in line 65, and use them in line 68. If we had multiple widgets, then we could use the same event list several times.


==Continued==
==Continued==

Revision as of 21:47, 13 December 2006

Author: Brad Hards, Sigma Bravo Pty Limited

Abstract

This article provides guidance on writing unittests for Qt4 and KDE4, based on the QtTestLib framework provided with Qt4.1. It provides an introduction to the ideas behind unit testing, tutorial material on the QtTestLib framework, and suggestions for getting the most value for your effort. This document is matched to Qt 4.1 and KDE 4.

About Unit Testing

A unit test is a test that checks the functionality, behaviour and correctness of a single software component. In Qt4 code (including KDE4 code) unit tests are almost always used to test a single C++ class (although testing a macro or C function is also possible).

Unit tests are a key part of Test Driven Development, however they are useful for all software development processes. It is not essential that all of the code is covered by unit tests (although that is obviously very desirable!). Even a single test is a useful step to improving code quality.

Even if they don't call them "unit tests", most programmers have written some "throwaway" code that they use to check an implementation. If that code was cleaned up a little, and built into the development system, then it could be used over and over to check that the implementation is still OK. To make that work a little easier, we can use test frameworks.

Note that it is sometimes tempting to treat the unit test as a pure verification tool. While it is true that unit tests do help to ensure correct functionality and behaviour, they also assist with other aspects of code quality. Writing a unit test requires a slightly different approach to coding up a class, and thinking about what inputs need to be tested can help to identify logic flaws in the code (even before the tests get run). In addition, the need to make the code testable is a very useful driver to ensure that classes do not suffer from close coupling.

Anyway, enough of the conceptual stuff - lets talk about a specific tool that can reduce some of the effort and let us get on with the job.

About QtTestLib

QtTestlib is a lightweight testing library developed by Trolltech and released under the GPL (a commercial version is also available, for those who need alternative licensing). It is written in C++, and is cross-platform. It is provided as part of the tools included in Qt 4.1, and earlier versions were provided separately. Note that earlier versions had a different API, and the examples will not work with that earlier version.

In addition to normal unit testing capabilities, QtTestLib also offers basic GUI testing, based on sending QEvents. This allows you to test GUI widgets, but is not generally suitable for testing full applications.

Each testcase is a standalone test application. Unlike CppUnit or JUnit, there is no Runner type class. Instead, each testcase is an executable which is simply run.

Tutorial 1: A simple test of a date class

In this tutorial, we will build a simple test for a class that represents a date, using QtTestLib as the test framework. To avoid too much detail on how the date class works, we'll just use the QDate. class that comes with Qt. In a normal unittest, you would more likely be testing code that you've written yourself.

The code below is the entire testcase.


Example 1. QDate test code

#include <QtTest>
#include <QtCore>

class testDate: public QObject
{
    Q_OBJECT
private slots:
    void testValidity();
    void testMonth();
};

void testDate::testValidity()
{
    // 11 March 1967
    QDate date( 1967, 3, 11 );
    QVERIFY( date.isValid() );
}

void testDate::testMonth()
{
    // 11 March 1967
    QDate date;
    date.setYMD( 1967, 3, 11 );
    QCOMPARE( date.month(), 3 );
    QCOMPARE( QDate::longMonthName(date.month()),
              QString("March") );
}


QTEST_MAIN(testDate)
#include "tutorial1.moc"

Stepping through the code, the first line imports the header files for the QtTest namespace. The second line imports the headers for the QtCore namespace (not strictly necessary, since QtTest also imports it, but it is robust and safe). Lines 3 to 9 give us the test class, testData. Note that testDate inherits from QObject and has the Q_OBJECT macro - QtTestLib requires specific Qt functionality that is present in QObject.

Lines 10 to 15 provide our first test, which checks that a date is valid. 3 the use of the QVERIFY macro, which checks that the condition is true. So if date.isValid() returns true, then the test will pass, otherwise the test will fail. QVERIFY is similar to ASSERT in other test suites.

Similarly, lines 16 to 24 provide another test, which checks a setter, and a couple of accessor routines. In this case, we are using QCOMPARE, which checks that the conditions are equal. So if date.month() returns 3, then that part of that test will pass, otherwise the test will fail.

Warning
Warning: As soon as a QVERIFY evaluates to false or a QCOMPARE does not have two equal values, the rest of that test will be skipped over. So in the example above, if the check at line 21 fails, then the check at lines 22 and 23 will not be run.


In a later tutorial we will see how to work around problems that this behaviour can cause.

Line 25 uses the QTEST_MAIN which creates an entry point routine for us, with appropriate calls to invoke the testDate unit test class.

Line 26 includes the Meta-Object compiler output, so we can make use of our QObject functionality.

The qmake project file that corresponds to that code is shown below. You would then use qmake to turn this into a Makefile and then compile it with make.

Example 2. QDate unit test project

CONFIG += qtestlib
TEMPLATE = app
TARGET += 
DEPENDPATH += .
INCLUDEPATH += .

# Input
SOURCES += tutorial1.cpp

This is a fairly normal project file, except for the addition of the CONFIG += qtestlib. This adds the right header and library setup to the Makefile.

This will produce an application that can then be run on the command line. The output looks like the following:

Example 3. QDate unit test output

$ ./tutorial1
********* Start testing of testDate *********
Config: Using QTest library 4.1.0, Qt 4.1.0-snapshot-20051003
PASS   : testDate::initTestCase()
PASS   : testDate::testValidity()
PASS   : testDate::testMonth()
PASS   : testDate::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped
********* Finished testing of testDate *********

Looking at the output above, you can see that the output includes the version of the test library and Qt itself, and then the status of each test that is run. In addition to the testValidity and testMonth tests that we defined, there is also a setup routine (initTestCase) and a teardown routine (cleanupTestCase) that can be used to do additional configuration if required.

Failing tests

If we had made an error in either the production code or the unit test code, then the results would show an error. An example is shown below:

Example 4. QDate unit test output showing failure

$ ./tutorial1
********* Start testing of testDate *********
Config: Using QTest library 4.1.0, Qt 4.1.0-snapshot-20051003
PASS   : testDate::initTestCase()
PASS   : testDate::testValidity()
FAIL!  : testDate::testMonth() Compared values are not the same
   Actual (date.month()): 4
   Expected (3): 3
    Loc: [tutorial1.cpp(25)]
PASS   : testDate::cleanupTestCase()
Totals: 3 passed, 1 failed, 0 skipped
********* Finished testing of testDate *********

Running selected tests

When the number of test functions increases, and some of the functions take a long time to run, it can be useful to only run a selected function. For example, if you only want to run the testMonth function, then you just specify that on the command line, as shown below:

Example 5. QDate unit test output - selected function

$ ./tutorial1 testValidity
********* Start testing of testDate *********
Config: Using QTest library 4.1.0, Qt 4.1.0-snapshot-20051003
PASS   : testDate::initTestCase()
PASS   : testDate::testValidity()
PASS   : testDate::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped
********* Finished testing of testDate *********        

Note that the initTestCase and cleanupTestCase routines are always run, so that any necessary setup and cleanup will still be done.

You can get a list of the available functions by passing the -functions option, as shown below:

Example 6. QDate unit test output - listing functions

$ ./tutorial1 -functions
testValidity()
testMonth()

Verbose output options

You can get more verbose output by using the -v1, -v2 and -vs options. -v1 produces a message on entering each test function. I found this is useful when it looks like a test is hanging. This is shown below:

Example 7. QDate unit test output - verbose output

$ ./tutorial1 -v1
********* Start testing of testDate *********
Config: Using QTest library 4.1.0, Qt 4.1.0-snapshot-20051003
INFO   : testDate::initTestCase() entering
PASS   : testDate::initTestCase()
INFO   : testDate::testValidity() entering
PASS   : testDate::testValidity()
INFO   : testDate::testMonth() entering
PASS   : testDate::testMonth()
INFO   : testDate::cleanupTestCase() entering
PASS   : testDate::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped
********* Finished testing of testDate *********

The -v2 option shows each QVERIFY, QCOMPARE and QTEST, as well as the message on entering each test function. I found this useful for verifying that a particular step is being run. This is shown below:

'Example 8. QDate unit test output - more verbose output

$ ./tutorial1 -v2
********* Start testing of testDate *********
Config: Using QTest library 4.1.0, Qt 4.1.0-snapshot-20051003
INFO   : testDate::initTestCase() entering
PASS   : testDate::initTestCase()
INFO   : testDate::testValidity() entering
INFO   : testDate::testValidity() QVERIFY(date.isValid())
    Loc: [tutorial1.cpp(17)]
PASS   : testDate::testValidity()
INFO   : testDate::testMonth() entering
INFO   : testDate::testMonth() COMPARE()
    Loc: [tutorial1.cpp(25)]
INFO   : testDate::testMonth() COMPARE()
    Loc: [tutorial1.cpp(27)]
PASS   : testDate::testMonth()
INFO   : testDate::cleanupTestCase() entering
PASS   : testDate::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped
********* Finished testing of testDate *********        

The -vs option shows each signal that is emitted. In our example, there are no signals, so -vs has no effect. Getting a list of signals is useful for debugging failing tests, especially GUI tests which we will see in the third tutorial.

Output to a file

If you want to output the results of your testing to a file, you can use the -o filename, where you replace filename with the name of the file you want to save output to.

Tutorial 2: Data driven testing of a date class

In the previous example, we looked at how we can test a date class. If we decided that we really needed to test a lot more dates, then we'd be cutting and pasting a lot of code. If we subsequently changed the name of a function, then it has to be changed in a lot of places. As an alternative to introducing these types of maintenance problems into our tests, QtTestLib offers support for data driven testing.

The easiest way to understand data driven testing is by an example, as shown below:

Example 9. QDate test code, data driven version

#include <QtTest>
#include <QtCore>


class testDate: public QObject
{
    Q_OBJECT
private slots:
    void testValidity();
    void testMonth_data();
    void testMonth();
};

void testDate::testValidity()
{
    // 12 March 1967
    QDate date( 1967, 3, 12 );
    QVERIFY( date.isValid() );
}

void testDate::testMonth_data()
{
    QTest::addColumn<int>("year");  // the year we are testing
    QTest::addColumn<int>("month"); // the month we are testing
    QTest::addColumn<int>("day");   // the day we are testing
    QTest::addColumn<QString>("monthName");   // the name of the month

    QTest::newRow("1967/3/11") << 1967 << 3 << 11 << QString("March");
    QTest::newRow("1966/1/10") << 1966 << 1 << 10 << QString("January");
    QTest::newRow("1999/9/19") << 1999 << 9 << 19 << QString("September");
    // more rows of dates can go in here...
}

void testDate::testMonth()
{
    QFETCH(int, year);
    QFETCH(int, month);
    QFETCH(int, day);
    QFETCH(QString, monthName);

    QDate date;
    date.setYMD( year, month, day);
    QCOMPARE( date.month(), month );
    QCOMPARE( QDate::longMonthName(date.month()), monthName );
}


QTEST_MAIN(testDate)
#include "tutorial2.moc"

As you can see, we've introduced a new method - testMonth_data, and moved the specific test date out of testMonth. We've had to add some more code (which will be explained soon), but the result is a separation of the data we are testing, and the code we are using to test it. The names of the functions are important - you must use the _data suffix for the data setup routine, and the first part of the data setup routine must match the name of the driver routine.

It is useful to visualise the data as being a table, where the columns are the various data values required for a single run through the driver, and the rows are different runs. In our example, there are four columns (three integers, one for each part of the date; and one QString ), added in lines 19 through 22. The addColumn template obviously requires the type of variable to be added, and also requires a variable name argument. We then add as many rows as required using the newRow function, as shown in lines 23 through 26. The string argument to newRow is a label, which is handy for determining what is going on with failing tests, but doesn't have any effect on the test itself.

To use the data, we simply use QFETCH to obtain the appropriate data from each row. The arguments to QFETCH are the type of the variable to fetch, and the name of the column (which is also the local name of the variable it gets fetched into). You can then use this data in a QCOMPARE or QVERIFY check. The code is run for each row, which you can see below:

Example 10. Results of data driven testing, showing QFETCH

$ ./tutorial2 -v2
********* Start testing of testDate *********
Config: Using QTest library 4.1.0, Qt 4.1.0-snapshot-20051020
INFO   : testDate::initTestCase() entering
PASS   : testDate::initTestCase()
INFO   : testDate::testValidity() entering
INFO   : testDate::testValidity() QVERIFY(date.isValid())
   Loc: [tutorial2.cpp(19)]
PASS   : testDate::testValidity()
INFO   : testDate::testMonth() entering
INFO   : testDate::testMonth(1967/3/11) COMPARE()
   Loc: [tutorial2.cpp(44)]
INFO   : testDate::testMonth(1967/3/11) COMPARE()
   Loc: [tutorial2.cpp(45)]
INFO   : testDate::testMonth(1966/1/10) COMPARE()
   Loc: [tutorial2.cpp(44)]
INFO   : testDate::testMonth(1966/1/10) COMPARE()
   Loc: [tutorial2.cpp(45)]
INFO   : testDate::testMonth(1999/9/19) COMPARE()
   Loc: [tutorial2.cpp(44)]
INFO   : testDate::testMonth(1999/9/19) COMPARE()
   Loc: [tutorial2.cpp(45)]
PASS   : testDate::testMonth()
INFO   : testDate::cleanupTestCase() entering
PASS   : testDate::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped
********* Finished testing of testDate *********

The QTEST macro

As an alternative to using QFETCH and QCOMPARE, you may be able to use the QTEST macro instead. QTEST takes two arguments, and if one is a string, it looks up that string as an argument in the current row. You can see how this can be used below, which is equivalent to the testMonth() code in the previous example.

Example 11. QDate test code, data driven version using QTEST

void testDate::testMonth()
{
    QFETCH(int, year);
    QFETCH(int, month);
    QFETCH(int, day);

    QDate date;
    date.setYMD( year, month, day);
    QCOMPARE( date.month(), month );
    QTEST( QDate::longMonthName(date.month()), "monthName" );
}      

In the example above, note that monthname is enclosed in quotes, and we no longer have a QFETCH call for monthname.

The other QCOMPARE could also have been converted to use QTEST, however this would be less efficient, because we already needed to use QFETCH to get month for the setYMD in the line above.

Running selected tests with selected data

In the previous tutorial, we saw how to run a specific test by specifying the name of the test as a command line argument. In data driven testing, you can select which data you want the test run with, by adding a colon and the label for the data row. For example, if we just want to run the testMonth test for the first row, we would use ./tutorial2 -v2 testMonth:1967/3/11. The result of this is shown below.

Example 12. QDate unit test output - selected function and data

$ ./tutorial2 -v2 testMonth:1967/3/11
********* Start testing of testDate *********
Config: Using QTest library 4.1.0, Qt 4.1.0-snapshot-20051020
INFO   : testDate::initTestCase() entering
PASS   : testDate::initTestCase()
INFO   : testDate::testMonth() entering
INFO   : testDate::testMonth(1967/3/11) COMPARE()
   Loc: [tutorial2.cpp(44)]
INFO   : testDate::testMonth(1967/3/11) COMPARE()
   Loc: [tutorial2.cpp(45)]
PASS   : testDate::testMonth()
INFO   : testDate::cleanupTestCase() entering
PASS   : testDate::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped
********* Finished testing of testDate *********

Tutorial 2: Testing Graphical User Interfaces

In the previous two tutorials, we've tested a date management class. This is an pretty typical use of unit testing. However Qt and KDE applications will make use graphical classes that take user input (typically from a keyboard and mouse). QtTestLib offers support for testing these classes, which we'll see in this tutorial.

Again, we'll use an existing class as our test environment, and again it will be date related - the standard Qt QDateEdit class. For those not familiar with this class, it is a simple date entry widget (although with some powerful back end capabilities). A picture of the widget is shown below.

Figure 1. QDateEdit widget screenshot

The way QtTestLib provides GUI testing is by injecting QInputEvent events. To the application, these input events appear the same as normal key press/release and mouse clicks/drags. However the mouse and keyboard are unaffected, so that you can continue to use the machine normally while tests are being run.

An example of how you can use the GUI functionality of QtTestLib is shown below.

Example 13. QDateEdit test code

#include <QtTest>
#include <QtCore>
#include <QtGui>
Q_DECLARE_METATYPE(QDate)

class testDateEdit: public QObject
{
    Q_OBJECT
private slots:
    void testChanges();
    void testValidator_data();
    void testValidator();
};

void testDateEdit::testChanges()
{
    // 11 March 1967
    QDate date( 1967, 3, 11 );
    QDateEdit dateEdit( date );

    // up-arrow should increase day by one
    QTest::keyClick( &dateEdit, Qt::Key_Up );
    QCOMPARE( dateEdit.date(), date.addDays(1) );

    // we click twice on the "reduce" arrow at the bottom RH corner
    // first we need the widget size to know where to click
    QSize editWidgetSize = dateEdit.size();
    QPoint clickPoint(editWidgetSize.rwidth()-2, editWidgetSize.rheight()-2);
    // issue two clicks
    QTest::mouseClick( &dateEdit, Qt::LeftButton, 0, clickPoint);
    QTest::mouseClick( &dateEdit, Qt::LeftButton, 0, clickPoint);
    // and we should have decreased day by two (one less than original)
    QCOMPARE( dateEdit.date(), date.addDays(-1) );

    QTest::keyClicks( &dateEdit, "25122005" );
    QCOMPARE( dateEdit.date(), QDate( 2005, 12, 25 ) );

    QTest::keyClick( &dateEdit, Qt::Key_Tab, Qt::ShiftModifier );
    QTest::keyClicks( &dateEdit, "08" );
    QCOMPARE( dateEdit.date(), QDate( 2005, 8, 25 ) );
}

void testDateEdit::testValidator_data()
{
    qRegisterMetaType<QDate>("QDate");

    QTest::addColumn<QDate>( "initialDate" );
    QTest::addColumn<QString>( "keyclicks" );
    QTest::addColumn<QDate>( "finalDate" );

    QTest::newRow( "1968/4/12" ) << QDate( 1967, 3, 11 )
                                 << QString( "12041968" )
                                 << QDate( 1968, 4, 12 );

    QTest::newRow( "1967/3/14" ) << QDate( 1967, 3, 11 )
                                 << QString( "140abcdef[" )
                                 << QDate( 1967, 3, 14 );
    // more rows can go in here
}

void testDateEdit::testValidator()
{
    QFETCH( QDate, initialDate );
    QFETCH( QString, keyclicks );
    QFETCH( QDate, finalDate );

    QDateEdit dateEdit( initialDate );
    // this next line is just to start editing
    QTest::keyClick( &dateEdit, Qt::Key_Enter );
    QTest::keyClicks( &dateEdit, keyclicks );
    QCOMPARE( dateEdit.date(), finalDate );
}

QTEST_MAIN(testDateEdit)
#include "tutorial3.moc"

Much of this code is common with previous examples, so I'll focus on the new elements and the more important changes as we work through the code line-by-line.

Lines 1 to 3 import the various Qt declarations, as before.

Line 4 is a macro that is required for the data-driven part of this test, which I'll come to soon.

Lines 5 to 12 declare the test class - while the names have changed, it is pretty similar to the previous example. Note the testValidator and testValidator_data functions - we will be using data driven testing again in this example.

Our first real test starts in line 13. Line 16 creates a QDate, and line 17 uses that date as the initial value for a QDateEdit widget.

Lines 19 and 20 show how we can test what happens when we press the up-arrow key. The QTest::keyClick function takes a pointer to a widget, and a symbolic key name (a char or a Qt::Key). At line 20, we check that the effect of that event was to increment the date by a day. The QTest:keyClick function also takes an optional keyboard modifier (such as Qt::ShiftModifier for the shift key) and an optional delay value (in milliseconds). As an alternative to using QTest::keyClick, you can use QTest::keyPress and QTest::keyRelease to construct more complex keyboard sequences.

Lines 23 to 29 show a similar test to the previous one, but in this case we are simulating a mouse click. We need to click in the lower right hand part of the widget (to hit the decrement arrow - see Figure 1), and that requires knowing how large the widget is. So lines 23 and 24 calculate the correct point based off the size of the widget. Line 26 (and the identical line 27) simulates clicking with the left-hand mouse button at the calculated point. The arguments to Qt::mouseClick are:

  • a pointer to the widget that the click event should be sent to.

the mouse button that is being clicked.

  • an optional keyboard modifier (such a Qt::ShiftModifier), or 0 for no modifiers.
  • an optional click point - this defaults to the middle of the widget if not specified.
  • an optional mouse delay.

In addition to QTest::mouseClick, there is also QTest::mousePress, QTest::mouseRelease, QTest::mouseDClick (providing double-click) and QTest::mouseMove. The first three are used in the same way as QTest::mouseClick. The last takes a point to move the mouse to. You can use these functions in combination to simulate dragging with the mouse.

Lines 30 and 31 show another approach to keyboard entry, using the QTest::keyClicks. Where QTest::keyClick sends a single key press, QTest::keyClicks takes a QString (or something equivalent, in line 30 a character array) that represents a sequence of key clicks to send. The other arguments are the same.

Lines 32 to 34 show how you may need to use a combination of functions. After we've entered a new date in line 30, the cursor is at the end of the widget. At line 32, we use a Shift-Tab combination to move the cursor back to the month value. Then at line 33 we enter a new month value. Of course we could have used individual calls to QTest::keyClick, however that wouldn't have been as clear, and would also have required more code.

Data-driven GUI testing

Lines 50 to 60 show a data-driven test - in this case we are checking that the validator on QDateEdit is performing as expected. This is a case where data-driven testing can really help to ensure that things are working the way they should.

At lines 52 to 54, we fetch in an initial value, a series of key-clicks, and an expected result. These are the columns that are set up in lines 39 to 41. However note that we are now pulling in a QDate, where in previous examples we used three integers and then build the QDate from those. However QDate isn't a registered type for {{qt|QMetaType}, and so we need to register it before we can use it in our data-driven testing. This is done using the Q_DECLARE_METATYPE macro in line 4 and the qRegisterMetaType function in line 38.

Lines 42 to 47 add in a couple of sample rows. Lines 42 to 44 represent a case where the input is valid, and lines 45 to 47 are a case where the input is only partly valid (the day part). A real test will obviously contain far more combinations than this.

Those test rows are actually tested in lines 55 to 59. We construct the QDateEdit widget in line 55, using the initial value. We then send an Enter key click in line 57, which is required to get the widget into edit mode. At line 58 we simulate the data entry, and at line 59 we check whether the results are what was expected.

Lines 61 and 62 are the same as we've seen in previous examples.

Re-using test elements

If you are re-using a set of events a number of times, then it may be an advantage to build a list of events, and then just replay them. This can improve maintainability and clarity of a set of tests, especially for mouse movements.

The key class for building a list of test events is imaginatively known as QTestEventList. It is a QList of QTestEvents. The normal approach is to create the list, and then use various member functions to add key and mouse events. The normal functions that you'll need are addKeyClick and addMouseClick, which are very similar to the QTest::keyClick and QTest::mouseClick functions we used earlier in this tutorial. For finer grained operations, you can also use addKeyPress, addKeyRelease, addKeyEvent, addMousePress, addMouseRelease, addMouseDClick and addMouseMove to build up more complex event lists. You can also use addDelay to add a specified delay between events. When the list has been built up, you just call simulate on each widget.

You can see how this works in the example below, which is the QDateEdit example (from above) converted to use QTestEventList.

Example 14. QDateEdit test code, using QTestEventList

       
#include <QtTest>
#include <QtCore>
#include <QtGui>
Q_DECLARE_METATYPE(QDate)

class testDateEdit: public QObject
{
    Q_OBJECT
private slots:
    void testChanges();
    void testValidator_data();
    void testValidator();
};

void testDateEdit::testChanges()
{
    // 11 March 1967
    QDate date( 1967, 3, 11 );
    QDateEdit dateEdit( date );

    // up-arrow should increase day by one
    QTest::keyClick( &dateEdit, Qt::Key_Up );
    QCOMPARE( dateEdit.date(), date.addDays(1) );

    // we click twice on the "reduce" arrow at the bottom RH corner
    // first we need the widget size to know where to click
    QSize editWidgetSize = dateEdit.size();
    QPoint clickPoint(editWidgetSize.rwidth()-2, editWidgetSize.rheight()-2);
    // build a list that contains two clicks
    QTestEventList list1;
    list1.addMouseClick( Qt::LeftButton, 0, clickPoint);
    list1.addMouseClick( Qt::LeftButton, 0, clickPoint);
    // call that list on the widget
    list1.simulate( &dateEdit );
    // and we should have decreased day by two (one less than original)
    QCOMPARE( dateEdit.date(), date.addDays(-1) );

    QTest::keyClicks( &dateEdit, "25122005" );
    QCOMPARE( dateEdit.date(), QDate( 2005, 12, 25 ) );

    QTestEventList list2;
    list2.addKeyClick( Qt::Key_Tab, Qt::ShiftModifier );
    list2.addKeyClicks( "08" );
    list2.simulate( &dateEdit );
    QCOMPARE( dateEdit.date(), QDate( 2005, 8, 25 ) );
}

void testDateEdit::testValidator_data()
{
    qRegisterMetaType<QDate>("QDate");

    QTest::addColumn<QDate>( "initialDate" );
    QTest::addColumn<QTestEventList>( "events" );
    QTest::addColumn<QDate>( "finalDate" );

    QTestEventList eventsList1;
    // this next line is just to start editing
    eventsList1.addKeyClick( Qt::Key_Enter );
    eventsList1.addKeyClicks( "12041968" );

    QTest::newRow( "1968/4/12" ) << QDate( 1967, 3, 11 )
                                 << eventsList1
                                 << QDate( 1968, 4, 12 );

    QTestEventList eventsList2;
    eventsList2.addKeyClick( Qt::Key_Enter );
    eventsList2.addKeyClicks( "140abcdef[" );

    QTest::newRow( "1967/3/14" ) << QDate( 1967, 3, 11 )
                                 << eventsList2
                                 << QDate( 1967, 3, 14 );
    // more rows can go in here
}

void testDateEdit::testValidator()
{
    QFETCH( QDate, initialDate );
    QFETCH( QTestEventList, events );
    QFETCH( QDate, finalDate );

    QDateEdit dateEdit( initialDate );

    events.simulate( &dateEdit);

    QCOMPARE( dateEdit.date(), finalDate );
}

QTEST_MAIN(testDateEdit)
#include "tutorial3a.moc"

This example is pretty much the same as the previous version, up to line 25. In line 26, we create a QTestEventList. We add events to the list in lines 27 and 28 - note that we don't specify the widget we are calling them on at this stage. In line 30, we simulate each event on the widget. If we had multiple widgets, we could call simulate using the same set of events.

Lines 31 to 34 are as per the previous example.

We create another list in lines 35 to 37, although this time we are using addKeyClick and addKeyClicks instead of adding mouse events. Note that an event list can contain combinations of mouse and keyboard events - it just didn't make sense in this test to have such a combination. We use the second list at line 38, and check the results in line 39.

You can also build lists of events in data driven testing as well, as shown in lines 41 to 70. The key difference is that instead of fetching a QString in each row, we are fetching a QTestEventList. This requires that we add a column of QTestEventList, rather than QString (see line 45). At lines 47 to 50, we create a list of events. At line 52 we add those events to the applicable row. We create a second list at lines 54 to 56, and add that second list to the applicable row in line 58.

We fetch the events in line 65, and use them in line 68. If we had multiple widgets, then we could use the same event list several times.


Continued

This tutorial has not been fully migrated yet. Please continue porting from http://developer.kde.org/documentation/tutorials/writingunittests/tute3.html

Tip
Note: This page is about KDE 4. It isn't applicable for KDE 3 development.