Wayland and Qt 5.4

Since nobody else has done the honors yet, I'm happy to announce that - as decided at the Qt Contributors Summit this year - support for running applications under a Wayland compositor will be seeing its initial release with Qt 5.4. That is, the QtWayland repository is finally going to stop sitting in the corner, sulking. :)

There's a few "buts", though.

Firstly, it should be noted that support for QWidget-Based applications (and other desktop-based usecases) may be far from ideal, and quality may not be great. This is a consequence of most development on QtWayland having been driven from mobile/embedded viewpoints to date, and is not, in general, an inherent limitation on the windowing system. It's also something of a reflection on Wayland itself, which is only now starting to mature for desktop use (through xdg-shell etc etc.)

tl;dr: Think of this as a technical preview, keep your expectations realistic, and if you want to use it, expect to roll up your sleeves a bit and get dirty from time to time.

Secondly, the QtCompositor API in the QtWayland module (allowing you to write your own Wayland compositor) will not be seeing a release at this time. The API is not frozen, and has not seen the usual polish/quality that you might expect from Qt APIs. As this API is only of use to a limited number of people (those looking to implement an embedded/mobile device, typically, or write their own DE) this should not impact too many people.

tl;dr: If you want to write a compositor, you get to keep both pieces if it breaks. If you want to use applications under an existing Wayland compositor, you're fine.

Future work to QtWayland is largely an open story, but some obvious candidates come to mind:

• Continued work on xdg-shell support
• Plugin based window decorations (to enable environment-specific look and feel) this has now landed in the 5.4 branch :)
• Integration with the rest of Qt's autotests (I spent a while getting tests fixed or at least runnable under window-compositor, but it would be nice to automate this)
• "Official" subsurface protocol support

If there's something you would like to see happen, here or not, you're more than welcome to pitch in. If you'd like to talk to the other people hacking on QtWayland, please pop by on #qt-lighthouse on freenode, and talk to the folks there :-)

I'd also like to take a moment to thank everyone for their contributions to QtWayland. In particular, I'd like to say thanks to the following, in no particular order (and I'm extremely sorry if I've missed someone, please let me know and I'll happily add you to the list):

• Kristian Høgsberg & Jesse Barnes, for their initial work on the port, sponsored by Intel,
• Jørgen Lind, Samuel Rødal, Andy Nichols, Laszlo Agocs, and Paul Olav Tvete for continuing work on it excellently and admirably,
• Nokia for sponsoring a good deal of the development up until their abrupt departure from the Qt world,
• Digia for continuing to help out after Nokia left,
• Andrew Knight, for ably shepherding problems encountered by Jolla for quite a long time,
• Jolla for sponsoring a large chunk of work on QtWayland (past and present),
• Gunnar Sletta for rewriting integration with rendering (especially QtQuick), removing a large number of bugs & improving performance,
• Giulio Camuffo for numerous fixes, improvements and interaction with the wider Wayland community.

As a conclusion, I'd like to note that I'm really happy to see this finally happen - I've wanted it for a very long time now - and for Wayland to keep moving on for bigger and better things. Hopefully, this release will achieve its intended result (that more eyes/hands get exposed to the code, and start to use it, and help out with it).

profiling is not understanding

When software goes slow, generally, the first reaction is to profile. This might be done through system tools (like Instruments on OS X, perf/valgrind/etc on Linux, VTune, etc). This is fine and good, but just because you have the output of a tool does not necessarily correlate to understanding what is going on.

This might seem like an obvious distinction, but all too often, efforts at improving performance focus on the small picture ("this thing here is slow") and not the bigger picture ("why is this so slow"). At Jolla, I had the pleasure of running into one such instance of this, together with Gunnar Sletta, my esteemed colleague, and friend.

As those of you who are familiar with Jolla may know, we had been working on upgrading to a newer Qt release. This also involved quite a bit of work for us, both in properly upstreaming work we had done on the hurry to the late-2013 release, and in isolating problems and fixing them properly in newer code (the new scenegraph renderer, and the v4 javascript engine in particular have been an interesting ride to get both at once!).

As a part of this work, we noted that touch handling was quite slow (something which we had worked around for our initial release, but now wanted to solve properly). This was due to the touch driver on the Jolla introducing touchpoints faster than the display was updating, that is, while the display might be updating at 57 hz (yes, the Jolla is weird, it doesn't do 60 hz) - we might be getting input events a lot more frequently than that.

This was, in turn, causing QtQuick to run touch processing (involving costly item traversals, as well as the actual processing of touch handling) a lot more frequently than the display was updating. As these took so much time, this in turn slowed rendering down, meaning even more touch handling was going on per frame. A really ugly situation.

Figure 1: Event tracing inside the Sailfish OS Compositor

Figure 1 demonstrates this happening at the compositor level. The bottom slice (titled "QThread") is the event delivery thread, responsible for reading events from evdev The peaks there are - naturally - when events are being read in. The top thread is the GUI thread, and the high peaks there are touch events being processed and delivered to the right QtQuick item (in this case, a Wayland client, we'll get to that later). The middle slice is the compositor's scenegraph rendering (using QtQuick).

With the explanation out of the way, let's look at the details a bit more. It's obvious that the event thread is regularly delivering events at around-but-not-quite twice the display update. Our frame preparation on the GUI thread looks good, despite the too-frequent occurrence of event delivery, though, and the render thread is coping too.

But this isn't a major surprise - the compositor in this case is dead simple (just showing a fullscreen client). What about the client? Let's take a look at it over the same timeframe...

Figure 2: Event tracing for the client (Silica's component gallery, in this case)

Figure 2 focuses on two threads in the client: the render thread (top), and the GUI thread (bottom). Touch events are delivered on the GUI thread, QtQuick processes them there while preparing the next frame for the render thread.

Here, it's very clear that touch processing is happening way too often, and worse than that, it's taking a very long time (each touch event's processing is taking ~4ms), not leaving much time for rendering - and this was on a completely unloaded device. In a more complicated client still, this impact would be much, much worse, leading to frame skipping (which we saw, on some other applications).

Going back to my original introduction here, if we had used traditional profiling techniques, we'd have seen that touch handling/preparation to render was taking a really long time. And we might have focused on optimizing that. Instead, thanks to some out-of-the-box thinking, we looked at the overall structure of application flow, and were able to see the real problem: doing extra work that wasn't necessary.

As an aside to this, I'm happy to announce that we worked out a neat solution to this: QtQuick now doesn't immediately process touch events, instead, choosing to wait until it is about to prepare the next frame for display - as well as "compressing" them to only deal with the minimal number of sensible touch updates per frame. This should have no real impact on any hardware where touch delivery was occurring at a sensible rate, but for any hardware where touch was previously delivering too fast, this will no longer be a problem as of Qt 5.4.

(Thanks to Gunnar & myself for the fix, Carsten & Mikko for opening my eyes about performance tooling, and Jolla for sponsoring this work.

P.S. If you're looking for performance experts, Qt/QML/etc expertise or all round awesome, Gunnar and myself are currently interested in hearing from you.)

writing a layout in QML

Sometimes, for whatever reason, the layouts provided "out of the box" in QML just don't cut it. lately, I've been doing a few rather different things for experimentation and learning purposes that have meant I've run into quite a lot of these cases.

when this happens, the first instinct is to fall into despair - but there's really no need for that. writing your own layout really isn't that hard. Here's a small, fairly self contained example doing just that.

MyLayout.qml:

import QtQuick 2.0

Item {
    id: layout

    property bool ready: false

    onChildrenChanged: performLayout()
    onWidthChanged: performLayout()
    onHeightChanged: performLayout()

    /* the meat of the layout */
    function performLayout() {
        /* nothing to layout? don't bother then */
        if (layout.children.length == 0)
            return

        var currentX = 0

        console.log("DOING LAYOUT FOR " + layout.children.length + " ITEMS")

        /* first real step of doing anything: go over all the children */
        for (var i = 0; i < layout.children.length; ++i) {
            var obj = layout.children[i]

            /* in the real world, we'd probably do something a lot more complex,
             * but let's just position our children along a row.
             */
            console.log("Positioning at " + currentX + " to " + (currentX + obj.width))
            obj.x = currentX
            currentX += obj.width
        }

        console.log("LAYOUT DONE")
    }
}

MyItem.qml:

import QtQuick 2.0

Rectangle {
    width: 100
    height: 50
    color: "black"

    Rectangle {
       width: 90
       height: 40
       anchors.centerIn: parent

       color: "red"
    }
}

main.qml

import QtQuick 2.0

Rectangle {
    width: 1000
    height: 100

    MyLayout {
        MyItem { }
        MyItem { }
        MyItem { }
        MyItem { }
        MyItem { }
        MyItem { }
        MyItem { }
    }
}

This is obviously very simplified for demonstration purposes, to name a few things that it doesn't do:

• it omits things like margins, wrapping
• it will break if MyItem is ever anchored
• it doesn't use anchoring (which might make for a more optimal implementation in this particular case - at the least, it wouldn't have to relayout if the width/height of the layout changed)
• it doesn't relayout if the geometry of the children change
• it relayouts whenever properties changes, which isn't optimal if e.g. the layout is animating a change, instead, it should delay a relayout using a Timer

All of these are left to the reader, but hopefully it's of some help in getting started.

on the importance of doing nothing

I've been meaning to write about this for a while, but I've only just now been driven over the edge by having to go and basically run sed over code again for no good reason.

When you're programming, always make sure you question *why* things are done. Qt provides three functions, helpfully named qMalloc/qRealloc/qFree. Despite the 'q' in front of their names, these functions do absolutely nothing useful, they just wrap around their stdlib friends. This was originally done to enable replacement of the allocator inside Qt (but there are better ways to do that, without getting sidetracked from my central point), but in reality, doesn't have much use. That's why I'm trying to deprecate them.

Now, you might ask, "what impact could a simple function call have, anyway"? I'm glad you asked. Benchmark time (spoiler for the lazy: ~10% extra overhead for small allocation sizes, ~0-5% for larger allocation sizes).
virgin:~/mallocbench% cat main.cpp

#include <QtCore>
#include <qtest.h>
#include <qcoreapplication.h>
#include <qdatetime.h>

class MallocBenchmark : public QObject
{
Q_OBJECT
private slots:
    void qtMalloc();
    void qtMalloc_data();
    void regularMalloc();
    void regularMalloc_data();
};

void MallocBenchmark::qtMalloc_data()
{
    QTest::addColumn<int>("size");
    QTest::newRow("1") << 1;
    QTest::newRow("10") << 1;
    QTest::newRow("100") << 100;
    QTest::newRow("10000") << 10000;
    QTest::newRow("1000000") << 1000000;
    QTest::newRow("10000000") << 10000000;
}

void MallocBenchmark::qtMalloc()
{
    QFETCH(int, size);

    QBENCHMARK {
        void *p = ::qMalloc(size);
        ::qFree(p);
    }
}

void MallocBenchmark::regularMalloc_data()
{
    qtMalloc_data();
}

void MallocBenchmark::regularMalloc()
{
    QFETCH(int, size);

    QBENCHMARK {
        void *p = malloc(size);
        free(p);
    }
}

QTEST_MAIN(MallocBenchmark)

#include "main.moc"

And now, the results on my machine:

********* Start testing of MallocBenchmark *********
Config: Using QTest library 5.0.0, Qt 5.0.0
PASS   : MallocBenchmark::initTestCase()
RESULT : MallocBenchmark::qtMalloc():"1":
     0.000059 msecs per iteration (total: 62, iterations: 1048576)
RESULT : MallocBenchmark::qtMalloc():"10":
     0.000062 msecs per iteration (total: 66, iterations: 1048576)
RESULT : MallocBenchmark::qtMalloc():"100":
     0.000087 msecs per iteration (total: 92, iterations: 1048576)
RESULT : MallocBenchmark::qtMalloc():"10000":
     0.000083 msecs per iteration (total: 88, iterations: 1048576)
RESULT : MallocBenchmark::qtMalloc():"1000000":
     0.0043 msecs per iteration (total: 72, iterations: 16384)
RESULT : MallocBenchmark::qtMalloc():"10000000":
     0.0063 msecs per iteration (total: 52, iterations: 8192)
PASS   : MallocBenchmark::qtMalloc()
RESULT : MallocBenchmark::regularMalloc():"1":
     0.000053 msecs per iteration (total: 56, iterations: 1048576)
RESULT : MallocBenchmark::regularMalloc():"10":
     0.000051 msecs per iteration (total: 54, iterations: 1048576)
RESULT : MallocBenchmark::regularMalloc():"100":
     0.000082 msecs per iteration (total: 86, iterations: 1048576)
RESULT : MallocBenchmark::regularMalloc():"10000":
     0.000076 msecs per iteration (total: 80, iterations: 1048576)
RESULT : MallocBenchmark::regularMalloc():"1000000":
     0.0043 msecs per iteration (total: 71, iterations: 16384)
RESULT : MallocBenchmark::regularMalloc():"10000000":
     0.0060 msecs per iteration (total: 99, iterations: 16384)
PASS   : MallocBenchmark::regularMalloc()
PASS   : MallocBenchmark::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped
********* Finished testing of MallocBenchmark *********

Around 10% extra time per iteration on smaller allocation sizes, 0-5% on larger sizes (most likely explained by glibc falling back to using mmap for larger allocations, which is going to take an awful long time compared to a single function call). These, obviously, aren't huge numbers. But remember: this is overhead you're taking for no reason at all. Don't do it. Your CPU cycles will thank me.

QFileSystemWatcher internals in Qt 5

Just thought I'd share some details on some of the recent changes I've pushed to Qt 5 a few weeks ago. (Yes, this post is rather overdue, I've been a bit slack with writing it). If you were in Tampere when I gave a short, completely underprepared Q&A on Qt 5 a few days ago, this won't be news to you, but I will go into a bit more detail.

tl;dr, all in all, a lot of code was deleted, and things still function more or less the same, except a bit better. That's quite a common story for Qt 5, I hope... :)

First of all, platform support: as with Qt 5 itself, Symbian support is no longer a goal. Since I wanted to make some changes to internals, and wasn't able to even remotely come close to building the Symbian code, it was removed.

On Linux, the (ancient, and no longer used by default) dnotify backend also met its maker. Since inotify has been around for some 6-7 years, it was about time, especially as the dnotify backend had some interesting bugs in behaviour.

The OS X FSEvents backend (also unused for quite some time, due to bugs, and not being a recommended way of working apparently) joined to make for a trinity of dead implementations. OS X's watching is survived by kqueue, which it shares with BSD platforms.

The currently supported backends are:

• inotify (on Linux)
• kqueue (on BSD and OS X)
• WaitForMultipleObjects on Windows, which I need to become more familiar with. Not having a Windows machine has meant that I'm not really able to do much here...

Aside from backend support, there were some more 'fun' changes which went in. First, some detail on implementation. Each QFileSystemWatcher has an 'engine' associated with it, which is backend-specific, and does the actual monitoring. The backend is responsible for communicating changes to the 'frontend' QFileSystemWatcher, which then sends the notifications to the API user.

In the past, QFileSystemWatcher engines used to be run in a thread. I'm not sure why this was done originally, but it pretty much never made much sense - monitoring file changes is not a particularly intensive operation, so this is just a waste of resources (thread stack, time to start the thread, etc) - which was compounded by this being a thread per engine, meaning that if you have a few different libraries monitoring files, they'd each start their own thread.

Another nasty side effect of this thread was resource consumption caused by monitoring. If you monitored a large number of paths, but couldn't consume events faster than the OS was throwing them at you, then that engine thread would happily sit there and keep on reading them and turning them into Qt signals for the QFileSystemWatcher/user code. But because that code was on a different thread, and unable to keep up, you'd just keep getting more, and more, and more signals, and memory usage would keep growing and growing.

This thread has now been removed, so changes are implicitly rate-limited to the thread the QFileSystemWatcher lives in, meaning that all of these are no longer a problem. Kudos should also go to Bradley Hughes for fixing a few issues which I missed on platforms other than Linux after it was integrated.

Brad also took this work a step further: QFileSystemWatcher has never been documented as being thread-safe, but the engines may have happened to be more or less thread-safe thanks to living on a different thread to the QFileSystemWatcher, through mutexing. One part inside Qt itself actually needed this for autotests to function correctly, too: QFileSystemModel. He fixed this requirement, and was thus able to remove the mutexes from the engines. Thanks!

I'd also like to thank Brad, João Abecasis, and anyone I've forgotten for helping to review these changes and get them integrated.

(One thing I neglected to mention above - the thread story is a little more complicated on Windows. Windows still has threads inside the engine (although the engine itself is no longer a thread, so there's still one less). This is necessary because WaitForMultipleObjects can only process up to MAXIMUM_WAIT_OBJECT handles at a time, unless you use multiple threads to do the monitoring, so that's exactly what it does. It spawns multiple threads on-demand as soon as it can't find a thread with a spare slot. But this is nothing new.)

why I avoid QRegExp in Qt 4 and so should you

A few times, when I've been working on something performance-critical, I've had people suggest (or ask me to review code) using QRegExp. I usually tell them that "this is a bad idea, QRegExp is slow/unmaintained", but I never actually sat down to do benchmarks. Well, in Qt 5, the subject of replacing QRegExp has been discussed a bit, and we have a volunteer: Giuseppe D'Angelo, a Qt hacker and italian student living in the UK (looking for work, by the way!).

One of the first steps he has taken has been to quantify the performance of two of our leading candidates for replacements: PCRE, and ICU's regex engine. He also took the liberty of benchmarking Qt's existing QRegExp - here's the results:

(Caveat: Giuseppe has asked for someone more familiar with ICU to look over the code there to make sure the results aren't negatively impacted.)


RESULT : REBenchmark::PCREBenchmark():"URI":
     1,123 msecs per iteration (total: 1,123, iterations: 1)
RESULT : REBenchmark::PCREBenchmark():"Email":
     1,798 msecs per iteration (total: 1,798, iterations: 1)
RESULT : REBenchmark::PCREBenchmark():"Date":
     99 msecs per iteration (total: 99, iterations: 1)
RESULT : REBenchmark::PCREBenchmark():"URI|Email":
     2,650 msecs per iteration (total: 2,650, iterations: 1)

RESULT : REBenchmark::ICUBenchmark():"URI":
     11,674 msecs per iteration (total: 11,674, iterations: 1)
RESULT : REBenchmark::ICUBenchmark():"Email":
     17,056 msecs per iteration (total: 17,056, iterations: 1)
RESULT : REBenchmark::ICUBenchmark():"Date":
     392 msecs per iteration (total: 392, iterations: 1)
RESULT : REBenchmark::ICUBenchmark():"URI|Email":
     30,552 msecs per iteration (total: 30,552, iterations: 1)

RESULT : REBenchmark::QRegExpBenchmark():"URI":
     21,579 msecs per iteration (total: 21,579, iterations: 1)
RESULT : REBenchmark::QRegExpBenchmark():"Email":
     55,426 msecs per iteration (total: 55,426, iterations: 1)
RESULT : REBenchmark::QRegExpBenchmark():"Date":
     1,357 msecs per iteration (total: 1,357, iterations: 1)
RESULT : REBenchmark::QRegExpBenchmark():"URI|Email":
     77,224 msecs per iteration (total: 77,224, iterations: 1)

(These tests were run on: Ubuntu 10.04 LTS 32 bit, Core(TM)2 Duo CPU P9600  @ 2.66GHz, 4GB RAM, GCC 4.4.3, Qt 4.7.4, PCRE 8.20, ICU 4.8.1.1)

As we see from these results, PCRE is the clear winner, probably thanks to its use of JIT in evaluating the expressions. We see ICU showing decent results, and getting second place (see the caveat above). Leading the rear of the pack, we see QRegExp, 10x-30x slower than PCRE - and these are hardly complicated regular expressions.

In conclusion, for now, if you want performance, you should probably steer clear of QRegExp, and find other ways to do what you want, either using QString directly if it's a simple job, or looking into a light shim around another regex engine until Qt 5, when hopefully, your problems will go away with a shiny new class.