profiling is not understanding

When software goes slow, generally, the first reaction is to profile. This might be done through system tools (like Instruments on OS X, perf/valgrind/etc on Linux, VTune, etc). This is fine and good, but just because you have the output of a tool does not necessarily correlate to understanding what is going on.

This might seem like an obvious distinction, but all too often, efforts at improving performance focus on the small picture ("this thing here is slow") and not the bigger picture ("why is this so slow"). At Jolla, I had the pleasure of running into one such instance of this, together with Gunnar Sletta, my esteemed colleague, and friend.

As those of you who are familiar with Jolla may know, we had been working on upgrading to a newer Qt release. This also involved quite a bit of work for us, both in properly upstreaming work we had done on the hurry to the late-2013 release, and in isolating problems and fixing them properly in newer code (the new scenegraph renderer, and the v4 javascript engine in particular have been an interesting ride to get both at once!).

As a part of this work, we noted that touch handling was quite slow (something which we had worked around for our initial release, but now wanted to solve properly). This was due to the touch driver on the Jolla introducing touchpoints faster than the display was updating, that is, while the display might be updating at 57 hz (yes, the Jolla is weird, it doesn't do 60 hz) - we might be getting input events a lot more frequently than that.

This was, in turn, causing QtQuick to run touch processing (involving costly item traversals, as well as the actual processing of touch handling) a lot more frequently than the display was updating. As these took so much time, this in turn slowed rendering down, meaning even more touch handling was going on per frame. A really ugly situation.

Figure 1: Event tracing inside the Sailfish OS Compositor

Figure 1 demonstrates this happening at the compositor level. The bottom slice (titled "QThread") is the event delivery thread, responsible for reading events from evdev The peaks there are - naturally - when events are being read in. The top thread is the GUI thread, and the high peaks there are touch events being processed and delivered to the right QtQuick item (in this case, a Wayland client, we'll get to that later). The middle slice is the compositor's scenegraph rendering (using QtQuick).

With the explanation out of the way, let's look at the details a bit more. It's obvious that the event thread is regularly delivering events at around-but-not-quite twice the display update. Our frame preparation on the GUI thread looks good, despite the too-frequent occurrence of event delivery, though, and the render thread is coping too.

But this isn't a major surprise - the compositor in this case is dead simple (just showing a fullscreen client). What about the client? Let's take a look at it over the same timeframe...

Figure 2: Event tracing for the client (Silica's component gallery, in this case)

Figure 2 focuses on two threads in the client: the render thread (top), and the GUI thread (bottom). Touch events are delivered on the GUI thread, QtQuick processes them there while preparing the next frame for the render thread.

Here, it's very clear that touch processing is happening way too often, and worse than that, it's taking a very long time (each touch event's processing is taking ~4ms), not leaving much time for rendering - and this was on a completely unloaded device. In a more complicated client still, this impact would be much, much worse, leading to frame skipping (which we saw, on some other applications).

Going back to my original introduction here, if we had used traditional profiling techniques, we'd have seen that touch handling/preparation to render was taking a really long time. And we might have focused on optimizing that. Instead, thanks to some out-of-the-box thinking, we looked at the overall structure of application flow, and were able to see the real problem: doing extra work that wasn't necessary.

As an aside to this, I'm happy to announce that we worked out a neat solution to this: QtQuick now doesn't immediately process touch events, instead, choosing to wait until it is about to prepare the next frame for display - as well as "compressing" them to only deal with the minimal number of sensible touch updates per frame. This should have no real impact on any hardware where touch delivery was occurring at a sensible rate, but for any hardware where touch was previously delivering too fast, this will no longer be a problem as of Qt 5.4.

(Thanks to Gunnar & myself for the fix, Carsten & Mikko for opening my eyes about performance tooling, and Jolla for sponsoring this work.

P.S. If you're looking for performance experts, Qt/QML/etc expertise or all round awesome, Gunnar and myself are currently interested in hearing from you.)

writing a layout in QML

Sometimes, for whatever reason, the layouts provided "out of the box" in QML just don't cut it. lately, I've been doing a few rather different things for experimentation and learning purposes that have meant I've run into quite a lot of these cases.

when this happens, the first instinct is to fall into despair - but there's really no need for that. writing your own layout really isn't that hard. Here's a small, fairly self contained example doing just that.

MyLayout.qml:

import QtQuick 2.0

Item {
    id: layout

    property bool ready: false

    onChildrenChanged: performLayout()
    onWidthChanged: performLayout()
    onHeightChanged: performLayout()

    /* the meat of the layout */
    function performLayout() {
        /* nothing to layout? don't bother then */
        if (layout.children.length == 0)
            return

        var currentX = 0

        console.log("DOING LAYOUT FOR " + layout.children.length + " ITEMS")

        /* first real step of doing anything: go over all the children */
        for (var i = 0; i < layout.children.length; ++i) {
            var obj = layout.children[i]

            /* in the real world, we'd probably do something a lot more complex,
             * but let's just position our children along a row.
             */
            console.log("Positioning at " + currentX + " to " + (currentX + obj.width))
            obj.x = currentX
            currentX += obj.width
        }

        console.log("LAYOUT DONE")
    }
}

MyItem.qml:

import QtQuick 2.0

Rectangle {
    width: 100
    height: 50
    color: "black"

    Rectangle {
       width: 90
       height: 40
       anchors.centerIn: parent

       color: "red"
    }
}

main.qml

import QtQuick 2.0

Rectangle {
    width: 1000
    height: 100

    MyLayout {
        MyItem { }
        MyItem { }
        MyItem { }
        MyItem { }
        MyItem { }
        MyItem { }
        MyItem { }
    }
}

This is obviously very simplified for demonstration purposes, to name a few things that it doesn't do:

• it omits things like margins, wrapping
• it will break if MyItem is ever anchored
• it doesn't use anchoring (which might make for a more optimal implementation in this particular case - at the least, it wouldn't have to relayout if the width/height of the layout changed)
• it doesn't relayout if the geometry of the children change
• it relayouts whenever properties changes, which isn't optimal if e.g. the layout is animating a change, instead, it should delay a relayout using a Timer

All of these are left to the reader, but hopefully it's of some help in getting started.

Avoiding graphics flicker in Qt / QML

It's very common when writing QML applications to write a small stub, something like the following:


int main(int argc, char **argv)
{
    QApplication application(argc, argv);
    QDeclarativeView view;
    view.setSource(QUrl("qrc:/qml/main.qml"));
    view.showFullScreen();
    return a.exec();
}

What's wrong with this? It's a very subtle problem. I'll give you a moment to think about it, and a video to see if you notice the problem. Make sure you don't cheat.

(demonstrating removal of flicker in QML)

Back already? Have you figured it out? That's right, it flickers. Horrifically.

So what causes this? By default, QWidgets are drawn parent first, with parents drawing children. When a widget is drawn, first, it draws its background, then it draws the actual content. That background proves to be a problem, in this case.

If we add the following lines to the above example, the flicker goes away, and my eyes no longer want to bleed:

    view.setAttribute(Qt::WA_OpaquePaintEvent);
    view.setAttribute(Qt::WA_NoSystemBackground);
    view.viewport()->setAttribute(Qt::WA_OpaquePaintEvent);
    view.viewport()->setAttribute(Qt::WA_NoSystemBackground);

NB: I'm not completely sure that adding it to both the view, and the viewport is completely necessary, but it can't harm at least. Make sure to re-set it if you change viewports.

For completeness, here's the full, fixed example:

int main(int argc, char **argv)
{
    QApplication application(argc, argv);
    QDeclarativeView view;
    view.setSource(QUrl("qrc:/qml/main.qml"));
    view.setAttribute(Qt::WA_OpaquePaintEvent);

    view.setAttribute(Qt::WA_NoSystemBackground);
    view.viewport()->setAttribute(Qt::WA_OpaquePaintEvent);
    view.viewport()->setAttribute(Qt::WA_NoSystemBackground);

    view.showFullScreen();
    return a.exec();
}

(If you're curious, Qt::WA_OpaquePaintEvent basically implies that you'll repaint everything as necessary yourself (which QML is well behaved with), and Qt::WA_NoSystemBackground tells Qt to nicely not paint the background.)

NB: on Harmattan (and Nemo Mobile) at least, make sure you always use QWidget::showFullScreen(). The compositor in use there unredirects fullscreen windows (meaning no compositor in the way), so you get faster drawing performance, and every frame counts.

(obligatory thanks to Daniel Stone of X and Collabora fame, for telling me to stop blaming X, and start blaming the crappy toolkits ☺)

Fast UI with Qt 4 on mobile

For device manufacturers, and those targeting device manufacturers like us in the Mer¹ and Nemo Mobile².communities, we need a performant base, and Qt's default configuration on Linux is ..not really that performant. It uses what is known as the 'native' graphics system, which uses X (and XRender) to do a lot of the grunt work. Unfortunately, XRender isn't exactly what you'd call speedy in many cases, and making loads of round trips to ask X to draw things probably doesn't help either.

There's another option in the 'raster' graphics system, which does all rendering client-side in your application using (as you'd expect) Qt's software rasterizer, which is adequate enough for performance on most desktops, but still not quite optimal on desktops: hardware acceleration is the missing goodie.

Qt as of 4.8 also includes what's known as the MeeGo graphics system - as the name implies, it's used on the Nokia N9. It uses hardware acceleration (plus some additional EGL extensions) to perform absolute magic and make your pixels (especially QML :)) fly, so if you're working on getting a device together using Qt 4, I'd highly recommend you look at it.

Because no blog post is complete without a video, here's one:

(comparing the performance of raster and MeeGo graphics systems on a Lenovo S10-3t)

This sounds great, but there's one caveat. If you're on certain types of graphics hardware (SGX in particular), then you'll probably not to just want to enable the MeeGo graphics system for everything, because you'll end up with a lot of GL contexts allocated, which is not good for two reasons; they're scarce resources, and they take up a large chunk of RAM (something in the order of 5-10mb, depending on your PVR configuration file. I never actually looked).

There is good news at hand, though! Qt has *another* graphics system, specifically designed to proxy everything through another system, and allow for runtime switching back to raster. In our case, obviously, we want it to use MeeGo's graphics system by default, but fall back to raster to avoid taking extra resources when not needed.

There's also some bad news: it didn't work out of the box with Qt 4.8 RC1.

But some more good news: I fixed it (and yes, I upstreamed the patches)!
http://qt.gitorious.org/qt/qt/commit/a7c77bd46ef85bae624e829cb2a02110ec60b318
and
http://qt.gitorious.org/qt/qt/commit/0ceab866c76e0d9eb17bc1f3d42af06c0033560b
are the two commits you'll want.

After that, configure Qt with -graphicssystem runtime, and -runtimegraphicssystem meego (or set QT_DEFAULT_RUNTIME_SYSTEM=meego), and it'll work beautifully, provided your system has the required GL extensions.

Alternatively, you can use Mer, which already includes these patches right now, and, as of this week, has this beautiful magic enabled by default on the Nokia N900/N950/N9 hardware ports, where this is really needed (and works well). I'm working on getting it enabled by default on other systems that can support it, too, like the Lenovo S10-3t, as time permits.

MeeGo CE work

With the governance for MeeGo CE now being turned over to the community, and increasingly, the work on MeeGo also being done by the community, I thought that perhaps now was a good time to look at where we are, and where I think our priorities should be. I've already started working on these myself, and will continue to do so.

Present:

• CE Fall Release is a good start, a lot faster than 1.2
• We still use too much memory running daemons for things we (mostly) don't use, like Tracker. We should look at ways to minimise this penalty, or remove the offenders.
• We are lacking in applications in a number of areas
• Our (MTF-provided) theming is really ugly IMHO

Future:

• It was looking like we were more or less going to get a new UI'for free' from Intel with their tablet work, but signs show that they seem to have abandoned this work, so I don't think we should rely on it, and given that it isn't at all optimised for our screen sizes and resource constraints, we can't really reuse it wholesale, either. That thankfully doesn't mean we can't reuse pieces (lipstick uses parts of meego-ux-launcher. :)
• We should investigate applauncherd to (further) speed up application load times. This will have a memory hit, so we should be careful, and measure to make sure it is really worth it, but this could also be minimised by only running the QML booster plugin.
• Qt needs investigation to make sure graphicssystem switching is actually working; I don't think it is at present, and this may mean a lot of wasted RAM with GL contexts being held by backgrounded applications.
• Software tuning is required on all areas of the stack, one simple example found in a few minutes of investigation is messageserver, which links to QtGui unnecessarily.
• It would be nice to revisit theming, and make things prettier. This is possible on an application-by-application process (e.g. I personally think things like lipstick and qmlcalc are the start of some good directions), but we need to make sure we have an up-to-date, OSS licensed Qt Components theme available which *looks good*, and also *really* nice looking icons for our applications.
• More applications! I've started work on a clock / alarm application, we'll see where it goes in the next week or two. I'm waiting for Tom Swindell (alterego) of MeeGo fame to help me with a nice QML theme for lipstick, and there's a few other projects I have on the boil too - watch this space...

Sorry for the brain-dump. I hope someone has found it helpful/useful, it's at least helped me prioritise things a bit for myself, maybe for you too? :)

It should go without saying, but should anyone want to help out on CE at getting an open source mobile OS into top gear, please, by all means get in contact with us on #meego-arm on freenode, and we'll find you something to do.