GSoC: Phonon QML Iteration2 & Meego

In my quest of bringing Phonon, the best multimedia abstraction library from KDE, to QML and Qt Quick I have reached another big iteration.

Following iteration 1 there is a now a new branch called ‘qml-i2‘ for iteration 2.

While i1 was using only existing experimental technology of Phonon, namely the VideoDataOutput class, i2 has moved away from this and now features a closer relation with the Phonon backend (currently only GStreamer).

Also I spent half an hour on creating an alternative appearance for the demo player. It is using Nokia Meego 1.2 Harmattan Qt Quick Components and I already ran it on a Nokia N950, the developer version of the Nokia N9 – looks really slick.

Hot new stuff:

  • Closer to backend
  • Redrawing only of space occupied by video frame
  • Pull instead of push frame access -> no memcopy
  • Faster due to above
  • Still raster/qimage based frame drawing
  • Demo player in demos/qml/videoplayer much improved
  • Harmattan demo player

The overall architecture is really simple to explain:
There are 3 QML Elements for Audio, Video and the Media control itself. The Video element uses a class called VideoGraphicsObject which implements the drawing logic. The VideoGraphicsObject connects to a backend implementation via a well defined interface. The backend implementation does magic in order to obtain raw video data from the video pipeline, emits a signal that a new frame is ready and the VideoGraphicsObject does the drawing.

What is particularly interesting is the way we currently access the frame data. The backend implementation holds exactly one frame, consequently either the pipeline or the VideoGraphicsObject hold a lock on the frame. This has three particular advantages for the time being: a) it does avoid any sort of object copy b) it allows the pipeline to adapt to drawing speed and frequency c) always the most current frame is drawn, even if the drawing operation was delayed.

To test qml-i2 you’ll need both the qml-i2 branch from the Phonon git repository as well as the qml-i2 branch from the Phonon GStreamer git repository. After installing both you should be able to run the demo player in Phonon’s demos/qml/videplayer folder.

GSoC: Details on Phonon in QML

Native Phonon QML Player

As mentioned in my previous blog post I am working on bringing Phonon, the best multimedia abstraction library from KDE, to QML and Qt Quick.

Now I’d like to go more into detail about what the point of all this is and how this is going to rock our world.

Before Qt Quick we had widgets (imagine them as boxes). These were easy to use for Phonon, because every box on just about every operating system has some way to draw video content on it. Therefore this was nicely implemented in the frameworks underneath Phonon, so only the high-level Qt integration had to be done. With Qt Quick we don’t have proper boxes anymore, not in the way the operating systems like them anyway. Consequently we need to do some work to get from where we are to Qt Quick.

How will this work you might ask. Really it is actually very simple. In Qt Quick rather than having a random box where you can draw your video frames on, you have declarative items, which in essence also are boxes but unknown to the operating system. In Qt Quick those boxes are rendered using Qt internal technology. Right now that would be a software rasterizing approach or similar magic supported by QPainter, in Qt 5 this will actually all be done using OpenGL to enable much more awesome and complex applications as essentially the whole user interface will be rendered using GL.

Even though Qt 5 will do this vastly different, the concept of making video playback happen with Qt Quick is the same in both cases. You get your video frame, hand it to your Qt Quick item and in there you draw the frame depending on available capabilities and what not.

Ok, clearly there is various degrees of complexity involved on both ends (video/audio sync? drawing speed? …). Some of the more interesting programming challenges I will blog about next week.

Anyhow, people who know me a bit will notice that this post is unusually long, and clearly I would not blog that much blah without offering something to try 😉 So, get the Phonon QML branch iteration1 and take it for a test drive if you want. There is an audio and video demo in demos/qml (just make sure you have audio.wav or vidoe.ogv in the demo folders).

Iteration1 actually uses only technology that is already available (namely stock Phonon QObjects for Media and Audio and VideoDataOutput from experimental for Video). Future iterations will get rid of the dependency on VideoDataOutput as the QML branch is moving parts of the magic in the Phonon backends to allow for more ways to optimize the entire drawing process. In fact my local branches already have something that draws without the overhead of VideoDataOutput 😉

If you want to watch development at large you might want to check out the main qml branch.

I’ll blog soon about the architecture (once I have figured it out better ^^), meanwhile you can take a look at this picture (for Phonon GStreamer).

Phonon Family 4.4.4

The Phonons are very proud to announce the immediate availability of a new minor release, bumping Phonon as well as the GStreamer, Xine and DirectShow backends to version 4.4.4. Also Phonon VLC is now available as 0.3.2.

by tarotastic@flickr

Everyone in the team put tremendous effort into this new release, focusing especially on improving reliability of the backends. But also a couple of new features entered GStreamer as I presented in my last blog post.

Even though these are only minor releases, they mark a new era, as from now on all Phonon backends can and will be released independently of the Phonon library itself. This enables us to bring bugfixes and new features quicker to distributors and user.

Last but not least we changed the default priorities of backends on Linux. To get the best experience you should now either use GStreamer or VLC. The former preferred choice Xine is as of now considered unmaintained and should not be used anymore.
Further more it is highly recommended to use Phonon in combination with PulseAudio as it improves user experience a lot and makes smarter choices about default output devices.

Get it while it is hot

I have been using 4.4.4 for quite some time now and I find it simply awesome 😀