Ubuntu One – Technical Aspects

Following my post on Ubuntu One in KDE I would now like to take a closer look at how Ubuntu One works using KDE technologies and what the current state of affairs is. So be prepared for a lot of technical blah :)

Something I need to make clear up front is that the KDE client does not connect to the Ubuntu One servers itself. This will probably sound odd to you, so let me explain just how both the KDE and GNOME clients generally work. There is a helper application for authentication (in up to 10.04 called ubuntuone-auth, in 10.10 and above ubuntu-sso-client) that handles authentication to Ubuntu One. Secondly there is a synchronization component called SyncDaemon, as one can assume it handles synchronization of data and is essentially the gateway to the the servers. Thirdly there are clients that talk to the SyncDaemon. Such a client could be a tray icon that informs you whether your SyncDaemon is actually connected or not. But it could just as well be an application to manage your shares. I implemented 2 of those using KDE or Qt technologies, namely the client structures and the authentication helper. I did not reimplement the SyncDaemon!

Let me quickly reason why redoing the SyncDaemon did not make much sense to me. The most obvious thing is that it does not depend on GNOME bits (for the better part) so there would be considerable less gain from rewriting it in Qt or KDE. Also since the SyncDaemon is essentially the central component with which everything stands or falls you do not want to have 2 implementations floating around as a fix applied to one will not be applied to the other and due to that increase potential issues the user has. Also the things SyncDaemon does are not trivial enough to assume everything will just work out after some initially high effort. That is why ubuntuone-kde is using the SyncDaemon from the regular ubuntuone-client which mostly works just fine since the daemon exposes all essential API via DBus which is very cross-platform and cross-desktop and cross-language (i.e. I like the approach).

What I created in my GSoC project can be divided into 3 parts:

  1. Authentication
  2. Libraries
  3. User interfaces

1. Authentication

What originally started off as ubuntuone-auth, when I began the project is now called ubuntu-sso-qt. The GNOME version (ubuntu-sso-client) just like the SyncDaemon exposes DBus interfaces through which authentication is handled. The basic work flow at this point is that a client requests authentication to an OAuth realm, SSO will look for an appropriate token entry in the supported keyrings and yield success if it found one, otherwise it will get an authentication process going and then yields success (or failure, depending on the outcome ;)). Suffice to say creating this was not terribly difficult to do but nonetheless very interesting.

One of the key aspects of this application is that it starts an own web server to make the OAuth experience better than what we are used to. I am sure everyone hit a dialog asking you to press OK once you have authenticated and authorized the application, upon which a browser opens with some website to ask you to authenticate something (you can see that sort of thing with the Facebook KIPI plugin for example). Instead of dropping such a dialog Ubuntu SSO will just launch a minimal HTTP server and tell the SSO server (or rather the Ubuntu One one) to send the browser back to that local server with OAuth token in the URL. The local server then extracts the OAuth information and redirects the browser the the proper Ubuntu One web interface. At this point OAuthentication is basically done and applications/libraries can use the token to access Ubuntu One.

In case you are interested, the HTTPDaemon is here.

The second nice thing about the Qt implementation is that it provides a plugin interface for secret storage services. That way it is possible to use any keyring/wallet as long as a plugin for it is available. Clearly this is a very good thing because without a lot of programming effort it now supports KWallet and GNOME Keyring while being itself implemented in Qt only, so technically one could also use it on OS X and Windows and one would just need to write additional plugins. It also helps with the fact that (hopefully) soon a new implementation will become available as proposed at freedesktop.org for which just another plugin is necessary.

And the best thing about my implementation is that, unlike the GNOME version, it is not written in Python and hence not eating away your memory right after startup for no good reason at all. In a KDE system it will use about 0.1 MiB when idle and about 1 MiB when doing an authentication. Finally, just to make my point of Python not necessarily being the better choice: the undynamic C++ version supports KWallet and GNOME Keyring while the Python version only does GNOME Keyring :P

More recently I became aware that the Ubuntu One team is working on enhancing the client to actually provide GUIs and what not, to totally eliminate the need for a browser window, which is probably a good idea. So, that is something one could look into for the future. Also it should be made central gateway to tokens, while right now each consumer will still lookup the received tokens directly via the keyring/wallet it would make a lot more sense to have Ubuntu SSO send the token in encrypted manner via DBus. I certainly think that using the proposed secrets storage API for inspiration on that would not be the worst of things.

2. Libraries

I created 2 libraries for use with Ubuntu One clients in a KDE system.

One is for read-only access to the REST server API with which one can get data about the authenticated user, his subscriptions, his authorized devices and his quota. This library is actually very simple to use but got some fine internals, in fact I spent hours just looking at the code and scrolling through its glory (because there was a bug ;-)). I think two things are very much worth mentioning about this library. First of all it is working asynchronous through and through (how else considering it is talking to a REST API). The consumer basically opens an interface, waits until it reports readyness (which it reaches once authorized and received a token) and then just tells it what data to get, one received the library emits appropriate signals and the consumer can use the data. Communication between library and server happens using JSON which gets nicely deserialized using QJSON. For more information on deserialization challenges with QJSON please revisit my blog post ♥ QJSON ♥, which explains how I solved (or rather worked around) the problem of deserializing nested objects from JSON to QObjects.
Currently the only user of this library is the information KCM (KDE configuration module) as seen in the General section of System Setting’s Ubuntu One category and in KInfoCenter.

Obviously what is missing here is some caching system. After all, since it is asynchronous it will take a considerable amount of time until the interface comes back with data, especially on slow connection. Yet for data like user information and quota it would absolutely make sense to cache them and use the cached data right away and if necessary emit update signals once the server returned.

The second library is used for accessing the local SyncDaemon and hence the real heart of ubuntone-kde. Central master piece of this library is a Qt DBus interface to the SyncDaemon which then gets wrapped into own API interfaces. The DBus interfaces do not get exposed directly to the library users but instead each method/signal needs an appropriate counterpart in the actual library. This was done to be able to ensure ABI and API stability from inside the library, so that applications built against libubuntuone-kde would not need to be recompiled just because an interface in the actual SyncDaemon was renamed, also it has the advantage of allowing us to apply Qt/KDE API naming standards and finally it makes it less problematic to work around naming clashes (for example one method of the SyncDaemon is called delete, which conflicts with C++’ delete).

What is used quite a bit in the library is the Qt DBus type system. The “problem” with the SyncDaemon’s DBus interfaces is that they use a lot of more-or-less complex data types that are almost all represented via QVariantMaps. Now as those maps are not exactly guaranteeing binary compatibility they need to be deserialized into proper objects, which is not that difficult in Qt but a bit nifty to get working properly. I will probably blog about this in detail sometime soon since I think that some things ought to be pointed out for future uses.
Currently the library implements complete status control over the SyncDaemon as well as control over shares and folders (folders in this context means folders that are synchronized with Ubuntu One).

Through all those library stuff I learned a lot about d and q pointers (never mind me if you do not know what those are) and also about library design in general.

3. User interfaces

Last but not least the user interfaces… Currently builing is a Status Notifier (system tray icon), a set of KCMs, a helper application to setup shares as well as a somewhat usable Dolphin plugin. The only really interesting thing here are the KCMs (also probably only because they contain a lot of stuff that could really go into a GUI library for Ubuntu One).

The Status Notifier is very straight forward since most of the things are happening in libubuntuone-kde. Basically what is going on is that the Status Notifier initializes the library and asks it to connect, then it just hooks up with appropriate signals from the status part of the library and takes appropriate actions upon incoming signals. Actions are mostly as simple as setting a new icon and tooltip and state. And that is essentially all that is going on there. Other than that it delegates all jobs to other parts. Lazy thing…

Ubuntuone-share is the name of a helper application that guides the user through setting up a new Ubuntu One share. It is a simple dialog in 3 steps. First the user must select the folder she wants to share, then she has to select the contact(s) to share this folder with and finally give the share a name and mark it readable or writable. Contact selection is provided by using an Akonadi contacts view/model so the user has direct access to all her Akonadi stored address book data which should make creating a share super easy.

KCMs are nice widgets primarily meant for configuration. KCMs are used in all sorts of places but mostly in System Settings and I want to be honest with you, I am pretty much in love with KCMs, because, since they are essentially just fancy QWidgets you can use them just like that and for example nest one KCM in another KCM without anyone ever knowing that it is a KCM of its own. Additionally KInfoCenter uses KCMs.
Suffice to say, I have plent of KCMs and I also nested one in anther … The General KCM which displays some general information about the user’s Ubuntu One account and throttling and autostart settings is in fact embedding the info KCM which is used in KInfoCenter and just adds throttling and autostart below the Info KCM, very funky I must say. But that is not all the Ubuntu One KCM part has to offer. In fact right now the KCM folder seems more like a GUI extension to libubuntuone-kde since it contains plain widgets for various types of data as well as custom model/views for folder management and the like. In general if I recall correctly all KCMs are using at least one plain widget and just create surrounding to it, so I expect that some of the stuff in there could be moved to a library in the future when more GUI classes grow.
Only through GSoC I fell in love with KCMs :-*

I did not write about the Dolphin plugin in my previous post, because I do not feel that it is anywhere near production quality, not only did I not test it a lot but also is it a bit limited and hackish. This becomes very apparent if you look at the header file … yes, the Ubuntu One Dolphin plugin is a VCS plugin ;) That is however more because Dolphin does currently not support any other plugin type, but I also must say the difference between Ubuntu One and a VCS are not that big from a file manager point of view. I will try to get a more appropriate plugin interface going for KDE SC 4.6, so that one can create proper file sync plugins. For the time being however I cannot advertise the plugin for ethic reasons.

There is also a lot things left to be done in the user interface department. For example Share management is not yet implemented also general UI love is always needed.

If you feel like helping out, please mail to kubuntu-devel@lists.ubuntu.com or ping me on IRC :)

About these ads

7 thoughts on “Ubuntu One – Technical Aspects”

  1. I really wish Kubuntu didn’t feel and act like the red-headed stepchild of the Ubuntu family. How many KDE fans really take Kubuntu seriously?

  2. Great job and an interesting read, even for a non-programmer. Keep up the great work – I can’t wait to try it out (can’t right now since I’m on Mint 9 KDE which is based on 10.04…)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s