KDE neon With Testing Translations

For the longest time, the plan was to equip KDE neon’s Developer Editions with translations. As the Developer Editions are built directly from our Git repositories and we do not maintain translations alongside the source code, there is a bit of a problem as the build somehow needs to bridge the gap between code and translations.

It’s fortunate that I also happen to work on ReleaseMe, a KDE tarball release application, and rebuilt it from scratch years ago already, so it supports third party usage of some of its functionality.

At this year’s FOSDEM, Plasma developer David Edmundson asked for translations in the Developer Editions. And, so, here we are. Both KDE neon Developer Editions now include translations live from our Subversion repository. They also include our x-test language allowing you to easily find improperly internationalized strings. Coverage is currently limited to KDE Frameworks and Plasma software.

The majority of tech to accomplish this is hidden in the internals of ReleaseMe itself. On the high-level this entails nothing more than resolving the KDE project and then getting its translations into the Git tree.

projects = ReleaseMe::Project.from_repo_url(url)
unless projects.size == 1
  raise "failed to resolve project #{repo_name} :: #{projects}"
end
project = projects[0]

l10n = ReleaseMe::L10n.new(l10n_origin, project.identifier,
project.i18n_path)
l10n.default_excluded_languages = [] # Include even x-test.
l10n.get(Dir.pwd)

(Underneath there’s, of course, lots of fiddly nonsense going on ;))

Enjoy!

Aptly Systemd Socket Activation

In the previous blog post I talked about Unix domain socket enablement in Aptly, a very popular deb repository management solution. In addition to adding socket support, I’ve also added support for systemd socket activation.

Systemd’s socket activation is an inordinately fancy feature of systemd where it will take over socket creation and management before the actual service is running. It ensures that incoming requests aren’t lost if the service crashes or isn’t started yet. When one has the opportunity to use systemd, letting it do socket activation for Aptly is super handy as it also gives easy access to chown capabilities.

Using socket activation requires you to create two systemd units. A service unit, describing the aptly service itself, and a socket unit, for listening for traffic. Examples files may look like this:

# aptly.service
[Unit]
Description=Aptly archive service

[Service]
ExecStart=/home/archive/bin/aptly api serve
WorkingDirectory=/home/archive
Restart=always
# aptly.socket
[Unit]
Description=Aptly archive service socket

[Socket]
ListenStream=/home/archive/aptly.sock
SocketMode=0660
SocketUser=archive
SocketGroup=archivesocket

[Install]
WantedBy=sockets.target

Enabling and starting the socket unit will create the socket file at /home/archive/aptly.sock and start the service upon incoming traffic, which you could cause, for example, using curl --unix-socket /home/archive/aptly.sock http:/api/version.

Fairly exciting already. But if you’ve previously been listening on a TCP port you may wish to establish a smoother migration path without having to change your entire infrastructure to work with sockets all at once. Aptly however only supports listening on one socket, so you can’t just have it listen on both a TCP socket and a unix socket. Fortunately, systemd also has a neat solution for this, called systemd-socket-proxyd. Using the socket activation systemd itself can proxy traffic from a TCP socket to our new aptly socket file.

Again example units for illustration:

# aptly-proxy.service
[Unit]
Description=Aptly Socket (Proxy Listener)

[Service]
ExecStart=/lib/systemd/systemd-socket-proxyd /home/archive/aptly.sock
PrivateTmp=yes
# aptly-proxy.socket
[Unit]
Description=Aptly Socket (Proxy Listener)
After=aptly.socket

[Socket]
ListenStream=127.0.0.1:8080

[Install]
WantedBy=sockets.target

When enabling and starting this socket unit systemd will listen on 127.0.0.1:8080 for traffic and proxy it through systemd-socket-proxyd to our actual Aptly socket file. At this point Aptly will be able to receive traffic from both the unix socket and the TCP socket (although of course effectively Aptly is only listening on the unix socket).

Now if only systemd could play audio it’d be quite the bundle of usefulness 😉

Aptly API over Unix Sockets

ruby

Last week I’ve added a bunch of code to enable use of Aptly’s API over Unix domain sockets.

Aptly is a Debian repository management software we use for KDE neon‘s deb repository. It is one of the more popular software for repo management as it is easy to deploy and features a handy REST API.

One of the more annoying shortcomings of Aptly’s API is the fact that it has no authentication mechanism. This makes using the API from remote hosts a fairly unwieldy affair as you first have to harden the beast.

Your options are basically limited to either run Apache (or some other HTTP server) that proxies Aptly’s API and adds HTTPAuth to it, or you pipe everything through SSH. Both of which are fine solutions, but they hide a fairly awkward security problem.

Aptly would be listening to a port on localhost and the lack of access control means that if any user gets compromised the Aptly instance is effectively compromised as well. It also means that if you do not read the documentation and explicitly set Aptly to listen on localhost only, you end up with a publicly writable Aptly instance. Ugh.

There is a really simple way out though: listening to a Unix domain socket rather than a TCP socket. Unix sockets are pseudo-files and therefore file access control applies. This makes access control easy to manage through the system’s user/group/all permissions of the socket file.

srw-rw----  1 user group    0 Mar  6 14:23 aptly.sock

Making use of this is super easy too!

Socket Listening

Aptly API’s serve invocation now supports a URI scheme for Unix sockets.

aptly api serve -listen="unix://tmp/aptly.sock"

This will (re)create a Unix domain socket at the defined path and listen to it.

Using

Using this you can restrict full access to the API service down to a user level. More fine-grained control still needs a frontend add-on such as an HTTP server with HTTPAuth, but for most purposes a per-user access control should be sufficient and vastly more secure than the previous TCP socket binding.

With OpenSSH’s direct-streamlocal@openssh.com SSH protocol extension you can then even communicate with a remote Aptly instance through a forwarded socket.

To enable myself to use this I’ve also added support for this to the Ruby net/ssh gem and my aptly-api gem.

Using latest master of everything involved we can now serve the API on a socket and then access that remotely via SSH like so:

require 'aptly'
require 'net/ssh'

Thread.new do
  Net::SSH.start('remotehost', 'user') do |ssh|
    ssh.forward.local_socket('/tmp/local.sock', '/home/user/aptly.sock')
    loop { ssh.process(0.1) }
  end
end

sleep 2 # wait for open

Aptly.configure do |c|
  c.uri = URI.parse('unix:///tmp/local.sock')
end
Aptly::Repository.list

Lovely, isn’t it?