On the beauty of libarchive

In your applications, you might have to deal with compressed files: ISO images of installers, e-book or e-comic types based on ZIP files, video DVD images.

libarchive makes things easier by allowing you not to have to deal with external commands to extract those few files you care about.

The API feels a bit antiquated, compared to using GLib/GIO for files handling, but it's generally easier than dealing with potential security issues launching external tools, or even dealing with shell argv quoting.

Examples

totem-pl-parser uses libarchive to determine what type of video disc image are hidden inside an ISO image.

gnome-epub-thumbnailer (as well as its siblings, the Krita and OpenRaster thumbnailers I talked about more recently) uses the ZIP handling to extract particular files, and figure out which file is the cover image.

Other uses and limitations

Boxes could use libarchive to extract files from ISO images for its auto-installer, evince could use it to handle CBZ e-comics.

There's a couple of limitations though. ISO support doesn't handle UDF images (which just means weird filenames, not inaccessible files), and RAR support is still quite young.

I hope that this post can spur on bug fixes for the RAR support, new UDF support, or even a GIO-style wrapper around the library.

The upstream authors have been particularly good at fixing bugs that only showed themselves with broken files, and I'd like to thank them for their very useful work.

Send to Pocket using GNOME

I'm a big fan of Pocket, the "Read Later" service.

I regularly save blog posts, videos, tweets and articles to read later, and then consume them on my iPad 1 (I hope they don't cut off the old app yet), my phone, or using the Kobo e-book reader.

So it's only normal that I'd try to make my experience of using it with GNOME, a bit more integrated than a simple Javascript bookmarklet in my browser.

Online Accounts support

The first step was writing the GNOME Online Accounts support for Pocket accounts. This isn't quite finished, and there was some ugliness due to the way Pocket's authentication works. It's not complicated, but it's neither OAuth 1, nor OAuth 2.

The patch also adds a new type of service that you can toggle on/off in the settings, see about that below.

Browser support

I don't really read articles on my laptop, and I'm Linux-tablet-less (the WeTab you might remember is now with gnome-shell developer Florian Müllner). So my main concern was saving articles to the service.

The UI is still a bit in flux, but this is what it looks like on my machine right now. In the future, we might want to try and show the status of the page (has it already been saved?) or a way to edit tags after having added the page.

Other services

There are other services similar to Pocket, such as Instapaper, or even the free and open source Poche.

The good news is that adding support for those services should be easier, as you'd only need to add a new gnome-online-accounts backend, and write a little bit of backend code in Epiphany (eg. 2 out of the 4 steps in adding support for Pocket).

The infrastructure is, or more accurately, will soon be there.

Update: The Epiphany/Web bug for the browser integration is here. Doh!

Week-end hacks #3

Mo' thumbnailers

Simple enough, but I still managed to make 2 broken releases ;)

There's now a Krita and OpenRaster/MyPaint thumbnailer in GNOME git, and as tarball releases.

Freebox TV streaming

My ISP, Free, gives its customers an access point and TV set-top box. The access point is also a UPnP, Samba, AFP server and Bittorrent client amongst other things.

It's also responsible for handling IPTV, streaming to the TV set-top box. You can also watch TV on your laptop streaming RTSP to local clients.

This Grilo plugin, once all the issues are fixed, should allow us to show the TV channels in Totem.

Bluetooth panel redesign

Another week, another panel refresh.

Rather than the 2-pane approach, and a separate setup interface we used to have, we've gone for a single pane device list, as you've probably seen on your smartphone.

We also do away with the "Discoverable" switch (your computer will be visible when this panel is opened, invisible if not), and nearby devices will show up at the bottom of the list. Simply click on one to set it up.

Clicking on an already setup device will bring up the properties, allowing you to connect to the device if necessary, or link to related preferences.

Finally, the biggest part of the work was making sure that the new setup mechanism worked at least as well as the stand-alone wizard. This means that I got 17 of my most representative devices out, and set up every single one of them. Edge case.

There's a good chance that we'll make some additional, minor, adjustments to the wording, spacing and behaviour of this panel before the GNOME 3.12 release. I'd particularly like to make clicking on a device connect to it if already setup and offer some other way of accessing properties.

Bluetooth file sharing (ObexPush) in GNOME 3.10

As you might remember, GNOME 3.10 switched to using BlueZ 5.x as its Bluetooth backend.

Switching to BlueZ 5.x meant that the old obex-data-server (which was used in both the gvfs ObexFTP backend, and gnome-user-share) couldn't be used anymore. The previously stand-alone obexd was to be used.

Its API is quite different, and it obviously didn't get much testing apart from its target use case, the single-user phone case.

I fixed a number of bugs this week-end, which should make Obex Push server-side (sending files from your phone to your computer) work as expected.

Distributors' homework

First, distributors will need to do a bit of work for you:
- Ship and apply this (not yet upstreamed) patch if you don't have a systemd-based session, so that obexd can be started via D-Bus.
- And ship this patch to have obexd write to the user's cache dir by default.

With both of those patches to BlueZ and gnome-user-share 3.10.1, you should be golden.

Note that the first patch is also required if you want to send files using bluetooth-sendto.

ObexFTP

You'll notice that we didn't mention ObexFTP yet, but we'll do, one last time. ObexFTP support client side hasn't seen any updates for a couple of years, and the server side support for it in obexd didn't match our expectations (such as the inability to kill existing, already made connections).

So ObexFTP support was never finished porting and re-enabled in gnome-user-share. And given that apart from computers and very few phones, the client side support was lacking, we decided to kill the support for it in gnome-user-share.

TL;DR

ObexPush server support is fixed in gnome-user-share 3.10.1, and ObexFTP server support is gone.

Reducing wake-ups, the 2013 edition

While doing work on UPower, trying to reduce its wake-ups when idle, I realised that a couple of applications on my desktop were waking up far more than should be necessary.

Detecting wake-ups

First, we need to detect wake-ups. This is a fairly well-known, and old, process, using the venerable PowerTop.

# powertop --html=/tmp/foo.html --time=60

You can increase the number of seconds to get a more realistic view of your machine's idleness, but this is enough to show the main culprits.

In my case, evolution, gnote and devhelp were all waking up about 30 times a second whilst mostly idle. Evolution might be an outlier, as it also talks to the network, and is a bigger application to debug, so I started with devhelp.

$ strace -vvv -p `pidof devhelp`
Process 19069 attached
restart_syscall(<... resuming interrupted call ...>) = 0
recvfrom(6, 0x23fece4, 4096, 0, 0, 0)   = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=3, events=POLLIN}], 3, 17) = 0 (Timeout)
recvfrom(6, 0x23fece4, 4096, 0, 0, 0)   = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=3, events=POLLIN}], 3, 17) = 0 (Timeout)

And the screen fills up with EAGAIN errors. This looks a lot like a a timeout being called too often.

Debugging wake-ups

I started sprinkling debug in g_main_context_prepare(), the function that prepares the various timeout and idle sources for dispatch, and calculates the timeouts for each poll() operation.

Something like:

if (source_timeout > 0 && source_timeout <= 20)

  g_message ("Source '%s' has very low timeout %d", g_source_get_name (source), source_timeout);

The problem is we end up getting a null source name for almost all of the sources. This is where g_source_set_name() and its sibling g_source_set_name_by_id() come in handy.

timeout_id = g_timeout_add (timeout, myfunction, mydata);

g_source_set_name_by_id (timeout_id, "[module-name] myfunction");

And we start doing that all over GTK+. As you can see from the patches in the bug, there's not just timeouts added by g_timeout_add() that we need to name.

In custody

The huge amount of debug shown when running our application with the gmain.c debug above tells us:

GLib-Message: Source 0x2d4f380 '[gtk+] gdk_frame_clock_paint_idle' has very low timeout 17

even when the window doesn't change, is in the background, and not updating. About 30 times a second.

Who are you gonna call?

Or by whom have you been called, rather. This is a small section of my ~/.gdbinit which will break on a particular function, print a backtrace, and continue. It makes it easier to interact with the logs after the fact, especially if they are calls that happen often and you're not interested in all the calls.

set breakpoint pending on

break gdk_frame_clock_begin_updating

commands

bt

continue

end

We did the same for gdk_frame_clock_begin_updating and found a backtrace similar to the one in Bugzilla. We only needed to start reading some code after that, and figuring out what was going on. The result was a bug in GTK+, likely a regression from GTK+ 3.8.

Your laptop should last a bit longer when the updates hit.

TL;DR

Name your timeouts with g_source_set_name_by_id(), run powertop, and file bugs against broken applications.

Update: Fixed powertop command-line.

More power management changes

As is becoming common, we will have some more power management changes in GNOME 3.12, though those changes will also affect other desktops, whether they use UPower's D-Bus interface, or libupower-glib, the helper library.

The goals of the exercise were simple:

• reduce wake-ups on the daemon and on the client side
• reduce code duplication amongst desktop environments, and even within the same environment (composite battery, anyone?)
• moving some policy actions to a lower level (one could not request hibernation or suspend when multiple users were logged in without interaction and passwords)

All those changes are now in UPower master ready for testing.

Out with the old

The deprecated interfaces for Suspend, Hibernate, etc. are finally removed, after being obsoleted by logind. We've also removed the QoS interface that nobody was using, and the out-dated battery recall support. It's not that batteries don't explode any more, it's that they don't all come from known-bad batches.

In with the new

We have 2 new properties on each of the devices.

WarningLevel which uses daemon-side configurations to tell you whether a device's battery level is low, critically low, or whether we're about to take action on that critical level.

We also have IconName, which replaces some cut'n'pasted code between desktop components. If your desktop environment has many more icons for all types of devices on low battery, for example, you can ignore this property and use the code you always have.

Using those new properties usefully is the new DisplayDevice object. It groups all the batteries and UPSes in the daemon into one, easy to use object that you can use to display a single status icon in your shell chrome. Obviously, if you want to show more devices, the individual batteries and UPSes are still available through the usual means. And it obviously has the 2 new properties mentioned above, so your session daemon can get told when to show notifications for low batteries.

And finally, using that new combined DisplayDevice is the critical battery action policies. As mentioned above, multi-user systems could not hibernate without requiring the user to enter an administrator password, which is less than convenient when your machine is running out of UPS power fast. The configuration for that policy is now in the daemon itself, with sane defaults, and it will hibernate the machine for you.

And to the modernisation

libupower-glib now uses GDBus, even if the daemon doesn't. The daemon however sends PropertiesChanged signals which means that modern D-Bus bindings will automatically get the new values for properties, instead of polling the daemon. The DeviceChanged and Changed signals have thus been removed.

API changes

They are numerous, too many to mention here. I've posted to the device-kit mailing-list with a list of changes that were made, reply there if you have any questions regarding using UPower in your application or session daemons.

Miscellaneous

systemd >= 207 will save your brightness settings across reboots, and the upcoming systemd 209 will have support for saving keyboard backlight across reboots.

I've made attempts at supporting Intel Rapid Start in systemd, but this will actually require kernel changes. Hopefully we should be able to land this by the time GNOME 3.12 is released.