GNOME Web hacks

After the Pocket integration earlier this week, I've cooked up or landed a couple more patches.

Mailman passwords

Those darn mailman administrator passwords. There's no "forgot password" button and the password is shared amongst all the administrators of a mailing-list. Now Epiphany remembers them and I don't need to go through my inbox trying to recover them.

Glow button fixes

Epiphany was using Totem's glow button, a button which glows a couple of times to bring your attention to it. It's used in Totem's browser plugin, to show that it's ready to play, and Epiphany to show finished downloads. It broke due to GTK+ changes, and it's now reimplemented using CSS animations instead of horrible hacks.

Analytics removal

A little privacy hack, inspired by the PureURL Firefox extension. This removes tracking information from URLs when tracking is disabled in the preferences.

On the beauty of libarchive

In your applications, you might have to deal with compressed files: ISO images of installers, e-book or e-comic types based on ZIP files, video DVD images.

libarchive makes things easier by allowing you not to have to deal with external commands to extract those few files you care about.

The API feels a bit antiquated, compared to using GLib/GIO for files handling, but it's generally easier than dealing with potential security issues launching external tools, or even dealing with shell argv quoting.

Examples

totem-pl-parser uses libarchive to determine what type of video disc image are hidden inside an ISO image.

gnome-epub-thumbnailer (as well as its siblings, the Krita and OpenRaster thumbnailers I talked about more recently) uses the ZIP handling to extract particular files, and figure out which file is the cover image.

Other uses and limitations

Boxes could use libarchive to extract files from ISO images for its auto-installer, evince could use it to handle CBZ e-comics.

There's a couple of limitations though. ISO support doesn't handle UDF images (which just means weird filenames, not inaccessible files), and RAR support is still quite young.

I hope that this post can spur on bug fixes for the RAR support, new UDF support, or even a GIO-style wrapper around the library.

The upstream authors have been particularly good at fixing bugs that only showed themselves with broken files, and I'd like to thank them for their very useful work.

Send to Pocket using GNOME

I'm a big fan of Pocket, the "Read Later" service.

I regularly save blog posts, videos, tweets and articles to read later, and then consume them on my iPad 1 (I hope they don't cut off the old app yet), my phone, or using the Kobo e-book reader.

So it's only normal that I'd try to make my experience of using it with GNOME, a bit more integrated than a simple Javascript bookmarklet in my browser.

Online Accounts support

The first step was writing the GNOME Online Accounts support for Pocket accounts. This isn't quite finished, and there was some ugliness due to the way Pocket's authentication works. It's not complicated, but it's neither OAuth 1, nor OAuth 2.

The patch also adds a new type of service that you can toggle on/off in the settings, see about that below.

Browser support

I don't really read articles on my laptop, and I'm Linux-tablet-less (the WeTab you might remember is now with gnome-shell developer Florian Müllner). So my main concern was saving articles to the service.

The UI is still a bit in flux, but this is what it looks like on my machine right now. In the future, we might want to try and show the status of the page (has it already been saved?) or a way to edit tags after having added the page.

Other services

There are other services similar to Pocket, such as Instapaper, or even the free and open source Poche.

The good news is that adding support for those services should be easier, as you'd only need to add a new gnome-online-accounts backend, and write a little bit of backend code in Epiphany (eg. 2 out of the 4 steps in adding support for Pocket).

The infrastructure is, or more accurately, will soon be there.

Update: The Epiphany/Web bug for the browser integration is here. Doh!

Week-end hacks #3

Mo' thumbnailers

Simple enough, but I still managed to make 2 broken releases ;)

There's now a Krita and OpenRaster/MyPaint thumbnailer in GNOME git, and as tarball releases.

Freebox TV streaming

My ISP, Free, gives its customers an access point and TV set-top box. The access point is also a UPnP, Samba, AFP server and Bittorrent client amongst other things.

It's also responsible for handling IPTV, streaming to the TV set-top box. You can also watch TV on your laptop streaming RTSP to local clients.

This Grilo plugin, once all the issues are fixed, should allow us to show the TV channels in Totem.

Bluetooth panel redesign

Another week, another panel refresh.

Rather than the 2-pane approach, and a separate setup interface we used to have, we've gone for a single pane device list, as you've probably seen on your smartphone.

We also do away with the "Discoverable" switch (your computer will be visible when this panel is opened, invisible if not), and nearby devices will show up at the bottom of the list. Simply click on one to set it up.

Clicking on an already setup device will bring up the properties, allowing you to connect to the device if necessary, or link to related preferences.

Finally, the biggest part of the work was making sure that the new setup mechanism worked at least as well as the stand-alone wizard. This means that I got 17 of my most representative devices out, and set up every single one of them. Edge case.

There's a good chance that we'll make some additional, minor, adjustments to the wording, spacing and behaviour of this panel before the GNOME 3.12 release. I'd particularly like to make clicking on a device connect to it if already setup and offer some other way of accessing properties.

Bluetooth file sharing (ObexPush) in GNOME 3.10

As you might remember, GNOME 3.10 switched to using BlueZ 5.x as its Bluetooth backend.

Switching to BlueZ 5.x meant that the old obex-data-server (which was used in both the gvfs ObexFTP backend, and gnome-user-share) couldn't be used anymore. The previously stand-alone obexd was to be used.

Its API is quite different, and it obviously didn't get much testing apart from its target use case, the single-user phone case.

I fixed a number of bugs this week-end, which should make Obex Push server-side (sending files from your phone to your computer) work as expected.

Distributors' homework

First, distributors will need to do a bit of work for you:
- Ship and apply this (not yet upstreamed) patch if you don't have a systemd-based session, so that obexd can be started via D-Bus.
- And ship this patch to have obexd write to the user's cache dir by default.

With both of those patches to BlueZ and gnome-user-share 3.10.1, you should be golden.

Note that the first patch is also required if you want to send files using bluetooth-sendto.

ObexFTP

You'll notice that we didn't mention ObexFTP yet, but we'll do, one last time. ObexFTP support client side hasn't seen any updates for a couple of years, and the server side support for it in obexd didn't match our expectations (such as the inability to kill existing, already made connections).

So ObexFTP support was never finished porting and re-enabled in gnome-user-share. And given that apart from computers and very few phones, the client side support was lacking, we decided to kill the support for it in gnome-user-share.

TL;DR

ObexPush server support is fixed in gnome-user-share 3.10.1, and ObexFTP server support is gone.

Reducing wake-ups, the 2013 edition

While doing work on UPower, trying to reduce its wake-ups when idle, I realised that a couple of applications on my desktop were waking up far more than should be necessary.

Detecting wake-ups

First, we need to detect wake-ups. This is a fairly well-known, and old, process, using the venerable PowerTop.

# powertop --html=/tmp/foo.html --time=60

You can increase the number of seconds to get a more realistic view of your machine's idleness, but this is enough to show the main culprits.

In my case, evolution, gnote and devhelp were all waking up about 30 times a second whilst mostly idle. Evolution might be an outlier, as it also talks to the network, and is a bigger application to debug, so I started with devhelp.

$ strace -vvv -p `pidof devhelp`
Process 19069 attached
restart_syscall(<... resuming interrupted call ...>) = 0
recvfrom(6, 0x23fece4, 4096, 0, 0, 0)   = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=3, events=POLLIN}], 3, 17) = 0 (Timeout)
recvfrom(6, 0x23fece4, 4096, 0, 0, 0)   = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=3, events=POLLIN}], 3, 17) = 0 (Timeout)

And the screen fills up with EAGAIN errors. This looks a lot like a a timeout being called too often.

Debugging wake-ups

I started sprinkling debug in g_main_context_prepare(), the function that prepares the various timeout and idle sources for dispatch, and calculates the timeouts for each poll() operation.

Something like:

if (source_timeout > 0 && source_timeout <= 20)

  g_message ("Source '%s' has very low timeout %d", g_source_get_name (source), source_timeout);

The problem is we end up getting a null source name for almost all of the sources. This is where g_source_set_name() and its sibling g_source_set_name_by_id() come in handy.

timeout_id = g_timeout_add (timeout, myfunction, mydata);

g_source_set_name_by_id (timeout_id, "[module-name] myfunction");

And we start doing that all over GTK+. As you can see from the patches in the bug, there's not just timeouts added by g_timeout_add() that we need to name.

In custody

The huge amount of debug shown when running our application with the gmain.c debug above tells us:

GLib-Message: Source 0x2d4f380 '[gtk+] gdk_frame_clock_paint_idle' has very low timeout 17

even when the window doesn't change, is in the background, and not updating. About 30 times a second.

Who are you gonna call?

Or by whom have you been called, rather. This is a small section of my ~/.gdbinit which will break on a particular function, print a backtrace, and continue. It makes it easier to interact with the logs after the fact, especially if they are calls that happen often and you're not interested in all the calls.

set breakpoint pending on

break gdk_frame_clock_begin_updating

commands

bt

continue

end

We did the same for gdk_frame_clock_begin_updating and found a backtrace similar to the one in Bugzilla. We only needed to start reading some code after that, and figuring out what was going on. The result was a bug in GTK+, likely a regression from GTK+ 3.8.

Your laptop should last a bit longer when the updates hit.

TL;DR

Name your timeouts with g_source_set_name_by_id(), run powertop, and file bugs against broken applications.

Update: Fixed powertop command-line.