Saturday 21 December 2013

GNOME Web hacks

After the Pocket integration earlier this week, I've cooked up or landed a couple more patches.

Mailman passwords

Those darn mailman administrator passwords. There's no "forgot password" button and the password is shared amongst all the administrators of a mailing-list. Now Epiphany remembers them and I don't need to go through my inbox trying to recover them.

Glow button fixes

Epiphany was using Totem's glow button, a button which glows a couple of times to bring your attention to it. It's used in Totem's browser plugin, to show that it's ready to play, and Epiphany to show finished downloads. It broke due to GTK+ changes, and it's now reimplemented using CSS animations instead of horrible hacks.

Analytics removal

A little privacy hack, inspired by the PureURL Firefox extension. This removes tracking information from URLs when tracking is disabled in the preferences.

Thursday 19 December 2013

On the beauty of libarchive

In your applications, you might have to deal with compressed files: ISO images of installers, e-book or e-comic types based on ZIP files, video DVD images.

libarchive makes things easier by allowing you not to have to deal with external commands to extract those few files you care about.

The API feels a bit antiquated, compared to using GLib/GIO for files handling, but it's generally easier than dealing with potential security issues launching external tools, or even dealing with shell argv quoting.

Examples

totem-pl-parser uses libarchive to determine what type of video disc image are hidden inside an ISO image.

gnome-epub-thumbnailer (as well as its siblings, the Krita and OpenRaster thumbnailers I talked about more recently) uses the ZIP handling to extract particular files, and figure out which file is the cover image.

Other uses and limitations

Boxes could use libarchive to extract files from ISO images for its auto-installer, evince could use it to handle CBZ e-comics.

There's a couple of limitations though. ISO support doesn't handle UDF images (which just means weird filenames, not inaccessible files), and RAR support is still quite young.

I hope that this post can spur on bug fixes for the RAR support, new UDF support, or even a GIO-style wrapper around the library.

The upstream authors have been particularly good at fixing bugs that only showed themselves with broken files, and I'd like to thank them for their very useful work.

Monday 16 December 2013

Send to Pocket using GNOME

I'm a big fan of Pocket, the "Read Later" service.

I regularly save blog posts, videos, tweets and articles to read later, and then consume them on my iPad 1 (I hope they don't cut off the old app yet), my phone, or using the Kobo e-book reader.

So it's only normal that I'd try to make my experience of using it with GNOME, a bit more integrated than a simple Javascript bookmarklet in my browser.

Online Accounts support

The first step was writing the GNOME Online Accounts support for Pocket accounts. This isn't quite finished, and there was some ugliness due to the way Pocket's authentication works. It's not complicated, but it's neither OAuth 1, nor OAuth 2.

The patch also adds a new type of service that you can toggle on/off in the settings, see about that below.



Browser support

I don't really read articles on my laptop, and I'm Linux-tablet-less (the WeTab you might remember is now with gnome-shell developer Florian Müllner). So my main concern was saving articles to the service.

The UI is still a bit in flux, but this is what it looks like on my machine right now. In the future, we might want to try and show the status of the page (has it already been saved?) or a way to edit tags after having added the page.



Other services

There are other services similar to Pocket, such as Instapaper, or even the free and open source Poche.

The good news is that adding support for those services should be easier, as you'd only need to add a new gnome-online-accounts backend, and write a little bit of backend code in Epiphany (eg. 2 out of the 4 steps in adding support for Pocket).

The infrastructure is, or more accurately, will soon be there.

Update: The Epiphany/Web bug for the browser integration is here. Doh!

Saturday 14 December 2013

Week-end hacks #3

Mo' thumbnailers

Simple enough, but I still managed to make 2 broken releases ;)

There's now a Krita and OpenRaster/MyPaint thumbnailer in GNOME git, and as tarball releases.

Freebox TV streaming

My ISP, Free, gives its customers an access point and TV set-top box. The access point is also a UPnP, Samba, AFP server and Bittorrent client amongst other things.

It's also responsible for handling IPTV, streaming to the TV set-top box. You can also watch TV on your laptop streaming RTSP to local clients.

This Grilo plugin, once all the issues are fixed, should allow us to show the TV channels in Totem.

Monday 9 December 2013

Bluetooth panel redesign

Another week, another panel refresh.

Rather than the 2-pane approach, and a separate setup interface we used to have, we've gone for a single pane device list, as you've probably seen on your smartphone.




We also do away with the "Discoverable" switch (your computer will be visible when this panel is opened, invisible if not), and nearby devices will show up at the bottom of the list. Simply click on one to set it up.

Clicking on an already setup device will bring up the properties, allowing you to connect to the device if necessary, or link to related preferences.


Finally, the biggest part of the work was making sure that the new setup mechanism worked at least as well as the stand-alone wizard. This means that I got 17 of my most representative devices out, and set up every single one of them. Edge case.



There's a good chance that we'll make some additional, minor, adjustments to the wording, spacing and behaviour of this panel before the GNOME 3.12 release. I'd particularly like to make clicking on a device connect to it if already setup and offer some other way of accessing properties.

Tuesday 12 November 2013

Bluetooth file sharing (ObexPush) in GNOME 3.10

As you might remember, GNOME 3.10 switched to using BlueZ 5.x as its Bluetooth backend.

Switching to BlueZ 5.x meant that the old obex-data-server (which was used in both the gvfs ObexFTP backend, and gnome-user-share) couldn't be used anymore. The previously stand-alone obexd was to be used.

Its API is quite different, and it obviously didn't get much testing apart from its target use case, the single-user phone case.

I fixed a number of bugs this week-end, which should make Obex Push server-side (sending files from your phone to your computer) work as expected.

Distributors' homework

First, distributors will need to do a bit of work for you:
- Ship and apply this (not yet upstreamed) patch if you don't have a systemd-based session, so that obexd can be started via D-Bus.
- And ship this patch to have obexd write to the user's cache dir by default.

With both of those patches to BlueZ and gnome-user-share 3.10.1, you should be golden.

Note that the first patch is also required if you want to send files using bluetooth-sendto.

ObexFTP

You'll notice that we didn't mention ObexFTP yet, but we'll do, one last time. ObexFTP support client side hasn't seen any updates for a couple of years, and the server side support for it in obexd didn't match our expectations (such as the inability to kill existing, already made connections).

So ObexFTP support was never finished porting and re-enabled in gnome-user-share. And given that apart from computers and very few phones, the client side support was lacking, we decided to kill the support for it in gnome-user-share.



TL;DR

ObexPush server support is fixed in gnome-user-share 3.10.1, and ObexFTP server support is gone.

Tuesday 22 October 2013

Reducing wake-ups, the 2013 edition

While doing work on UPower, trying to reduce its wake-ups when idle, I realised that a couple of applications on my desktop were waking up far more than should be necessary.

Detecting wake-ups

First, we need to detect wake-ups. This is a fairly well-known, and old, process, using the venerable PowerTop.

# powertop --html=/tmp/foo.html --time=60

You can increase the number of seconds to get a more realistic view of your machine's idleness, but this is enough to show the main culprits.

In my case, evolution, gnote and devhelp were all waking up about 30 times a second whilst mostly idle. Evolution might be an outlier, as it also talks to the network, and is a bigger application to debug, so I started with devhelp.

$ strace -vvv -p `pidof devhelp`
Process 19069 attached
restart_syscall(<... resuming interrupted call ...>) = 0
recvfrom(6, 0x23fece4, 4096, 0, 0, 0)   = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=3, events=POLLIN}], 3, 17) = 0 (Timeout)
recvfrom(6, 0x23fece4, 4096, 0, 0, 0)   = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=3, events=POLLIN}], 3, 17) = 0 (Timeout)

And the screen fills up with EAGAIN errors. This looks a lot like a a timeout being called too often.

Debugging wake-ups

I started sprinkling debug in g_main_context_prepare(), the function that prepares the various timeout and idle sources for dispatch, and calculates the timeouts for each poll() operation.

Something like:
if (source_timeout > 0 && source_timeout <= 20)
  g_message ("Source '%s' has very low timeout %d", g_source_get_name (source), source_timeout);

The problem is we end up getting a null source name for almost all of the sources. This is where g_source_set_name() and its sibling g_source_set_name_by_id() come in handy.

timeout_id = g_timeout_add (timeout, myfunction, mydata);
g_source_set_name_by_id (timeout_id, "[module-name] myfunction");

And we start doing that all over GTK+. As you can see from the patches in the bug, there's not just timeouts added by g_timeout_add() that we need to name.

In custody

The huge amount of debug shown when running our application with the gmain.c debug above tells us:
GLib-Message: Source 0x2d4f380 '[gtk+] gdk_frame_clock_paint_idle' has very low timeout 17
even when the window doesn't change, is in the background, and not updating. About 30 times a second.

Who are you gonna call?

Or by whom have you been called, rather. This is a small section of my ~/.gdbinit which will break on a particular function, print a backtrace, and continue. It makes it easier to interact with the logs after the fact, especially if they are calls that happen often and you're not interested in all the calls.

set breakpoint pending on
break gdk_frame_clock_begin_updating
commands
bt
continue
end

We did the same for gdk_frame_clock_begin_updating and found a backtrace similar to the one in Bugzilla. We only needed to start reading some code after that, and figuring out what was going on. The result was a bug in GTK+, likely a regression from GTK+ 3.8.

Your laptop should last a bit longer when the updates hit.

TL;DR

Name your timeouts with g_source_set_name_by_id(), run powertop, and file bugs against broken applications.

Update: Fixed powertop command-line.

Thursday 17 October 2013

More power management changes

As is becoming common, we will have some more power management changes in GNOME 3.12, though those changes will also affect other desktops, whether they use UPower's D-Bus interface, or libupower-glib, the helper library.

The goals of the exercise were simple:

  • reduce wake-ups on the daemon and on the client side
  • reduce code duplication amongst desktop environments, and even within the same environment (composite battery, anyone?)
  • moving some policy actions to a lower level (one could not request hibernation or suspend when multiple users were logged in without interaction and passwords)
All those changes are now in UPower master ready for testing.

Out with the old

The deprecated interfaces for Suspend, Hibernate, etc. are finally removed, after being obsoleted by logind. We've also removed the QoS interface that nobody was using, and the out-dated battery recall support. It's not that batteries don't explode any more, it's that they don't all come from known-bad batches.

In with the new

We have 2 new properties on each of the devices.

WarningLevel which uses daemon-side configurations to tell you whether a device's battery level is low, critically low, or whether we're about to take action on that critical level.

We also have IconName, which replaces some cut'n'pasted code between desktop components. If your desktop environment has many more icons for all types of devices on low battery, for example, you can ignore this property and use the code you always have.

Using those new properties usefully is the new DisplayDevice object. It groups all the batteries and UPSes in the daemon into one, easy to use object that you can use to display a single status icon in your shell chrome. Obviously, if you want to show more devices, the individual batteries and UPSes are still available through the usual means. And it obviously has the 2 new properties mentioned above, so your session daemon can get told when to show notifications for low batteries.

And finally, using that new combined DisplayDevice is the critical battery action policies. As mentioned above, multi-user systems could not hibernate without requiring the user to enter an administrator password, which is less than convenient when your machine is running out of UPS power fast. The configuration for that policy is now in the daemon itself, with sane defaults, and it will hibernate the machine for you.

And to the modernisation

libupower-glib now uses GDBus, even if the daemon doesn't. The daemon however sends PropertiesChanged signals which means that modern D-Bus bindings will automatically get the new values for properties, instead of polling the daemon. The DeviceChanged and Changed signals have thus been removed.

API changes

They are numerous, too many to mention here. I've posted to the device-kit mailing-list with a list of changes that were made, reply there if you have any questions regarding using UPower in your application or session daemons.

Miscellaneous

systemd >= 207 will save your brightness settings across reboots, and the upcoming systemd 209 will have support for saving keyboard backlight across reboots.

I've made attempts at supporting Intel Rapid Start in systemd, but this will actually require kernel changes. Hopefully we should be able to land this by the time GNOME 3.12 is released.

Monday 23 September 2013

GNOME 3.10 is coming!

The new release is coming! As has been the case for the past couple of releases, I've mostly been shepherding great work by other contributors, and I'll detail my limited contributions beyond mere bug fixing.

Wayland support

I've done some work on enabling clutter-gtk applications to be able to run on  Wayland though the harder work of implementing sub-surfaces is still pending.

Giovanni has done incredible work on mutter to start moving some of the X11 dependent code inside the compositor, which should allow you to run a (cut down) Wayland session using gnome-shell.

This also means that Thomas Wood's redesigned Displays panel has Wayland support. A perfect storm of changes for one of the only panels that received little attention since the GNOME 2.x days.


The new displays panel with a TV that claims to be oh so small


Date & Time redesign

Zeeshan, through his work on Geoclue2, and Kalev, through his Summer of Code project, have completely redesigned the Date & Time panel. Aside from being easier to setup, it means that we can finally implement the automatic timezone switching depending on your location.


The new Date & Time panel


BlueZ 5 support

GNOME is the first major desktop to ship with BlueZ 5 support, thanks to work by Gustavo Padovan and Emilio Pozuelo Monfort.

The older version was not supported anymore, and the new version allows us to support things like "Just Works" pairing, better support of audio devices (though the PulseAudio 5.x release to support this is only coming shortly after GNOME 3.10) and a much better architecture for a more stable operation.

GNOME 3.12 should see a redesigned Bluetooth panel, to match current best practices on other platforms (such as merging the management and pairing wizard UIs into one).


Bluetooth devices in use


Miscellaneous


Intuos 4 OLEDs

OLED support for Wacom Intuos 4 tablets (as seen above, thanks Przemo), media keys support for MPRIS applications such as Spotify (thanks to Michael Wood and Lars Uebernickel), updated UI for the Universal Access panel (the ever present Matthias Clasen), support for many more fingerprint readers in libfprint (thanks Vasily Khoruzhick).


Redesigned Universal Access panel



And to my contributions

More work on Videos. Totem 3.10 is still based on the same interface as in GNOME 3.8, but some work has been on the master branch towards the new UI, with some of the features getting backported. We have:
  • new session management for when Totem crashes
  • support for chapters within files (such as Matroska videos)
  • Wayland bug fixes in GTK+, clutter and the combined clutter-gtk
  • a completed GDBus port
  • Working overlaid controls (though their behaviour isn't quite up to scratch)
  • Remote files support in Grilo, including support for Recent files
  • Started work on merging the various sidebars within the main view (which included landing GtkSearchBar in GTK+)
  • libquvi 0.9 support
On top of which you'll find the usual mix of bug fixes, small featuresitch scratching, and swamp-draining in finger-pointing fests.

I also spent quite a bit of time on a side project that didn't come to fruition at this time, but I hope to be able to post some details soon.

Tuesday 6 August 2013

GUADEC Hardware giveaway: Update

Broadcom CrystalHD 70015

Fabiano Fidêncio will be working on integrating the GStreamer plugin for it upstream, and making sure that the default experience with those devices is good in Fedora. The less powerful 70012 card is now in my 6-year old Dell laptop to test Fabiano's work.

Dell Wireless 5520 3G modem

The 3G modem that's been ousted from the Dell laptop to make room for the CrystalHD will go to Aleksander Morgado, in his "torture chamber" (his words) for ModemManager.

iPod Touch 2G

The iPod is going to Adam Reviczky who's going to look into Notes syncing over IMAP (hopefully, it looks like the device might be too old) and for mobile website testing.

Sunday 21 July 2013

Week-end hacks #2

More thumbnailer work, as it's something I can do 20 minutes of, and then go and do something else.

As my thumbnailers were starting to get samey (either the code would spit out PNG/JPEG data, or would create an actual GdkPixbuf), I've created a skeleton file that handles most of the dirty work. It's available on github.

I've also created a MOBI thumbnailer, that's integrated to gnome-epub-thumbnailer and in release 1.2.



Saturday 20 July 2013

GUADEC[1] Hardware giveaway

I'm cleaning up my hardware chest, and giving away some hardware to a good home. I intend on travelling with those that I found a new home for at GUADEC. All of them are in good working condition. If there is a charger or power supply, it will be a UK one.

Drop me a mail with your intended usage (preferably GNOME or kernel related), or need some more info about the devices.

Up for grabs


Palm Pilot Tungsten E2



The predecessor to all your new-fangled smartphones. This one could even do Bluetooth syncing using gnome-pilot, all those years ago. Might be nice as a remote control of some sort, or legacy support for Pilots.

D-Link DIR-615 Wi-Fi N router



Works with DD-WRT. Would be great to work with DD-WRT or associated on a way to configure those through a GNOME UI à-la Airport base stations.

HP iPaq 914



Euro plug. Apparently this can't run Linux... Yet!

DXR3 card


Offload your MPEG2 decoding to this PCI card.

iPod Touch 2G


Too old to run any recent iOS, but good enough to show off your web apps skills, or work on Notes sync with IMAP servers.

Broadcom Crystal HD mini-PCIE 70015 and 70012


2 video decoder cards usable with Linux. You'd need to port the GStreamer plugin to GStreamer 1.0 to get those (or one of those at least).

Plantronics and Motorola Bluetooth headsets


Not the newest devices, but they work.

Red Hat branded power adapter


USB to Nokia/Motorola with this retractable extension lead.

On their way to a good home [2]


Logitech MX 5000 pack and diNovo keyboard


Space-age mouse and keyboard set. Benjamin Tissoires will be getting the (now not so much space age as grubby and outdated) pack to hopefully implement HID++ 1.0 in the kernel.
The diNovo keyboard is a nice little Bluetooth keyboard for a media PC or the likes, even has the tiniest of trackpads.

Logitech 9000, PS3 Eye and Creative OV511-based webcams


Hans de Goede will get those to make them work out-of-the-box in Fedora, and upstream, trying to clean up some hacks he gave me for those a long time ago.

No-name USB GPS dongle and Tom-Tom Bluetooth GPS


For Zeeshan, just in case he gets bored implementing geoclue2.

Nokia N82 and Palm Centro


For Dan Williams. ModemManager's testing gear is growing.

Belkin G and Dell 1450 Wi-Fi USB dongles


Giovanni and Jasper will enjoy those Wi-Fi dongles that will create bugs in gnome-shell's network menu and the new aggregate menu.

[1]: Or near enough for some of the items :)
[2]: I made my pre-selection based on the possible uses for the hardware.

Friday 19 July 2013

Did somebody say "FreeFA"?

Little birdy tells me that we're playing football again this year, so don't forget to sign up!

Monday 15 July 2013

Week-end hacks

Last week, I picked up a Kobo Mini e-reader from my local FNAC store for (less than) 40€ and loaded it up with books from the latest Humble eBook Bundle. As generic document icons aren't really that great to recognise the books, I wrote a small thumbnailer for it, which is now available in GNOME git.

Some DRM-free e-books.

The release is available on the GNOME FTP site, and somebody packaging it up for Fedora would be greatly appreciated.

The other week-end hack was a way to run a program with user-defined DNS servers, rather than relying on the system's /etc/resolv.conf file. It only supports IPv4, but it was good enough to run a few command-line utilities with those specific DNS servers.

Thursday 4 April 2013

Geocluing the desktop, slowly

Over the past couple of months, Satabdi has been working during her Outreach Program for Women on geocode-glib, and Zeeshan more recently joined us to help with cleaning up some of the code.

As Satabdi's program is now finished (though not her involvement!), and a new GNOME development cycle has started, I'll try to explain where geocode-glib fits in, and answer some questions on the future of Geoclue.

(Reverse-)Geocoding

First and foremost, geocode-glib is a geocoding and reverse geocoding library. It uses Yahoo! web services for this, though we're investigating using Nominatim for this in the future.

This replaces the convoluted implementations for those two services in geoclue (3 if you include Address as a service). The API should be mostly stable now, and we'll soon start porting a few applications to it (Evolution and Empathy come to mind).

The library also includes a GeocodeLocation object. This will be useful later.

GeoIP

geocode-glib, thanks to Satabdi's work, includes a GeoIP server, to be installed on GNOME servers eventually, which uses data from MaxMind to  locate a user connected to the Internet from the IP address. We also have a client library to access this server.

This is usually good enough to locate a user in a city, or a country, which would help us with many integration points in GNOME, such as the upcoming Date and Time panel re-design.

But this code isn't really for you, app developers.

Geoclue

A fine project, but the codebase is showing its age (dbus-glib!), and the decision, well, not to take decisions on which backends to use for various services make it fragile. This is a maintenance problem, both for in terms of making sure all the services are kept working, and that geoclue itself is kept stable.

So we'll start a reimplementation of Geoclue. The goals are:

  • Trimmed down API, just for positioning
  • Smaller, but more integrated, selection of ways to get the positioning (GPS from your integrated WWAN modem, Wi-Fi AP data, IP address, no discrete GPS or manual location)
  • Power-saving, by aggregating requests from all the applications
  • and finally, privacy, where only applications that you allow to request your location can get it, and only with an accuracy as needed for the application.
The code currently in geocode-glib for IP geocoding will likely move there when the project has been kickstarted.

I hope this clears any misconceptions people might have about geocode-glib, or, more likely, about the future of geoclue.

Friday 1 February 2013

Power management in GNOME 3.8

In the past couple of weeks, apart from reviewing very many patches for gnome-control-center (especially for new and re-designed panels), I've been working on updating the power management handling in GNOME.

Test suite

The first change is that we have a test suite (currently with 15 separate tests) to test interaction between gnome-settings-daemon's power management and various session and system components. This is thanks to Martin Pitt, and his work on python-dbusmock.

We'll try and add new tests as bug reports come in to avoid regressions, although some cases will remain untested because of limitations in our logging.

 All clear

Screensaver and backlight interaction

With gnome-shell becoming the sole screensaver (after the removal of fallback mode, and the obsoletion of gnome-screensaver), we've been able to streamline the code handling the various screen backlight power levels.

Your screen will now turn off as soon as the screensaver kicks in, moving your mouse in the screensaver will turn it back on for 20 seconds before turning off again, and when to dim (if you've chosen so) is dependent on whether you're on battery or not, and the default idle time (eg. if your screen turns off after 5 minutes of inactivity, the screen will dim after 4). This makes the behaviour more consistent, and predictable, compared to the mish-mash of settings we had before, where some delays were available for change in the UI, and others only through GSettings or gnome-tweak-tool.

Those constants are separate from the code, and exported to the test suite so they are flexible and can be changed if the behaviour doesn't exactly match what users are expecting.

The other change relating to that, is that the screen shield will now always pop down when the screensaver kicks in (thanks to Giovanni for the gnome-shell work). This doesn't mean that you'll have to enter your password each time, but only after the "lock delay" if you've set one.

We've also added a number of nice touches, like the screen turning back on for a short period when you plug or unplug your laptop, made sure that your laptop screen gets turned off and your session locked when closing the lid and turn off the backlight for machines where suspend causes the backlight to come back on temporarily (as seen on MacBooks).

Very very idle

We've also added a long-requested feature: the ability to force logout after a period of idle. This is useful in kiosk and computer lab situations, and is only available through GSettings. As we've added support for this feature (warning prior to logging out, with the screen turning on for a couple of seconds when the warning shows up), we've realised that the infrastructure is the same for automatic suspend/hibernate situation. This means I expect to change the default "long idle" behaviour to suspending. This will still be changeable in the Power preferences. This should land after 3.7.5, and don't worry, we'll make this change very visible in the release notes :)

*I* am not suspending by default

Inhibit

But you don't want to suspend, you really don't.

GNOME supports the draft FreeDesktop "Idle inhibition" specification, as implemented by KDE, which hopefully means that more third-party applications should start behaving better when playing back films, in presentation mode, or for large overnight downloads. This should hopefully get out of draft status before the GNOME 3.8 release.

We also have a gnome-session-inhibit tool available in gnome-session for your scripting needs.

Colophon
 
All the changes mentioned should be available in GNOME 3.7.5, and I will be available to take complaints at FOSDEM this week-end.

Monday 28 January 2013

Office-runner 1.0


Office Runner 1.0 is now available, downloadable at the usual location.

We had some pretty good comments on the original announcement, even though some people took this opportunity to start slinging insults, which is pretty uncool.

There's plenty of related power management work that's been happening in preparation of GNOME 3.8, and we're mostly waiting on GNOME Shell changes to land before I can talk to you about those. I hope it gets the same (positive) comments as Office Runner did, though I imagine some people will get angry about changes, as they are wont to do.

This version requires systemd for the suspend inhibition.

And don't forget to post your high scores ;)