Pipewire Hackfest 2018

Good morning from Edinburgh, where the breakfast contains haggis, and the charity shops have some interesting finds.

My main goal in attending this hackfest was to discuss Pipewire integration in the desktop, and how it will eventually replace PulseAudio as the audio daemon.

The main problem GNOME has had over the years with PulseAudio relate mostly to how PulseAudio was a black box when it came to its routing policy. What happens when you plug in an HDMI cable into your laptop? Or turn on your Bluetooth headset? I've heard the stories of folks with highly mobile workstations having to constantly visit the Sound settings panel.

PulseAudio has policy scattered in a number of places (do a "git grep routing" inside the sources to see that): some are in the device manager, then modules themselves can set priorities for their outputs and inputs. But there's nothing to take all the information in, and take a decision based on the hardware that's plugged in, and the applications currently in use.

For Pipewire, the policy decisions would be split off from the main daemon. Pipewire, as it gains PulseAudio compatibility layers, will grow a default/example policy engine that will try to replicate PulseAudio's behaviour. At the very least, that will mean that Pipewire won't regress compared to PulseAudio, and might even be able to take better decisions in the short term.

For GNOME, we still wanted to take control of that part of the experience, and make our own policy decisions. It's very possible that this engine will end up being featureful and generic enough that it will be used by more than just GNOME, or even become the default Pipewire one, but it's far too early to make that particular decision.

In the meanwhile, we wanted the GNOME policies to not be written in C, difficult to experiment with for power users, and for edge use cases. We could have started writing a configuration language, but it would have been too specific, and there are plenty of embeddable languages around. It was also a good opportunity for me to finally write the helper library I've been meaning to write for years, based on my favourite embedded language, Lua.

So I'm introducing Anatole. The goal of the project is to make it trivial to write chunks of programs in Lua, while the core of your project is written in C (we might even be able to embed it in Python or Javascript, once introspection support is added).

It's still in the very early days, and unusable for anything as of yet, but progress should be pretty swift. The code is mostly based on Victor Toso's incredible "Lua factory" plugin in Grilo. (I'm hoping that, once finished, I won't have to remember on which end of the stack I need to push stuff for Lua to do something with it ;)

GNOME 3.18, here we go

As I'm known to do, a focus on the little things I worked on during the just released GNOME 3.18 development cycle.

Hardware support

The accelerometer support in GNOME now uses iio-sensor-proxy. This daemon also now supports ambient light sensors, which Richard used to implement the automatic brightness adjustment, and compasses, which are used in GeoClue and gnome-maps.

In kernel-land, I've fixed the detection of some Bosch accelerometers, added support for another Kyonix one, as used in some tablets.

I've also added quirks for out-of-the-box touchscreen support on some cheaper tablets using the goodix driver, and started reviewing a number of patches for that same touchscreen.

With Larry Finger, of Realtek kernel drivers fame, we've carried on cleaning up the Realtek 8723BS driver used in the majority of Windows-compatible tablets, in the Endless computer, and even in the $9 C.H.I.P. Linux computer.

Bluetooth UI changes

The Bluetooth panel now has better « empty states », explaining how to get Bluetooth working again when a hardware killswitch is used, or it's been turned off by hand. We've also made receiving files through OBEX Push easier, and builtin to the Bluetooth panel, so that you won't forget to turn it off when done, and won't have trouble finding it, as is the case for settings that aren't used often.

Videos

GNOME Videos has seen some work, mostly in the stabilisation, and bug fixing department, most of those fixes were also landed in the 3.16 version.

We've also been laying the groundwork in grilo for writing ever less code in C for plugin sources. Grilo Lua plugins can now use gnome-online-accounts to access keys for specific accounts, which we've used to re-implement the Pocket videos plugin, as well as the Last.fm cover art plugin.

All those changes should allow implementing OwnCloud support in gnome-music in GNOME 3.20.

My favourite GNOME 3.18 features

You can call them features, or bug fixes, but the overall improvements in the Wayland and touchpad/touchscreen support are pretty exciting. Do try it out when you get a GNOME 3.18 installation, and file bugs, it's coming soon!

Talking of bug fixes, this one means that I don't need to put in my password by hand when I want to access work related resources. Connect to the VPN, and I'm authenticated to Kerberos.

I've also got a particular attachment to the GeoClue GPS support through phones. This allows us to have more accurate geolocation support than any desktop environments around.

A few for later

The LibreOfficeKit support that will be coming to gnome-documents will help us get support for EPubs in gnome-books, as it will make it easier to plug in previewers other than the Evince widget.

Victor Toso has also been working through my Grilo bugs to allow us to implement a preview page when opening videos. Work has already started on that, so fingers crossed for GNOME 3.20!

GNOME 3.16 is out!

Did you see?

It will obviously be in Fedora 22 Beta very shortly.

What happened since 3.14? Quite a bit, and a number of unfinished projects will hopefully come to fruition in the coming months.

Hardware support

After quite a bit of back and forth, automatic rotation for tablets will not be included directly in systemd/udev, but instead in a separate D-Bus daemon. The daemon has support for other sensor types, Ambient Light Sensors (ColorHug ALS amongst others) being the first ones. I hope we have compass support soon too.

Support for the Onda v975w's touchscreen and accelerometer are now upstream. Work is on-going for the Wi-Fi driver.

I've started some work on supporting the much hated Adaptive keyboard on the X1 Carbon 2nd generation.

Technical debt

In the last cycle, I've worked on triaging gnome-screensaver, gnome-shell and gdk-pixbuf bugs.

The first got merged into the second, the second got plenty of outdated bugs closed, and priorities re-evaluated as a result.

I wrangled old patches and cleaned up gdk-pixbuf. We still have architectural problems in the library for huge images, but at least we're up to a state where we know what the problems are, not being buried in Bugzilla.

Foundation building

A couple of projects got started that didn't reached maturation yet. I'm pretty happy that we're able to use gnome-books (part of gnome-documents) today to read Comic books. ePub support is coming!

Grilo saw plenty of activity. The oft requested "properties" page in Totem is closer than ever, so is series grouping.

In December, Allan and I met with the ABRT team, and we've landed some changes we discussed there, including a simple "Report bugs" toggle in the Privacy settings, with a link to the OS' privacy policy. The gnome-abrt application had a facelift, but we got somewhat stuck on technical problems, which should get solved in the next cycle. The notifications were also streamlined and simplified.

I'm a fan

Of the new overlay scrollbars, and the new gnome-shell notification handling. And I'm cheering on co-new app in 3.16, GNOME Calendar.

There's plenty more new and interesting stuff in the release, but I would just be duplicating much of the GNOME 3.16 release notes.

A look at new developer features

As the development window for GNOME 3.16 advances, I've been adding a few new developer features, selfishly, so I could use them in my own programs.

Connectivity support for applications

Picking up from where Dan Winship left off, we've merged support for application to detect the network availability, especially the "connected to a network but not to the Internet" case.

In glib/gio now, watch the value of the "connectivity" property in GNetworkMonitor.

Grilo automatic network awareness

This glib/gio feature allows us to show/hide Grilo sources from applications' view if they require Internet and LAN access to work. This should be landing very soon, once we've made the new feature optional based on the presence of the new GLib.

Totem

And finally, this means we'll soon be able to show a nice placeholder when no network connection is available, and there are no channels left.

Grilo Lua resources support

A long-standing request, GResources support has landed for Grilo Lua plugins. When a script is loaded, we'll look for a separate GResource file with ".gresource" as the suffix, and automatically load it. This means you can use a local icon for sources with the URL "resource:///org/gnome/grilo/foo.png". Your favourite Lua sources will soon have icons!

Grilo Opensubtitles plugin

The developers affected by this new feature may be a group of one, but if the group is ever to expand, it's the right place to do it. This new Grilo plugin will fetch the list of available text subtitles for specific videos, given their "hashes", which are now exported by Tracker.

GDK-Pixbuf enhancements

I can point you to the NEWS file for the latest version, but the main gains are that GIF animations won't eat all your memory, DPI metadata support in JPEG, PNG and TIFF formats, and, for image viewers, you can tell whether a TIFF file is multi-page to open it in a more capable viewer.

Batched inserts, and better filters in GOM

Does what it says on the tin. This is useful for populating the database quicker than through piecemeal inserts, it also means you don't need to chain inserts when inserting multiple items.

Mathieu also worked on fixing the priority of filters when building complex queries, as well as supporting more than 2 items in a filter ("foo OR bar OR baz" for example).

Videos 3.14 features

We've added a few, but nonetheless interesting features to Videos in GNOME 3.14.

Auto-rotation of videos

If you capture videos in portrait orientation on your phone, we are now able to rotate them automatically in the movie player, as well as in the thumbnails.

Better streaming

You can now seek anywhere inside streamed videos, even if we didn't download all the way to that point. That's particularly useful for long videos, or slow servers (or a combination of both).

Thumbnails generation

Finally, videos without thumbnails in your videos directory will have thumbnails automatically generated, without having to browse them in Files. This makes the first experience of videos more pleasing to the eye.

What's next?

We'll work on integrating Victor Toso's work on grilo plugins, to show information about the film or TV series on your computer, such as grouping episodes of a series together, showing genres, covers and synopsis for films.

With a bit of luck, we should also be able to provide you with more video content as well, through partners.

Post-GUADEC

• If you have an orientation sensor in your laptop that works under Windows 8, this tool might be of interest to you.
• Mattias will use that code as a base to add Compass support to Geoclue (you're on the hook!)
• I've made a hack to load games metadata using Grilo and Lua plugins (everything looks like nail when you have a hammer ;)
• I've replaced a Linux phone full of binary blobs by another Linux phone full of binary blobs
• I believe David Herrmann missed out on asking for a VT, and getting something nice in return.
• Cosimo will be writing some more animations for me! (and possibly for himself)
• I now know more about core dumps and stack traces than I would want to, but far less than I probably will in the future.
• Get Andrea to approve Timm Bädert's git account so he can move Corebird to GNOME. Don't forget to try out Charles, Timm!
• My team won FreeFA, and it's not even why I'm smiling ;)
• The cathedral has two towers!

Unfortunately for GUADEC guests, Bretzel Airlines opened its new (and first) shop on Friday, the last days of the BoFs.

(Lovely city, great job from Alexandre, Nathalie, Marc and all the volunteers, I'm sure I'll find excuses to come back :)

My GNOME 3.12 in numbers

1 new GNOME Videos, 1 updated Bluetooth panel, 2 new thumbnailers, 9 grilo sources, and 1 major UPower rework.

I'm obviously very attached to the GNOME Videos UI changes, the first major UI rework in its 12-year existence.

GNOME Videos watching itself

Go read the press release and follow to the release notes.

Extend GNOME Videos with Lua

As you've probably seen in my previous post, the new Videos UI has part of its interface focused on various channels from online sources, such as the Blip.tv, or videos from the Guardian.

Grilo recently grew support for Lua sources, which means you can write about 100 lines of lua, and integrate videos from an online source into Videos easily.

The support isn't restricted to videos, GNOME Music and GNOME Photos and a number of other applications will also be able to use this to be extended.

Small tutorial by example

Our example is that of a source that would fetch the list of Ogg Theora streams from Xiph.org's streaming directory.

First, define the "source": the name is what will show up in the interface, supported_keys lists the metadata fields that we'll be filling in for each media item, and supported_media mentions we only show videos, so the source can be skipped in music players.

source = { 
  id = 'grl-xiph-example',
  name = 'Xiph Example',
  supported_keys = { 'id', 'title', 'url', 'type' },
  supported_media = 'video',
}

We'll then implement one of the source methods, to browse/list items in that source. First, we cheat a bit and tell the caller that we don't have any more items if you need to skip some. This is usual for sources with few items as the front-end is unlikely to list items 2 by 2. If that's not the case, we fetch the page on the Xiph website and wait for the callback in fetch_cb

function grl_source_browse(media_id)
  if grl.get_options("skip") > 0 then
    grl.callback()
  else
    grl.fetch('http://dir.xiph.org/by_format/Ogg_Theora', 'fetch_cb')
  end
end

Here's the meat of the script, where we parse the web page into media items. Lua doesn't use regular expressions, but patterns. They're different, and I find them easier to grasp. Remember that the minus sign/dash is a reserved character, '%' is the escape character, and '()' enclose the match.

We create a new table called for each one of the streams in the HTML we scraped, with the metadata we said we'd give in the source definition, and send it back to Grilo. The '-1' there is the number of items remaining in the list.

Finally, we call grl.callback() without any arguments to tell it we're done fetching the items.

function fetch_cb(results)
  if not results then
    grl.callback()
  end 

  for stream in results:gmatch('<p class="stream%-name">(.-)</p>') do
    media = {}
    media.url = stream:match('href="(.-)" ')
    media.id = media.url
    media['type'] = 'video'
    media.title = stream:match('<a href.->(.-)</a>')

    grl.callback(media, -1) 
  end 

  grl.callback()
end

We're done! You just need to drop this file in ~/.local/share/grilo-plugins/grl-lua-factory, and you can launch Videos or the test application grilo-test-ui-0.2 to see your source in action.

Why Lua?

This screen scraping is what Lua is good at, with its powerful yet simple pattern matching. Lua is also easily embeddable, with very few built-in functions which means we can have better control over the API plugins use, a low foot-print, and all the benefits of an interpreted garbage-collected language.

I hear heckles of "Javascript" in the background, so I guess I better address those as well. I think Lua's pattern matching is better than Javascript regexes, and more importantly, Lua is more easily embeddable in big applications, because of its simplicity as a language and a VM. Basically, Javascript (and the gjs implementation we'd likely have used in particular) is too powerful for our use case.

Better sources

It's obviously possible to avoid this screen scraping when the online source provides data in an easily parseable format (such as Json for which we have Lua bindings). That will be the case of the Guardian videos source (once we've figured out a minor niggle with the 50 items query limit) thanks to the Guardian's Open Data work.

Hopefully it means that we'll have sources for the Wiki Commons Picture of the day (as requested by GNOME alumni Luis Villa) for use in the Background settings, or for Mediagoblin installations.

Videos sidenote

An aside, for those of you who have videos on a separate network storage, or not indexed by Tracker, there's a hidden configuration to show specific paths in the Channels tab in Videos.

gsettings set org.gnome.totem filesystem-paths "['smb://myserver.local/videos/']"

Epilogue

I'm looking forward to seeing more Grilo sources. I feel that this Lua source lowers the barrier to entry, enabling the evening hacker to integrate their favourite media source into GNOME, which probably means that we'll need to think about parental controls soon! ;)

Thanks to Victor Toso for his work on Lua sources, both during and after the Summer of Code, and Juan Suarez for his mentoring and patch reviewing.

Videos is here!

It's been some time in the making, with the redesign work started a couple of release cycles ago, but we finally reached a state where it's usable, and leaps and bounds easier to use than the previous versions.

I should note that I use Totem and Videos interchangeably, Totem is still the name of the project, code repository, but the user-visible name is Videos (or GNOME Videos if differentiation is necessary).

Discovery

The old UI made it particularly hard to consume media from various web sites, as you can see from the screenshot below. It's cramped, we had separate sidebars for search and browse, we didn't show icons for browse, etc.

And here's the new UI, browsing the same source (Apple Trailers).

This is definitely easier to find media. Totem also had a number of specific plugins to find media sources, usually from third-party developers. We don't support those anymore, and if you have been writing such a plugin, you should port them to grilo, now a hard-dependency.

I've also spent some time working on Grilo and its plugins, creating a few new sources along the process.

Amongst the new ones are the Freebox TV plugin, the under-powered Guardian Videos source, and the not-yet-fully-integrated Pocket videos list. Don't forget to check for blocker bugs if you're trying to test those!

Playback

This is also a pretty big upgrade. We now have video-specific menu item, the gear menu, and better looking pop-ups. This matches the design used in GNOME Documents for sliders. It's also better suited for touch: a mouse move will show the OSD for a short time, but a touchscreen tap will show the OSD until you tap it again.

The older version had some features only available in windowed, such as rotation, or zoom, and some we tried to cram into the context menu (subtitle or sound track selection for example). Here, there's no loss of accessibility for features, they're all in the same gear menu, whether fullscreen or windowed.

Bugs, bugs, bugs

With a few valiant testers and designers, we tried to fix a number of bugs. This release doesn't mean Videos is bug-free, far from it, but it's certainly robust and usable enough to make this development release.

There's some theming bugs, as can be seen in that last screenshot's previous/next buttons, there's bugs in grilo and grilo-plugins, and there's bugs in Videos itself.

Do file bugs when you see something amiss, it'll help designers and myself move items from our own TODO lists :)

What's next

A lot :)

You can see some of those in the design wireframes.

"Make available offline" is something I have a great interest in, especially coupled with the Pocket source. Selecting a bunch of items to watch later, on the train, or on the plane.

Better metadata, especially for films and series. This isn't just for films you snarfed from Usenet and torrent sites either. The already existing Rai.tv source has a number of films, and a BBC iPlayer source is planned.

Finally, remote playback, to "throw" videos from your laptop to the TV. Controls should still work, and we'll want to allow browsing through sources when playback is remote.

Notes on development

Half notes, half thanks. As mentioned in the introduction, this release has been some time in the making, but it also comes at a time when we've had the necessary plumbing to make all this possible.

To name but a few, we've made good use of gnome-documents' widgets to list videos, the GtkStack, GtkRevealer and GtkPopover. The GtkSearchBar and GtkSearchEntry widgets are also examples of widgets that moved to GTK+ for Videos' development.

Getting it

Soon in your development distributions, in totem's master git branch, and in GNOME's FTP server.

Week-end hacks #3

Mo' thumbnailers

Simple enough, but I still managed to make 2 broken releases ;)

There's now a Krita and OpenRaster/MyPaint thumbnailer in GNOME git, and as tarball releases.

Freebox TV streaming

My ISP, Free, gives its customers an access point and TV set-top box. The access point is also a UPnP, Samba, AFP server and Bittorrent client amongst other things.

It's also responsible for handling IPTV, streaming to the TV set-top box. You can also watch TV on your laptop streaming RTSP to local clients.

This Grilo plugin, once all the issues are fixed, should allow us to show the TV channels in Totem.

GNOME 3.10 is coming!

The new release is coming! As has been the case for the past couple of releases, I've mostly been shepherding great work by other contributors, and I'll detail my limited contributions beyond mere bug fixing.

Wayland support

I've done some work on enabling clutter-gtk applications to be able to run on  Wayland though the harder work of implementing sub-surfaces is still pending.

Giovanni has done incredible work on mutter to start moving some of the X11 dependent code inside the compositor, which should allow you to run a (cut down) Wayland session using gnome-shell.

This also means that Thomas Wood's redesigned Displays panel has Wayland support. A perfect storm of changes for one of the only panels that received little attention since the GNOME 2.x days.

The new displays panel with a TV that claims to be oh so small

Date & Time redesign

Zeeshan, through his work on Geoclue2, and Kalev, through his Summer of Code project, have completely redesigned the Date & Time panel. Aside from being easier to setup, it means that we can finally implement the automatic timezone switching depending on your location.

The new Date & Time panel

BlueZ 5 support

GNOME is the first major desktop to ship with BlueZ 5 support, thanks to work by Gustavo Padovan and Emilio Pozuelo Monfort.

The older version was not supported anymore, and the new version allows us to support things like "Just Works" pairing, better support of audio devices (though the PulseAudio 5.x release to support this is only coming shortly after GNOME 3.10) and a much better architecture for a more stable operation.

GNOME 3.12 should see a redesigned Bluetooth panel, to match current best practices on other platforms (such as merging the management and pairing wizard UIs into one).

Bluetooth devices in use

Miscellaneous

Intuos 4 OLEDs

OLED support for Wacom Intuos 4 tablets (as seen above, thanks Przemo), media keys support for MPRIS applications such as Spotify (thanks to Michael Wood and Lars Uebernickel), updated UI for the Universal Access panel (the ever present Matthias Clasen), support for many more fingerprint readers in libfprint (thanks Vasily Khoruzhick).

Redesigned Universal Access panel

And to my contributions

More work on Videos. Totem 3.10 is still based on the same interface as in GNOME 3.8, but some work has been on the master branch towards the new UI, with some of the features getting backported. We have:

• new session management for when Totem crashes
• support for chapters within files (such as Matroska videos)
• Wayland bug fixes in GTK+, clutter and the combined clutter-gtk
• a completed GDBus port
• Working overlaid controls (though their behaviour isn't quite up to scratch)
• Remote files support in Grilo, including support for Recent files
• Started work on merging the various sidebars within the main view (which included landing GtkSearchBar in GTK+)
• libquvi 0.9 support

On top of which you'll find the usual mix of bug fixes, small features, itch scratching, and swamp-draining in finger-pointing fests.

I also spent quite a bit of time on a side project that didn't come to fruition at this time, but I hope to be able to post some details soon.