X11 Applications and Unity 8

The buzzword convergence has been bandied about a lot in relation to Ubuntu.  That’s because the plan is to have one single Ubuntu that works the same way across phones, tablets, laptops, desktops, TVs, kiosks, and refrigerators. It’s an exciting idea that some other software environments have also aimed at, but so far have not been able to deliver on.

One of the key technologies that lets the same Ubuntu run on all kinds of devices from your pocket to your desktop is Unity 8, a graphical shell that presents a way to surface and launch applications and display some information.  Unity 8 has been designed to dynamically adapt to the available display and input devices present, so it can flow from a full-screen personality on a tiny phone with only touch input to a multiple-display, multiple-workspace windowed personality with a mouse and a keyboard attached to an engineer’s workstation.  Going hand-in-hand with Unity 8 is an entire SDK that can be used to build applications that are also flexible across all combinations of displays and inputs.  Part of the magic underlying Unity 8 is the Mir compositing display server, which replaces the venerable X11 display server.

Underneath the shell layer is a new kind of way of running the fundamental operating system, with a read-only system image that can be transactionally updated.  While that is a technical description of how things differ from the old DOS and Unix way, what it means in practice is that it’s very difficult to install malware that will take over your system and turn it into a spambot for some nefarious organization and it’s also difficult to get your system into a state in which it no longer boots up or runs improperly.  In the hostile world of today’s always-on always-connected devices, that’s a very good thing.  Also, no reinstalling the OS every few months when it starts to crawl, for those of you out there familiar with Microsoft Windows.

One final bit of change for Unity 8 is that instead of the factored deb packages in which you build your system up like a jigsaw puzzle, the newer systems use self-contained application packages, originally clicks and now snaps.

So, the major differences coming down the pipe with Unity 8 and convergence include the following.

1. Mir instead of X11.
2. Read-only system files.
3. No debs.

What, you may ask, about all the many gigabytes of existing deb packages everyone relies on in the Ubuntu archives, their organization’s private archives, and third-party archives?  Doesn’t this break the contract between Ubuntu and its users?

Libertine to the Rescue

The first answer to the above concern is that many of the more popular applications are being snapped as I write this.  That means they’re being packaged up so they can be installed and run on a read-only system image without deb support.

Another answer is that the most popular toolkits like GTK+ and Qt (and even libSDL, for things that really count like games) have been ported to Mir so most applications don;t have to care what the underlying display server technology is.

The final answer is that we’ve come up with  a cunning way to install and use your existing deb packaged X11 applications on a read-only deb-free system running Mir.  We call it libertine.

We chose the name libertine because the word in the English language refers to an individual who feels they need to flout rules and engage in risky and socially unacceptable behaviours.  What we’ve done is set up a container into which debs can be installed in which they can mess up the system in the container without affecting the real system, and they can run unconfined and unfettered without fear of stealing data or keystrokes and reporting home.

Libertine consists of a container with a minimal Ubuntu system installed in it, and a Mir client application that proxies the confined application.  In the case of an X11 application, that proxy is called XMir, and is actually an x.org server with a Mir-based DDX — which is to say, it’s a bog-standard X11 server that ends up drawing its output on a Mir surface.  Libertine also provides alternative proxies, such as a terminal application for venerable terminal applications such as Midnight Commander or good old vi.

The secret sauce is that actual application launching is taken care of by a tool we call the Ubuntu Application Launcher [UAL].  When you click or tap on an application icon in the Unity 8 Dash, a URI describing the application gets sent to UAL, which then figures out if it’s a native application, a libertine application, or even a snappy application, and then does the right thing to start the application.  In the case of a libertine application, that means starting the XMir server and then using the libertine tools to launch the applications in its container.  The libertine tools spin up the container as necessary, bind-mount various required devices and directories from the host system into the container, start some bridge daemons, and then start the application.  Then takes a break because it works hard.

Here’s picture saying the same thing for the right-brained among you.

An interesting thing to note here is that a separate XMir process is spawned for every libertine application launched.  This is the only secure way to use X11 in a hostile environment of keyloggers and data snoopers.  Data sharing previously done through the X11 server, like cut-and-paste and drag-and-drop, has to be done through a trusted arbitrator called the content hub and involves attestation steps.

The libertine project consists of a suite of tools:  a set of command-line utilities, a scope for discovering installed applications, a management GUI, and even a scope to browse the Ubuntu archives for packages to install.

Some of you may have heard the name Puritine bandied about.  Puritine is a pre-rolled container with a set of 5 curated X11 applications that comes pre-installed on certain devices as a demonstration of convergence technology.  The name was a play on libertine and puritan.

This was a brief high-level introduction to Libertine, our solution for running existing deb-packaged XApps under Unity 8.  I’ll be diving into more technical detail in some upcoming posts, but in th mean time you can actually play with libertine right now on your Ubuntu device or desktop.

Advertisement

Previewing the New Unity

Canonical announced a while ago — maybe measurable in multiple dozens of months — that they were rewriting their flagship Unity desktop shell so it can converge the mobile phone and tablet experience with the desktop and laptop experience.  The new Unity is dubbed Unity 8, for which I apologize because the name was my suggestion and now sounds a little dated after both Apple’s Mac OS and Microsoft’s Windows are up to version 10 now.  Unity 8 has been shipping on phones and tablets for at least a year now but it isn’t quite ready for the more demanding desktop consumer yet.

This is where you come in.  Our current goal is to make the Unity 8 desktop available in the Ubuntu 16.10 release coming up this October as an alternative login session on the ISO (installation image).  Before we get even that far we want to get more widespread testing and feedback from daring and confident Ubuntu users, users like you.

Installing and testing is easy, and if you really don’t like it you can just remove it and carry on, no harm done.  I recommend using the command line to install the shell, because I haven’t yet figured out how to use Ubuntu Software to find and install the appropriate packages — I’m really not a GUI guy at heart.

Installing Unity 8

I’m going to assume you’re either running Ubuntu 16.04 LTS or the pre-release Yakkety Yak version of Ubuntu.  Those are the only Ubuntu versions that support Unity 8 on desktop at the moment.  The procedure is the same for both.

First, you need to open a command-line terminal.  From Unity, pressing the control-alt-T combination should do the trick, but advanced users may have their own favourite technique.  From the terminal, just type the following commands.

sudo add-apt-repository ppa:ci-train-ppa-service/stable-phone-overlay
sudo apt update
sudo apt install unity8-desktop-session

If you don’t have the command add-apt-repository, it’s in the package software-properties-common which you may need to install first.

That’s it.  Log out, then from the LightDM prompt click on the circle-of-friends logo beside your name and select the entry that says “Unity 8”, enter your password, and you’re good to go.  Unity 8 on the desktop, baby, yeah!

Oh, a word of caution.  The proprietary binary blob video drivers from AMD, nVidia, and some VMs still do not support the Mir display server used by Unity 8, so you’re probably going to suffer a bit of disappointment.  Try using the open source video drivers for your card instead.

Reporting problems

Yeah, we know there are problems, that’s why Unity 8 is not the default on your desktop yet.  We know the window management is still a work in progress, that most of your favourite desktop programs aren’t available out of the box, and that there are still session management issues.

If you do run into a problem, errors get logged to various files in $HOME/.cache/upstart.

You can contact developers in the #ubuntu-unity irc channel on Freenode and file bugs in Launchpad (don’t forget to check to see if the bug has already been reported first).

Removing Unity 8

If you really don’t like Unity, or for some reason it’s messed up your system, the following commands will remove it and restore things to the way they were previously.

sudo apt-get remove unity8-desktop-session
sudo add-apt-repository --remove ppa:ci-train-ppa-service/stable-phone-overlay

 

Locally Integrated Menus for Ubuntu 14.04 LTS

Over the last few months some of the Unity team members have been working hard to get some new features landed in time to be included in the Trusty Tahr due out this April.  One of the most important features that Marco Trevisan has been working on has been what we call “locally integrated menus,” which refers to having application menus appear in the application window title bar (“locally”) as opposed to in the top panel bar (“globally”).  You can choose to use this new configuration on larger screens or on the newer higher-DPI screens so application menus are closer to where you’re working, or you can choose the current configuration.  It will be your choice.  I can tell you I miss the local menus when I go use my machines running older versions of Ubuntu.  this is an exciting change.

I think Marco is better at describing the history and technical issues behind this effort. I recommend you read his post.

The user interface is a dynamic thing and a description of this change in words does not do it justice.  If you’d like to see the locally integrated menus in action, we’ve prepared an introduction video for your viewing enjoyment as well.

Kudos to Marco for his persevering work on this and to Brandon Schaefer and Chris Townsend for their reviews, testing, and support.

More Trusty Excitement

I have another bit of exciting news on the Ubuntu 14.04 LTS front.

One  of the things we’ve done is managed to port the Simple Direct Media layer (SDL) to the Mir display server.  For those of you fortunate enough to not have a clue what the means, SDL is a portability layer used by many games, including many available on Valve’s Steam platform.  The consequences of that work is that a whole slew of really good games are going to be available immediately on Unity8 when it finally hits the desktop.

Wait!  There’s more.  SDL autodetects the actual video layer.  That means you can install a game from Steam on Unity7 running on X11 today on Ubuntu 13.10, and when you log in to your Unity8 session running on top of Mir on Ubuntu 14.04 LTS, it will just work.  Isn’t that just the bee’s knees?

There’s still more.  The SDL layer isn’t just used for games.  It’s used for professional applications like remote desktop interfaces and various visualizers.  They will all just work on the Unity8 desktop running on Mir.  I’m really excited by this, and you should be too, because it just might mean next year will really be the year of the Linux desktop. Heh.  Humour me.

The screenshot here is the game DOTA2 running on Mir, taken by Brandon Schaefer. He’s the wizard who did most of the work of making the wonderful and widespread SDL library work on Mir in cooperation with Valve and the SDL maintainers.  I’m sure it was tough having to play games during work hours, but he made the valiant effort out of a sense of duty when asked.  Unfortunately, now he has to get back to work fixing high-DPI issues in Unity7.

The Mir back end will be released with libSDL 2.0.2 in the very near future.

Edit: Brandon also has a video capture.

Progress on the Converged Desktop

It’s still very early days in Canonical’s efforts to unify the experience from phone to tablet to desktop to TV, but results are starting to trickle in.

Unity8 home screen on a Lenovo Yoga 2 Pro

One of the side projects on my plate is a Unity8 login session you can install alongside the regular Unity7 desktop (or other desktops, your choice). The idea is to have a preview session available for the Trusty Tahr that lets Unity 8 developers and Touch App developers tune the experience on the desktop. For example, there’s evidently still some work to be done to make Unity8 useful on high-DPI screens like my Lenovo Yoga 2 Pro (3200×1800 pixels on a 13″ laptop).

Sorry for the potato. I haven’t got screen capture working yet so I had to use an actual physical camera. Early days. You can see the high-DPI issues in this picture.  For example, see the teeny-tiny indicators at the top?  Didn’t think so.

The cool thing about this session is it runs on the Mir display server with no X11 in sight. LightDM (the program that accepts your password and logs you in in Ubuntu) uses X11 to run the Unity7 greeter, then shuts it down gracefully and starts a Mir server, then uses Upstart to run a Unity8 session. Gee whiz.

Right now you could just install this from a PPA and go at it, but there are some big wrinkles that still need to be worked out. For instance, there’s no cursor support so mouse and trackpad support is a little troublesome (works swell with a touchscreen), and keyboard input seems to have no effect — Unity8 has been developed with phones and tablets in mind, they don’t generally have a mouse, trackpad, or keyboard. It’s OK, there are plans in the works to add those, and that’s one of the reasons we have this session at this point in the Ubuntu release cycle.

Also, there is no way to log off from Unity8 (phones and tablets don’t generally support that concept) so that pretty much requires a reboot of the whole system unless you have an SSH session open.

Finally, there seems to be some trouble actually launching applications. No worries, that’ll get fixed too.

I think this is pretty exciting. That’s why I’m letting you all in on this project in its infancy. The Ubuntu community deserves to be kept abreast of the coming excitement that will be available in Ubuntu 14.04 LTS.

The future looks very small.

I have a new toy. I didn’t get it because I’m hip, although I am, I got it because we’re trying to prepare Unity 7 on the Trusty Tahr (Ubuntu 14.04 LTS) for the next generation of hardware that will be sitting on everyone’s desk (or lap, or table in the coffee shop) within a few years. I got a laptop with a high-DPI (dots per inch) 4K display and a sensitive touchscreen.

This particular piece of furniture is a Lenovo Yoga 2 Pro, sporting a 3200×1800 pixel 10-touch touchscreen in a 13 inch form factor. That works out to a pixel density of about 280 pixels per inch, much more refined than my main laptop (a Lenovo ThinkPad T410, 1440×900 at 14 inches) which sits at 120 pixels per inch and the external monitor I have attached to it (a Benq FP22W, 1680×1050 at 22 inches) at 95 pixels per inch. Sure, spec ennui, but it’s germane to the topic here.

The problem is that out of the box, most GUI software assumes it’s running on a display device, regardless of its dimensions, with a dot pitch of 96 dots per inch. It’s true for Microsoft Windows and it’s true for GNU/Linux, although I’ve been out of the Apple Macintosh world long enough to plead ignorance there. I know it’s true of Microsoft Windows because the Yoga 2 Pro came with Microsoft Windows 8.1 preinstalled by the manufacturer, and I had a brief chance to test it out before I got to work. IE displayed web pages in teeny weeny characters, and when I opened COMMAND.COM (or whatever the name of the command console is these days) to create a rescue image, it defaulted to using 8×8 bitmapped fonts. The eyestrain finding the reconfiguration option felt like it caused my corneas to bleed.

We have the same problem in Ubuntu. When I installed a prelease image of Trusty on the Yoga 2 Pro the GRUB2 menu was so tiny I couldn’t read it (bitmapped fonts again). Fortunately, the default was sensible and the system booted OK. Unity 7, of course, was similarly unusable, as was the Terminal, the Browser, and pretty much everything else. Ouch.

The problem is rooted in the fact that it’s an invalid assumption that all display devices have a dot pitch of 96 pixels per inch. I already experience this with my dual monitor setup, but it’s less noticeable with 120 vs. 95 DPI. This is just not a valid assumption.

See, a character in a 12 point font needs to appear to be 12 points. That’s 1 pica. One sixth of an inch. The size of a 12 point character should not vary depending on the resolution of your monitor. There’s a caveat, though, in that when I said ‘appear’ what I meant was at a comfortable viewing distance. Turns out that for the best human interface we do in fact want the absolute size of text to change depending on the viewing distance so that there is a constant angle subtended by the display. Er, that means things that are father away need to be bigger so they seem the same size. Got it? Think: projectors. Turns out people use phones and tablets up close, so smaller is OK, but they use their laptops and desktops farther away so smaller is no good.

This is where I need a diagram as a visual aid, but I’m afraid my drawing skills have rusted out and are at the shop for repairs. If someone wants to contribute one, that’d be great.

So, what we need to do for Unity running on the desktop is to figure out the physical dot pitch for each physical display connected to the system, and calcluate the scaling factor that would convert to a fixed 96 pixels per inch, then scale the fonts by that much. Other metrics need to be expressed in terms of ems (another measure based on the current font size — a term that comes from the days of hot metal) and graphics scaled accordingly.

But wait, we don’t want to scale windows if we don’t have to. We don’t want to waste the “retina” display, we just want text to be readable (and controls to be usable). At this point, we’re looking at making sure the Unity Launcher, the Unity panel, the Quicklists, and the Shortcuts are all usable out of the box on a high DPI display, because I have one and I tell you it’s not too usable right now.

A lot of applications are not going to work perfectly on high DPI, including the browsers and the office suites. We’re thinking of adding some optional window scaling through Compiz to help out with those but time is rapidly flowing and there’s a lot of work to do. Stand by for updates. As always, patches are welcome.

Ubuntu Desktop Convergence

This is where it starts to get exciting, folks.  The future starts now.

Ubuntu is an operating system for the server, the cloud, the desktop, and the mobile device.  One single OS.  That makes it different from Apple’s OSes (Mac OS X on the desktop, iOS on the mobile) and Microsoft’s current OSes (Windows 8 on the desktop, Windows RT on the mobile, and Windows Server 2012 on the server and in the cloud).

Screenshot of Unity7 running on Mir (click to embiggen).

At least that’s the plan.  It’s not yet the reality because there isn’t really a shipping Ubuntu for mobile.  Sure, there’s lots of preview releases, but only a shipping version is a shipping version.

So what needs to be done before we have full convergence to a single OS?

Well, one of the things Canonical has been working on is a replacement for the venerable X11 display server traditionally used in most GNU/Linux environments.  They’re developing something they’ve called Mir, from the Russian мир meaning “world” or “peace” (but not “world peace”) and the name of the Soviet space station, in keeping with a general space-exploration theme in and around Ubuntu, for reasons.  It’s also a German language pronoun (first person singular dative case), as in “show me,” because the original architect was a German national.  Really, I’m making this up as I go along and if you’re not one of the tl;dr crowd consider yourself appreciated. This new display server has been discussed in great detail elsewhere and is under active development.  The important take away is that in order to run effectively on the mobile form factors, the X11 server needs to be replaced.

This is where Unity8 comes in to the picture.  You may have read elsewhere about “Unity Next”, which was a working title for the new Unity in the queue.  It’s now more formally known as Unity 8, because it supersedes the current Unity 7 as the one single Unity (there can be only one).  Unity 8 is designed to run natively on the Mir compositor, fancy term for what X11 called a display server.  That’s what will be on the mobile offering, currently dubbed “Ubuntu Touch”.  It’s also what will be on the desktop in the fully converged world.

A replacement for X11 is fine for a phone or a tablet and all, but Ubuntu already has a good thing going with the classic GNU/Linux milieu on the desktop, which is heavily dependent on X.  We can’t just throw all that away.  So, we need an X11 server running on top of the Mir compositor so all those legacy applications folks and grown to know and love will continue to run just like forever.  This is where something dubbed Xmir comes in to play:  It’s an x.org driver that fits in to the x.org X11 server just like the nVidia, AMD, and Intel drivers do and lets the X11 server run on top of Mir.  Cast your gaze at the screencap in this article:  you’re seeing Unity 7 running on Mir on Saucy (the clue is the unity-system-compositor line in the process tree seen in the screencap).  This is just like magic.  And, because accelerated graphics are still a work in progress, it’s mighty slow magic at this pre-release point.  Oh, and yes I see the crash reporter running in the process listing.  I do these things so you don’t have to.

What we’re going to do for the Saucy Salamander release of Ubuntu is make a Unity8 Desktop preview available for those who want to take it for a test drive.  You’re going to be able to choose to have the option of logging in to either a regular Unity7 session with X11 running, or a Unity8 session without X11.  And that, folks, is where the excitement starts.

Unity Maintenance for Ubuntu “Saucy Salamander”

Once again Canonical has tasked a team with performing maintenance of the Unity (desktop shell) stack for the upcoming Saucy Salamander release of Ubuntu.  Once again, the focus is on user interface polish and bug fixes.  My team was dedicated exclusively to that effort for the Ubuntu 13.04 release cycle and judging from the complaints in the media (and, um, elsewhere involving a naked truth few of us wish to see) about how there was nothing exciting in Unity and it just worked, it was a big success.  Kudos go to Andrea, Brandon, Chris, Marco, Nick, the tireless Sam, to John and Didier, and to the many other faceless members of the Ubuntu community who did what they did to make Unity snappy and smooth.

We shipped Unity 7.0 in Ubuntu 13.04.  The version we’re going to ship in Ubuntu 13.10 is Unity 7.1.

Screenshot of “100 scopes” in action in Saucy Salamander (click to embiggen).

I don’t want to mislead you here:  there is some new functionality coming and in fact has already landed in release previews.  What most people will notice is something we’ve been calling “100 scopes”, which is a greatly expanded set of sources for your searches.  I’m not sure if there are actually one hundred sources or if it’s a good deal more or less, it’s just a name, but it does seem like there are significantly more data sources that are used to satisfy your queries.

One of the other aspects of this “100 scopes” work is that many of the back ends — processes that run in the background performing searches for you so the graphics user interface will continue to operate smoothly — were rewritten to reduce the amount of memory and CPU they consume.

Some of the other upcoming new functions being added include the ability to purchase items from their source directly through the Dash.  In other words, you can search for what you desire using the Dash, select a result, and purchase it directly from, say, Amazon, without even opening up your browser.

Some other changes are going on behind the scenes, mostly related to the exciting mobile-desktop convergence store that’s at the heart of Ubuntu’s future.  That, however, is another story for another day.

The bees are all wrapped snugly in their little home. I used a modified water heater jacket and the handyman’s secret weapon to give them extra protection from the anticipated cold since it often goes below -30 C here for at least a few days and sometimes weeks here in winter, and the hive is a little bit exposed to the wind.

The snow is added insulation.

The only concern is about moisture inside the hive, especially with the hive entrance buried beneath the snow cover.  We’ll find out in the spring.

Ubuntu Phone and the Ubuntu Desktop

Canonical announced today that there are phones running natively with the Ubuntu experience. I’m excited by this, because I think the Ubuntu experience on the phone is well thought out and coherent. I applaud the teams involved for their hard work and success in getting a working Ubuntu phone into peoples hands for some serious testing.

I’d also like to emphasize that the work that Canonical is doing for the phone form factor will not be affecting the work my team is doing on the Ubuntu desktop, at least not for Raring Ringtail. We have a single common Unity that runs across all the form factors, but part of the expression of Unity is that it works appropriately under different circumstances and what’s appropriate for the desktop is not always appropriate for the phone, and vice-versa. This is not Microsoft Windows 8, we’re not going to try to turn your desktop into a big touch-based phone without the dialy-talky bits. I fully expect improvements and enhancements that are a result of the work done for the phone to make their way onto the desktop over the next few releases, but for 13.04 we’re a team focused on bug fixes and usability enhancements.

But I am very excited about the phone.