jump to navigation

Application isolation with AppArmor – part II February 8, 2013

Posted by jdstrand in canonical, security, ubuntu.
add a comment

Last time I discussed AppArmor, gave some background and some information on how it can be used. This time I’d like to discuss how AppArmor is used within Ubuntu. The information in this part applies to Ubuntu 12.10 and later, and unless otherwise noted, Ubuntu 12.04 LTS (and because we also push our changes into Debian, much of this will apply to Debian as well).


  • Ubuntu follows the upstream policy layout and naming conventions and uses the abstractions extensively. A few abstractions are Ubuntu-specific and they are prefixed with ‘ubuntu’. Eg /etc/apparmor.d/abstractions/ubuntu-browsers.
  • Ubuntu encourages the use of the include files in /etc/apparmor.d/local in any shipped profiles. This allows administrators to make profile additions and apply overrides without having to change the shipped profile (will need to reload the profile with apparmor_parser, see /etc/apparmor.d/local/README for more information)
  • All profiles in Ubuntu use ‘#include <tunables/global>’, which pulls in a number of tunables: ‘@{PROC}’ from ‘tunables/proc’, ‘@{HOME}’ and ‘@{HOMEDIRS}’ from ‘tunables/home’ and @{multiarch} from ‘tunables/multiarch’
  • In addition to ‘tunables/home’, Ubuntu utilizes the ‘tunables/home.d/ubuntu’ file so that ‘@{HOMEDIRS}’ is preseedable at installation time, or adjustable via ‘sudo dpkg-reconfigure apparmor’
  • Binary caches in /etc/apparmor.d/cache are used to speed up boot time.
  • Ubuntu uses the /etc/apparmor.d/disable and /etc/apparmor.d/force-complain directories. Touching (or symlinking) a file with the same name as the profile in one of these directories will cause AppArmor to either skip policy load (eg disable/usr.sbin.rsyslogd) or load in complain mode (force-complain/usr.bin.firefox)


Policy load happens in 2 stages during boot:

  1. within the job of a package with an Upstart job file
  2. via the SysV initscript (/etc/init.d/apparmor)

For packages with an Upstart job and an AppArmor profile (eg, cups), the job file must load the AppArmor policy prior to execution of the confined binary. As a convenience, the /lib/init/apparmor-profile-load helper is provided to simplify Upstart integration. For packages that ship policy but do not have a job file (eg, evince), policy must be loaded sometime before application launch, which is why stage 2 is needed. Stage 2 will (re)load all policy. Binary caches are used in both stages unless it is determined that policy must be recompiled (eg, booting a new kernel).

dhclient is a corner case because it needs to have its policy loaded very early in the boot process (ie, before any interfaces are brought up). To accommodate this, the /etc/init/network-interface-security.conf upstart job file is used.

For more information on why this implementation is used, please see the upstart mailing list.


AppArmor profiles are shipped as Debian conffiles (ie, the package manager treats them specially during upgrades). AppArmor profiles in Ubuntu should use an include directive to include a file from /etc/apparmor.d/local so that local site changes can be made without modifying the shipped profile. When a package ships an AppArmor profile, it is added to /etc/apparmor.d then individually loaded into the kernel (with caching enabled). On package removal, any symlinks/files in /etc/apparmor.d/disable, /etc/apparmor.d/force-complain and /etc/apparmor.d are cleaned up. Developers can use dh_apparmor to aid in shipping profiles with their packages.

Profiles and applications

Ubuntu ships a number of AppArmor profiles. The philosophy behind AppArmor profiles on Ubuntu is that the profile should add a meaningful security benefit while at the same time not introduce regressions in default or common functionality. Because it is all too easy for a security mechanism to be turned off completely in order to get work done, Ubuntu won’t ship overly restrictive profiles. If we can provide a meaningful security benefit with an AppArmor profile in the default install while still maintaining functionality for the vast majority of users, we may ship an enforcing profile. Unfortunately, because applications are not designed to run under confinement or are designed to do many things, it can be difficult to confine these applications while still maintaining usability. Sometimes we will ship disabled-by-default profiles so that people can opt into them if desired. Usually profiles are shipped in the package that provides the confined binary (eg, tcpdump ships its own AppArmor profile). Some in progress profiles are also offered in the apparmor-profiles package and are in complain mode by default. When filing AppArmor bugs in Ubuntu, it is best to file the bug against the application that ships the profile.

In addition to shipped profiles, some applications have AppArmor integration built in or have AppArmor confinement applied in a non-standard way.


An AppArmor sVirt driver is provided and enabled by default for libvirt managed QEMU virtual machines. This provides strong guest isolation and userspace host protection for virtual machines. AppArmor profiles are dynamically generated in /etc/apparmor.d/libvirt and usually you won’t have to worry about AppArmor confinement. If needed, profiles for the individual machines can be adjusted in /etc/apparmor.d/libvirt/libvirt-<uuid> or in all virtual machines in /etc/apparmor.d/abstractions/libvirt-qemu. See /usr/share/doc/libvirt-bin/README.Debian.gz for details.


LXC in Ubuntu uses AppArmor to help make sure files in the container cannot access security-sensitive files on the host. lxc-start is confined by its own restricted profile which allows mounting in the container’s tree and, just before executing the container’s init, transitioning to the container’s own profile. See LXC in precise and beyond for details. Note, currently the libvirt-lxc sVirt driver does not have AppArmor support (confusingly, libvirt-lxc is a different project than LXC, but there are longer term plans to integrate the two and/or add AppArmor support to libvirt-lxc).


Ubuntu ships a disabled-by-default profile for Firefox. While it is known to work well in the default Ubuntu installation, Firefox, like all browsers, is a very complex piece of software that can do much more than simply surf web pages so enabling the profile in the default install could affect the overall usability of Firefox in Ubuntu. The goals of the profile are to provide a good usability experience with strong additional protection. The profile allows for the use of plugins and extensions, various helper applications, and access to files in the user’s HOME directory, removable media and network filesystems. The profile prevents execution of arbitrary code, malware, reading and writing to sensitive files such as ssh and gpg keys, and writing to files in the user’s default PATH. It also prevents reading of system and kernel files. All of this provides a level of protection exceeding that of normal UNIX permissions. Additionally, the profile’s use of includes allows for great flexibility for tightly confining firefox. /etc/apparmor.d/usr.bin.firefox is a very restricted profile, but includes both /etc/apparmor.d/local/usr.bin.firefox and /etc/apparmor.d/abstractions/ubuntu-browsers.d/firefox. /etc/apparmor.d/abstractions/ubuntu-browsers.d/firefox contains other include files for tasks such as multimedia, productivity, etc and the file can be manipulated via the aa-update-browser command to add or remove functionality as needed.


A complain-mode profile for chromium-browser is available in the apparmor-profiles package. It uses the same methodology as the Firefox profile (see above) with a strict base profile including /etc/apparmor.d/abstractions/ubuntu-browsers.d/chromium-browser whichcan then include any number of additional abstractions (also configurable via aa-update-browser).

Lightdm guest session

The guest session in Ubuntu is protected via AppArmor. When selecting the guest session, Lightdm will transition to a restrictive profile that disallows access to others’ files.


The libapache2-mod-apparmor package ships a disabled-by-default profile in /etc/apparmor.d/usr.lib.apache2.mpm-prefork.apache2. This profile provides a permissive profile for Apache itself but allows the administrator to add confinement policy for individual web applications as desired via AppArmor’s change_hat() mechanism (note, apache2-mpm-prefork must be used) and an example profile for phpsysinfo is provided in the apparmor-profiles package. For more information on how to use AppArmor with Apache, please see /etc/apparmor.d/usr.lib.apache2.mpm-prefork.apache2 as well as the upstream documentation.


AppArmor also has a PAM module which allows great flexibility in setting up policies for different users. The idea behind pam_apparmor is simple: when someone uses a confined binary (such as login, su, sshd, etc), that binary will transition to an AppArmor role via PAM. Eg, if ‘su’ is configured for use with pam_apparmor, when a user invokes ‘su’, PAM is consulted and when the PAM session is started, pam_apparmor will change_hat() to a hat that matches the username, the primary group, or the DEFAULT hat (configurable). These hats (typically) provide rudimentary policy which declares the transition to a role profile when the user’s shell is started. The upstream documentation has an example of how to put this all together for ‘su’, unconfined admin users, a tightly confined user, and somewhat confined default users.


aa-notify is a very simple program that can alert you when there are denials on the system. aa-notify will report any new AppArmor denials by consulting /var/log/kern.log (or /var/log/audit/audit.log if auditd is installed). For desktop users who install apparmor-notify, aa-notify is started on session start via /etc/X11/Xsession.d/90apparmor-notify and will watch the logs for any new denials and report them via notify-osd. Server users can add something like ‘/usr/bin/aa-notify -s 1 -v’ to their shell startup files (eg, ~/.profile) to show any AppArmor denials within the last day whenever they login. See ‘man 8 aa-notify’ for details.

Current limitations

For all AppArmor can currently do and all the places it is used in Ubuntu, there are limitations in AppArmor 2.8 and lower (ie, what is in Ubuntu and other distributions). Right now, it is great for servers, network daemons and tools/utilities that process untrusted input. While it can provide a security benefit to client applications, there are currently a number of gaps in this area:

  • Access to the DBus system bus is on/off and there is no mediation of the session bus or any other buses that rely on Unix abstract sockets, such as the accessibility bus
  • AppArmor does not provide environment filtering beyond having the ability to clear the very limited set of glibc secure-exec variables
  • Display management mediation is not present (eg, it doesn’t protect against X snooping)
  • Networking mediation is too coarse-grained (eg you can allow ‘tcp’ but cannot restrict the binding port, integrate with a firewall or utilize secmark)
  • LXC/containers support is functional, but not complete (eg, it doesn’t allow different host and container policy for the same binary at the same time)
  • AppArmor doesn’t currently integrate with other client technologies that might be useful (eg gnome-keyring, signon/gnome-online-accounts and gsettings) and there is no facility to dynamically update a profile via a user prompt like a file/open dialog

Next time I’ll discuss ongoing and future work to address these limitations.

Mirror, mirror… January 15, 2013

Posted by jdstrand in security, ubuntu.

Recently I wanted to mirror traffic from one interface to another. There is a cool utility from Martin Roesch (of Snort fame) called daemonlogger. It’s use is pretty simple (note I created the ‘daemonlogger user and group on my Ubuntu system since I wanted it to drop privileges after starting):
$ sudo /usr/bin/daemonlogger -u daemonlogger \
  -g daemonlogger -i eth0 -o eth1

Sure enough, examing tcpcump output, anything coming in on eth0 was mirrored to eth1. I then used a crossover cable and used tcpdump on the other system and it all worked as advertised. This could be very useful for an IDS.

Then I thought how cool it would be to do this for multiple interfaces, but then I quickly realized that would be confusing for whatever was trying to interpret that traffic. I read that someone used a combination of daemonlogger and vtun for a remote snort monitor to work in EC2, which got me playing….

In my test environment, I again used a crossover cable to connect two systems (let’s call them the ‘server’ and the ‘monitor’). The server system runs vtund. The monitor system then connects as a vtun client. When this happens, tap devices are created on the server and the monitor. The vtund server is setup to start daemonlogger on connect to mirror the traffic from eth0 to the tap0 device. The monitor system can then listen on the tap device and get all the traffic. I then created a second vtun tunnel and mirrored from eth1 to tap1. The monitor could monitor on both tap devices. Neat!

To do accomplish this, I used this /etc/vtund.conf on the server:options {
 type stand;
 port 5000;
 syslog daemon;
 ifconfig /sbin/ifconfig;
default {
  type ether;
  proto udp;
  compress no;
  encrypt no;
  multi yes;
  keepalive yes;
  srcaddr {
   iface eth2;
internal {
  password pass0;
  device tap0;
  up {
   ifconfig "%d up";
   program /usr/bin/daemonlogger "-u daemonlogger -g daemonlogger -i eth0 -o %d";
  down {
   ifconfig "%d down";
   program /usr/bin/pkill "-f -x '/usr/bin/daemonlogger -u daemonlogger -g daemonlogger -i eth0 -o %d'";
external {
  password pass1;
  device tap1;
  up {
   ifconfig "%d up";
   program /usr/bin/daemonlogger "-u daemonlogger -g daemonlogger -i eth1 -o %d";
  down {
   ifconfig "%d down";
   program /usr/bin/pkill "-f -x '/usr/bin/daemonlogger -u daemonlogger -g daemonlogger -i eth1 -o %d'";

Ther server’s /etc/default/vtun has:RUN_SERVER=yes

And here is the client /etc/vtund.conf:options {
  port 5000;
  ifconfig /sbin/ifconfig;
default {
  type ether;
  proto udp;
  compress no;
  encrypt no;
  persist yes;
internal {
  device tap0;
  password pass0;
  up {
   ifconfig "%d up";
  down {
   ifconfig "%d down";
external {
  device tap1;
  password pass1;
  up {
   ifconfig "%d up";
  down {
   ifconfig "%d down";

The monitor system’s /etc/default/vtun looks like this:CLIENT0_NAME=internal

With the above, I was able to start suricata (a snort-compatible rewrite that is multithreaded) to listen to both tap0 and tap1 on the monitoring system and see all the traffic. Very cool. :)

Now, this isn’t optimized for anything– sending 2 interfaces’ worth of traffic through one interface is likely going to result in droppoed packets to the monitor if the links are busy. Could maybe use compression to help there. Also, the server really needs to have an extra interface for vtun traffic that isn’t mirrored, otherwise you end up mirroring the vtun traffic itself (in my test environment, eth2 on the server used and the montior used Finally, vtun doesn’t offer any meaningful line encryption so if you are doing this for anything resembling production, you’re going to want to use a dedicated network segment between the server and the monitor (a crossover cable worked fine in my test environment).


Application isolation with AppArmor – part I January 11, 2013

Posted by jdstrand in canonical, security, ubuntu.
Tags: , ,
add a comment

A lot of exciting work has been going on with AppArmor and this multipart series will discuss where AppArmor is now, how it is currently used in Ubuntu and how it fits into the larger application isolation story moving forward.

Brief History and Background

AppArmor is a Mandatory Access Control (MAC) system which is a Linux Security Module (LSM) to confine programs to a limited set of resources. AppArmor’s security model is to bind access control attributes to programs rather than to users. AppArmor confinement is provided via profiles loaded into the kernel. AppArmor profiles can be in one of two modes: enforcement and complain. Profiles loaded in enforcement mode will result in enforcement of the policy defined in the profile as well as reporting policy violation attempts (either via syslog or auditd) such that what is not allowed in policy is denied. Profiles in complain mode will not enforce policy but instead report policy violation attempts. AppArmor is typically deployed on systems as a targeted policy where only some (eg high risk) applications have an AppArmor profile defined, but it also supports system wide policy.

Some defining characteristics of AppArmor are that it:

  • is root strong
  • is path-based
  • allows for mixing of enforcement and complain mode profiles
  • uses include files to ease development
  • is very lightweight in terms of resources
  • is easy to learn
  • is relatively easy to audit

AppArmor is an established technology first seen in Immunix and later integrated into SUSE and Ubuntu and their derivatives. AppArmor is also available in Debian, Mandriva, Arch, PLD, Pardus and others. Core AppArmor functionality is in the mainline Linux kernel starting with 2.6.36. AppArmor maintenance and development is ongoing.

An example AppArmor profile

Probably the easiest way to describe what AppArmor does and how it works is to look at an example, in this case the profile for tcpdump on Ubuntu 12.04 LTS:

#include <tunables/global>
/usr/sbin/tcpdump {
  #include <abstractions/base>
  #include <abstractions/nameservice>
  #include <abstractions/user-tmp>
  capability net_raw,
  capability setuid,
  capability setgid,
  capability dac_override,

  network raw,
  network packet,

  # for -D
  capability sys_module,
  @{PROC}/bus/usb/ r,
  @{PROC}/bus/usb/** r,

  # for finding an interface
  @{PROC}/[0-9]*/net/dev r,
  /sys/bus/usb/devices/ r,
  /sys/class/net/ r,
  /sys/devices/**/net/* r,

  # for tracing USB bus, which libpcap supports
  /dev/usbmon* r,
  /dev/bus/usb/ r,
  /dev/bus/usb/** r,

  # for init_etherarray(), with -e
  /etc/ethers r,

  # for USB probing (see libpcap-1.1.x/
  # pcap-usb-linux.c:probe_devices())
  /dev/bus/usb/**/[0-9]* w,

  # for -z
  /bin/gzip ixr,
  /bin/bzip2 ixr,

  # for -F and -w
  audit deny @{HOME}/.* mrwkl,
  audit deny @{HOME}/.*/ rw,
  audit deny @{HOME}/.*/** mrwkl,
  audit deny @{HOME}/bin/ rw,
  audit deny @{HOME}/bin/** mrwkl,
  owner @{HOME}/ r,
  owner @{HOME}/** rw,

  # for -r, -F and -w
  /**.[pP][cC][aA][pP] rw,

  # for convenience with -r (ie, read 
  # pcap files from other sources)
  /var/log/snort/*log* r,

  /usr/sbin/tcpdump r,

  # Site-specific additions and overrides. See 
  # local/README for details.
  #include <local/usr.sbin.tcpdump>

This profile is representative of traditional AppArmor profiling for a program that processes untrusted input over the network. As can be seen:

  • profiles are simple text files
  • comments are supported in the profile
  • absolute paths as well as file globbing can be used when specifying file access
  • various access controls for files are present. From the profile we see ‘r’ (read), ‘w’ (write), ‘m’ (memory map as executable), ‘k’ (file locking), ‘l’ (creation hard links), and ‘ix’ to execute another program with the new program inheriting policy. Other access rules also exists such as ‘Px’ (execute under another profile, after cleaning the environment), ‘Cx’ (execute under a child profile, after cleaning the environment), and ‘Ux’ (execute unconfined, after cleaning the environment)
  • access controls for capabilities are present
  • access controls for networking are present
  • explicit deny rules are supported, to override other allow rules (eg access to @{HOME}/bin/bad.sh is denied with auditing due to ‘audit deny @{HOME}/bin/** mrwkl,’ even though general access to @{HOME} is permitted with ‘@{HOME}/** rw,’)
  • include files are supported to ease development and simplify profiles (ie #include <abstractions/base>, #include <abstractions/nameservice>, #include <abstractions/user-tmp>, #include <local/usr.sbin.tcpdump>)
  • variables can be defined and manipulated outside the profile (#include <tunables/global> for @{PROC} and @{HOME})
  • AppArmor profiles are fairly easy to read and audit

Complete information on the profile language can be found in ‘man 5 apparmor.d’ as well as the AppArmor wiki.

Updating and creating profiles

AppArmor uses the directory heirarchy as described in policy layout, but most of the time, you are either updating an existing profile or creating a new one and so the files you most care about are in /etc/apparmor.d.

The AppArmor wiki has a lot of information on debugging and updating existing profiles. AppArmor denials are logged to /var/log/kern.log (or /var/log/audit/audit.log if auditd is installed). If an application is misbehaving and you think it is because of AppArmor, check the logs first. If there is an AppArmor denial, adjust the policy in /etc/apparmor.d, then reload the policy and restart the program like so:

$ sudo apparmor_parser -R /etc/apparmor.d/usr.bin.foo
$ sudo apparmor_parser -a /etc/apparmor.d/usr.bin.foo
$ <restart application>

Oftentimes it is enough to just reload the policy without unloading/loading the profile or restarting the application:

$ sudo apparmor_parser -r /etc/apparmor.d/usr.bin.foo

Creating a profile can be done either with tools or by hand. Due to the current pace and focus of development, the tools are somewhat behind and lack some features. It is generally recommended that you profile by hand instead.

When profiling, keep in mind that:

  • AppArmor provides an additional permission check to traditional Discretionary Access Controls (DAC). DAC is always checked in addition to the AppArmor permission checks. As such, AppArmor cannot override DAC to provide more access than what would be normally allowed.
  • AppArmor normalizes path names. It resolves symlinks and considers each hard link as a different access path.
  • AppArmor evaluates file access by pathname rather than using on disk labeling. This eases profiling since AppArmor handles all the labelling behind the scenes.
  • Deny rules are evaluated after allow rules and cannot be overridden by an allow rule.
  • Creation of files requires the create permission (implied by w) on the path to be created. Separate rules for writing to the directory of where the file resides are not required. Deletion works like creation but requires the delete permission (implied by w). Copy requires ‘r’ of the source with create and write at the destination (implied by w). Move is like copy, but also requires delete at source.
  • The profile must be loaded before an application starts for the confinement to take effect. You will want to make sure that you load policy during boot before any confined daemons or applications.
  • The kernel will rate limit AppArmor denials which can cause problems while profiling. You can avoid this be installing auditd or by adjusting rate limiting in the kernel:
    $ sudo sysctl -w kernel.printk_ratelimit=0


There is a lot of documentation on AppArmor (though some is still in progress):

Next time I’ll discuss the specifics of how Ubuntu uses AppArmor in the distribution.

Thanks to Seth Arnold and John Johansen for their review.

OpenSSH and multiple identities November 29, 2012

Posted by jdstrand in security, ubuntu.
add a comment

While this may be old news to some, I only just now figured out how to conveniently use multiple identities in OpenSSH. I have several ssh keys, but only two that I want to use with the agent: one for personal use and one for work. I’d like to be able to not specify which identity to use on the command line most of the time and just use ssh like so:

$ ssh <personal>
$ ssh <work>
$ ssh -i ~/.ssh/other.id_rsa <other>

where the first two use the agent with ~/.ssh/id_rsa and ~/.ssh/work.id_rsa respectively, and the last does not use the agent. ‘man ssh_config’ tells me that the agent looks at all the different IdentityFile configuration directives (in order) and also the IdentitiesOnly option. Therefore, I can set up my ~/.ssh/config to have something like:
# This makes it so that only my default identity
# and my work identity are used by the agent
IdentitiesOnly yes
IdentityFile ~/.ssh/id_rsa
IdentityFile ~/.ssh/work.id_rsa

# Default to using my work key with work domains
Host *.work1.com *.work2.com
    IdentityFile ~/.ssh/work.id_rsa

With the above, it all works the way I want. Cool! :)

Overfeeding the Planet April 15, 2011

Posted by jdstrand in canonical, security, ubuntu.

I just learned to my chagrin that wordpress.com would update my atom feed every time I clicked the ‘Update’ button in my blog. As a result, I have recently been spamming Planet Ubuntu pretty heavily lately, for which I apologize. Thanks to jcastro and especially nigelb for figuring out that the fix was simply to have the planet use the rss feed instead of the atom feed. While this post/apology could be argued to be yet more spam, I thought I’d pass this information along so others might check their feeds and adjust accordingly. The wiki has been updated for this change, and you can reference the change made to my planet-ubuntu settings in Launchpad if you want to check your settings.

Unity and Me April 8, 2011

Posted by jdstrand in ubuntu, uncategorized.

I have been wanting to write this post for a while, and thought I might do it now before Ubuntu releases Natty. :)

I’ll admit when Mark announced that Unity would be the new desktop for Ubuntu, I was skeptical. I always liked the indicators work, but had used an otherwise pretty standard Gnome desktop for years, and liked it. I upgraded to Natty very early in the cycle and have been using Unity for months. Things were quite rocky at first, with instability issues and features either gone or partially implemented. Of course that is to be expected since massive amounts of development work was being done on it. Today, the features are there and Unity is quite stable for me. It is getting very close to release, but AIUI the Unity developers are working very hard to squish the remaining stability bugs. You can read more about the decision to stick with Unity, but that is not what I am writing about today. I’m writing about why as an Ubuntu user I like Unity. Keep in mind, I am not a Unity developer and have just picked things up along the way and this isn’t meant to be an exhaustive list of features or bugs. Just what I like and and dislike.

The Cool
I will certainly miss some great stuff in Unity here, but I’ll mention what I know about:

  • Unity gets out of my way. It reminds me of minimalist desktops from ages ago, but is modern and beautiful
  • It is keyboard friendly
  • The datetime indicator. Not only does it show me the calendar when I click on it, it shows me my appointments for today
  • The global menu. I was pretty sure I didn’t like this at first, but I am actually at a point now where I like it. You always know where to go and it cuts down on wasted space. On my 1440×900 resolution latop screen, this is appreciated (as is the lack of bottom panel)
  • AppIndicators are cool. I’ve always liked them and they are better than ever in Natty. NetworkManager, sound, power, Me, social: all functional, easy to navigate and out of the way.
  • Launcher quicklists. It is easy to add them and they provide cool extra functionality for applications. Right click on evolution or the Applications list to see what I mean.
  • Software search via Alt+F2 or the BFB (aka Big Friendly Button in the upper left– you know, the Ubuntu logo :). While the search results aren’t always perfect, they often are and this functionality is way cool. Eg, try typing in ‘network’ and see what pops up.
  • The applications menu is really neat. Right click on the Launcher icon to see the familiar Applications menu folder entries. Clicking any of them brings you to a place with most recently used applications, installed applications for the category you selected, or suggestions for download.
  • Files and Folders is similarly implemented, with familiar places/categories available via right click. Selecting a place shows you things of that type you’ve edited today, yesterday or within the last week or more.
  • The trash is conveniently located. It doesn’t waste space on the Desktop or clutter the panel and you can drag and drop things onto it.
  • Update (2011/04/15): Superkey shortcuts (just learned about them :). Eg:
    • hold the ‘Super’ (aka ‘meta’) key and the first 10 Launcher icons will show a number. You can then do ‘Super+#’ to launch or bring to focus that application. ‘Super+shift+#’ will launch a second instance of the application.
    • ‘Super+s’ will bring up your workspaces, just as if you click the workspaces launcher. You can then navigate via the arrow keys to choose the workspace to move to.
    • ‘Super+w’ will give you an exploded view of all of your open applications, and again you can use the arrow keys to select the window to bring to focus. It is sorta like Alt+Tab, but with all your open applications.

Tweaks and other things you should know
There are probably also a lot more of these than what I will list here, but I’m just a simple man with simple tastes. ;)

  • To use Unity at all you need a 3D capable card that it supports. Apparently Unity/Compiz uses all kinds of OpenGL and if Unity doesn’t detect that it will run well on your computer, you can use Ubuntu Classic instead. unity-2d might be interesting as an alternative as well for those needing a strictly 2D experience.
  • Middle-clicking on a Launcher will launch another instance of the application
  • Use the power indicator/System Settings to get to the the old ‘System’ menu and preferences in the classic desktop
  • The Files & Folders menu provides similar functionality to the Places menu in the classic desktop
  • If you launch an application via Alt+F2 or the Applications place, you can keep it in the panel for later use by right clicking on the the icon and selecting ‘Keep in Launcher’
  • There is an open bug where when launching an application it shows up under the Launcher, so the Launcher autohides. This can be worked around via CompizConfig Settings Manager (Power Button/System Settings/Personal/CompizConfig Settings Manager, or ‘ccsm’ from a terminal). In ccsm, select Desktop and click on Ubuntu Unity Plugin and select ‘Hide Launcher: Never’. I’m told this bug is being fixed soon, but it might be good to know you can adjust the Launcher behavior anyway. Update (2011/04/09): This is fixed as of today. I confirmed it by setting autohide back to ‘Dodge Windows’ and opening a gagillion terminals. Yes, that’s right, a gagillion. :)
  • You can also adjust the Launcher icon size in ccsm. In the Ubuntu Unity Plugin settings, select the Experimental tab and adjust ‘Launcher icon size’. On my laptop the standard size is good, but on a big monitor, they are too large (might be neat if Unity could detect that somehow….)
  • I sorely missed my weather applet. Well, people stepped up and wrote ‘indicator-weather’
  • ‘Ctrl+Alt+t’ opens a terminal. Apparently this was around for awhile in Gnome but I didn’t know about it. Very handy regardless.
  • Use Alt+F2 to search for and run applications
  • The window selector:
    • right click: zooms in
    • left single click: selects window
    • left double click: selects window and zooms
  • Can drag and drop onto the trash
  • The Files & Folders menu provides similar functionality to the Places menu in the classic desktop
  • Custom launchers can be used, but are not readily supported via a GUI afaict. I use profiles in firefox and wanted to middle click on the firefox Launcher and launch the profile manager. This is how I did that:
    1. Create a new desktop file with:$ mkdir ~/bin/unity
      $ cp /usr/share/applications/firefox.desktop ~/bin/unity
      $ chmod 755 ~/bin/unity/firefox.desktop
    2. Update the ‘Exec=…’ entry in ~/bin/unity/firefox.desktop to have:Exec=firefox -Profile-Manager -no-remote %u
    3. Finally, open the file manager (Nautilus) and drag and drop ~/bin/unity/firefox.desktop onto the Launcher
  • Update (2011/04/10): Custom launchers for terminal applications can also be used, which can be very useful for applications such as irssi and mutt. The thing to remember is that you’ll want to specify a different window manager class for the terminal, otherwise after you start your application via the Launcher, it will show up with all your other terminals and you can’t use a superkey keyboard shortcut with it. For example, to create a launcher to login to another server, you can use something like the following for a .desktop file (see above for how to get this into the Launcher):[Desktop Entry]
    Name=My Server
    Comment=Login to my server
    Exec=gnome-terminal --sm-client-disable --disable-factory --class=MyServer -x ssh -t myserver.example.com
  • Update (2015/04/27): gnome-terminal removed ‘–disable-factory’ (https://bugzilla.gnome.org/show_bug.cgi?id=707899) so if you were relying on it to separate different window manager classes, this will no longer work. ‘mate-terminal’ seems to be a suitable replacement
  • Update (2011/04/09): There are a bunch of other tweaks and questions and answers at askubuntu.com:

The Uncool
Alas, there are a few things I miss and/or people might need to know about:

  • hamster-applet (a time tracker) integration in the panel. Someone is actually working on this, so we may have an implementation soon. Otherwise, simply adding the Time Tracker to the Launcher works ok enough
  • The workspace switcher isn’t as functional as the old Gnome one. It is certainly pretty, but if you use workspaces extensively, you may need to adjust your workflow (eg, by using superkey shortcuts). I actually use workspaces in the way that they are implemented in Unity, so this wasn’t a big deal for me. Work is ongoing in this area, but improved functionality won’t be in Natty release.
  • Unless an application has indicator support, it won’t be integrated into the panel. Things I miss here are the system monitor and sensors-applet. Someone has started work in this area as well. One could go really old school in the meantime (like I did) and use gkrellm to monitor things. There is a certain old-World charm there for such a modern desktop, but it would be nice to have these indicators already available.
  • Stability issues are much improved these days (they better be, we are almost at release! ;) and I haven’t had to use these in several weeks, but am listing them here for completeness:
    1. ‘compiz’ will restart Compiz, the 3D window manager (in the past I would use ‘compiz –replace’, but I’m told that is no longer the correct way)
    2. ‘unity –reset’ will completely reset Unity to shipped defaults and restart it (Note: any ccsm changes will have to be reapplied after running this command)
  • Update (2011/04/09): Focus follows mouse, aka point to focus, aka sloppy focus is not currently supported and using it will likely lead to frustration. Personally, I’ve been click to focus for many years now, so this didn’t get in my way….

I like Unity and to me Unity is clean, easy to use, out of the way, functional and fits my workflows. I didn’t know how much I liked it until I had to stop using it for a week and a half. The release schedule for Unity was very aggressive, but I am happy to say that if the stability bugs are addressed in time, Unity should be a very nice desktop for Natty users. If when trying it out you find a bug, please file it with:$ ubuntu-bug unity no matter where it is in the stack (eg, unity, compiz, global menu, etc). The developers are very responsive and I bet they will be adding more and more bug fixes before release as well as queuing up others for SRU after.


A bite out of the Apple December 27, 2010

Posted by jdstrand in ubuntu.

Being the “computer guy” for friends and family is both demanding and rewarding. On the one hand, I get to install Ubuntu on lots of computers. On the other hand, I end up having to deal with various hardware and devices. For many years I have successfully avoided Apple’s proprietary products, but recently I’ve had to deal with them. Let me first say that the hardware I’ve gotten my hands on is excellent, and let me then say that having worked with open source for so long, it is surprising at how hard Apple works at making sure you are locked into using only their stuff. Also, am I the only one who finds their faux-ubiquity in TV and movies annoying? I mean, how is it that virtually every laptop, monitor and computer has the Apple logo emblazoned on it? Clearly, their marketing is at least as excellent as their hardware, but I digress….

After an annoying foray into the world of the MacBook Pro for a friend (btw, with all the Ubuntu developers I see at UDS with Apple hardware, why isn’t it better supported in the default install? I mean, I felt like I stepped back 5 years with all the twiddling I had to do with the silly touchpad to get a halfway sane configuration — I’d fix it in Ubuntu myself, but the device is out of my hands now), I finally was confronted with trying to get a new, off-the-shelf iPod Touch to work with Ubuntu 10.10. Pretty and shiny as it was, let’s just say it did not work well with Ubuntu out of the box. Looking at https://help.ubuntu.com/community/PortableDevices/iPod and the video at http://www.libimobiledevice.org/ I thought it was going to be relatively easy, but there were several roadblocks:

  1. Currently, you can’t just open a new iPod and use it with a non-iTunes system (ie, Windows and OS X — last I checked, the latest iTunes doesn’t work too well in Wine). As soon as you plug it into a USB port, there is a helpful graphic telling you that you need to connect to iTunes. This step seems to be required to download the latest firmware for the device and register it.
  2. In addition to the above, you must do an initial music sync from iTunes to initialize the music databases on the device.
  3. This new iPod Touch only does USB 2.0 and won’t fallback to 1.1 — a frustrating tactic that works quite well at driving revenue since people all of a sudden feel that their perfectly good computer yesterday is old and outdated after getting their shiny new iPod today. Ranting aside, qemu-kvm (my preferred virtualization technology) does not provide a USB 2.0 host controller.
  4. The version of usbmuxd in Maverick (Ubuntu 10.10) is too old. Among other things, it installs a udev rule to start usbmuxd which is used to coordinate the communications between the device and Ubuntu (see the README file in /usr/share/doc/usbmuxd for a detailed explanation of an IMHO too complex protocol). The udev rule for Maverick did not catch the idProduct of my device (0x129e) and would not start usbmuxd.
  5. While hand-editing /lib/udev/rules.d/85-usbmuxd.rules and running ‘sudo udevadm control –reload-rules’ did get udev to start usbmuxd, usbmuxd didn’t know how to deal with my device. I guess there was a reason the udev rule was so specific to begin with. ;)
  6. Apple’s latest version of iOS (4.2.1) broke stuff for Linux users. AIUI, libimobiledevice is the (excellent) library that pulls all the disparate parts of dealing with iPods together and makes it so that music players like Rhythmbox work seamlessly with with the device. Upstream fixed the 4.2.1 problem quickly, but the version of libimobiledevice in Maverick is too old.
  7. As of the time of this writing, DbVersion 5 is not supported by libgpod, so after drag and dropping files in Rhythmbox to the device, the iTunes database on the iPod is not updated and the files don’t show up. This is also being tracked in Ubuntu as LP: #601251.

See what I mean about Apple trying very hard to make sure that their devices work with only their stuff? Merry Christmas Jamie, you get to fiddle with several highly proprietary systems for hours on end! *Sigh*. Of course this isn’t Ubuntu’s fault; Apple doesn’t provide iTunes for Linux and they keep changing things in incompatible ways while the open source developers furiously try to keep up (admirably).

So what did I do to get the thing to work? Well, I didn’t because of the libgpod DbVersion 5 support. That said, I am to the point now where I resolved everything except this, so when the new libgpod comes out, I’ll be ready. Here is what I did to address each of the above:

  • I didn’t have a Windows install handy, but do have a legal installation disk for Windows XP. I installed it via virt-manager, but ran into a bug where the domain xml used ‘kvm’ as the emulator instead of qemu-system-i386, so when I rebooted the machine after stage 1 of the install, I got a black screen (I still need to file a bug on this). Not sure if it is still required in Maverick, but I also disabled all the <features/> except <pae/> (ie, ACPI and APIC). I installed iTunes, plugged the device in and… nothing. qemu-kvm doesn’t provide a USB 2.0 controller. D’oh! Now, I could conceivably do PCI passthrough for the USB 2.0 controller on the host, but that required fiddling with the pci-stub kernel module so that the controller isn’t being used by the ehci_hcd module on the host. I’d like to play with this more, but haven’t found good documentation on it yet.
  • At this point, I still didn’t have Windows handy for what I needed. The non-OSE version of VirtualBox does provide a USB 2.0 controller if you also install the ‘Oracle_VM_VirtualBox_Extension_Pack-4.0.0-69151.vbox-extpack’. So I removed the kvm* modules from the kernel, installed VirtualBox, installed the extension pack, installed Windows XP in the guest, installed iTunes, plugged the device in, then told VirtualBox to make it available in the guest. This got me to some sort of wizard to install the firmware and register the device. Using iTunes on Windows XP in VirtualBox worked flawlessly.
  • From within iTunes, I did a music sync, ejected the device, then shutdown the VM
  • In Ubuntu, I added Paul McEnery’s ppa and performed an upgrade. This got me several library updates, but not everything (Update: it now has everything but a new libgpod). In particular, you need (all from the ppa, except where noted):
    • usbmuxd 1.0.6
    • libimobiledevice 1.0.4 (and be sure to install libimobiledevice-utils)
    • ipheth (for tethering, not strictly needed)
    • A version of libgpod4 that supports DbVersion 5 (in progress)

Now when I plug in the device in Ubuntu, I am prompted to open F-Spot and Rhythmbox, the device is mounted via gvfs, the device shows up on the Desktop and I can read anything on the device via nautilus. ideviceinfo works, and presumably so do all the other idevice* utilities. I can also copy files to the device via nautilus (though make sure you see the ‘Sync in Progress’ message on the iPod before unplugging!), but they don’t show up in the iPod (because I am not using libgpod). Now I just need a new libgpod to support the DbVersion 5 database so it all works in Rhythmbox (I can use iTunes in the VirtualBox VM using a shared folder to add files to the device in the meantime). Beyond getting Rhythmbox to work, I’d like to be able to copy videos and pictures to the device. Maybe once libgpod is in order I can use gtkpod (there are also notes in the aforementioned wiki documentation). If I can get this to all work right, I may document all this step by step.

While the person I am setting this up for is generally happy (and patiently waiting for proper Rhythmbox support), this whole experience has felt more like Apple took a bite out of me. Here’s to hoping this blog helps others with new iPods!

Update (2010/12/28): While I haven’t personally tried any of the following, I’m told:

Update (2010/12/29): As pointed out by Christophe in the comments, ideviceactivate does not work with the iTouch 4G, but patches have been submitted. Also, he says you only need to do the ihash/sync music step with DbVersion 4 devices, so other devices will be ok.

Show the date in Natty’s Unity December 13, 2010

Posted by jdstrand in ubuntu.

By default in Unity on Natty the clock only shows the time but doesn’t provide a GUI method to display the date. Since I never seem to know the date, and only sometimes the day, I like to have my clock display these to me. This is bug #601974. The quick fix:

$ gsettings set org.ayatana.indicator.datetime show-date true
$ gsettings set org.ayatana.indicator.datetime show-day true

Since this was my first foray into gsettings, I figured I’d also post how I figured this out. First I looked at the output of:

$ gsettings help

Then I drilled down until I found what I needed:

$ gsettings list-schemas|grep datetime
$ gsettings list-keys org.ayatana.indicator.datetime

$ gsettings get org.ayatana.indicator.datetime show-date
$ gsettings set org.ayatana.indicator.datetime show-date true
$ gsettings get org.ayatana.indicator.datetime show-date

I filed bug #689722 to request exposing this via the GUI.

Update: dconf-editor from dconf-tools can be used to edit gsettings in a graphical way (like gconf-editor for gconf). Thanks to mdeslaur for pointing this out! :)

Update: The gsettings path moved to /com/canonical/indicator/datetime in recent indicator-datetime versions. See bug #717318 for details.

Compiz vs Ubuntu Classic Desktop December 1, 2010

Posted by jdstrand in ubuntu.

I am running the development version of Ubuntu (the Natty Narwhal). I’ve tried the Unity desktop (and will continue to do so) but for reasons I won’t go into here, I need to use the Ubuntu Classic Desktop for now. After today’s update I could no longer login to a functional Ubuntu Classic Desktop because of bug #683686. There were a number of things that went wrong and I wasted an hour on trying to work it out (thank you to didrocks and seb128 for helping me). Here is what I’ve learned:

  • Do not disable the Unity plugin in CompizConfig Settings Manager while in Unity (or enable it when not in Unity)
  • To use Unity, login to GDM with ‘Ubuntu Desktop’
  • To use the traditional desktop, login to GDM with ‘Ubuntu Classic Desktop’
  • If after logging in to the Ubuntu Classic Desktop your window manager does not start, this might be bug #683686. To work around it, logout, move your ~/.config/compiz-1 aside (logging into a console first), then log back in with GDM like normal. This bug is actively being worked on.
  • There is a known bug with compiz and the gnome-panel that may cause applets to not load. Logging out and back in again usually solves this. This bug is actively being worked on.

Hope this helps anyone suffering from the same problems I did. Please file bugs if you are having other problems with Compiz, Unity or the Ubuntu Classic Desktop.

A fun hack: triggered reverse ssh connections for remote access to anywhere November 8, 2010

Posted by jdstrand in security, ubuntu.

So, like a lot people, I get asked to install Ubuntu on friend’s and family’s computers. I talk to them about what their use cases are and more often than not (by far) I install it on their systems. That’s cool. What is less cool is tech support for said installation. Not that I mind doing it or that there is a lot to do, but what becomes problematic is when they go home and are sitting behind their NAT router from their ISP, and I can’t just connect to them to fix something I forgot or to help them out of a jam. Before I go any farther let me say that what I am going to describe is never done on machines without the owner’s permission and that I am always upfront in that my account on his/her machine is an administrative account that has access to everything on the system (barring any encryption they use, of course). I should also mention that what I am describing is more in the ‘fun hack that other people might like’ category, and not in the ‘serious systems administration’ category. In other words, this is total crack, but it is fun crack. :)

Now, there are a lot of ways to do this, and I have tried some and surely missed many others. Here are a few I’ve tried:

  • Giving realtime tech support over the phone
  • Giving realtime tech support over some secure/encrypted chat mechanism
  • Email support
  • OpenSSH access from my machine to theirs, including adjusting their router for port forwarding
  • VPN access from their machine to my network, at which point I can OpenSSH to their machine

Rather than going through a comparison of all the different techniques, let’s just say they all have issues: realtime support almost always entails describing some obscure incantation to fix the problem, which is neither confidence inspiring for them and is extremely time consuming. Email is too slow. Routers get reset and straight OpenSSH doesn’t work when they are away from home. VPN access is not bad, especially with the use of OpenVPN client and server certificates. It has the added benefit of being opt-in by the user, and is easy to use with the network-manager-openvpn-gnome package. It is probably the most legitimate form of access, and should be highly considered, especially for corporate environments. The only real issues are that some draconian networks will block this VPN traffic and that my IP happens to change so they have to fiddle with their connection setting. So I started doing something different.

Remotely triggered reverse OpenSSH connections
The basic idea is this: on the client install OpenSSH, harden it a bit, install a firewall so that no one can connect to it, then create a cron job to poll an HTTP server you have access to for an IP address to connect to, then create a reverse SSH connection to that IP address. If this sounds a little shady and a bit like a botnet, well, you’d be right, but again, I did ask for permission first. :)

So, more specifically, on the remote machine (ie, the one you want to administer):

  • Install openssh-server and set /etc/ssh/sshd_config with the following:
    # Force key authentication (ie, no passwords)
    PasswordAuthentication no
    # Only allow logins to my account
    AllowUsers me
    Obviously, you will need to copy your ssh key over to this machine (man ssh-copy-id) before restarting OpenSSH and putting the above into effect.
  • Setup a firewall. Eg:$ sudo ufw enable
  • Create some passwordless ssh keys. Eg:
    $ ssh-keygen -f revssh.id_rsa
  • Create a script to poll some HTTP server, then create the reverse connection (this is an abridged script. I’ve omitted error checking and locking for brevity):#!/bin/sh
    hname=`hostname | cut -f 1 -d '.'`
    ip=`elinks -no-home -dump "http://your.web.server/$hname" | head -1 | awk '{print $1}' | egrep '^[1-9][0-9\.]*$'` || {
    #echo "Could not obtain ip" >&2
    exit 0
    echo "Connecting to $ip"
    # we use StrictHostKeyChecking=no so that keys gets added without prompting
    ssh -oStrictHostKeyChecking=no -i $HOME/.ssh/revssh.id_rsa -p 8080 -NR 3333:localhost:22 revssh@"$ip" sleep 30 || true

To remotely administer a machine, create a file on the server with the name of the hostname of the client to have only the external IP address of your network in it, and then the remote client will pick it up and try to setup a reverse ssh connection on port 8080 (usually open even on the most draconian firewalls, but you could also use 53/tcp, 80/tcp or 443/tcp), after which you can connect to the remote machine with something like:$ ssh -t -p 3333 localhost

This is hardly perfect. For one thing, the client is polling an HTTP server so it can easily be man-in-the-middled, but that isn’t a big deal because even if the attacker had full knowledge of this technique, all it gives is a connection to OpenSSH on the client, which is configured to only allow connections from ‘me’ and with my ssh key. This could of course be fixed by connecting with HTTPS and using connection.ssl.cert_verify=1 with elinks. Similarly, the HTTP server could be subverted and under attacker control. For the client, this is no different than the MITM attack in that the attacker really doesn’t have much to work with due to the OpenSSH configuration. Also, the client completely ignores the fingerprint of the server it is connecting to, but again, not huge deal because of our OpenSSH configuration on the client, but you will need to be extra careful in checking fingerprints when connecting to the reverse connection.

You need to remember to update your webserver (ie, just remove the file) so the client isn’t always trying to connect to you. Also, it is somewhat inconvenient that when you logout the reverse connection is still there, but instead of using ‘exit’ to logout of the remote machine, use something like: $ kill -9 `ps auxww | grep [s]sh | grep 3333 | awk '{print $2}'`

While is it relatively easy to setup the client, it is somewhat harder to setup the initiating end. First off, you need to have a webserver that can be accessed by the client. Then you need to have the ssh server on your local machine listen on port 8080 (done either via a port redirect or a separate ssh server). You also need to setup a non-privileged ‘revssh’ user (eg, /bin/false for the shell, a disabled password, etc) on your machine. If you are behind a firewall/router you need to allow connections through your firewall to your machine so that the connection to port 8080 from the client is not blocked. Finally, if you remotely administer multiple clients you will want to keep track of their ssh fingerprints, because when you connect to ‘-p 3333 localhost’ they will conflict with each other (most annoying, but workable). I have written a ‘revssh_allow’ script to automate the above for me (not included, as it is highly site-specific). It will: fire up an sshd server on port 8080 that is specially configured for this purpose, adjust my local firewall to open port 8080/tcp from the client, connect to my router to set up the port redirection to my machine, then poll (via ‘netstat -atn | grep “:3333.*LISTEN”‘) for the connection from the client, then remind me how to connect to the client and how to properly kill the connection.

So yeah, this is a fun hack. Is it something to put in production? Probably not. Does it work for administering friend’s and family’s computers? Absolutely, but I’ll have to see how well it works over the long haul.

Have fun!