jump to navigation

A bite out of the Apple December 27, 2010

Posted by jdstrand in ubuntu.

Being the “computer guy” for friends and family is both demanding and rewarding. On the one hand, I get to install Ubuntu on lots of computers. On the other hand, I end up having to deal with various hardware and devices. For many years I have successfully avoided Apple’s proprietary products, but recently I’ve had to deal with them. Let me first say that the hardware I’ve gotten my hands on is excellent, and let me then say that having worked with open source for so long, it is surprising at how hard Apple works at making sure you are locked into using only their stuff. Also, am I the only one who finds their faux-ubiquity in TV and movies annoying? I mean, how is it that virtually every laptop, monitor and computer has the Apple logo emblazoned on it? Clearly, their marketing is at least as excellent as their hardware, but I digress….

After an annoying foray into the world of the MacBook Pro for a friend (btw, with all the Ubuntu developers I see at UDS with Apple hardware, why isn’t it better supported in the default install? I mean, I felt like I stepped back 5 years with all the twiddling I had to do with the silly touchpad to get a halfway sane configuration — I’d fix it in Ubuntu myself, but the device is out of my hands now), I finally was confronted with trying to get a new, off-the-shelf iPod Touch to work with Ubuntu 10.10. Pretty and shiny as it was, let’s just say it did not work well with Ubuntu out of the box. Looking at https://help.ubuntu.com/community/PortableDevices/iPod and the video at http://www.libimobiledevice.org/ I thought it was going to be relatively easy, but there were several roadblocks:

  1. Currently, you can’t just open a new iPod and use it with a non-iTunes system (ie, Windows and OS X — last I checked, the latest iTunes doesn’t work too well in Wine). As soon as you plug it into a USB port, there is a helpful graphic telling you that you need to connect to iTunes. This step seems to be required to download the latest firmware for the device and register it.
  2. In addition to the above, you must do an initial music sync from iTunes to initialize the music databases on the device.
  3. This new iPod Touch only does USB 2.0 and won’t fallback to 1.1 — a frustrating tactic that works quite well at driving revenue since people all of a sudden feel that their perfectly good computer yesterday is old and outdated after getting their shiny new iPod today. Ranting aside, qemu-kvm (my preferred virtualization technology) does not provide a USB 2.0 host controller.
  4. The version of usbmuxd in Maverick (Ubuntu 10.10) is too old. Among other things, it installs a udev rule to start usbmuxd which is used to coordinate the communications between the device and Ubuntu (see the README file in /usr/share/doc/usbmuxd for a detailed explanation of an IMHO too complex protocol). The udev rule for Maverick did not catch the idProduct of my device (0x129e) and would not start usbmuxd.
  5. While hand-editing /lib/udev/rules.d/85-usbmuxd.rules and running ‘sudo udevadm control –reload-rules’ did get udev to start usbmuxd, usbmuxd didn’t know how to deal with my device. I guess there was a reason the udev rule was so specific to begin with. ;)
  6. Apple’s latest version of iOS (4.2.1) broke stuff for Linux users. AIUI, libimobiledevice is the (excellent) library that pulls all the disparate parts of dealing with iPods together and makes it so that music players like Rhythmbox work seamlessly with with the device. Upstream fixed the 4.2.1 problem quickly, but the version of libimobiledevice in Maverick is too old.
  7. As of the time of this writing, DbVersion 5 is not supported by libgpod, so after drag and dropping files in Rhythmbox to the device, the iTunes database on the iPod is not updated and the files don’t show up. This is also being tracked in Ubuntu as LP: #601251.

See what I mean about Apple trying very hard to make sure that their devices work with only their stuff? Merry Christmas Jamie, you get to fiddle with several highly proprietary systems for hours on end! *Sigh*. Of course this isn’t Ubuntu’s fault; Apple doesn’t provide iTunes for Linux and they keep changing things in incompatible ways while the open source developers furiously try to keep up (admirably).

So what did I do to get the thing to work? Well, I didn’t because of the libgpod DbVersion 5 support. That said, I am to the point now where I resolved everything except this, so when the new libgpod comes out, I’ll be ready. Here is what I did to address each of the above:

  • I didn’t have a Windows install handy, but do have a legal installation disk for Windows XP. I installed it via virt-manager, but ran into a bug where the domain xml used ‘kvm’ as the emulator instead of qemu-system-i386, so when I rebooted the machine after stage 1 of the install, I got a black screen (I still need to file a bug on this). Not sure if it is still required in Maverick, but I also disabled all the <features/> except <pae/> (ie, ACPI and APIC). I installed iTunes, plugged the device in and… nothing. qemu-kvm doesn’t provide a USB 2.0 controller. D’oh! Now, I could conceivably do PCI passthrough for the USB 2.0 controller on the host, but that required fiddling with the pci-stub kernel module so that the controller isn’t being used by the ehci_hcd module on the host. I’d like to play with this more, but haven’t found good documentation on it yet.
  • At this point, I still didn’t have Windows handy for what I needed. The non-OSE version of VirtualBox does provide a USB 2.0 controller if you also install the ‘Oracle_VM_VirtualBox_Extension_Pack-4.0.0-69151.vbox-extpack’. So I removed the kvm* modules from the kernel, installed VirtualBox, installed the extension pack, installed Windows XP in the guest, installed iTunes, plugged the device in, then told VirtualBox to make it available in the guest. This got me to some sort of wizard to install the firmware and register the device. Using iTunes on Windows XP in VirtualBox worked flawlessly.
  • From within iTunes, I did a music sync, ejected the device, then shutdown the VM
  • In Ubuntu, I added Paul McEnery’s ppa and performed an upgrade. This got me several library updates, but not everything (Update: it now has everything but a new libgpod). In particular, you need (all from the ppa, except where noted):
    • usbmuxd 1.0.6
    • libimobiledevice 1.0.4 (and be sure to install libimobiledevice-utils)
    • ipheth (for tethering, not strictly needed)
    • A version of libgpod4 that supports DbVersion 5 (in progress)

Now when I plug in the device in Ubuntu, I am prompted to open F-Spot and Rhythmbox, the device is mounted via gvfs, the device shows up on the Desktop and I can read anything on the device via nautilus. ideviceinfo works, and presumably so do all the other idevice* utilities. I can also copy files to the device via nautilus (though make sure you see the ‘Sync in Progress’ message on the iPod before unplugging!), but they don’t show up in the iPod (because I am not using libgpod). Now I just need a new libgpod to support the DbVersion 5 database so it all works in Rhythmbox (I can use iTunes in the VirtualBox VM using a shared folder to add files to the device in the meantime). Beyond getting Rhythmbox to work, I’d like to be able to copy videos and pictures to the device. Maybe once libgpod is in order I can use gtkpod (there are also notes in the aforementioned wiki documentation). If I can get this to all work right, I may document all this step by step.

While the person I am setting this up for is generally happy (and patiently waiting for proper Rhythmbox support), this whole experience has felt more like Apple took a bite out of me. Here’s to hoping this blog helps others with new iPods!

Update (2010/12/28): While I haven’t personally tried any of the following, I’m told:

Update (2010/12/29): As pointed out by Christophe in the comments, ideviceactivate does not work with the iTouch 4G, but patches have been submitted. Also, he says you only need to do the ihash/sync music step with DbVersion 4 devices, so other devices will be ok.


1. ethana2 - December 27, 2010

I think we should sue them for the Launchpad trademark and settle for a native Ubuntu version of iTunes.

2. Timo Aaltonen - December 28, 2010

Hah, I had to do the same this weekend :/ Didn’t feel like jailbreaking the device (which was not mine, but my 10yo nephew’s), so I used iTunes exclusively from the original winxp installation on my laptop. Couldn’t get banshee (nor Rhythmbox) to work with it, but seems like I didn’t have all the correct pieces (new libgpod). Now I need to update their home computer running lucid to handle the device, bah.. and it’s 160km away.

Also, at least one upstream could be better informed: http://sourceforge.net/tracker/?func=detail&aid=3109738&group_id=158745&atid=809061 (patch rejected as invalid, though several iPod ID’s were added last summer)

3. Christophe Fergeau - December 29, 2010

* pmcenery ppa now has the needed versions of libimobiledevice and usbmuxd.
* The “have to upload one song or use ihash” step is needed for DBVersion 4 devices, it’s useless for DBVersion 5
* saying that ligpod will soon have DBVersion 5 support is misleading, at best it will be an unsatisfying workaround.
* some patches (sent upstream) are needed for ideviceactivate to work with an itouch 4g

jdstrand - December 29, 2010

I’ve updated the post with your comments. Thanks! Actually, I didn’t say that libgpod will soon have DbVersion 5 support– I said only that it seems like it is being worked on and that as soon as it is available, I will be ready. Can you expand on your comment “at best it will be an unsatisfying workaround”?

Christophe Fergeau - December 29, 2010

Since I now have a github account, I pushed the changes I needed for the itouch4 to https://github.com/cfergeau/ideviceactivate
For DBVersion 5, initial setup might or might not be needed, I don’t really know. Unsatisfying workaround is something depending on a binary blob. And my first name has an “h” after the C :)

jdstrand - December 29, 2010

Thanks for the reply. Hopefully some of the libgpod work involves reversing any needed blobs…. Just now fixed up your name. Sorry about the misspelling!

4. Jim Doyle - January 6, 2011

Thanks for this excellent writeup, it exactly reflects my experiences after our daughters were given ipod touches this Christmas! It summarizes all the points that were causing me trouble and references all the links one would want to follow up.

One of the most misleading ubuntu.com pages I came across is https://help.ubuntu.com/community/PortableDevices/iPhone. It seems to clearly state that 10.04 supports ipod touch 4g out of the box, which it definitely doesn’t.

I posted recently to the gtkpod-devel mailing list where libgpod development is discussed, asking what had to be implemented in libgpod and why. Here are some of the answers:

[me] I’m not 100% sure I understand why changing the database version back [a jailbroken ipod hack that changes it to DBVersion 2] helps…

[cf] DBVersion 2 will make it use the same hashes as a Nano 5g which we know how to compute.

[cf] The iTouch 3g has DBVersion 4. Basically there are 3 hashes, DBVersion 3 used by the iPod Classic and Nano 3g+, DBVersion 4 used by older iOS devices, and DBVersion 5 used by A4 iDevices. DBVersion 3 and 4 are known, DBVersion 5 isn’t.

What I still don’t understand is who figured out DBVersion 3 and 4, and who is in a position to reverse engineer DBVersion 5. It sounds like it’s not the libgpod developers who would do the reverse engineering.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: