Back Dat NAS Up

Things I have learned this weekend:

  • mysqldump is an important part of your backup workflow, but be sure to flush.
  • CRON WILLS IT, but only if you incant the ancient prayer with “-e“.
  • NFS root_squash is not a vegetable, but it can turn a capable man into one.
  • ZFS listens to xattr and ACL, but you probably haven’t heard of it.
  • cp can make hard links automatically, but don’t tell Chris Hansen.
  • backintime is the flux capacitor on the rsync DeLorean, but without the Libyan terrorists.

Make Me Krazy Very

Sometimes, I really, really hate software.MakeMKV-too-oldI installed this copy of MakeMKV from the source tarballs two weeks ago. Two. When I attempted to convert a disc this morning, I got this. This is the second time this software has “expired” on me. Funny, it worked just fine before the expiration date.

Numb Dive Thumb Drive Dumb Live

A few days after purchasing my 2012 Honda Civic, I’m beginning to learn a few of its quirks. The car comes stock with an audio unit that plays AM/FM, CD, Line-In, as well as USB support for thumbdrives and iPod. “How cool is that?” I asked. So I loaded up one of my spare thumbdrives with a miscellany of tunes from my collection, hopped in the car, plugged it in, and started scrolling through the list.

What I found was an unsorted collection that had neither rhyme nor reason in its sort order. Tracks from Tool were sorted before Combichrist. All the tracks on The Knife’s “Bleeding the Habitual” were completely out of order. The hell?

I did some digging around. Apparently, the MP3 playback feature sorts all of the files on the thumbdrive according to when they were copied onto the device. Yeah, you read that right. The thumbdrive must be formatted FAT32, an old hard disk format written by Microsoft before it made NTFS the default in all its OSes. If you open a command line window and change drives to one with a FAT32 format and then do ‘dir’, what you’ll find is a somewhat unordered directory listing. Why is that?

Under the hood, FAT32 stores references to all its files one-by-one in a data structure on the disk; when you add a file to a FAT32 directory, the reference to that file is appended to the end of that data structure. When software gets a directory listing, it will, in the very least, get an exact copy of this list. The indexes, you see, are exactly as written: one-by-one, appended to the end. If you want to sort the files on any kind of criteria such as modification date or alphabetical by filename, you must first read in this directory list and sort it yourself with your own software. This is very low-level stuff.

I can understand where the Honda firmware designers opted to sort files based on FAT32 directory entry; it’s cheaper, processor-wise and memory-wise, to query the directory structure and run with that instead of having to read it in and then create the data objects in memory to sort things alphabetically. It takes less time and memory for the radio to do so, and it requires less software engineering talent and money to write and test the code. However, it’s a rather lazy thing to do so, considering that even the smallest of modern embedded computer systems can have as much power and space as necessary if the engineer is crafty enough.

All told, I think it was a budgetary decision on Honda’s part; if their engineers can save $5 per unit for however many millions of units sold, then they’ve made their upper management very, very happy. Someone’s getting bonuses. The downside is that the end user gets a higher degree of confusion which is exactly the kind of thing nobody needs when they’re behind the wheel. To find their music in their collection, they’d have to move their eyes from the road to the screen to scrutinize page-by-page to find the music they want. This is a bad thing — distracted driving kills.

So. How do we work around this shortcoming? Well, luckily for us there is a Linux command-line utility called Fatsort. You can use it on any FAT32-formatted partition to rewrite all of the directory entries so that they’re in alphabetical order. A typical invocation is as follows:

fatsort -f -I -c /dev/sdg1

What this does is to apply an alphabetic sort while (-c) ignoring case, (-I) ignoring prefixes like “a” and “the”, and (-f) forcing the reorder on the mounted filesystem at /dev/sdg1. Your invocation will most likely differ.

After dumping all the music I could fit on the thumbdrive, I processed it with fatsort and plugged it into the head unit. Now all my artists, albums, and songs are sorted alphabetically so I can keep a mental map of where to scroll without taking my eyes off the road too much. I can now know with confidence that King’s X is just before KMFDM.

I wonder if Honda has an updated firmware ROM that fixes this sort of problem. One can only hope. Then again, asking a specialist mechanic at the dealership might be like talking to a wall. Not everybody pays attention to low-level details.

Swap-o-Rama (¡UUID Es Me Llamo!)

(Just for the sake of remembering my own damn self — and maybe helping someone else.)

Earlier, I attempted to hibernate my desktop. Never done it in the 15 months I’ve had Ubuntu 8.04 LTS on here, so I figured “Hell, why not?” I hit the dropdown, and the system commenced its hibernating shutdown action. But it neither shut off nor came back to life. I ssh’d into it to discover via the system logfile that the hibernate was halfway through its operations when the power daemon discovered “Oh Hell! There’s no resume device on this machine!” at which point it ceased and attempted to raise the machine from the dead.

After manually power-cycling the system, I did some digging. Turns out the UUID of the swap/resume partition had been changed somewhere along the way, so both /etc/fstab and /etc/initramfs-tools/conf.d/resume were stuck with the old UUID from way back when the system was installed. That UUID no longer exists.

I’m not sure what caused it to change; such an idea is disturbing, considering this is supposed to be the super-stable, LTS build of Ubuntu. The UUID of the swap partition is usually set with the mkswap -U command, and can be set on an ext2/ext3 partition with the tune2fs command. But anything can go wrong.

So. How to make fix? A quick check with sudo blkid will list the block devices on the system:

oldman@Deskie:~$ sudo blkid
/dev/sda1: UUID="c97b2e80-a5b2-4371-a60e-126a8d1402a5" TYPE="ext3"
/dev/sda5: TYPE="swap" UUID="f3a3cd16-f0c8-40a1-8a7d-578c57296c6b"
/dev/sda6: UUID="775a56e3-0244-453f-8d8a-fd1eb42a82f6" /
SEC_TYPE="ext2" TYPE="ext3"

The swap partition has the UUID f3a3cd16-f0c8-40a1-8a7d-578c57296c6b and is the device /dev/sda5. Copy and paste the UUID into the fstab and resume config files to replace the UUID that was there. Execute sudo swapon -a to mount your swap partition, and you should be golden. Verify with free -m to check for a nonzero swap size.

oldman@Deskie:~$ free -m
             total       used       free     shared    buffers     cached
Mem:          3962       1784       2178          0        154        660
-/+ buffers/cache:        969       2993
Swap:        11601          0      11601

The value 11601 tells me I’m gettin’ my swap on. Aw yeah.

Ubuntu Have No Idea How Debian I Am On Freshness

I spent some time Saturday night nerding out by upgrading my laptop’s OS from Ubuntu 9.04 to Debian Testing (Squeeze).

I say “upgrade” because moving to Debian is a major step up from Ubuntu. Canonical, Inc., the company that publishes Ubuntu, essentially takes a snapshot of the entire Debian project and all its software packages and over the span of a few months sorts out which packages are worthy of going into the next Ubuntu release, and which ones are too incomplete to make the cut. Meaning that by the time a Ubuntu version is released, the software within, especially from the active projects, is at least a few months old.

If you’re one of those late adopters, like me, by the time the dust settles after a release and you finally decide to install the new version, you could easily be trying to tolerate software that’s over a year old. And if you’re using one of the LTS (Long Term Support) versions, like I am on my desktop, then you’re looking at 2 years with no functionality improvements; nothing but security updates. That’s unacceptable.

The straw that broke the camel’s back is the crappy bluetooth stack that was packaged with Ubuntu 9.04. They froze the BlueZ package during a time when the project was undergoing a major push in improvement, so the version that got frozen was largely incomplete and had gaping holes in its functionality and stability. Again, this was largely unacceptable, because the best versions were posted, available for installation, right on the project’s website and was available as a package on Debian Testing, but nowhere to be found on Ubuntu.

See, I come from the Windows world, which to me means I can grab any piece of software, no matter how fresh or stale, and install it for use at any time. I’m used to that paradigm. But that kind of capability isn’t part of the Ubuntu/Debian package management ethos, and it sticks in my craw. Case in point, my desktop system is saddled with Firefox version 3.0.16 — the latest is 3.5. All I’ve gotten for the past 2 years is security updates. On any other OS, I could be rocking with the latest version’s bells and whistles. But not Ubuntu. This is weak sauce.

Hopefully sticking with the Testing branch of Debian will help me get the freshest packages; once any project publishes a new version with bugfixes and functionality improvements, Debian picks it up, does some essential testing during the Unstable phase, and then pushes it upstream to Testing. Following this track means taking some measured risk; in trying to keep up with the newest stuff, breaking something fundamental to the system’s function is a possibility (incompatible libraries being the biggest risk). It also means more frequent updates are required to stay current. But, right now, I feel it’s a worthwhile gamble.

Ubuntu’s coup de grâce is that they take the bother of finding the right drivers for most hardware and package them into their distribution media so most computer systems function right out of the box. This is their strength. They also take the hassle of putting together a cohesively-matched set of packages that work well together with hardly any configuration necessary. A beginner can throw Ubuntu on a box and within the evening be doing their thing with almost no effort. That’s commendable. Contrast that with the driver confusion I experienced (video and wireless drivers weren’t loaded so automatically), and you can understand why people who say “Yeah, I need to learn Linux” try Ubuntu first.

But I’ve already taken that class. Now it’s time for my bluetooth mouse and I to graduate.