Shell

Enabled the Zsh completion system

I’ll admit it: I’m a bit slow when it comes to the shell. I use it a lot, but never feel like I’m using it as well as I could. But today, I figured out how to turn on Z Shell completion system. And it is very, very good.

This guide helped me, leading me to add the following to my .zshrc file:


autoload -U compinit
compinit

Once I’d done that, I could begin completing various commands and parameters. Within a few moments, using the tutorial above, I’d already completed:

  • command names
  • file and directory paths
  • changing a directory with cd, listing only directories
  • listing each directory that would be extracted from a tar archive
  • ssh destination, including user and host
  • changing to a directory three-levels deep with cd, using only the first letter of each of the three paths (i.e. “/u/l/b” for “/usr/local/bin”).

This is cool.

Read the Mac OS X edition and version from property lists

When scripting for system administration, it’s often helpful to know what edition and version of Mac OS X is installed.

By edition, I mean whether you are using Mac OS X or Mac OS X Server — Server generally being a superset of the client operating system. The version, of course, is an indentifier such as “10.4.9” or “10.3” or “10.2.8.”

Given the version number, you can deduce the marketing name of the system software: “Tiger,” “Panther,” and “Jaguar,” respectively. Interestingly, I’m aware of no string in the system that identifies that marketing name; even “About This Mac” in the Apple menu does not display it. (However, you can find all of the names in the Mac OS X entry in Wikipedia.)

You can obtain the version information in a number of ways. From a scriptability standpoint, the quickest way to the chase is probably the sw_vers utility. The basic usage and output looks like this:

$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.4.9
BuildVersion: 8P2137

… but you can also, as of Mac OS X 10.3 or later, narrow it down to get just the string you want. (Note that you can only run sw_vers, without modification, in Jaguar and earlier.) So, if you just need the ProductVersion, ask for it:

$ sw_vers -productVersion
10.4.9

One drawback of this solution is that you can only get the version information for a running copy of Mac OS X. According to its man page, sw_vers
“prints version information about the Mac OS X or Mac OS X Server operating
system running on the local machine.” It can’t detect the version of a
non-running Mac OS X installation.

Happily, the same information is stored in two property list files in the /System directory. Therefore, you can read the version data directly from these plist files.

  • For Mac OS X workstation or client: $volume/System/Library/CoreServices/SystemVersion.plist
  • For Mac OS X Server:
    $volume/System/Library/CoreServices/ServerVersion.plist

… where $volume is the path to the target volume. The ServerVersion.plist file does not exist on workstation/client edition of Mac OS X. The mere presence of the ServerVersion.plist, therefore, indicates that you have encountered Mac OS X Server.

So, from the command line, you could enter:

$ defaults read /System/Library/CoreServices/SystemVersion 'ProductVersion'
10.4.9

… or even …

$ defaults read '/Volumes/Mac OS X Install DVD/System/Library/CoreServices/SystemVersion' 'ProductVersion'
10.4

Reading the plists comes in handy if you’re trying to detect the version of Mac OS X on a disk other than the current startup disk, where sw_vers fails. This means you can even detect the version of Mac OS X installed on Apple’s boot CDs and DVDs!

The plist method is also useful if you’re using a scripting language — such as Python, with its plistlib module (I’m sure there’s something for Perl, too … and you could also try defaults as above) — that can read them. If the files can be read directly, you don’t need to call out to a shell command, even one as simple as sw_vers or defaults, from within the scripting language. That means you won’t be spawning an additional process, and your scripts can therefore gain a slight speed increase.

Here is example defaults output from a running Mac OS X workstation (a Core 2 Duo-based MacBook Pro):

{
ProductBuildVersion = 8P2137;
ProductCopyright = "1983-2007 Apple Inc.";
ProductName = "Mac OS X";
ProductUserVisibleVersion = "10.4.9";
ProductVersion = "10.4.9";
}

And from a Tiger install DVD, originally for PowerPC:

{
ProductBuildVersion = 8A428;
ProductCopyright = "1983-2005 Apple Computer, Inc.";
ProductName = "Mac OS X";
ProductUserVisibleVersion = "10.4";
ProductVersion = "10.4";
}

To show that this is not a fluke, here’s the same result from a Panther install CD (you must use Disc 1, as it is the startup disc in the set):

{
ProductBuildVersion = 7B85;
ProductCopyright = "Apple Computer, Inc. 1983-2003";
ProductName = "Mac OS X";
ProductUserVisibleVersion = "10.3";
ProductVersion = "10.3";
}

I expect that this applies all the way back to Mac OS X 10.0 from March 24, 2001. To date, though, I’ve verified this behavior as far back as a Jaguar install CD:

{
ProductBuildVersion = 6C115;
ProductCopyright = "Apple Computer, Inc. 1983-2002";
ProductName = "Mac OS X";
ProductUserVisibleVersion = "10.2";
ProductVersion = "10.2";
}

Update: I did run the same test on a Mac OS X 10.0 install CD from 2001. The results show that you can use the property list reading method to obtain the version information for any release of Mac OS X:

{
ProductBuildVersion = 4K78;
ProductName = "Mac OS X";
ProductVersion = "10.0";
}

However, note that the ProductUserVisibleVersion and ProductCopyright keys are not in the plist of the Cheetah system.

The information above was adapted from my+similar+post+on+the+MacEnterprise+list.

Shell with color in Tiger

Here is how I colorized my shell environment in Mac OS X Tiger:

  1. Added “CLICOLOR” and set it to “1” in the ~/.MacOSX/environment.plist, using the handy environment editor in SSHKeychain
  2. Changed my terminal type to “dtterm” in Terminal’s “Preferences” window.NOT FOUND: TerminalZshAndTerm.jpg

I don’t recall exactly how I got to this point, but I’d never seen a hint that provided color with this level of ease — and I recently fell into a pique of wanting but not having colorized shell output.

Updating a base Mac OS X system image using shadow files

When deploying system software with disk images, it is helpful to have various checkpoint images that you can revert to while you’re building up a fully-fledged template computer. This is something they teach you in school (really, I was taught it in a systems administration class) and it’s more or less encoded in the solution accelerator documentation for Microsoft’s Business Desktop Deployment 2007 for Windows.

However, if you’re updating images, keeping the base and intermediate images can strain your storage capacity. Mac OS X lacks the compelling live editing features of Microsoft’s new WIM image format — which if it had appeared first on the Mac, I’d be trumpeting loudly, so I feel compelled to at least give a nod to Microsoft here.

Since I’m always struggling with storage capacity and I prefer having an up-to-date base image, I thought about this problem a bit in the context of Mac OS X imaging and have come upon what seems to be a unique solution: the use of shadow files.

Here’s the basic idea:

  1. Create a base operating system install on a partition of your template computer’s disk
  2. Capture a compressed, read-only image of it immediately (go wild, save the extra percent or two, and compress it with the Tiger-only bzip2 scheme … you can afford to do this if you have time and are only deploying with a Tiger startup disk)
  3. Scan the base image for Apple Software Restore’s block-level checksums
  4. Attach that base image to the filesystem — honoring ownership and specifying a shadow file — so that it acts as if it is writable, using hdiutil
  5. Install the latest Mac OS X combo update on it
  6. Create a new read-only compressed image of the mounted volume
  7. Prep the new updated base image for ASR.

Congratulate yourself on this use of shadow files, because you’ve saved at least one intermediate step and the space required for a full read-write disk image — or worse, an extra local partition needed only for restoration and updating the base image.

Unmount that volume and throw away the shadow file at this point if you want, because you’ve now got two system images ready for deployment. One has the base system software, and serves as a checkpoint that you can return to later; it’s the base for all future updates of that major revision of Mac OS X. The other image has the latest version of Mac OS X. If you’re deploying that image with ASR, the result will be a more secure system because it’s closer to being fully patched — and it should take less time to update it with the additional security updates and application installs — whether you use installers or Radmind or another solution — because you’ve got the bulk of the operating system done.

Unfortunately, many updates can only be installed on the startup disk and thus cannot be included in the updated base image. Beyond the combo operating system updates, few of Apple’s other installers will work on a non-startup volume. But you can install them after deploying the updated base image, using your tool of choice. For reasons like this, Geoff doesn’t see updating the base image as valuable, but in some IT environments, it may be very worthwhile.

My next step is to script this process and tie it to a watched folder. Imagine dropping a combo update into a watched folder … and letting a script generate the new, updated image for you.

Thoughts on Mac OS X 10.4.9 from the MacEnterprise extended KB article

Based on my reading of the MacEnterprise.org extended knowledgebase article on Mac OS X 10.4.9, some interesting changes have been made. Here are some of my comments on items listed in that article:

  • “Includes iChat support for USB Video Class webcams,” which seems to mean that UVC webcams will work without additional drivers on this version of the OS … so I doubt we’ll see a replacement for the standalone Apple iSight and I wouldn’t doubt we’ll just have to make do with the third-party opportunities this presents.
  • “Resolves an issue when using Kerberos authentication with Active Directory if the user is a member of many groups,” which may mean that my problem with binding to Active Directory over UDP using an account with a large TokenGroups attribute is resolved … presumably, if it is fixed, the kpasswd utility now supports TCP as well (although I do not see a change to /usr/bin/kpasswd in a Radmind transcript, the Kerberos framework did change).
  • “Adds support for WPA2 encryption in Network Diagnostics,” which is an option that is coming at just the right time for me.
  • “Includes updated security certificates,” which should mean that the root SSL certificates database has been updated (I note in a Radmind transcript that both X509Anchors and X509Certificates has changed with this update).
  • I’m not exactly hip with whatever change happened in /private/var/db/sudo. Based on what I’m seeing, a subdirectory for each sudo user will appear; so far, I’ve only seen this on Radmind model systems, and so I see one subdirectory with a blank “ttyp1” file inside. I think I might want to put this new directory in a negative transcript for Radmind.

[Via Philip Rinehart on the MacEnterprise.org list.]

No books

I’ve just done a quick search on zsh at Amazon. I think I’ve finally hit the final frontier — or is it just rock bottom? I’m obsessed with a technology without any books specifically devoted to it. Ha!

The closest is “From Bash to Z Shell,” as far I can tell.

On the way to Site5

I’ve decided to follow Sthomas’ referral to Site5, taking them up on their $5 hosting deal. It simply came down to a price I was willing to pay.

I figure that I’ll probably save around that much electricity per month by not having to run my server at home. I’ll regain some upstream bandwidth on my Internet connection, which can be put to other uses. And, my son will not have so many whirring fans running in his room, since we haven’t finished remodeling the room all the technology is moving to.

I had been hoping to get:

  • a several GB storage allotment
  • a high bandwidth allotment
  • High or unlimited hosted sites
  • Full DNS control (for potential use with DNS-SD, perhaps)
  • Dedicated IP address
  • SSL
  • Greater than 5 hosted databases (well, just because)
  • SSH/SFTP
  • Shell access
  • Python 2.3 or later, preferably with mod_python (using FastCGI seems to be a big workaround)
  • WebDAV
  • QuickTime/Darwin Streaming Server
  • Reliable, consistent server performance
  • Good customer service (even if you never need it, it should be there)

I had to compromise on several points, but ultimately the price point was important to me. I had to balance some items that I wanted versus what I felt was needed, and consider some items which could be added onto an account locally rather than by the hosting provider.

It’ll take a little time for me to sort it out, but I’ll be moving this site to my new hosting account shortly. Hopefully, I can make it happen without an outage.

I had also considered the following hosting companies, and each had compelling offerings that were just not outweighed by $5 per month. I’ll list some of the benefits and drawbacks for each of these well-reviewed major hosting firms.

  • Dreamhost: WebDAV, QuickTime/Darwin Streaming Server, more quota space
  • WebFaction: considered very good for Python (including mod_python, which seems to be a rarity) and open source hosting, but had some of the most limited stats (like only three hosted dynamic sites for $7/month with a two year commitment)
  • Bluehost: considered good for Python, good stats.

In the future, I'm still considering some arrangement with Rsync.net or another off-site backup/storage provider. I think it’s an interesting time to be on the Internet, when you can really start to take advantage of some truly useful hosted services.

Pondering the switch to zsh

Geoff got me thinking about zsh last fall, and it’s on my mind again. I’m pondering a switch to it.

I’m still stuck on tcsh because it’s what I started with for my heavy interactive shell use. All my shell config files and knowledge reside in tcsh, because that’s what you used by default in Mac OS X 10.0. But, I dislike that the tcsh syntax is different than what I’m used to when it comes to shell scripting with bash. Bash is what I learned when I first took up scripting. Well, more to the point, I learned sh, and have grown a bit into using some bash-isms.

So I’m reading more about zsh, which means I’ll probably be firmly rooted in analysis paralysis for a while. But it looks good, and my chances in converting will improve if I can get my .tcshrc custom-tailoring moved over with reasonable ease.

In the meantime, here’s a search for zsh on Del.icio.us.

Compressed Mac OS X disk image statistics

I wanted to determine how much space I’d save—and what the compression time tradeoff might be—to compress Mac OS X system images with gzip versus bzip2. Both are options in Mac OS X Tiger, but the bzip2 compression is new in that release (and thus not backward compatible with Panther and earlier). Given that Apple’s own disk imaging is the bedrock of almost all imaging solutions from Mac OS X, including those from third party computer management solutions providers, this information is valuable and widely applicable on the Mac platform.

If the time were the same but I saved some disk space, I’d count that as a win for one compression choice or the other. For significant space savings, I’d even sacrifice time to a longer conversion operation because disk image compression and ASR prep can be handled in the background during downtime or operational lulls. A Folder Action or launchd task might be ideal for that, and might even be a useful option if you are just archiving images.

The two disk image formats I tested are UDZO and UDBZ, as defined by the hdiutil command and its man page.

For an UDZO version of the image (gzip compression at level 9—or best),
I got the following results from hdiutil convert:

Elapsed Time: 1h 1m 50.751s
File size: 4226869141 bytes, Checksum: CRC32 $8CF3F127
Sectors processed: 20184610, 19980489 compressed
Speed: 2.6Mbytes/sec
Savings: 59.1%

I ran the hdiutil command with time (a handy utility if you haven’t used it), to get these figures:

real 61m52.133s
user 51m7.575s
sys 2m47.700s

For the same image converted to UDBZ (bzip2 compression):

Elapsed Time: 2h 16m 28.057s
File size: 3956667545 bytes, Checksum: CRC32 $8CF3F127
Sectors processed: 20184610, 19980489 compressed
Speed: 1.2Mbytes/sec
Savings: 61.7%

The time results for that were:

real 136m29.205s
user 106m1.832s
sys 8m30.739s

You may draw your own conclusions, as appropriate to the data and your situation. The source image was the same size, nearly 10 GB, which included Mac OS X Tiger and a suite of applications with a small number of user accounts. The source and converted files were both stored on an external FireWire 400 hard disk, and converted with a 1.5-GHz 12-inch PowerBook G4.

Although the ASR-prepped (i.e. asr --imagescan) gzip and bzip2 images were not that different in the end, the bzip2 version definitely came out smaller. It has in ever test I’ve ever run. However, it did take twice as long to create the image. So, if you’re strapped for disk space to collect your images and CPU time means less to you than storage (which is often the case for me, given that I have access to a lot of G4-class Mac hardware), I would consider bzip2. However, gzip at level 9 is perfectly acceptable (still providing almost 60% savings in this case), is faster to create, and has the benefit of working on older versions of Mac OS X, as well.

I should try gzip at other settings (like levels 1 and 6, for example), because you can’t choose level 9 in the Disk Utility interface; you must use the command line -imagekey flag. (You can get bzip2 from the GUI if you enable the hidden advanced disk image formats, but I still don’t see a way to choose a higher gzip level.)

Syndicate content