warning: Creating default object from empty value in /home/jaharmic/public_html/jaharmi/modules/taxonomy/ on line 33.

Rethought zshrc

I’ve been using zsh for a while as my preferred shell. I have a hacked-together zshrc file, and yet really wanted to use it across multiple systems. Some of those systems are running Mac OS X, others Solaris, and still others Linux. Executables are in a different locations and even have different switches across this range of systems, so my cobbled zshrc was not helping me.

As I was about to fall asleep last night, it finally hit me that fixing my zshrc would be a good thing to do. I jotted down some notes about an idea to reorganize it, and did something about it today.

Of course, since I’ve checked my zshrc and other dotfiles into a [Bad link] repository, I could experiment without fear.

I created three top-level functions, with one “case” statement in each. [Bad link], but they are one of the things I like about shell scripting. These statements do allow the script to make choices based on the host, operating system, or shell that it was running in. (Yeah, it’s a zshrc, but I sometimes do stupid things — like sourcing it in bash on the one Linux system that won’t let me switch to zsh. [Bad link])

I separated all the important sections of my zshrc into their own individual function calls. Each of those function calls was placed into one of the applicable case statements.

The case statement functions figure out the conditions the zshrc is running in, and then run the other functions to set up my environment.

The changes tested well from first try across the various platforms and hosts I log in to. I did have a minor problem with the hostname command, because Solaris doesn’t have a “-s” flag for it. Eventually, I solved that — and the odd “uname: error in setting name: Not owner” error I got, even though I wasn’t directly running uname there — by replacing hostname with uname.

Thankfully, it works for me, and it should be a little easier to manage changes in the future.

Compiling a five-way Universal Binary of Rsync 3 with LLVM-GCC on Leopard

The Xcode 3.1 Developer Tools install both the standard GNU compiler, gcc, and the LLVM compiler, llvm-gcc. I thought I’d try my hand at compiling something with [Bad link], so naturally I decided to update my instructions for compiling Rsync 3 as a Universal Binary for Leopard. Let’s try Rsync 3.0.3!

The following set of instructions, of course, requires that you have already installed the Xcode 3.1, [Bad link]. To choose LLVM as your C compiler temporarily for this compile, specify the CC=“/Developer/usr/bin/llvm-gcc” variable (as explained in [Bad link]).

$ cd /tmp
$ curl -O curl -O
$ tar -xzvf rsync-3.0.3.tar.gz
$ rm rsync-3.0.3.tar.gz
$ curl -O
$ tar -xzvf rsync-patches-3.0.3.tar.gz
$ rm rsync-patches-3.0.3.tar.gz
$ cd rsync-3.0.3
$ patch -p1 <patches/fileflags.diff
$ patch -p1 <patches/crtimes.diff
$ ./prepare-source
$ env CFLAGS="-O -g -isysroot /Developer/SDKs/MacOSX10.5.sdk -arch i386 \
-arch ppc7400 -arch ppc970 -arch ppc64 -arch x86_64" LDFLAGS="-arch i386 \
-arch ppc7400 -arch ppc970 -arch ppc64 -arch x86_64" \
CC="/Developer/usr/bin/llvm-gcc" \
./configure --prefix=/usr/local --disable-dependency-tracking

$ make
$ ./rsync --version
rsync version 3.0.3 protocol version 30
Copyright (C) 1996-2008 by Andrew Tridgell, Wayne Davison, and others.
Web site:
64-bit files, 32-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, no iconv, symtimes, file-flags
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
$ file ./rsync
./rsync: Mach-O universal binary with 5 architectures
./rsync (for architecture i386): Mach-O executable i386
./rsync (for architecture ppc7400): Mach-O executable ppc
./rsync (for architecture ppc970): Mach-O executable ppc
./rsync (for architecture ppc64): Mach-O 64-bit executable ppc64
./rsync (for architecture x86_64): Mach-O 64-bit executable x86_64
$ lipo -info ./rsync
Architectures in the fat file: ./rsync are: i386 ppc7400 ppc970 ppc64 x86_64
$ sudo make install

Warning: I have not tested Rsync 3 compiled with LLVM to ensure that it operates at all or is compatible with Rsync compiled by GCC. I haven’t compared the compilation speed against GCC’s performance, determined whether the files are larger/smaller, or have produced a more optimized binary. You use the software compiled with these instructions at your own risk.

Most of the original instructions were [Bad link]. My contributions are simply:

  • Universal Binary support: four- or five-way, depending on whether you want to use “-arch i386 -arch ppc7400 -arch ppc970 -arch ppc64 -arch x86_64” (which gives specific optimized binaries for PowerPC G4 and G5 systems in 32-bit mode) or “-arch i386 -arch ppc -arch ppc64 -arch x86_64” (for more general 32-bit PowerPC support, as I did in my previous article).
  • LLVM compilation: use of the llvm-gcc front end, which hopefully produces a working Rsync 3.

Start Directory Service debug logging automatically after the next restart on Mac OS X

There can be times when you need to troubleshoot Directory Services in Mac OS X during system starts up. Directory Services apparently sits above and below the kernel — depending on what each of those components needs to do — so having debugging capability early on in the boot process can help you hone in on problems.

To enable debug logging at startup, run the following at the command line:

$ sudo touch /Library/Preferences/DirectoryService/.DSLogDebugAtStart

… and then, on the next restart, debug logging will be enabled. This is equivalent to running:

$ sudo killall -USR1 DirectoryService

… but the advantage is is that debug logging begins at startup, rather than whenever you can log in and start it manually.

Remove the file from the path above when you want to disable the logging for the subsequent restart. As a file whose name starts with a dot, you won’t see it listed by default in the Finder. (The Finder hides dot files by default.)

$ sudo rm /Library/Preferences/DirectoryService/.DSLogDebugAtStart

For more details, see the Apple article on [Bad link]. (Note that directory service logging works on the workstation version of Mac OS X, not just Mac OS X Server.)

Is that site running Drupal, curl edition

The [Bad link] Web site has a clever way to help answer the question of “[Bad link]” Angie Byron mentions that the HTTP “Expires” header returned by Drupal corresponds to a specific default date. Look for that date in the HTTP headers, and you can make a reasonable guess that a site is a Drupal site — or at least one that hasn’t modified one of the core files.

Some commenters posted notes about how to do the same thing with wget and then curl. I expanded on the curl instructions to make them a little more robust (especially in the case of redirects or URL rewriting), and here’s the result:

$ curl -fsIL 2>&1 | grep -q -m 1 "Expires: Sun, 19 Nov 1978 05:00:00 GMT" && echo "Yes, this appears to be a Drupal site." || echo "No, this does not appear to be a Drupal site."
Yes, this appears to be a Drupal site.

Navigating through files with less

It’s taken about a decade but I finally read the man page for “less.” Here are some important keyboard commands for navigating through files with it that I wanted to note for my own future use. If someone else gets a benefit from this, great, but it’s already helping me jump through man pages with more ease. (And it gave me an opportunity to write a post with a really ambiguous title.)

Key Function
f Move forward one screen
b Move backward one screen
j Move forward one line
y Move backward one line
= Show the number of lines and your progress through the file
< Move to the beginning of the file
> Move to the end of the file
% Scroll to the position in the file represented by the number before the percent sign

These commands had eluded me for a long time, even though it would have been great to know them. After all, less is the default pager in Mac OS X. Thanks to the wonders of air travel, I had some time to read some off-line documentation.

Of course, I’ve long used “/” followed by some text to search through man pages, and that’s worth knowing if you don’t already. One useful trick I picked up was the search “/^EXAM” (props to [Bad link] for mentioning the caret and making my recipe one character longer, but more specific) to search for the examples section.

Python to remove commands module and some Mac-specific modules

Drat! I’ve learned that the commands module for Python, which I use, [Bad link]. That means I can no longer safely call commands.getstatusoutput() anymore. I’ve frequently used this call in the past because it seemed the sanest, easiest way to call for a shell utility and get both its output and return status (for success or error).

I’ll have to find some other way to perform the same function — preferably one that will work on Python 2.3 from Mac OS X Tiger, Python 2.5.1 from Leopard, and future Pythons. The stated replacement for a number of similar modules (including popen2, which frankly kind-of frightened me off with its name) is the [Bad link], but I don’t know if that will work for my purposes.

There are also a bunch of Mac-specific modules being removed. I don’t use any of them right now, but that doesn’t mean they wouldn’t have been useful.

This kind of thing is spirit-crushing to me for some reason. I’m especially annoyed that Python has been around for so long and it is still reorganizing the ways it calls shell commands. Just settle on something! It seems hard to take it seriously as a system administration scripting language when things like this happen.

On the other hand, I love so much of the Python Standard Library, which has afforded me a lot for system administration …

Radmind server logging and the repo command

Under normal circumstances, the latest Radmind tools that communicate with the server report client status updates in the Radmind server’s system log. These standard messages can include ones like:

May 8 03:14:56 RadmindServerHost radmind[7890]: report - - ktcheck No updates needed
May 15 03:15:25 RadmindServerHost radmind[24531]: report - - ktcheck Updates retrieved
May 15 03:21:48 RadmindServerHost radmind[24534]: report - - lapply Changes applied successfully
May 15 03:31:07 RadmindServerHost radmind[24356]: report CertificateCN - lapply Error, changes made

The Radmind repo, or “report,” tool provides the ability to send arbitrary messages to the Radmind server process. But how are these messages formatted and sent?

$ repo -e "Debug" -h -w2 "Test message"

… results in the system log message:

May 15 03:31:56 RadmindServerHost radmind[25236]: report CertificateCN - Debug Test message

Here, we can see that an entry created with repo looks like the standard Radmind log messages above. The client hostname and IP address are reported after the “report” text. The CertificateCN for the client — if the highest authorization level is specified (with the -w2 flag) — is also listed; if not, a dash takes its place. I haven’t seen a case where the second dash is substituted, however.

Finally, where the Radmind command/tool used would normally be, the “event” specified by repo will printed. After that, the message text appears.

The value proposition is that if you’re using Radmind, the repo command can help you send arbitrary messages to the server for logging. As bonus, if you've taken the time and effort to build the certificate infrastructure for Radmind, you can send these messages securely between the clients and the server cloaked in SSL.

If you’re using multiple servers, you may want to combine their logs in one location so that you can get all of the clients’ reports in one location. You may also want or need to retain these reports for more time. In either case, determine what policies you should apply to the syslog or Apple System Logger (ASL, for Mac OS X) configuration for your server systems.

Whether or not you use repo, it’s good to know that the tools do some logging. The logging can be followed to try to determine the status of your clients, or whether they are failing their updates.

Unfortunately, the most common client failures I have seen tend to involve the lapply tool, and the default level of detail I’ve seen reported back to the server does not provide an indication of what problem has been encountered. You see only that there was an error. Still, even though you may not get enough detail to remotely resolve the problem, it’s something for you to go by find problems in the first place.

Python string method for title case, versus the Daring Fireball script

I ran a quick test of the Python “title” string method (from Python 2.5.1 in Leopard); it fails the tests on the edge cases mentioned in the [Bad link] post at Daring Fireball. It really only uppercases the first letter of every word and lowercases everything else.

Not much better than looping through a string with ucfirst() in Perl, except you don't have to do the looping yourself.

Drat. I was expecting better.

Computational groups for Leopard directory services

There are occasionally times in system administration where I feel as if I’m on the bleeding edge. They are not frequent … which probably puts me somewhere much closer to the lunatic fringe. These are also the times when no one else probably cares about the same things — but no matter, I generally have my own pragmatic reasons for them. This is one of those times, and I have waited for this particular feature for a while. I'm happy to have discovered it.

Let’s set up a scenario: you have a Mac OS X Tiger system bound to an Active Directory. All of the thousands of Active Directory users are available on that Tiger system; any one of them could log in to that computer. This isn’t bad if you only allow logins at the login window. However, if you enable any sharing services, the game changes and your exposure widens.

What if you don’t want everyone in your enterprise directory to log in? This gets a little scary to think about. What if you want to limit those who can log in, particularly for services like SSH? How do you let some users in but keep others out — even going to the point of denying access to whole classes of users?

There are a number of things you can do, some at the client and some in the directory. Unfortunately, some kinds of configuration must be done at the client. There isn’t a great way I’m aware of to distinguish between local and network accounts, complicates the situation for the administrator. With Open Directory in Tiger, any user from any directory service was as valid as any other. There will probably be some manual effort — ongoing effort at that — required to restrict access.

Enter Mac OS X Leopard. It now has “computational groups” in its Open Directory client. They are like smart groups in your local node. The memberships of these groups are calculated, rather than static. They are more like query-based groups.

For the scenario above, the two computational groups that provide the most utility are probably “localaccounts” and “netaccounts.” They allow you to distinguish users who are in the DSLocal database from those in Active Directory. This magic gives the glue we need to maintain finer-grained access controls automatically over time; you could use these groups for the sshd server configuration or in the authorization database at /private/etc/authorization, as examples.

Let’s try the groups out with dsmemberutil, with which we can determine whether or not a user is a member of a specific group. What about the first local user, typically created by the Mac OS X Setup Assistant?

$ dsmemberutil checkmembership -u 501 -G localaccounts
user is a member of the group
$ dsmemberutil checkmembership -u 501 -G netaccounts
user is not a member of the group

Digging further, all of the computational groups I’ve found had the same comment. We can search for that to find the rest, to find all of the computational groups that ship with a Leopard client.

$ dscl . -search /Groups Comment "Group membership calculated by system" | awk '/user|account/ { print $1 }'

From that list, we can determine more information about the groups:

Short name Real name GID Notes
authedusers Authenticated Users 50
consoleusers Terminal Server User 53
interactusers Interactive 51
netaccounts Network Accounts 61 Accounts from a non-local database
localaccounts Local Accounts 62 Accounts residing only in one of the local databases
netusers Network 52

Unfortunately, I couldn’t find any other documentation about the computational groups. If you do find some, I’d love to hear about it.

Get the UUID for a user quickly with dsmemberutil on Leopard

Never let it be said that Apple’s Open Directory command lines tools make complete sense. After all, why would you think to use the new dsmemberutil tool in Leopard to find the Universally Unique Identifier (UUID) for a user account?

You would have if you had read the first line of its man page, maybe, where it outlines exactly how miscellaneous — and I say that in the kindest possible way — the command is: “dsmemberutil -- various operations for the membership APIs.”

Enough lead in; the UUID for a user (as well as a group) can be found with this tool:

$ dsmemberutil getuuid -u 501

Choose a user with either -u with the userid, or -U with the short username.

One way this can be exceedingly useful is if you want to track down the shadow password file corresponding to a specific user account:

$ sudo ls -1 /private/var/db/shadow/hash | grep dsmemberutil getuuid -u <em>501</em>

Hey, look, the UUIDs match, saving a trip to dscl!

Syndicate content