Jaharmi’s Irreality

Is that site running Drupal, curl edition

2008-07-02T09:45:02-04:00

The Lullabot Web site has a clever way to help answer the question of “Is that site running Drupal?” Angie Byron mentions that the HTTP “Expires” header returned by Drupal corresponds to a specific default date. Look for that date in the HTTP headers, and you can make a reasonable guess that a site is a Drupal site — or at least one that hasn’t modified one of the core files.

Some commenters posted notes about how to do the same thing with wget and then curl. I expanded on the curl instructions to make them a little more robust (especially in the case of redirects or URL rewriting), and here’s the result:

$ curl -fsIL http://jaharmi.com/ 2>&1 | grep -q -m 1 "Expires: Sun, 19 Nov 1978 05:00:00 GMT" && echo "Yes, this appears to be a Drupal site." || echo "No, this does not appear to be a Drupal site." Yes, this appears to be a Drupal site.

Navigating through files with less

2008-06-16T12:07:01-04:00

It’s taken about a decade but I finally read the man page for “less.” Here are some important keyboard commands for navigating through files with it that I wanted to note for my own future use. If someone else gets a benefit from this, great, but it’s already helping me jump through man pages with more ease. (And it gave me an opportunity to write a post with a really ambiguous title.)

Key	Function
f	Move forward one screen
b	Move backward one screen
j	Move forward one line
y	Move backward one line
=	Show the number of lines and your progress through the file
<	Move to the beginning of the file
>	Move to the end of the file
%	Scroll to the position in the file represented by the number before the percent sign

These commands had eluded me for a long time, even though it would have been great to know them. After all, less is the default pager in Mac OS X. Thanks to the wonders of air travel, I had some time to read some off-line documentation.

Of course, I’ve long used “/” followed by some text to search through man pages, and that’s worth knowing if you don’t already. One useful trick I picked up was the search “/^EXAM” (props to Adam for mentioning the caret and making my recipe one character longer, but more specific) to search for the examples section.

Ambivalence, survival, and confidence

2008-06-16T12:04:05-04:00

Despite my general feeling of ambivalence toward Apple WWDC 2008 this year, I have woken up on the day after the conclusion of my trip with a feeling of confidence.

I’m sure that will wear off in the coming hours, days, and months, but my state of mind right now is very positive. I wanted to remember I had it. Perhaps the biggest single contributor to this feeling is enduring of 22 hours of travel — and 37 straight hours awake — on the way back. Survival can really lift your spirits.

I woke up this morning with a craving for music by Mike Doughty, so I played “Looking at the World from the Botton of a Well”. It hit the spot. (“27 Jennifers” is on the WWDC 2008 iMix, but it doesn’t fit my mood nearly as well.)

It was great to see everyone this year — some old friends and some new friends. The relationships truly make the conference for me, now that I’ve been to so many sessions that just change incrementally each year. This is true even if I didn’t follow everyone to Dave’s afterwards; maybe that’s for another year.

I was surprised at the amount of movement within and between organizations … so many people I know have different jobs (and even homes) since I last saw them in 2006. The Mac landscape is changing. Even with its growth, it could be maturing. That kind of positive sign can breed confidence, too.

It was especially meaningful to me to be able to visit some friends of the family on the first weekend, and then my best friend’s family on the second.

Comparing Radmind command file timestamps between client and server

2008-06-08T23:58:44-04:00

In trying to determine the age of the command file on a Radmind client — in this case, a Mac OS X 10.5 Leopard system — I learned that its modification date is the same as on the server. This is true on multiple clients I examined, so I believe it to be both consistent and intentional. It also makes sense.

What timestamp, then, can you use to determine how old your command file is, or when your client last updated?

Let’s look into this. For example, here is the command file on a client:

[RadmindClientA]$ ls -ul /private/var/radmind/client | grep command.K -rw-r--r-- 1 root wheel 667 Jun 3 05:30 command.K [RadmindClientA]$ ls -cl /private/var/radmind/client | grep command.K -rw-r--r-- 1 root wheel 667 May 29 10:28 command.K [RadmindClientA]$ ls -l /private/var/radmind/client | grep command.K -rw-r--r-- 1 root wheel 667 May 29 10:21 command.K

And the same file examined on the server:

[RadmindServer]$ ls -ul /private/var/radmind/command | grep selected-client-command.K -rw-r----- 1 root wheel 667 Jun 3 05:27 selected-client-command.K [RadmindServer]$ ls -cl /private/var/radmind/command | grep selected-client-command.K -rw-r----- 1 root wheel 667 May 29 10:21 selected-client-command.K [RadmindServer]$ ls -l /private/var/radmind/command | grep selected-client-command.K -rw-r----- 1 root wheel 667 May 29 10:21 selected-client-command.K

The status change time (ls -cl) is a little bit harder to understand. I’ve seen it minutes apart from the modification time on a client, but it is the same as the modification time when viewed on the server.

The access time (ls -ul) of the command file on the client, however, simply corresponds to when ktcheck last ran.

For good measure, another client:

[RadmindClientB]$ ls -ul /private/var/radmind/client/ | grep command.K -rw-r--r-- 1 root wheel 1188 Jun 5 05:08 command.K [RadmindClientB]$ ls -cl /private/var/radmind/client/ | grep command.K -rw-r--r-- 1 root wheel 1188 Jun 3 14:13 command.K [RadmindClientB]$ ls -l /private/var/radmind/client/ | grep command.K -rw-r--r-- 1 root wheel 1188 Jun 3 14:09 command.K

And its command file on the server:

[RadmindServer]$ ls -ul /private/var/radmind/command | grep another-selected-client-command.K -rw-r--r-- 1 root wheel 1188 Jun 4 05:13 another-selected-client-command.K [RadmindServer]$ ls -cl /private/var/radmind/command | grep another-selected-client-command.K -rw-r--r-- 1 root wheel 1188 Jun 3 14:09 another-selected-client-command.K [RadmindServer]$ ls -l /private/var/radmind/command | grep another-selected-client-command.K -rw-r--r-- 1 root wheel 1188 Jun 3 14:09 another-selected-client-command.K

I looked into this because I was wondering why ktcheck -C (to “clean” the /private/var/radmind/client directory) wasn’t having the effect I expected. I thought it cleared out the client directory and started with fresh files, which would all have new timestamps because they were new to the client. When I didn’t get the new timestamps I anticipated, I worked through what I saw until I settled on this explanation.

The result? The main means you have of determining whether the command file has been updated is its access time. The access timestamp corresponds with the date and time of the last ktcheck action. This appears to be the case no matter which flags I’ve used with ktcheck — even with ktcheck -n (where “no files are modified”).

The modification time, for all intents and purposes, only indicates when the file was last changed on the server. Testing this out — with command files that have been overwritten with the exact same contents (using cat), as well as command files deleted on the client before ktcheck — I have reached the conclusion that the modification time changes only when the file contents do.

So, I don’t have solid explanations for all of this. But, I’m now armed with the knowledge that the access timestamp is the most important when you’re looking for the age — or at least the last refresh time — of a Radmind client’s command file.

Change the name of an application in your LaunchBar configuration

2008-06-08T23:48:15-04:00

If you can’t remember the specific name of an application, but you often think of it by another name, you can change how it is referred to in your LaunchBar configuration.

I used to do something like this with Microsoft Excel; I’d change its name in the LaunchBar configuration to “XL.” You could also refer “Lineform” as “Illustrator” without changing the application’s name in the Finder.

To make the change, open LaunchBar. If you can remember the application, perform a search for it. When it is selected in the LaunchBar bar, choose Configuration > Reveal in Configuration. Enter a new name for the selected app in the “Name” column in the window that appears.

I’m not aware, however, of a way to have an application to respond to two or more names in LaunchBar. There may be a way; I just don’t know it.

Hunting Snow Leopard

2008-06-05T06:59:19-04:00

I’ve been thinking for a few months that Mac OS X 10.6 (or the like) would be introduced to developers at WWDC 2008. I’ve said as much to those unfortunate enough to be within earshot. I haven’t mentioned it here … and in recent days I’ve felt less confident about this gut feeling based on the public WWDC session schedule.

If Apple was trying to return to an 18-month release cycle for Mac OS X, my thought process went (after the CEO announced just such a push), a developer preview would almost have to be shown at this WWDC. That has lingered in the back of my mind. It also provided a reason to go to the conference if you were interested more in the Mac track than the iPhone.

Now there are rumors flying around, based on various strings in Apple software and from other sources, that there will be a preview release of Mac OS X 10.6 at WWDC. And it’s being referred to with the moniker, “Snow Leopard.”

If, as these rumors say, the upgrade will focus on reliability and security, then the general lack of room for schedules on it is more plausible. The name distinguishes it little from Leopard, but maybe just enough.

However, unless this release were more like the free-with-$20 shipping Mac OS X 10.1 update, who would buy it? Would Apple charge $129 retail for stability and security? It seems that there would have to be more.

I can definitely see that security could be enhanced by greater adoption of certain features — some reasonable candidates for further enhancement since Leopard — which are the focus of several sessions at WWDC, according to the published schedule.

Since I’m always watching for when old rumors swing back around, I’d guess that one additional change we could see is the mythical “Illuminous” user interface. (Assuming, of course, that this is not the unified interface style we already have in Leopard.) It usually takes two years for fun old rumors to become reality, if they ever do — that’s enough time for many to have forgotten about them and for actual development work to have taken place. A new interface with a new name would jack up the value of a new OS in some people’s minds. Moreso if it actually works better than the old one.

If the lack of PowerPC support were true, then this would be an astounding announcement. It would cut off upgrades for a large (but ever-decreasing) percentage of the Mac population. Already, Leopard itself was limited to G5s and the newest G4s. This could have an interesting effect on those institutional customers who bought G5s for their compute power. Let’s recall that although the Intel Macs have been out since January 2006 and the transition was relatively quick, the high end Power Mac and Xserve systems were the last to be replaced. For a while yet, there can still be Apple PowerPC-based systems that are less than three years old.

Ah, we’ll see what happens Monday. You never know with rumors. I have no inside information and even I wouldn’t make any important decisions based on these musings.

Limit Launchd LaunchAgents to specific session types in Leopard

2008-05-29T18:28:17-04:00

In Leopard, launchd has some options that can tailor your now-working LaunchAgents for specific circumstances. For example, if you want your LaunchAgent to run only when a user has logged in at the Aqua console — rather than SSH or other login sessions — you can use the “LimitLoadToSessionType” key:

LimitLoadToSessionType Aqua

Note that to find events that have been limited in this way, you must use launchctl’s “-S” flag to specify the session type. Otherwise, launchctl won’t find jobs that are specified for a different kind of session than the current one, as per Levi Brown’s post.

I also found the “LimitLoadToHosts” and “LimitLoadFromHosts” keys when looking this topic up in the launchd.plist man page.

Thanks to James Bucanek for mentioning this on the launchd-dev list.

WWDC 2008 iMix on iTunes Store

2008-05-29T06:12:53-04:00

It’s that time of year again. My WWDC 2008 iMix has been posted.

It started as a gag but I’ve had occasional encouragement from others who thought it was a good idea. Plus, I generally like the music at the conference so it’s nice to save some record of it. It’s hard to find iMixes because you have to go into the iMix section of the store to search for them, so I wanted to help myself by linking to it here.

If you’re at the conference, feel free to send me submissions; information is in the iMix description itself.

Update: I had to replace the earlier iMix with a new version so I could add to it. It appears that if you create an iMix on one computer, you can't update it on another.

Python to remove commands module and some Mac-specific modules

2008-05-28T21:46:43-04:00

Drat! I’ve learned that the commands module for Python, which I use, is deprecated and removed in Python according to PEP 3108. That means I can no longer safely call commands.getstatusoutput() anymore. I’ve frequently used this call in the past because it seemed the sanest, easiest way to call for a shell utility and get both its output and return status (for success or error).

I’ll have to find some other way to perform the same function — preferably one that will work on Python 2.3 from Mac OS X Tiger, Python 2.5.1 from Leopard, and future Pythons. The stated replacement for a number of similar modules (including popen2, which frankly kind-of frightened me off with its name) is the subprocess module from PEP 324, but I don’t know if that will work for my purposes.

There are also a bunch of Mac-specific modules being removed. I don’t use any of them right now, but that doesn’t mean they wouldn’t have been useful.

This kind of thing is spirit-crushing to me for some reason. I’m especially annoyed that Python has been around for so long and it is still reorganizing the ways it calls shell commands. Just settle on something! It seems hard to take it seriously as a system administration scripting language when things like this happen.

On the other hand, I love so much of the Python Standard Library, which has afforded me a lot for system administration …

Radmind server logging and the repo command

2008-05-28T21:13:35-04:00

Under normal circumstances, the latest Radmind tools that communicate with the server report client status updates in the Radmind server’s system log. These standard messages can include ones like:

May 8 03:14:56 RadmindServerHost radmind[7890]: report radmind-client.example.com 192.168.7.42 - - ktcheck No updates needed May 15 03:15:25 RadmindServerHost radmind[24531]: report radmind-client.example.com 192.168.7.42 - - ktcheck Updates retrieved May 15 03:21:48 RadmindServerHost radmind[24534]: report radmind-client.example.com 192.168.7.42 - - lapply Changes applied successfully May 15 03:31:07 RadmindServerHost radmind[24356]: report radmind-client.example.com 192.168.7.42 CertificateCN - lapply Error, changes made

The Radmind repo, or “report,” tool provides the ability to send arbitrary messages to the Radmind server process. But how are these messages formatted and sent?

$ repo -e "Debug" -h radmindserverhost.example.com -w2 "Test message"

… results in the system log message:

May 15 03:31:56 RadmindServerHost radmind[25236]: report radmind-client.example.com 192.168.7.42 CertificateCN - Debug Test message

Here, we can see that an entry created with repo looks like the standard Radmind log messages above. The client hostname and IP address are reported after the “report” text. The CertificateCN for the client — if the highest authorization level is specified (with the -w2 flag) — is also listed; if not, a dash takes its place. I haven’t seen a case where the second dash is substituted, however.

Finally, where the Radmind command/tool used would normally be, the “event” specified by repo will printed. After that, the message text appears.

The value proposition is that if you’re using Radmind, the repo command can help you send arbitrary messages to the server for logging. As bonus, if you’ve taken the time and effort to build the certificate infrastructure for Radmind, you can send these messages securely between the clients and the server cloaked in SSL.

If you’re using multiple servers, you may want to combine their logs in one location so that you can get all of the clients’ reports in one location. You may also want or need to retain these reports for more time. In either case, determine what policies you should apply to the syslog or Apple System Logger (ASL, for Mac OS X) configuration for your server systems.

Whether or not you use repo, it’s good to know that the tools do some logging. The logging can be followed to try to determine the status of your clients, or whether they are failing their updates.

Unfortunately, the most common client failures I have seen tend to involve the lapply tool, and the default level of detail I’ve seen reported back to the server does not provide an indication of what problem has been encountered. You see only that there was an error. Still, even though you may not get enough detail to remotely resolve the problem, it’s something for you to go by find problems in the first place.

Format numbers with the Python locale module

2008-05-26T23:00:09-04:00

I’m constantly astounded by the breadth of features available in the Python Standard Library. Although the functions I find there are not always easy to grasp, it is almost always worth searching around a bit for a function or method in the standard library before I write my own code to do something someone else has probably had to do before.

Take the “locale” module. It lets you format certain kinds of data based on your locale and its customs. Numbers (including currency) happen to be one of its specialties. Since I had a need to output long numbers whose digits were grouped with commas — which makes them easier to read — and locale.format() does just that. Even better, it’s internationalized and formats them for your system’s own locale.

>>> import locale >>> a = {'size': 123456789, 'unit': 'bytes'} >>> print(locale.format("%(size).2f", a, 1)) 123456789.00 >>> locale.setlocale(locale.LC_ALL, '') # Set the locale for your system 'en_US.UTF-8' >>> print(locale.format("%(size).2f", a, 1)) 123,456,789.00

In trying to use it in the bundled Python 2.5.1 on Mac OS X Leopard, I noticed that the default for scripts and the interpreter doesn’t format numbers as I expected for my locale. I found that I needed to set a locale to get the expected formatting. To do this, I run locale.setlocale(), as above. I’m not sure if this is required for other Python installations, but it’s worth mentioning.

One difficulty I created for myself was when I tried mixing more into my format string. For example, locale.format() would fail to reformat a number when I added in the string for the unit from my original dictionary:

>>> print(locale.format("%(size).2f %(unit)s", a, 1)) 123456789.00 bytes

In retrospect, this makes total sense, but it took Mark looking over my code to discover my error.

When I ruled the world?

2008-05-24T07:59:13-04:00

Does anyone else think that Apple has some ulterior motive for promoting Viva la Vida, the track on the new iTunes ad featuring Coldplay? That maybe its lyrics are indicative of something going on at Apple?

“When I ruled the world,” indeed.

No matter. I find it a fantastic visual treat and now the haunting music is stuck in my head. I would not be surprised if it were featured at WWDC 2008 in a few weeks.

Python string method for title case, versus the Daring Fireball script

2008-05-20T16:55:31-04:00

I ran a quick test of the Python “title” string method (from Python 2.5.1 in Leopard); it fails the tests on the edge cases mentioned in the Title Case post at Daring Fireball. It really only uppercases the first letter of every word and lowercases everything else.

Not much better than looping through a string with ucfirst() in Perl, except you don’t have to do the looping yourself.

Drat. I was expecting better.

Get Python installation information from Distutils

2008-05-20T10:07:04-04:00

The Distutils Python module includes functions to obtain information about the Python installation. This may be useful for system administrators, and it certainly caught my eye when I read about it.

The results below are from Apple’s bundled build of Python 2.5.1 in Mac OS X Leopard. Credit for the comments describing each function comes from the distutils.sysconfig documentation.

>>> import distutils.sysconfig >>> distutils.sysconfig.get_python_version() # Get the major Python version without patchlevel '2.5' >>> distutils.sysconfig.get_python_lib(standard_lib=True) # Return the directory containing the Python library; if 'standard_lib' is true, return the directory containing standard Python library modules '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5' >>> distutils.sysconfig.get_python_lib() # Otherwise, return the directory for site-specific modules '/Library/Python/2.5/site-packages' >>> distutils.sysconfig.get_python_lib(plat_specific=True) # Return any platform-specific modules from a non-pure-Python module distribution '/Library/Python/2.5/site-packages'

You’d install your own modules for system-wide use in the directory returned by distutils.sysconfig.get_python_lib().

Determine if a string starts with one of a tuple of strings in Python

2008-05-20T08:47:06-04:00

In Python, there is a built-in string method, “startswith,” that lets you determine whether a line starts with a character. I’ve used it before because I tend to want to use the features of the standard library, despite having awesome features like slicing at my disposal. What I didn’t realize immediately, though, was how to compare the string against two or more sets of characters. All I knew is that entering a list as the prefix — instead of the more common single string — didn’t work.

Luckily, my question was answered right away in the string method documentation. I found that to perform these comparisons together, you must supply the “startswith” string method with a tuple. It works the same for the “endswith” method. However, this does require Python 2.5.

>>> starts_with = ('f', 'a') >>> def does_startwith(selected_text): ... if selected_text.startswith(starts_with): ... print True ... >>> does_startwith('z is a letter') >>> does_startwith('f is a letter') True >>> does_startwith('b is a letter') >>> does_startwith('a is a letter') True

This has some application to a larger question I had, so I wanted to note it.