Jaharmi’s Irreality renewable hyperlinks with 0% transfat since 1999

I came across an interesting “problem” with Active Directory binding on Mac OS X Leopard. The symptoms were:

• No Active Directory user accounts could log into the computer from the loginwindow.
• Some of the attempted logins involved cached mobile accounts from the Active Directory.
• The account login failures happened even though loginwindow’s “network accounts are available” indicator was green.
• The login problem persisted it the computer been unbound and rebound to the domain.
• The same Active Directory users could log in on other Macs.
• Local users could log in to the affected computer.
• Using “su” to switch users from a local user to an Active Directory user worked in Terminal.
• Lookups using “dscl” and other DirectoryService tools worked.

Since I’ve written (what seems like a) a book about Active Directory troubleshooting, I threw the book at this problem. It ended up taking quite some time to troubleshoot, and the answer ended up being very simple. However, it wasn’t on my normal list of culprits.

The biggest clue I found, besides the symptoms above, was that the DirectoryService debug logs yielded this during Active Directory logins from loginwindow:

It didn’t seem like a smoking gun, but I’d never come across this “false” response on a bound system before. So, what group was so important to the login process that the DirectoryService debug logs cared enough to note the failure? I was darned if I knew, and I had no other promising clues at that point.

So, I investigated that group further, and found it by its UUID using dsmemberutil:

Well, that helped a little, but the name would have helped a lot more. I had to find which group corresponded to the GID of 200. That GID was not at all familiar to me, but it was under 500, so there was a pretty good chance it came from Mac OS X.

This was my eureka! moment. I wasn’t entirely sure, but I was pretty confident that the “com.apple.access_loginwindow” group was the access control list group for the loginwindow process. Loginwindow controls all graphical logins to Mac OS X, and is the parent process of each GUI login session.

Looking up the group’s description confirmed that it was the ACL group. I did the lookup in Workgroup Manager, which was set to view the DSLocal directory service. While I was there, I also checked the membership: it listed only the computational group “localaccounts.” The “localaccounts” group is essentially a query that returns all accounts in the local directory service.

Well, that would certainly prevent Active Directory users from logging in with loginwindow. The ACL consulted the membership of the “com.apple.access_loginwindow” group to determine who was allowed to log in via the GUI. Because it contained only the “localaccounts” group, the ACL was preventing all non-local users from logging in.

Not knowing how this group was handled or even what had last edited it, I compared the affected system to a different AD-bound Leopard computer, which also had Workgroup Manager. (It’s handy to have the Mac OS X Server Admin Tools deployed out to your computers even if you don’t have a server to maintain.) The second computer didn’t have the group at all, which perplexed me a bit.

However, that made me reasonably sure I could simply delete that group. I backed it up from the filesystem at the command line, just to make sure, and then deleted it with Workgroup Manager on the affected computer.

After that, logins for all Active Directory accounts I tried proceeded normally at the loginwindow on that system.

With the problem solved, I sought more information about the workings of the “com.apple.access_loginwindow” group. I confirmed that it is created when the “Allow network users to login in at login window”
option is turned on in System Preferences > Accounts > Login Options. This should be turned on by default, and that initial state results in no “com.apple.access_loginwindow” group at all.

Since the option is on by default, the really simple solutions to this kind of problem are:

1. Don’t turn off the “Allow network users to login in at login window” option in System Preferences > Accounts > Login Options.

   

2. If “Allow network users to login in at login window” has been turned off, either:
   1. delete the group named above, or
   2. toggle the option back on.

Deleting the “com.apple.access_loginwindow” group removes it completely and reinstates login capability for both local and network user accounts.

Toggling the System Preferences option back on, adds the “netaccounts” group to the “com.apple.access_loginwindow” group, reenabling login for both local and network users. It does not, however, remove the group “com.apple.access_loginwindow,” which remains on the system afterwards.

Here's what that looks like in Workgroup Manager:

To prevent this on managed clients, I could see a system administrator proactively creating and managing the membership of the “com.apple.access_loginwindow” group. To ensure that managed clients bound to an Active Directory allow both local and network users to log in, make sure the group is populated with the appropriate nested groups: “localaccounts” and “netaccounts.”

I had interesting things happen when I ran out of disk space today.

The most notable one was that I saw “Process completed” — or some variant of that — every time I tried to open a Terminal window with a new shell session. I briefly staved it off by specifying Bash as my shell, but then it came right back after opening another tab or two in Terminal. Consulting Google led me to this “Terminal application quits” thread at Apple Discussions. On a lark, I tried deleting /usr/bin/login as one poster suggested. It worked!

…But only for a little while. The problem returned. In the meantime, I had freed up some disk space because I’d realized I couldn’t save files anywhere (“But ~/Pictures is writable!”). Clearly something else had become an issue, because disk space was available.

Then I found another thread, “Terminal’s ’Process Completed’ message and /usr/bin/login,” on Apple Discussions. The more permanent solution from that thread appears to be the removal of corrupted Apple System Log databases. Once I did that and restarted the ASL service, all was well and has stayed that way so far.

Filling up my disk must have corrupted the logs as they were being written or rotated, and led to this cascade failure. Like I said, interesting!

One of the posters in the second Apple Discussions thread indicated that the underlying database corruption issue is addressed in Snow Leopard. However, it seems that you could still see this on Leopard — my experience was with a recently-patched Mac OS X 10.5.8 system.

It should come as no surprise that Apple Installer installation packages can contain scripts. These scripts are supposed to conduct important operations during the course of the software installation.

However, when you are the system administrator of more than one Mac, you find that developers sometimes miss a good balance between what you think should be in the installer payload versus what should be in its scripts. The payload of a installer, by definition, are the files and links that should be installed, along with information on where they should be installed as well as how (i.e. ownership, permissions).

Therefore, developers should not need to run scripts that create or delete files — they should be created from the payload itself, and if a file must be deleted during the install then consider that perhaps you’re doing it wrong. Likewise, there should be little need move or copy files, because as many copies as desired can be installed from the paylod. Similarly, the need to change ownership or modify permissions should be taken care of in the payload.

Perhaps I’m being a purist here. I’m certainly accused of that, from time to time. However, this just makes sense to me and I happen to think that many developers are similarly logical people. They just aren’t the kind of logical people who happen to spend effort on software installation, especially the kind that results in a deployment-friendly installer package.

So how do we as administrators verify the quality of the scripts in installers? Is there a way we can quickly peer into them to decide if any of the scripts’ steps will be superfluous or even (gasp!) harmful?

Well, I have a quick suggestion for scanning packaged installers. The following one-liner shell command will search an installer package or metapackage for scripts that have the kinds of steps outlined above.

Note that this will only work for the traditional installer packages; it will not work with Leopard-style flat packages (which are documented so badly by Apple that the best description comes from reverse engineering by Iceberg's author). The one-liner will currently only find the defined install operations scripts: preflight, preinstall, preupgrade, postinstall, postupgrade, and postflight. (Any other scripts are likely to be called by one of those six.) It assumes those scripts will be shell scripts, currently, even though any of them could be written in other scripting languages installed with Mac OS X, like Python, Perl, or Ruby. It will also not work on the JavaScript-based system and volume requirements portions of the installation.

However, it’s a start. The output displays the offending file and line number, so you can conduct more careful examination of the matches it finds.

I haven’t run this on an exhaustive list of installation packages, but I have already seen at least one installer that produces worrisome output.

Update: I’ve changed the regex for the pre/postflight script so that it is more general that what I originally posted. I’m also having some problems with the snippet working with a certain installer whose scripts I know have cp and chmod commands. So, I may be back to the drawing board with this; comments are welcome.

In this MacEnterprise list thread about printing authentication, Greg Neagle mentions that:

Under Leopard, all local users are members of lpadmin, but I think network users are not. So this method won't grant network users CUPS rights.

To confirm Greg's suspicions, I ran the following shell snippet.

This loops through the fictional accounts, "mobile_account_user," "network_account_user," and "local_account_user." These accounts are, as you might expect, as a locally-cached mobile account from a network directory, a wholly network directory-based account, and a simple local admin account. While the accounts presented here are fictional, the results were confirmed on a live system bound to a directory service.

The rest of the snippet determines if the accounts are members of any of the specified computational groups that debuted in Leopard. The last group checked is the "lpadmin" group. By looking at these group memberships, we can determine whether Leopard thinks that the account being tested is a local or network account.

Running the snippet above, with the right accounts available, produces the following output:

So, it appears mobile and local users get added to the lpadmin group automatically in Leopard, but network accounts do not.

Note that I didn’t check whether membership in the “admin” group made a difference or not. I also didn’t isolate for that factor.

I found it interesting that the mobile account is a member of “netaccounts” but not “localaccounts.” (By group membership alone, I’m not sure you could identify whether an account was a mobile account or not. Yet, that kind of test is part of the point of having these computational groups in the first place.)

A question came up on the AppleScript-Users mailing list and I wanted to make note if it, because I came up with a quick Python-based way to answer it. Here, I reiterate that answer on how to get the number of pages from a PDF.

This Python 2.5 sequence (shown as run interactively from the Terminal on Leopard — not as a script) works for me using both a local file and a file from an AFP-mounted volume. It uses the Quartz 2D bindings for Python, which are part of the Python install on Mac OS X (since 10.3?), so it is not pure Python.

Substitute your own path for the pdf_filename variable’s contents, and you can try it yourself. (Above, $ is the shell prompt, and >>> is the Python interpreter’s command prompt, so don’t copy/paste those.)

Since I was doing this in Terminal, I could also type "pdf_filename = '" then drag and drop a file from the Finder into Terminal window to insert its path, complete the quoting, and pressed Return. (Saves some time and potential for mistyping.)

You could wrap this sequence into one longish command line and run it from AppleScript with “do shell script.” Or you could save it as a Python script file and again run it with “do shell script.” Or, you could skip AppleScript and just use Python, which I prefer over AppleScript. (Python seemed very natural to me with my minor AppleScript background, and I'm pretty sure I've written more of it than AppleScript by this point.)

This is derived from “Example 2: Splitting a PDF File” in the Using Python with Quartz 2D on Mac OS X document at the Apple Developer Connection.

That ADC example also shows how you could specify a PDF file’s path at the command line (with sys.argv), rather than hardcoding it as I did for my example.

There may be a shorter way to do this since I’m no expert on Quartz 2D. (I just appreciate that the bindings are there for a scripting language I happen to like a lot.) I honestly don’t know what’s happening when the provider and pdf objects are being set, but I really don’t need to know for this.

Nigel and Jeff present Mac OS X Laptop Deployments with Puppet in the MacIT track at Macworld Expo 2009. They are two of the first Mac system administrators I knew of using Puppet, and both had a background in Radmind.

I’ve been reading through James Turnbull’s Pulling Strings with Puppet, since our library had a copy. I had hoped to get through it during our winter break, but illness and other factors (no Puppet pun intended) conspired to get in the way. From what I’ve read about it already, Puppet is clearly interesting. Nigel was very enthusiastic about it when we talked at WWDC 2008.

To me, it seems that it would take some effort to model what you want in it and build up a repository of what you want managed. Perhaps I’m feeling like an old dog trying to learn new tricks. Grin.

One point that Nigel and Jeff made in their presentation slides that struck me is that they needed a solution that works when offline, which Puppet does. Radmind can work offline but I daresay that’s not the way that most people would think to use it (lapply with its “-n” flag would be the most basic change).

Kyle also mentioned to me that he’s been using Puppet in conjunction with Radmind. I believe he has Puppet managing configurations and Radmind managing the bulk of the filesystem.

You can find out if an SSL certificate has expired with the command below. I’ve found it useful to be able to check for expired certificates in my use of Radmind, where you can uniquely identify clients to the server with them.

$ openssl x509 -in /path/to/cert.pem -noout -checkend 0

I mention this command primarily because I reviewed the the OpenSSL x509 man page (“man x509”) that comes with Mac OS X Leopard, and it didn’t show the “checkend” option for the command. That was odd, because that option was just what I needed.

I did, however, find it documented in the usage statement-style help for the command:

$ openssl x509 --help

In that usage statement, the “checkend” option is described (with little punctuation) as a way to “check whether the cert expires in the next arg seconds [sic] exit 1 if so, 0 if not.” So, using zero seconds shows you if the certificate has already expired, while an integer greater than zero will show if it will expire in the future. No matter how many seconds you check against, you must examine the results from the exit code (the “$?” shell variable) to see if the certificate is or has expired.

I find this is tremendously useful knowledge when dealing with certificates in Radmind, where an expired certificate can mean the failure of a client to connect to the Radmind server. It could be beneficial in other circumstances, of course — but I don't have those circumstances.

Taking this further, you could check for certificate expiration on a Radmind server — if your certificates are stored in the Radmind special directory for each hostname of a managed client. (Substitute one of your own managed clients’ hostnames for “hostname” in the path below.)

$ openssl x509 -in /var/radmind/special/hostname/private/var/radmind/cert/cert.pem -noout -checkend 0

Since you can do it for one client certificate, you could also loop through all of the certificates on a Radmind server. In this example, I’ll continue to use the path of /var/radmind even though, on Mac OS X, I’d generally prefer to specify the full /private/var/radmind; your Radmind server may not be on Mac OS X even if your clients are. Also, you may need to modify the “depth” parameter on your search to accommodate the paths on your server. Finally, I’ll change the “checkend” parameter to 604800, for seven days (60*60*24*7=604800). That produces something along the lines of:

Change the last line to “done | grep expiring” if you only want to see the expiring certificates.

It’s great to get just the CN of the certificate in these circumstances, since it’s likely you’ll want to act on just those that need attention. One way to do this relatively cleanly is to use OpenSSL x509’s “subject” and “nameopt” options, and then parse the output. Below, I’ll use awk for that. (Again, substitute one of your own managed clients’ hostnames for “hostname” in the path below.)

$ openssl x509 -in /private/var/radmind/special/hostname/private/var/radmind/cert/cert.pem -noout -subject -nameopt sep_multiline | awk '/CN/ {split($1,elements,"=") ; print elements[2] ;}'

Beyond checking for expiration on the server, it may be valuable to do so in your Radmind client scripts, especially if you favor SSL connections. If you find an expired certificate, you can take some remedial action right away that might allow the client to communicate with the server.

I thought about this a while, and the easiest way I came up with — after having already developed more complex logic — was to simply rename or remove the expired certificate from its normal path. Then, allow the client to connect with another authorization level where the client certificate is unnecessary. (Use of a client certificate implies Ramind’s “-w2” authorization level, while a lesser level would mean you’re performing hostname/DNS rather than certificate verification.) This would probably mean you have multiple Radmind server processes running, each on its own port, to accept such incoming requests on the server.

Irrational Exuberance has An Epic Introduction to PyObjC and Cocoa. Skimming over it, it really does seem to start at the ground level — and that’s what I need. I have high hopes it will help me figure out PyObjC.

[Via PyObjC Development mailing list.]

While trying to troubleshoot what I’d done to mess up the Mercurial repositories managing my Drupal installations last weekend, I really would have liked a way to see what files had changes in specific revisions. Each revision to a Mercurial repository affects some files, of course, but it seems awfully hard to figure what files changed in that check-in.

I have since found a way to do that by customizing the output of Mercurial. To customize output, you can create templates on the command line (with --template) or for more powerful reformatting, create an output style file.

I struggled for a while to figure out how to use style files, and eventually came up with something that works for me so far.

Since I’ve installed Mercurial from Lee Cantey’s standard binary package for Mac OS X Leopard, I created the file “map-cmdline.changedfiles” at the “/Library/Python/2.5/site-packages/mercurial/templates” path. (Where you put the file may vary depending on where Mercurial is installed, and I’m sorry but I don’t know where it gets installed on other systems.) The contents of “map-cmdline.changedfiles” are below, along with my possibly inept description of what each line is doing:

# Get all of the files in the selected revision
# and stringify them, whatever that means
# but do not 'tabindent' or wrap them to 68/76 columns
# Without first setting changeset to the list of files
# you won't get output from subsequent lines
changeset = '{files|stringify}'
# List modified files, one per line
# preceded by M to mimic `hg status`
file = 'M {file}\n'
last_file = 'M {file}\n'
# List added files, one per line
# preceded by A to mimic `hg status`
file_add = 'A {file_add}\n'
last_file_add = 'A {file_add}\n'
# List deleted files, one per line
# preceded by ! to mimic `hg status`
file_del = '! {file_del}\n'
last_file_del = '! {file_del}\n'

I don’t know why the “map-cmdline.” portion of the filename is there, but as long as I have it, I can call the style file from the command line with what follows the period. So, I can call the style with “--style changedfiles” — and that tiny bit of voodoo seems reasonable enough to me. (The other styles in the directory above, many of which end in “.tmpl” extensions, seem related to the Mercurial Web server, hgweb. I tried, but I couldn’t use their names at the command line, with or without their extensions. Plus, their contents looked HTML-ish.)

With the “map-cmdline.changedfiles” style file saved in that location, I can call Mercurial’s “log” command:

$ hg log --style changedfiles -r tip

… which gives me a list of the files changed in the “tip” (or latest revision) of the repository. I could substitute in any revision identifier for “tip.”

I haven’t actually seen the “file_add” and “file_del” keywords in action; every time I’ve used this style file in the manner described, I’ve only seen files marked as “M” — even if I’m looking at a revision where new files were first checked into the repo. I’m confused by that, but I’m not going to let it sour my day at this point.

There might have been an easier way to do this but I didn’t find one last weekend. It took me some time to figure even this bit out, and I hope writing this post saves someone new to Mercurial from future frustration.

While you could clear the various LaunchServices databases by deleting files, I prefer to use a more targeted command line tool when possible. My assumption is that the command line tool is canonical and will therefore produce more reliable results.

In Mac OS X Tiger, you could clear the LaunchServices database using the following “lsregister” command:

$ /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/LaunchServices.framework/
Versions/Current/Support/lsregister -kill -r -domain system -domain local -domain user

But, in Leopard, the LaunchServices framework’s path has moved; it is now in the CoreServices framework bundle. You can find it with the “find” command:

$ find /System -name lsregister -type f -print 2>/dev/null
/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/
Versions/A/Support/lsregister

Once you find it, you can get the usage statement for the “lsregister” command in Leopard, which will at least tell you what options it has:

$ /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/
Versions/A/Support/lsregister
lsregister: [OPTIONS] [ ... ]
[ -apps [,domain]... ]
[ -libs [,domain]... ]
[ -all [,domain]... ]
Paths are searched for applications to register with the Launch Service database.
Valid domains are "system", "local", "network" and "user". Domains can also
be specified using only the first letter.
-kill Reset the Launch Services database before doing anything else
-seed If database isn't seeded, scan default locations for applications and libraries to register
-lint Print information about plist errors while registering bundles
-convert Register apps found in older LS database files
-lazy n Sleep for n seconds before registering/scanning
-r Recursive directory scan, do not recurse into packages or invisible directories
-R Recursive directory scan, descending into packages and invisible directories
-f force-update registration even if mod date is unchanged
-u unregister instead of register
-v Display progress information
-dump Display full database contents after registration
-h Display this help

Note that if you’re having Launch Services-related problems in one user account but not another, you’ll likely want to start with the “user” domain. (It pays to understand the domain structure of Mac OS X, so that you only make changes where you need to.)