Jaharmi’s Irreality renewable hyperlinks with 0% transfat since 1999

I came across an interesting “problem” with Active Directory binding on Mac OS X Leopard. The symptoms were:

• No Active Directory user accounts could log into the computer from the loginwindow.
• Some of the attempted logins involved cached mobile accounts from the Active Directory.
• The account login failures happened even though loginwindow’s “network accounts are available” indicator was green.
• The login problem persisted it the computer been unbound and rebound to the domain.
• The same Active Directory users could log in on other Macs.
• Local users could log in to the affected computer.
• Using “su” to switch users from a local user to an Active Directory user worked in Terminal.
• Lookups using “dscl” and other DirectoryService tools worked.

Since I’ve written (what seems like a) a book about Active Directory troubleshooting, I threw the book at this problem. It ended up taking quite some time to troubleshoot, and the answer ended up being very simple. However, it wasn’t on my normal list of culprits.

The biggest clue I found, besides the symptoms above, was that the DirectoryService debug logs yielded this during Active Directory logins from loginwindow:

It didn’t seem like a smoking gun, but I’d never come across this “false” response on a bound system before. So, what group was so important to the login process that the DirectoryService debug logs cared enough to note the failure? I was darned if I knew, and I had no other promising clues at that point.

So, I investigated that group further, and found it by its UUID using dsmemberutil:

Well, that helped a little, but the name would have helped a lot more. I had to find which group corresponded to the GID of 200. That GID was not at all familiar to me, but it was under 500, so there was a pretty good chance it came from Mac OS X.

This was my eureka! moment. I wasn’t entirely sure, but I was pretty confident that the “com.apple.access_loginwindow” group was the access control list group for the loginwindow process. Loginwindow controls all graphical logins to Mac OS X, and is the parent process of each GUI login session.

Looking up the group’s description confirmed that it was the ACL group. I did the lookup in Workgroup Manager, which was set to view the DSLocal directory service. While I was there, I also checked the membership: it listed only the computational group “localaccounts.” The “localaccounts” group is essentially a query that returns all accounts in the local directory service.

Well, that would certainly prevent Active Directory users from logging in with loginwindow. The ACL consulted the membership of the “com.apple.access_loginwindow” group to determine who was allowed to log in via the GUI. Because it contained only the “localaccounts” group, the ACL was preventing all non-local users from logging in.

Not knowing how this group was handled or even what had last edited it, I compared the affected system to a different AD-bound Leopard computer, which also had Workgroup Manager. (It’s handy to have the Mac OS X Server Admin Tools deployed out to your computers even if you don’t have a server to maintain.) The second computer didn’t have the group at all, which perplexed me a bit.

However, that made me reasonably sure I could simply delete that group. I backed it up from the filesystem at the command line, just to make sure, and then deleted it with Workgroup Manager on the affected computer.

After that, logins for all Active Directory accounts I tried proceeded normally at the loginwindow on that system.

With the problem solved, I sought more information about the workings of the “com.apple.access_loginwindow” group. I confirmed that it is created when the “Allow network users to login in at login window”
option is turned on in System Preferences > Accounts > Login Options. This should be turned on by default, and that initial state results in no “com.apple.access_loginwindow” group at all.

Since the option is on by default, the really simple solutions to this kind of problem are:

1. Don’t turn off the “Allow network users to login in at login window” option in System Preferences > Accounts > Login Options.

   

2. If “Allow network users to login in at login window” has been turned off, either:
   1. delete the group named above, or
   2. toggle the option back on.

Deleting the “com.apple.access_loginwindow” group removes it completely and reinstates login capability for both local and network user accounts.

Toggling the System Preferences option back on, adds the “netaccounts” group to the “com.apple.access_loginwindow” group, reenabling login for both local and network users. It does not, however, remove the group “com.apple.access_loginwindow,” which remains on the system afterwards.

Here's what that looks like in Workgroup Manager:

To prevent this on managed clients, I could see a system administrator proactively creating and managing the membership of the “com.apple.access_loginwindow” group. To ensure that managed clients bound to an Active Directory allow both local and network users to log in, make sure the group is populated with the appropriate nested groups: “localaccounts” and “netaccounts.”

Jesper Noehr explains why {l,r}strip are considered harmful for removing extensions from filenames with Python. I think he’s absolutely right on that score, and I would agree. The lstrip() and rstrip() methods shouldn’t be used for this purpose.

However, like the only commenter on that post, I’d also recommend os.path.splitext() as the proper tool for the extension-removing job.

Let’s take some example filenames you might come across on Mac OS X Snow Leopard:

If we had a list of filenames (or file paths) like this — perhaps created by os.walk() or some other generator-based process — we couldn’t easily use Jesper’s recommended solution. The replace() string method would give us a much harder time dealing with the range of filenames and extensions in that list. In the case where you don’t know the filename extensions in advance, replace() breaks down. The replace() method would have to be looped with many possible filename extensions.

What we need is a way to split filename from extension, even if we don’t know the extension beforehand. The os.path.splitext() alternative does just that, returning a tuple. Here, I’ll import the os module and then use a list comprehension to run os.path.splitext() through every filename in the list above.

It becomes a simple matter to get just the filename from the tuple, as I do here by modifying the list comprehension to just get the zeroth item from it:

Note that several interesting conditions are handled by os.path.splitext():

• long filename extensions, including “.pages” and “.networkConnect”
• spaces in filenames, as in “Property List.plist”
• parentheses, as in “VPN (Cisco VPN).networkConnect”
• periods within the filename, as seen in “Dated Spreadsheet 2010.02.17.xlsx”

Mac OS X Snow Leopard appears to roll in the functionality of the separate Keychain Minder tool. Keychain Minder has provided a way for system administrators to help keep the passphrase in sync with the login account password. That can be very helpful for users in a directory services environment, because users may change their password in ways outside Mac OS X, thereby leaving the keychain passphrase out of sync.

The keychain passphrase is separate from the password used to log in to a Mac OS X user account. By default, however, the password on the login account is set as the passphrase for that user’s default keychain. When the password and passphrase get out of sync, it can cause a lot of confusion for those who don’t understand what’s going on.

I’d wager it’s a rare Mac OS X user that intentionally sets their login account password and keychain passphrase to be different, as I do. Therefore, keeping the two in sync is a benefit in a large percentage of cases.

Snow Leopard implements this feature as a preference item in Keychain Access, under the First Aid tab. It’s labeled “Synchronize login keychain password with account.” (I would have rephrased that as “default keychain” since keychains by other names can be the default keychain; the default name just happens to be “login” nowadays.)

Keychain Minder stored its settings in the com.afp548.KeychainMinder.plist preferences file. This doesn’t seem to have any impact, one way or another, on this particular keychain preference.

So, I looked for and eventually discovered that the new built-in feature of Snow Leopard stores its state in the SyncLoginPassword key of the com.apple.keychainaccess.plist file. You can see this change by use of the defaults command in Terminal:

You will want to have this preference disabled on any user accounts — likely power users — whose login account passwords will differ from their keychain passphrases. Otherwise, they will get prompted regularly to “Synchronize,” “Create New,” or “Continue” during the login process.

After using Acquia Drupal for a while, I took advantage of a trial subscription to the Acquia Network. The network’s services showed me that I had files present in my install that the agent could not account for.

I suspected this was happening because of the way I manage my Acquia Drupal installation with Mercurial. So, I’ve modified my previous process (and updated my instructions) to extract the downloaded tar archive with the --recursive-unlink option. This option appears to successfully remove the contents of every directory before putting new files back into them.

When the archive is extracted in this way, my repository’s working directory shows modified, unknown, and deleted files. This allows me to treat each category of files individually before I commit the changes for a Drupal update as a revision.

$ hg status

The modified files will be tracked normally because they’ve already been added to the Mercurial repository, so I don’t need to do anything special for them.

The unknown files are ones that are completely new, and have not appeared in the same position in a previous revision. They have yet to be tracked by Mercurial, so I have to add them to the repository. To add just those unknown files, then, I have to pick them out from the status listing:

$ hg status --unknown

In order to operate just on those files to add them to the repository, I run a for loop:

This changes the “?” status to “A,” because the files were successfully being tracked by Mercurial.
I use the “--no-status” flag on the “status” command so that just the file paths are printed; the actual status code is not, which is appropriate for the target of the “add” command in the loop.

I do the same basic steps with deleted files. These are files that were in the previous revisions but have been deleted by the --recursive-unlink option from the tar extraction and not replaced with the extraction of the new Acquia Drupal tar archive. If the deleted files had been replaced by the tar extraction, they would either be unchanged (which would not show up in the “status” output) or marked as modified.

To remove the files that are marked as deleted from the repository’s working directory:

However, that may be the same as simply using the following, which I have to explore further:

$ hg remove --after

So, to follow all of these changes in the repository, I run the loop for the uknown files and the loop for the deleted files. The modified files are already tracked, so I don’t need to do anything additional for them. After that, a “commit” will record all of the changes — modifications, additions, and deletions — in the repo.

These commands are based on my current understanding of Mercurial, and they do work for me right now. There could certainly be another better way to do this in one fell swoop — or at least fewer steps. I would welcome that, so if you’re aware of a way, feel free to comment or contact me.

Update: I found that the “hg addremove” command cleanly replaces all of the shell loops I mentioned above. Therefore, I recommend using it instead of the “for” loops I described.

Apache Solr provides a Web service front end for the Apache Lucene indexing and search engine library. Both Solr and Lucene (upon which Solr depends) are Java-based, which has implications for shared Web hosting.

Drupal is an open source CMS, and I happen to use it on a shared Web hosting provider as of this writing. Drupal is gaining support for Apache Solr through a module that has had a lot of input from Acquia (the “Red Hat” of Drupal).

Dries Buytaert of Acquia has some interesting perspective on search for the Web and CMSes in some recent articles on his site. Specifically, he talks about Acquia Search, a Solr-based search service that is being offered to Drupal sites on the Acquia Network. He discusses the advantages afforded by good search capabilities for both visitors to a Drupal Web site and for site administrators.

I’ve used Acquia Search (in beta), and it has been great. It’s very fast compared to the core Drupal Search module. The ability to perform faceted searches, word stemming, spell checking, and more is all tremendous. (You can see it in action in the search field in the site sidebar, as long as my Acquia Network subscription from the beta lasts.)

But Acquia Search part of a larger service offering — the Acquia Network — which ultimately makes it too expensive for me on my personal sites. It’s priced out of reach for me — more costly for one year than two years of Web hosting, domain registrations, and separate e-mail hosting for my domains are today. I think it’s clear that Acquia is aiming at a different market, and that’s fine.

My idle thought, however, is that search by itself is a compelling feature even for small Web sites like mine. It’s as compelling as hosting files, like HTML or PHP or images, or serving databases, like MySQL and PostgreSQL.

If it’s important as Dries notes in his posts — that the search market is so large and growing, and of such universal importance — then great search is a compelling feature to have for many levels of sites. After serving the files and serving the database, it may be the next big service that a Web hosting provider could offer. And today, Web hosting offers a range of pricing (and service levels) to meet various needs.

I could see advertising that for some monthly fee, a Web host offers 55 GB of storage and 550 GB of monthly data transfer and unlimited MySQL databases — and oh, by the way, some reasonable level of indexing/search with Apache Solr and/or Sphinx or whatever. Although I hate to suggest it, search could even be an optional add-on, as many providers treat dedicated IP addresses or SSL or the like.

There may be an additional win, in that separate servers could be optimized for search to offload that processing from the Web server. It could even be something that a Web host contracts out or partners with another to provide — maybe even with a company like Acquia that’s already set up their infrastructure to scale on Amazon EC2.

Especially if other CMSes, such as WordPress, get Solr integration — as with this WordPress Solr plugin — then the case for Web hosts offering something like Solr search gets convincing.

It should come as no surprise that Apple Installer installation packages can contain scripts. These scripts are supposed to conduct important operations during the course of the software installation.

However, when you are the system administrator of more than one Mac, you find that developers sometimes miss a good balance between what you think should be in the installer payload versus what should be in its scripts. The payload of a installer, by definition, are the files and links that should be installed, along with information on where they should be installed as well as how (i.e. ownership, permissions).

Therefore, developers should not need to run scripts that create or delete files — they should be created from the payload itself, and if a file must be deleted during the install then consider that perhaps you’re doing it wrong. Likewise, there should be little need move or copy files, because as many copies as desired can be installed from the paylod. Similarly, the need to change ownership or modify permissions should be taken care of in the payload.

Perhaps I’m being a purist here. I’m certainly accused of that, from time to time. However, this just makes sense to me and I happen to think that many developers are similarly logical people. They just aren’t the kind of logical people who happen to spend effort on software installation, especially the kind that results in a deployment-friendly installer package.

So how do we as administrators verify the quality of the scripts in installers? Is there a way we can quickly peer into them to decide if any of the scripts’ steps will be superfluous or even (gasp!) harmful?

Well, I have a quick suggestion for scanning packaged installers. The following one-liner shell command will search an installer package or metapackage for scripts that have the kinds of steps outlined above.

Note that this will only work for the traditional installer packages; it will not work with Leopard-style flat packages (which are documented so badly by Apple that the best description comes from reverse engineering by Iceberg's author). The one-liner will currently only find the defined install operations scripts: preflight, preinstall, preupgrade, postinstall, postupgrade, and postflight. (Any other scripts are likely to be called by one of those six.) It assumes those scripts will be shell scripts, currently, even though any of them could be written in other scripting languages installed with Mac OS X, like Python, Perl, or Ruby. It will also not work on the JavaScript-based system and volume requirements portions of the installation.

However, it’s a start. The output displays the offending file and line number, so you can conduct more careful examination of the matches it finds.

I haven’t run this on an exhaustive list of installation packages, but I have already seen at least one installer that produces worrisome output.

Update: I’ve changed the regex for the pre/postflight script so that it is more general that what I originally posted. I’m also having some problems with the snippet working with a certain installer whose scripts I know have cp and chmod commands. So, I may be back to the drawing board with this; comments are welcome.

After the news hit about Time Warner Cable’s intent to charge different rates for tiers of monthly data transfer — and an enormous $1/GB fee for overages — it seems eminently sane to consider the competition.

In Rochester, that competition is Frontier DSL. For a long time, that basically meant there was no competition, I’m very sorry to say.

However, the changes to TWC’s fee structure may be so extreme that even that level of competition is good. While I don’t think our household monthly data transfer is excessive, I’m reasonably sure (based on what I’ve seen from the data I’ve collected from our broadband router) that we’ll blow right past the 5 GB/month tier and maybe the 10 GB/month one. We would have to — and by that I mean, I would have to, really — develop some more austere usage of the family Internet connection that we’re accustomed to. Thus, I’m examining the pro and con positions for Frontier’s high-speed Internet service.

Pro

With Frontier DSL, my family should:

• Not have to deal with the stress of the upcoming 5/10/40/100 GB-per-month tiers from TWC, which will reportedly take effect in Rochester in November 2009
• Get to send a clear message to TWC that metered Internet access is a terrible idea
• Get the peace of mind that unlimited, unmetered Internet access provides — but only if Frontier’s existing 5 GB-per-month transfer cap is eliminated
• Benefit from the refer-a-friend affiliate program — both parties get the $20 referral credit
• Be able to combine the billing with our Plain Old Telephone Service (POTS).

Con

However, there are some drawbacks to Frontier DSL. My family would be concerned about:

• A potential monthly DSL modem rental fee
• A two-year commitment with a $200-300 (I’ve see both figures!) early termination fee
• The 5 GB-per-month unenforced data transfer cap (but that cap may be dropped entirely in an effort to better compete with TWC)
• Blocking port 25 — while there are workarounds, this is just aggravating
• The unknown quantity of Frontier’s technical support, whereas TWC’s has been reasonably good over the years.

Anyway, while we’re mulling this over, the news is playing out on sites like StopTheCap and StopTWC! Meanwhile, I’m more than a little annoyed at the traditional news media avoiding some of the other angles surrounding this topic — the pricing change as a way to protect cable television revenues, the local monopoly (and how cable infrastructure compares to its telephone equivalent), the impact on increasingly Internet-dependent households during a recession, how this might change the habits of people (including employees working at home), and so on.

It was extremely satisfying to see the following dialog about the volume expansion when logging into my Infrant ReadyNAS:

The overall process of expansion took about four days, converting from four 320 GB drives to four 1 TB drives. (For reference, I selected the Hitachi 7K1000.B 0A38016 drive from ZipZoomFly, and did so almost entirely on price — despite my longstanding misgivings about IBM/Hitachi drives.)

The capacity expansion took longer than strictly necessary, because I wrote zeros across each of the drives before installing them, one at a time, in the ReadyNAS. (I ended up writing zeros to each drive twice, switching from the Disk Utility to the command line equivalent.)

Near the end of the process, I found out that the automatic X-RAID™ expansion doesn’t happen until you reboot the ReadyNAS after upgrading the last drive. I had also enabled a snapshot on the ReadyNAS, which also prevented the automatic volume capacity expansion, so I had to delete that.

Here is a sequence of commands and output that show how I keep the Acquia Drupal open source content management system up to date with Mercurial, the open source distributed version control system.

In the example below, my Mercurial repositories for Drupal are located in the “drupal” subdirectory of my “repo” folder. Once I’ve moved into that directory, I download the Acquia Drupal distribution with curl and then extract it into my previously-created Mercurial working directory, “acquia_drupal,” using tar.

(Update: I added the --recursive-unlink option after I noticed that the Acquia Network control panel keeps track of extra — possibly unneeded — files and folders you have in your install. The recursive unlink option seems to avoid having stray files from old versions of modules hanging around in your repository after you install updates.)

After extracting Acquia Drupal my Mercurial working directory, I get the status of the repository. It shows there are changes from the last version I checked in — and this includes new files, denoted by a “?” at the beginning of their line.

Since there are new files, I have to add them so they’ll be tracked by the repository. I only need to add in the parent directory for any changed files, and any new files within it will also be added for tracking.

Excellent; the new files have been added. After this, I just need to accommodate the deleted files that no longer need to be tracked (created when using the “--recursive-unlink” option on tar). For that, see my newer instructions.

Now that the right files are being tracked, I need to commit the changes — modified, added, and deleted files — to the repository. This will create a new revision in the repository’s history, which I’ll tag with the text “Acquia Drupal 1.2.0.”

Once this revision is checked in, I can use it to propagate changes to other repositories. I keep the main Acquia Drupal distribution in its own repository, and then use the “hg fetch” command to pull its changes into one where I track contributed modules. That second repository is then pulled into a third repository which stores just the changes for my production Web site. The use of three repositories in this way modularizes and isolates the updates.

Nigel and Jeff present Mac OS X Laptop Deployments with Puppet in the MacIT track at Macworld Expo 2009. They are two of the first Mac system administrators I knew of using Puppet, and both had a background in Radmind.

I’ve been reading through James Turnbull’s Pulling Strings with Puppet, since our library had a copy. I had hoped to get through it during our winter break, but illness and other factors (no Puppet pun intended) conspired to get in the way. From what I’ve read about it already, Puppet is clearly interesting. Nigel was very enthusiastic about it when we talked at WWDC 2008.

To me, it seems that it would take some effort to model what you want in it and build up a repository of what you want managed. Perhaps I’m feeling like an old dog trying to learn new tricks. Grin.

One point that Nigel and Jeff made in their presentation slides that struck me is that they needed a solution that works when offline, which Puppet does. Radmind can work offline but I daresay that’s not the way that most people would think to use it (lapply with its “-n” flag would be the most basic change).

Kyle also mentioned to me that he’s been using Puppet in conjunction with Radmind. I believe he has Puppet managing configurations and Radmind managing the bulk of the filesystem.