Software

Remove filename extensions with os.path.splitext instead of the {lr}strip or replace methods in Python

Jesper Noehr explains why {l,r}strip are considered harmful for removing extensions from filenames with Python. I think he’s absolutely right on that score, and I would agree. The lstrip() and rstrip() methods shouldn’t be used for this purpose.

However, like the only commenter on that post, I’d also recommend os.path.splitext() as the proper tool for the extension-removing job.

Let’s take some example filenames you might come across on Mac OS X Snow Leopard:

>>> a = [ 'Document.pages', 'Property List.plist', 'Application.app', 'Word.docx', 'VPN (Cisco VPN).networkConnect', 'Dated Spreadsheet 2010.02.17.xlsx']

If we had a list of filenames (or file paths) like this — perhaps created by os.walk() or some other generator-based process — we couldn’t easily use Jesper’s recommended solution. The replace() string method would give us a much harder time dealing with the range of filenames and extensions in that list. In the case where you don’t know the filename extensions in advance, replace() breaks down. The replace() method would have to be looped with many possible filename extensions.

What we need is a way to split filename from extension, even if we don’t know the extension beforehand. The os.path.splitext() alternative does just that, returning a tuple. Here, I’ll import the os module and then use a list comprehension to run os.path.splitext() through every filename in the list above.

>>> import os
>>> [os.path.splitext(x) for x in a]
[('Document', '.pages'), ('Property List', '.plist'), ('Application', '.app'), ('Word', '.docx'), ('VPN (Cisco VPN)', '.networkConnect'), ('Dated Spreadsheet 2010.02.17', '.xlsx')]

It becomes a simple matter to get just the filename from the tuple, as I do here by modifying the list comprehension to just get the zeroth item from it:

>>> [os.path.splitext(x)[0] for x in a]
['Document', 'Property List', 'Application', 'Word', 'VPN (Cisco VPN)', 'Dated Spreadsheet 2010.02.17']

Note that several interesting conditions are handled by os.path.splitext():

  • long filename extensions, including “.pages” and “.networkConnect”
  • spaces in filenames, as in “Property List.plist”
  • parentheses, as in “VPN (Cisco VPN).networkConnect”
  • periods within the filename, as seen in “Dated Spreadsheet 2010.02.17.xlsx”

Access to this item is restricted

I had an odd situation over the weekend that resulted in the inability to view the passwords associated with keys in my Mac OS X user keychain. Every time I clicked on the “Show password” checkbox in a key’s detail window, I’d get an “Access to this item is restricted” dialog.

Needless to say, this was disconcerting. I happened to have a lot of data in that keychain — this is what I get for keeping the same one around since Mac OS X 10.0 or 10.1. While I could revert to a backup, the newest backup wasn’t as recent as I would like. Plus, I just wanted to know why the problem had cropped up.

So, I asked about my problem on the Apple-CDSA mailing list. If anyone would be able to help with the obscure corners of keychains, I figured the people there would.

Very promptly, I got a reply from Ken McLeod, which led me to check the validity of the code signature on the Keychain Access utility.

$ codesign -vvv /Applications/Utilities/Keychain\ Access.app
/Applications/Utilities/Keychain Access.app: code or signature modified

Clearly, the signature and the application didn’t match. Something was amiss.

I reinstalled Mac OS X 10.6.2 on the system, using the latest combo update installer package, and cleared up the problem signature mismatch.

$ codesign -vvv /Applications/Utilities/Keychain\ Access.app
/Applications/Utilities/Keychain Access.app: valid on disk
/Applications/Utilities/Keychain Access.app: satisfies its Designated Requirement

In retrospect, although I wouldn’t have thought of this being a problem, this breakage between the signature and the app — and its affect on my ability to view stored passwords — gives me confidence that thought has been put into the code signing mechanism in Mac OS X. You wouldn’t want a compromised app displaying your unencrypted keychain items, after all.

Weird things happen when you run out of disk space

I had interesting things happen when I ran out of disk space today.

The most notable one was that I saw “Process completed” — or some variant of that — every time I tried to open a Terminal window with a new shell session. I briefly staved it off by specifying Bash as my shell, but then it came right back after opening another tab or two in Terminal. Consulting Google led me to this “Terminal application quits” thread at Apple Discussions. On a lark, I tried deleting /usr/bin/login as one poster suggested. It worked!

…But only for a little while. The problem returned. In the meantime, I had freed up some disk space because I’d realized I couldn’t save files anywhere (“But ~/Pictures is writable!”). Clearly something else had become an issue, because disk space was available.

Then I found another thread, “Terminal’s ’Process Completed’ message and /usr/bin/login,” on Apple Discussions. The more permanent solution from that thread appears to be the removal of corrupted Apple System Log databases. Once I did that and restarted the ASL service, all was well and has stayed that way so far.

Filling up my disk must have corrupted the logs as they were being written or rotated, and led to this cascade failure. Like I said, interesting!

One of the posters in the second Apple Discussions thread indicated that the underlying database corruption issue is addressed in Snow Leopard. However, it seems that you could still see this on Leopard — my experience was with a recently-patched Mac OS X 10.5.8 system.

The new Apple Education Licensing Program

The new Apple Education Licensing Program replaces the Apple Maintenance Program (AMP) as the primary way to purchase ongoing software upgrade rights for a school, college, or university’s fleet of Macintosh computers. The Apple Education Licensing Program (AELP? or just ELP?) is a yearly, renewable license.

I think this is a big deal because it’s awfully close to what I’ve asked Apple for year after year — especially at venues like WWDC (at least when they have presented the opportunity for giving constructive feedback).

The ELP licensing for Mac OS X is actually a bundle — called the “Mac Software Collection” — of the operating system combined with the iLife and iWork suites. (It aligns with the same bundle of software in the standalone “Mac Box Set.”) I believe that the bundling of iLife is the biggest win, because it provides upgrade rights to software that is bundled with every new Mac but frequently updated. If you are managing a larger group of Macs over several years, the licensing and deployment issues involved with iLife could be complex — moreso because it’s the kind of software that people want on their computers. It also doesn’t hurt that Keynote is included — anecdotally, it appears to sell a lot of Macs in organizations all by itself.

The ability to keep the operating system and core applications from the Mac Software Collection up-to-date across a range of Macs with a single renewable license could be a tremendous savings in administrative overhead.

Organizations must cover 100 percent of their installed base of owned or leased Macs, at least for the Mac Software Collection. This may be a deal-breaker for a number of universities and perhaps other organizations that tend to be decentralized in their operations.

Apple’s other software can also be obtained through ELP, but can be purchased to cover 100 percent of computers at the departmental level. It is possible that the exact nature of the organizational unit is at the organization’s discretion.

ELP allows the flexibility to cover faculty/staff home use as well as students, should the organization elect to pay for that additional coverage.

Each ELP purchase appears to incur a one-time 10 percent enrollment fee. If an organization doesn’t renew for a year but then buys again later, it may be subject to the enrollment fee again. And, if you have a lot of departments seeking Apple’s Pro Apps or IT-related software, those 10 percent charges could add up to significant overhead. I’m not sure if there’s a clear way to allow two or three departments to get the software they want without requiring an entire educational institution’s Macs be covered.

As part of the introduction of ELP, educational institutions are being encouraged to move to the new licensing program before December 13, 2009, through a waiver of the enrollment fee. It appears possible to crossgrade from AMP to ELP, at least if you talk to your Apple account team.

I am not aware whether Apple has rolled out something similar to organizations outside education.

AppleScript date and time format parsing change in Snow Leopard

I had a handy script I wrote on a plane years ago that let me block off my Entourage calendar at specific times each weekday for a given week. The times for these events were created by concatenating some strings and converting the result into an AppleScript date object. I mention that merely for background, and because it was an incredibly geeky way to automate the tedious process of blocking off lunch on my calendar (without resorting to recurring events).

I found that my script didn’t work in Snow Leopard — despite flawless operation across several successive major versions of Mac OS X. The dates themselves remained correct, but the times were all coming up as 12:00 a.m. instead of what was expected.

For example, here’s a simplified reproduction scenario you can try on Snow Leopard:

set vEventDateString to "Friday, August 28, 2009"
set vEventStartTime to "10:00 a.m."
log vEventStartTime
(*10:00 a.m.*)
set vEventStartDate to the (date (vEventDateString & " " & vEventStartTime))
log vEventStartDate
(*date Friday, August 28, 2009 12:00 AM*)

It turns out that the fix is easy if not exactly obvious: remove the periods from “a.m.” and “p.m.” before converting strings to date objects. (I use the periods because I follow the Associated Press Stylebook!)

set vEventDateString to "Friday, August 28, 2009"
set vEventStartTime to "10:00 am"
log vEventStartTime
(*10:00 am*)
set vEventStartDate to the (date (vEventDateString & " " & vEventStartTime))
log vEventStartDate
(*date Friday, August 28, 2009 10:00 AM*)

So, there is a workaround in the unlikely event you encounter the same problem with your scripts.

Core Graphics bindings on 64-bit Python in Snow Leopard

Mac OS X Snow Leopard does not include Core Graphics bindings (CGBindings) for 64-bit Python.

The SWIG-based Python CGBindings originally shipped with Mac OS X 10.3, which bundled Python 2.3. Since that time, these bindings — specific to the system’s bundled framework build of Python — had allowed access to Core Graphics objects and commands from within scripts.

They were one of the reasons I decided to use Python in the first place. I thought they would be fun to learn and use, particularly with the then-new PDF Services feature of Mac OS X. The Core Graphics bindings also provided much, much more power than the command line sips tool and had an advantage over other alternatives by being bundled with the operating system. I thought they offered the possibility of growing with Mac OS X’s graphics hardware acceleration. I even found a way to use them to create better screenshots with drop shadows, a task where I’d previously employed Ambrosia’s Snapz Pro X.

Here’s an example of what you’ll see on Snow Leopard if you try to “import CoreGraphics” in 64-bit Python:

# Explicitly enable 64-bit execution for Python
$ export VERSIONER_PYTHON_PREFER_32_BIT=no  
$ python
Python 2.6.1 (r261:67515, Jul  7 2009, 23:51:51)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from CoreGraphics import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/BinaryCache/CoreGraphicsBindings/CoreGraphicsBindings-26~139/Root/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/CoreGraphics/__init__.py", line 7, in <module>
ImportError: /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/CoreGraphics/_CoreGraphics.so: no appropriate 64-bit architecture (see "man python" for running in 32-bit mode)
>>> ^D

With 32-bit Python on Snow Leopard:

# Explicitly enable 32-bit execution for Python
$ export VERSIONER_PYTHON_PREFER_32_BIT=yes
$ python                                  
Python 2.6.1 (r261:67515, Jul  7 2009, 23:51:51)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from CoreGraphics import *
>>> ^D

While the CGBindings are still available to 32-bit Python in Snow Leopard, you must use PyObjC to replace their functionality for 64-bit Python. Since 64-bit Python is the default in Snow Leopard, it makes sense to transition from the bindings to PyObjC as soon as possible. This means there is some porting work for scripts that used the Core Graphics bindings. I guess I’m glad I didn’t do as much with them as I’d planned.

I see this change as something of a loss. (Is this what Carbon developers are experiencing? Hm.) The Core Graphics bindings were relatively easy to use and felt reasonably Pythonic, even if the documentation was almost nonexistent. PyObjC feels more foreign to me when I attempt to use it — even though it’s clearly the future.

iPhone above all other mobile phones in the Snow Leopard Address Book

The Address Book application in Mac OS X Snow Leopard has a new telephone number label for contacts: “iPhone.” (Credit to Jeff Carlson for bringing it to my attention.)

macosx-workstation-snowleopard-addressbook-iphonelabel.png

What I find just as interesting as its existence is its order in the list. The “iPhone” label comes before “mobile,” the label I used for all cellular numbers. Is that a subtle dig to put the iPhone above other mobile phones?

It is also the only label that rates a capital letter.

I have no idea how or if this label will survive through Entourage Sync Services and Exchange synchronization. Hm.

Update: I wouldn’t advise using this label right now if you use Sync Services or otherwise sync data elsewhere. I’ve already lost cell numbers in Entourage with Sync Services enabled when I flipped an existing “mobile” number to “iPhone” and back.

Update: I have tested it twice with dummy contacts and could not reproduce the problem that resulted in the number being removed from both Entourage and Address Book. (I believe this is odd because it happened to two of my existing contacts when I flipped them to “iPhone” and back to “mobile.”) However, the “iPhone” number does definitely get removed from Entourage through Sync Services, which means you would lose the number on anything connected to an Exchange account (if Entourage is synchronizing with Exchange). … Such as an iPhone with Exchange ActiveSync.

Python 32-bit execution on Snow Leopard

The default installation of Python on Mac OS X Snow Leopard is version 2.6.1. According to the man page for Python on Snow Leopard, Python 2.6 executes as a 64-bit application by default.

If, for some reason, you need to run it as a 32-bit application, this can be changed at the command line:

# Prefer 32-bit execution for Python 2.6.1 on Snow Leopard
$ defaults write com.apple.versioner.python Prefer-32-Bit -bool yes

The preference can be set in either the User or Local filesystem domain in Mac OS X, following the normal precedence rules. To unset it, presumably you would change the boolean to “no” — or perhaps even delete the “Prefer-32-Bit” key.

There is also an environment variable that can override this preference.

Sync the keychain passphrase with the login account password in Snow Leopard

Mac OS X Snow Leopard appears to roll in the functionality of the separate Keychain Minder tool. Keychain Minder has provided a way for system administrators to help keep the passphrase in sync with the login account password. That can be very helpful for users in a directory services environment, because users may change their password in ways outside Mac OS X, thereby leaving the keychain passphrase out of sync.

The keychain passphrase is separate from the password used to log in to a Mac OS X user account. By default, however, the password on the login account is set as the passphrase for that user’s default keychain. When the password and passphrase get out of sync, it can cause a lot of confusion for those who don’t understand what’s going on.

I’d wager it’s a rare Mac OS X user that intentionally sets their login account password and keychain passphrase to be different, as I do. Therefore, keeping the two in sync is a benefit in a large percentage of cases.

Snow Leopard implements this feature as a preference item in Keychain Access, under the First Aid tab. It’s labeled “Synchronize login keychain password with account.” (I would have rephrased that as “default keychain” since keychains by other names can be the default keychain; the default name just happens to be “login” nowadays.)

macosx-workstation-snowleopard-keychainaccess-syncaccountandkeychain.png

Keychain Minder stored its settings in the com.afp548.KeychainMinder.plist preferences file. This doesn’t seem to have any impact, one way or another, on this particular keychain preference.

So, I looked for and eventually discovered that the new built-in feature of Snow Leopard stores its state in the SyncLoginPassword key of the com.apple.keychainaccess.plist file. You can see this change by use of the defaults command in Terminal:

# Synchronize disabled in the Keychain Access preferences dialog
$ defaults read com.apple.keychainaccess SyncLoginPassword
0
# Synchronize enabled in the Keychain Access preferences dialog
$ defaults read com.apple.keychainaccess SyncLoginPassword
1

You will want to have this preference disabled on any user accounts — likely power users — whose login account passwords will differ from their keychain passphrases. Otherwise, they will get prompted regularly to “Synchronize,” “Create New,” or “Continue” during the login process.

View Kerberos tickets with Ticket Viewer in Snow Leopard

Throughout the history of Mac OS X’s inclusion of Kerberos, there has been a Kerberos application available in /System/Library/CoreServices. This utility was the graphical interface for managing Kerberos tickets &mdash. It put a Mac OS X face on the MIT Kerberos command line tools like kinit, klist, and kdestroy.

In Snow Leopard, that utility is replaced by a new application called “Ticket Viewer.” I’m not sure of the reason for the change, as it seems arbitrary, but it is what it is. (And it’s not as if Apple hasn’t changed more heavily-used application’s names — the Print Center vs. Printer Setup Utility situation springs to mind.)

macosx-workstation-snowleopard-ticketviewer-window.png

Those of you who may have linked to the Kerberos application — I liked having a symlink in /Applications/Utilities, for example — should update those links. (I also have to update my explicit indexing of that application for LaunchBar, because it doesn’t scan all of /System/Library/CoreServices by default.)

The Ticket Viewer is also available from the application menu in Keychain Access. It has the keyboard shortcut of Command-Option-K.

macosx-workstation-snowleopard-keychainaccess-appmenu-ticketviewer.png

Syndicate content