Learning

Lessons and tools for personal enhancement.

Change the activity name in Harmony Remote Software

I’m updating the settings for my Harmony 520 and discovered that I couldn’t change the names of activities in Harmony Remote Software v7.7.0. I am not alone, but I couldn’t find a solution. While I had the problem on Mac OS X 10.6.8, others reported problems on various other operating systems, including Microsoft Windows.

The software wouldn’t let me to change the name of more than one activity. Instead, it would let me select and highlight the text, but not edit it. It was maddening, especially when it would beep instead of accepting characters I typed.

Then, by happenstance, I clicked over to Safari to do something while I was frustrated. When I flipped back to the Harmony Remote Software, it finally accepted my typed changes.

Therefore, if you have come across this problem, try switching to another application and back to the Harmony software again.

I’m continually astounded at how aggravating the Harmony remote can be, after all the hype. I suppose it is still better than many, if not most, of the alternatives, but that isn’t saying much.

Mystery of the failed SSH logins solved by discovery of empty files

I had a repeat of a problem that I’ve probably had many times over the years. I wanted to write down the solution before I forgot it, because it had been long enough since the last time that I’d forgotten the fix.

Let’s set the stage: I wanted to SSH into a remote system. In my case, that remote system was a Mac OS Snow Leopard system. The client was also Mac OS X.

On that remote system, I had enabled sshd. This is done on Snow Leopard with System Preferences > Sharing > Remote Login. (In the Sharing System Preferences, you set up the users that are allowed to remote log in, either allowing all users or setting up an ACL that only allows specific users. If you set up the ACL, it should supercede whatever additional user/group login restrictions you’ve configured in /private/etc/sshd_config.)

Then, I tried to SSH from my client system. Instead of a successful login, I was told the following — before I was prompted for a password.

$ ssh user@server.example.com
Connection closed by 192.168.1.10.

Scratching your head, you add verbosity to the SSH connection attempt.

$ ssh userid@server.example.com
[snip]
debug1: SSH2_MSG_KEXINIT sent
Connection closed by 192.168.1.10.

I searched for other people having the same problem with “debug1: SSH2_MSG_KEXINIT sent,” because other people always have the same problem and some of them wrote about it and some of those solved it. Right?

Well, in this case, the other solutions I found were not helpful to my specific situation. But many people responding to pleas for help did mention the always-good advice to “check the server logs.” Which I could do, so I did.

In the secure.log, I found several groups of lines like this corresponding to the times I had tried to log in:

Nov 5 18:47:20 server sshd[1239]: error: Could not load host key: /etc/ssh_host_key
Nov 5 18:47:20 server sshd[1239]: error: Could not load host key: /etc/ssh_host_rsa_key
Nov 5 18:47:20 server sshd[1239]: error: Could not load host key: /etc/ssh_host_dsa_key
Nov 5 18:47:20 server sshd[1239]: Disabling protocol version 1. Could not load host key

That reminded me of the common problem with SSH host keys on Radmind-managed computers. See, Mac OS X will try to create the host keys if they are missing, but not if they are zero-length. On Radmind-managed computers, it was trivially easy to get zero-length SSH host key files in /private/etc because the tendency was to manage them with negative transcripts. Files listed in negative transcripts would be created if they were missing, but they would be created as empty files (by design).

Empty SSH host key files will prevent you from logging into that system with SSH.

I checked the server and — sure enough — the host key files were zero length. I deleted them, then stopped and restarted Remote Login for good measure. This solved the problem, and I could log in from the client.

Reinstall Xcode if easy_install fails with missing files like python.h

I was trying to install Dulwich on a Mac OS X Lion system this week and ran into difficulty. I kept getting installation failures that included a missing “python.h” and, eventually, llvm-gcc-4.2 failed to compile the module.

I found the situation frustrating, partly because I pretty much own the top search hits about how to install Dulwich and Hg-Git on Mac OS X Lion, thanks to some earlier article.

It turns out that I had reinstalled Lion about two weeks ago, and had not reinstalled Xcode 4. So, I updated to Xcode 4.2 and this completely eliminated my problem. Presumably, it would also work for you — and for future me, since I’m likely to repeat this — even if Lion hadn’t been reinstalled in between.

Things that didn’t work included but were not limited to:

  • cursing under my breath
  • stomping
  • hand-waving
  • complaining on Twitter
  • the silent treatment
  • waiting for 4, 12, and then 24 hours to see if it would fix itself
  • installing the latest version of setuptools, v0.6c11, from an egg
  • installing the current version of Distribute, new hotness or not
  • any steps involving Linux distribution package managers like apt-get.

Read preferences from a property list within a Mac OS X Python script

I had a need to read some settings from a Python script on Mac OS X recently. I wanted to be able to change selected parameters for the script — some of which could be site or implementation specific — without embedding them directly in the code. Since the script was Mac OS X-only, using a property list seemed like a good idea.

With customizable settings from a property list, a script could become useful and more customizable for a wider community — or even different internal audiences.

I thought reading preferences was going to take a lot of effort. I was, however, pleasantly surprised at how easily I was able to accomplish it.

Asking others who had been down this route before resulted in some links to Apple docs. I wanted a more concrete example of how it was done in Python, and I got a great one.

That example came from reading the source to munkilib from the open source project, Munki. Since Munki had its own preference file, it needed to read from it, and its example was very enlightening.

Frogor directed me to a specific spot in the Munki code. That spot demonstrated how to read a preferences file with CoreFoundation.

Munki also has its own internal defaults for preferences. The munkilib/munkicommon.py example showed how to implement default settings in their absence in a property list. That provides a fallback position so that you always have some value available. In Munki, setting the defaults was done within a function. It seemed that it would be more generic if those defaults were separated out of the preference-reading function.

My own example of how to read a plist is outlined below. This focuses on just what you need in order to read the preferences and provide default settings for a larger script. A single set of preferences can be shared between scripts, and each script can encode its set own defaults. While you can create your own keys and values, for the example below, I will use the keys “StringPreference,” “BooleanPreference,” “ArrayPreference,” and “DictionaryPreference.”

  1. Create a new property list. There are several ways to do this, including the property list editor that has been rolled into Xcode (and is no longer a standalone application) in v4.
  2. Save the file. The name will be used later.
  3. Set up the basic shell of the script that will read the plist and import CoreFoundation.
    #!/usr/bin/env python

    from Foundation import CFPreferencesCopyAppValue
  4. Add in a variable for the script’s bundle ID. The bundle ID uniquely identifies your script’s preferences. It is written in reverse DNS notation, which should be familiar to almost anyone who has dealt with property lists before. It’s handy to have this defined globally for your script so that you can refer back to it as needed.
    #!/usr/bin/env python

    from Foundation import CFPreferencesCopyAppValue

    this_bundle_id = ‘com.example.your-script-here’
  5. Create a function to read a preference by its bundle ID. The business end of the function is the use of CFPreferencesCopyAppValue get a value for a key.
    #!/usr/bin/env python

    from Foundation import CFPreferencesCopyAppValue

    this_bundle_id = ‘com.example.your-script-here’

    def get_preference(preference_key, bundle_id=this_bundle_id):
        """
        Get the preference value for a given combination of preference
        key and bundle ID. Returns the requested preference value.
        """

        # Get the specified preference key from the specified preference
        # bundle ID using CoreFoundation
        preference_value = CFPreferencesCopyAppValue(preference_key, bundle_id)
        return preference_value
  6. Tie the get_preferences function together with a default_preferences object. This way, any missing preferences will fall back to the defaults encoded in your script. Any preferences that are set in a property list will override the defaults.
    #!/usr/bin/env python

    from Foundation import CFPreferencesCopyAppValue

    this_bundle_id = ‘com.example.your-script-here’
    default_preferences = {
        ‘StringPreference’: False,
        ‘BooleanPreference’: False,
        ‘ArrayPreference’: list(),
        ‘DictionaryPreference’: dict(),
    }

    def get_preference(preference_key, bundle_id=this_bundle_id):
        """
        Get the preference value for a given combination of preference
        key and bundle ID. Returns the preference value.
        """

        # Get the specified preference key from the specified preference
        # bundle ID using CoreFoundation
        preference_value = CFPreferencesCopyAppValue(preference_key, bundle_id)
        # If the value is not set in the property list, get a default value
        # from the default_preferences objects
        if preference_value == None:
            preference_value = default_preferences.get(preference_key)
        return preference_value

The script won’t do anything yet, until we create a main() or other functions to call the get_preferences function. Since you’d be reading preferences as part of a larger script, this is fine for now. However, with what’s written already, you can test out reading preferences interactively with the shell and Python interpreter.

  1. Start by checking the property list in the shell. In my case, I created an example property list that sets two of the four preferences.
    $ defaults read com.example.your-script-here
    {
        BooleanPreference = True;
        StringPreference = abc;
    }
  2. Open Terminal, run “python” at the command prompt, and paste the script code above at the interactive Python “>>>” prompt.
  3. Run the get_preferences function and print its output.
    >>> print(get_preference(‘StringPreference’))
    abc
    >>> print(get_preference(‘BooleanPreference’))
    True
    >>> print(get_preference(‘ArrayPreference’))
    []
    >>> print(get_preference(‘DictionaryPreference’))
    {}

That’s it, the expected results were returned. Notice that the output for StringPreference and BooleanPreference are taken from the property list, while the empty array and dictionary come from the default_preferences in the script.

Update: Greg pointed out that there is a certain degree of danger using #!/usr/bin/env python as the shebang line in the script. Should a system have a non-Apple installation of Python, then /usr/bin/env python might return that. Another Python is probably less likely to be able to bridge CoreFoundation, and thus wouldn’t be able to use the CFPreferencesCopyAppValue call to read preferences.

I think the overall shebang danger is small because few Mac OS X systems will have an alternative Python installed, but clearly your chances of that increase in certain situations. To eliminate this risk, do what Munki does and insert the #!/usr/bin/python shebang instead.

Parse a vendor RSS feed to get the latest available product version

There may be times when you want to obtain the number of the latest available version — not just the latest installed version — of a software package through automated means. If the vendor or project provides a syndication feed (either RSS or Atom) that describes new releases, then you may be able to parse that data and get the newest release from it.

As an example, let’s examine the RSS feed for Group Logic’s ExtremeZ-IP. Other developers provide RSS/Atom feeds for their releases, but the EZIP feed is a good one to start a demonstration with because it is generally structured well.

We can break apart the EZIP feed with the Universal Feed Parser module for Python, which you must obtain separately.

Update: Because of Mark Pilgrim situation (also described here), the Universal Feed Parser Web site is no longer available. There is an alternative source for the Universal Feed Parser at Google Code, and I have cloned the Universal Feed Parser repository to Bitbucket from there.

import feedparser
ezip_feed = feedparser.parse(‘https://www.grouplogic.com/ezipreleases.xml’)
ezip_feed[‘feed’][‘title’]
u‘ExtremeZ-IP Latest Releases’
ezip_feed.version
‘rss20’

As you can see, the “ExtremeZ-IP Latest Releases” feed is automatically recognized as RSS 2.0. I prefer to use HTTPS for fetching these feeds whenever possible, so if the developer has an HTTP feed, I try to see if it also works with HTTPS.

Next, let’s find out where the version numbers are kept in the feed. It looks like they are in the entry title, based on reading the feed in Safari RSS. I can confirm that with the Universal Feed Parser. We’ll want to examine the title of every feed item so we can better handle both current and future entries from the feed. There are more entries to the EZIP than I will print out.

for entry in ezip_feed.entries:
    entry[‘title’]
u‘ExtremeZ-IP File and Print Server - Version 7.1.1x94’
u‘ExtremeZ-IP File and Print Server - Version 7.1x14’
u‘ExtremeZ-IP File and Print Server - Version 7.0x41’

We get Unicode strings as output from the Universal Feed Parser. That’s why the quoted strings are preceded with a “u” character.

I’d like to strip out the version string from the title element in each entry. I’m going to do so by splitting on whitespace and getting the last group of characters from the string. (This doesn’t account for text like “Hot Fix,” as seen in the EZIP feed, but it is still a good enough starting point for my purposes.)

for entry in ezip_feed.entries:
    entry[‘title’].split()[-1]
u‘7.1.1x94’
u‘7.1x14’
u‘7.0x41’

By stripping out the build number after “x” in the version string, you potentially lose some data. In the EZIP feed, there are entries where two consecutive version numbers are the same except for the build number after the “x.” However, depending on your needs, it may still be useful to eliminate that part of the version string, so we’ll do that next.

for entry in ezip_feed.entries:
    entry[‘title’].split()[-1].partition(‘x’)[0]
u‘7.1.1’
u‘7.1’
u‘7.0’

We really only need the most current or “top” item in the feed, since that should give us the newest release number. The newest version in this particular feed should be in the first entry. That’s “entries[0]” below, because we’re using Python and it zero references the first item in lists.

ezip_current_release = ezip_feed.entries[0].title
ezip_current_release_version = ezip_current_release.split()[-1].split()[-1]
ezip_current_release_version_stripped = ezip_current_release_version.partition(‘x’)[0]
u‘7.1.1’

There, we now have the version number of the most current release, 7.1.1, for the product. We have drawn it straight from the developer’s syndicated feed, so it is as current as the developer makes it.

How could that be useful? The output can be compared against other data, like the currently-installed version. The comparison, in turn, could be made part of a monitoring workflow, so you could get alerts if you fall behind.

If we didn’t strip the build number after the “x,” we would be left with a complex version number. Some Python tools, like distutils, will not currently handle the trailing characters in the version number well.

I have found that you can improve upon distutils’ StrictVersion/LooseVersion version number handling by switching to parse_version in pkg_rsources (“from pkg_resources import parse_version as V”). More coverage of that topic appears in PEP 386. If you are comparing the original version strings from the EZIP feed with similarly complex output from elsewhere, then I would probably use the pkg_resources module.

Download Sony GPS Assist data automatically when a memory card mounts with a LaunchAgent

Brix Anderson’s post about Downloading Sony GPS Assist Data Manually includes a Perl script that downloads the GPS Assist data for Sony cameras.

The GPS Assist data is stored on the memory card. When a memory card with GPS Assist data is inserted into a compatible GPS-equipped Sony camera — like the awesome Alpha SLT-A55V I got a few months ago — the camera has much faster lock-on times when searching for its location. The lock-on may take 10 seconds instead of requiring minutes.

I had been working without GPS Assist data since I bought the camera. I didn’t understand its value, but then I had captured photos where most of the shoot was incorrectly geotagged. After investigating this, I found the GPS Assist data should help, but that it required software on the desktop to update that data. Sony did not supply the relevant package for Mac OS X to do this.

Since I wanted to make sure I got updated location information on my photos, I did some searching and came across Brix’s page. I was honestly surprised to find that the GPS Assist data was written to the same memory cards as the photos, but it does make sense after some more thought.

Brix’s Perl script there certainly did the trick; my SLT-A55V picked up on the new GPS Assist data on the first card I tried it with. I have a limited number of cards, so I wrote a wrapper shell script that would run the Perl downloader script if certain volumes were present in /Volumes. I control the names of the cards, so I also control their paths.

I then call that wrapper script via a LaunchAgent. The LaunchAgent is triggered via “StartOnMount,” so it watches for new volumes to be mounted by Mac OS X. When a volume is mounted, the LaunchAgent runs, calls the wrapper script, the wrapper script checks for volume names, and if there is a match, runs the GPS Assist Perl script.

Now, every time I insert my memory cards to download photos, I get freshly-updated GPS Assist data for my camera.

Enable the Drupal Token Filter module to insert the current year into a copyright block

In the course of a conversation with Jesse today, I launched into action to fix the copyright statement in my site’s footer block. It still listed the copyright dates as “1999-2009” when the year had clearly rolled over into 2010 some time ago. Ahem.

Drupal 6, to my knowledge, doesn’t have a built-in way to insert a variable into the body of a text field in a node or block. However, it does have the very-powerful Token API, of which I’m a big fan. Tokens let you insert variable data. It’s just not always easy to figure out how a mere mortal — rather than a module developer — can take advantage of tokens in Drupal. (Variable insertion was a key feature of the UserLand Manila CMS I started using in the late 90’s, and I miss that sort of capability.)

In Drupal 5, I had previously used the Copyright module. Copyright module integrated with the Token API so you could use tokens in the block it created. The module was never upgraded for Drupal 6, so my footer copyright block became static text.

I realized that an Input Filter that worked with the Token API would probably let me do what I want, and I found a module that provided that missing link.

Here’s what I did so I could insert a constantly-advancing date variable into my copyright footer block:

  1. Downloaded Token Filter module.
  2. Installed Token Filter module.
  3. Enabled Token Filter module via the Admin Menu: Site Building > Modules > List.
  4. Added Token Filter to my standard Input Filter used across the site.
  5. Changed “2009” in my copyright block to “[site-date-yyyy]” which didn’t work, and then read the Token Filter docs to find out that should have been “2012” which then worked.

Now, my copyright block should always show the current year in the date range, like this: 1999-2012.

Access to this item is restricted

I had an odd situation over the weekend that resulted in the inability to view the passwords associated with keys in my Mac OS X user keychain. Every time I clicked on the “Show password” checkbox in a key’s detail window, I’d get an “Access to this item is restricted” dialog.

Needless to say, this was disconcerting. I happened to have a lot of data in that keychain — this is what I get for keeping the same one around since Mac OS X 10.0 or 10.1. While I could revert to a backup, the newest backup wasn’t as recent as I would like. Plus, I just wanted to know why the problem had cropped up.

So, I asked about my problem on the Apple-CDSA mailing list. If anyone would be able to help with the obscure corners of keychains, I figured the people there would.

Very promptly, I got a reply from Ken McLeod, which led me to check the validity of the code signature on the Keychain Access utility.

$ codesign -vvv /Applications/Utilities/Keychain\ Access.app
/Applications/Utilities/Keychain Access.app: code or signature modified

Clearly, the signature and the application didn’t match. Something was amiss.

I reinstalled Mac OS X 10.6.2 on the system, using the latest combo update installer package, and cleared up the problem signature mismatch.

$ codesign -vvv /Applications/Utilities/Keychain\ Access.app
/Applications/Utilities/Keychain Access.app: valid on disk
/Applications/Utilities/Keychain Access.app: satisfies its Designated Requirement

In retrospect, although I wouldn’t have thought of this being a problem, this breakage between the signature and the app — and its affect on my ability to view stored passwords — gives me confidence that thought has been put into the code signing mechanism in Mac OS X. You wouldn’t want a compromised app displaying your unencrypted keychain items, after all.

AppleScript date and time format parsing change in Snow Leopard

I had a handy script I wrote on a plane years ago that let me block off my Entourage calendar at specific times each weekday for a given week. The times for these events were created by concatenating some strings and converting the result into an AppleScript date object. I mention that merely for background, and because it was an incredibly geeky way to automate the tedious process of blocking off lunch on my calendar (without resorting to recurring events).

I found that my script didn’t work in Snow Leopard — despite flawless operation across several successive major versions of Mac OS X. The dates themselves remained correct, but the times were all coming up as 12:00 a.m. instead of what was expected.

For example, here’s a simplified reproduction scenario you can try on Snow Leopard:

set vEventDateString to "Friday, August 28, 2009"
set vEventStartTime to "10:00 a.m."
log vEventStartTime
(*10:00 a.m.*)
set vEventStartDate to the (date (vEventDateString & " " & vEventStartTime))
log vEventStartDate
(*date Friday, August 28, 2009 12:00 AM*)

It turns out that the fix is easy if not exactly obvious: remove the periods from “a.m.” and “p.m.” before converting strings to date objects. (I use the periods because I follow the Associated Press Stylebook!)

set vEventDateString to "Friday, August 28, 2009"
set vEventStartTime to "10:00 am"
log vEventStartTime
(*10:00 am*)
set vEventStartDate to the (date (vEventDateString & " " & vEventStartTime))
log vEventStartDate
(*date Friday, August 28, 2009 10:00 AM*)

So, there is a workaround in the unlikely event you encounter the same problem with your scripts.

Core Graphics bindings on 64-bit Python in Snow Leopard

Mac OS X Snow Leopard does not include Core Graphics bindings (CGBindings) for 64-bit Python.

The SWIG-based Python CGBindings originally shipped with Mac OS X 10.3, which bundled Python 2.3. Since that time, these bindings — specific to the system’s bundled framework build of Python — had allowed access to Core Graphics objects and commands from within scripts.

They were one of the reasons I decided to use Python in the first place. I thought they would be fun to learn and use, particularly with the then-new PDF Services feature of Mac OS X. The Core Graphics bindings also provided much, much more power than the command line sips tool and had an advantage over other alternatives by being bundled with the operating system. I thought they offered the possibility of growing with Mac OS X’s graphics hardware acceleration. I even found a way to use them to create better screenshots with drop shadows, a task where I’d previously employed Ambrosia’s Snapz Pro X.

Here’s an example of what you’ll see on Snow Leopard if you try to “import CoreGraphics” in 64-bit Python:

# Explicitly enable 64-bit execution for Python
$ export VERSIONER_PYTHON_PREFER_32_BIT=no  
$ python
Python 2.6.1 (r261:67515, Jul  7 2009, 23:51:51)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from CoreGraphics import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/BinaryCache/CoreGraphicsBindings/CoreGraphicsBindings-26~139/Root/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/CoreGraphics/__init__.py", line 7, in <module>
ImportError: /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/CoreGraphics/_CoreGraphics.so: no appropriate 64-bit architecture (see "man python" for running in 32-bit mode)
>>> ^D

With 32-bit Python on Snow Leopard:

# Explicitly enable 32-bit execution for Python
$ export VERSIONER_PYTHON_PREFER_32_BIT=yes
$ python                                  
Python 2.6.1 (r261:67515, Jul  7 2009, 23:51:51)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from CoreGraphics import *
>>> ^D

While the CGBindings are still available to 32-bit Python in Snow Leopard, you must use PyObjC to replace their functionality for 64-bit Python. Since 64-bit Python is the default in Snow Leopard, it makes sense to transition from the bindings to PyObjC as soon as possible. This means there is some porting work for scripts that used the Core Graphics bindings. I guess I’m glad I didn’t do as much with them as I’d planned.

I see this change as something of a loss. (Is this what Carbon developers are experiencing? Hm.) The Core Graphics bindings were relatively easy to use and felt reasonably Pythonic, even if the documentation was almost nonexistent. PyObjC feels more foreign to me when I attempt to use it — even though it’s clearly the future.

Syndicate content