Technology

Compiling Rsync 3 as a Universal Binary for Leopard

When I come across software I might need to add into Mac OS X that requires compilation, I typically want to produce one Universal Binary. Make it a four-way UB and you get both 32- and 64-bit support.

A single binary is ideal for a Radmind transcript (or other package, if you wanted to bundle it into an installer) that can be deployed on both PowerPC and Intel Macs on Leopard.

Since rsync 3.0.2 with some patches is apparently working quite well at preserving Mac OS X data and metadata — passing the Backup Bouncer tests — I thought I’d try my hand at a four-way Universal Binary.

What worked for me, using a Mac Pro 4x2.8 GHz with Mac OS X 10.5.2 and Xcode 3.0, was to start with Mike Bombich’s instructions and modify them with some fairly standard Universal Binary build steps. The configure and compile were both less than a minute on this system.

$ ./prepare-source
$ env CFLAGS="-O -g -isysroot /Developer/SDKs/MacOSX10.5.sdk -arch i386 -arch ppc -arch ppc64 -arch x86_64" \
LDFLAGS="-arch i386 -arch ppc -arch ppc64 -arch x86_64" \
./configure --prefix=/usr/local --disable-dependency-tracking

$ make
$ sudo make install

I have seen the use of “-Wl,-syslibroot,/Developer/SDKs/MacOSX10.5.sdk,” in the LDFLAGS environment variable when compiling some applications but this did not work for me with rsync; when I removed it, rsync 3.0.2 configured successfully for me.

The result of the above build process appears to be a full four-way UB:

$ lipo -info /usr/local/bin/rsync
Architectures in the fat file: /usr/local/bin/rsync are: i386 ppc7400 ppc64 x86_64

A local transfer on the build system appears to have worked correctly. I did not test with Backup Bouncer, sync with a non-Mac system, or when shuttling data between architectures. So, accept these results with a grain of salt; I’m just happy I got rsync to compile for now.

Manipulate Apple Type Services settings in Leopard with atsutil

Aaron Hall noted+on+the+MacEnterprise+list that the font caches in Leopard “seem to be under /var/folders now, but you can manipulate them with the cool new atsutil in Leopard.”

The atsutil command is quite an interesting find to me, so I wanted to point it out.

Microsoft Office 2008 Service Pack 1 released

Microsoft released Service Pack 1 to update Office 2008 for Macintosh on Tuesday, May 13. The update makes a significant number of changes — spread across Entourage, Word, Excel, and PowerPoint — so if you are a system administrator, I would recommend examining the full description of Office 2008 for Mac Service Pack 1 (12.1.0) in its release notes.

The update is downloadable via Microsoft AutoUpdate (invoke that helper program with Help > Check for Updates in any of the Office applications, or look inside /Library/Application Support/Microsoft in the file system) as well as the Microsoft Office for Mac Web site.

List directory nodes available to a local Mac OS X host with dscl

It took me a little while to figure out how I could tell what DirectoryService directory nodes are available to a Mac OS X system. I could perform the task interactively, just issuing an “ls” command at the dscl “>” prompt. But for a non-interactive one-liner, I was bedeviled by what I thought was a pretty basic function for the dscl tool.

I wanted to find all of the top-level nodes — because, for example, that could tell+me+if+I+really+had+access+to+an+Active+Directory+or+not.

It turns out that I was missing the use of “localhost” as the datastore parameter, which reliably returns the information I wanted in a parsable form. Rather than “/” (simply doesn’t work) or just a period (only uses the local directory) — or even listing “/Active\ Directory” explicitly — “localhost” specifies all of the directories available to the local host. I was incorrectly assuming “localhost” would tie me to the local directory, as with “localonly,” and I thus avoided trying it until I felt like an utter failure.

$ dscl localhost -list .

This produces the following output on a Leopard-based system which only has local accounts in DSLocal:

$ dscl localhost -list .
BSD
Local
Search
Contact

If your system happens to be bound to a Microsoft Active Directory, you’ll instead see it prepended:

$ dscl localhost -list .
Active Directory
BSD
Local
Search
Contact

Or if you’re on Tiger — again bound to Active Directory — you’ll still see “Bonjour,” “NetInfo,” and “SLP” in the mix:

$ dscl localhost -list .
Active Directory
Bonjour
NetInfo
SLP
Search
Contact

Example of mismatched UIDs between a calendar event and cancellation in Microsoft Entourage

I had a good example today of how mismatched UIDs between Exchange meeting updates and existing events can affect Microsoft Entourage’s processing of those incoming updates.

I used the same drag-and-drop-to-export methodology as in my previous article. The iCalendar .ics file for the existing event that I dropped out of my Entourage calendar contained these unique identifiers:

UID:CD0000008B9511D182D800C04FB1625D717DE62091129A469FCB6E9EDDBF2075
X-ENTOURAGE_UUID:D26822E7-E4B7-49BB-9284-B18A2D42B77F

The e-mail cancellation from the meeting organizer, when dropped into the Finder as a .eml file, had the following unique identifiers:

UID:A449CD26-8099-11DC-8144-000393C78A56
X-ENTOURAGE_UUID:A449CD26-0991-DC81-4000-93C78A560000

As we can see, even though these were supposedly for the same event, neither the UID nor the X-ENTOURAGE_UUID properties matched. I emphasize the UID property because, as per my post on matching UIDs between Exchange meeting updates and existing events in Microsoft Entourage, I believe it to be the most important.

Both I and the meeting organizer are Entourage users in this case. That may be why both the event and the cancellation had X-ENTOURAGE_UUID properties at all. The UID property has appeared consistently for me in both events and their updates, no matter what their source.

Because of the discrepancy between unique identifiers, Entourage apparently couldn’t determine whether the cancellation was for the event that existed in my calendar. When Entourage “processed” the cancellation, it removed nothing from my calendar.

This seems entirely reasonable, although I would personally like Entourage to do deeper matching if it could: how about “is there an existing event by the same organizer and/or with the same name at the time of the update or cancellation?” Matching UIDs has got to be simpler.

How does this happen? Well, for me, it could have happened any number of ways: There could have been a past error when:

  • Entourage was trying to synchronize to my Exchange account
  • my Entourage calendar was synchronizing with Sync Services on Tiger or Leopard
  • Missing Sync for Palm OS, which I use with my Palm Treo 650, was performing its synchronization with Sync Services on Tiger or Leopard
  • I was using Microsoft Outlook in cached mode with my Exchange account
  • I used Duplicate Killer for Outlook from 4Team to remove massive numbers of duplicate calendar entries from my Outlook/Exchange calendar (which could have resulted in the loss of events with their original UIDs).

As far as I can determine, when you have data synchronization problems, you’re going to have one of the following outcomes:

  • data is added
  • data is deleted
  • data is modified
  • nothing happens.

Not all of these are pleasant, mind you — and that can depend on the circumstances involved. Deleting data is perfectly acceptable in some scenarios where it needed to be removed, but not others where it means it has just been lost. Adding data is great, unless it results in unwanted duplicates.

Anyway, I hope this helps you understand how Microsoft Entourage works a bit better.

Flat panel wake up and on screen display delay

Now that I have my dual-monitor Gateway FPD2485W setup, I’ve got a few complaints. Of course!

The two monitors take an awfully long time to wake up from their power-saving mode. Then, when they finally wake up — invariably at different times, the newer one first — I get the on-screen display (OSD) overlay telling me that they’ve chosen to accept the DVI input.

Well, duh, that’s the only video source hooked up to them, so it’s not helpful. This wouldn’t be so bad, but the OSD overlay stays on the screen for what seems like eons. Since the overlays are smack dab in the middle of the screen and are opaque, they block important visual elements like Mac OS X’s login window.

So far, I haven’t been able to find out how to get rid of the OSD overlay. If I could do that, I think I’d tolerate the wake up delay more readily.

Certainly, some of this is Gateway’s fault. I guess I can’t blame them much since they specifically don’t support Macs and that’s what I’ve hooked the flat panels up to. I could have gone with some brand that did advertise Mac support, but I didn’t. This must be my payback. Grin.

Mac OS X keychain and password storage

I’ve found that trying to explain the Mac OS X keychain at all tends to make peoples’ eyes glaze over. The keychain is poorly-understood overall, perhaps because it tries to bridge the gap between security and convenience.

A few thoughts:

  • A keychain has its own password which may or may not be set to the same as the password for the login account.
  • The keychain password is completely independent of the login account’s password, even if it is the same text as the login account’s password. They can be changed independently. When they are the same, they are just two passwords which happen to be the same.
  • User keychains are created within user home directories, and are protected by file system permissions while they are enforced.
  • Keychain keys are further protected by 3DES encryption. Directory or metadata information is in cleartext.
  • The Mac OS X Setup Assistant and Accounts System Preferences both create accounts whose login passwords and keychain passwords will match.
  • If the password is shared between the login account and that account’s default keychain, the keychain will be unlocked during the login process. This is the default for accounts created by the Mac OS X Setup Assistant and Accounts System Preferences.
  • If the default keychain’s password does not match the login account’s password, the keychain will not be unlocked automatically during the login process. The user may be prompted to unlock it, using the keychain password, if other applications require a key stored within.
  • The only time that a password change for a login account changes that user’s default keychain password is when the login account is logged in and changes its own password through Accounts System Preferences.
  • If the computer is bound to a directory service, a login account may be tied to that. However, the keychain is not. Changing the login account’s password through a directory service does not reset the keychain’s password. The keychain’s existing password will remain until or unless it is changed.
  • A third-party software utility, Keychain Minder, can help to keep login and keychain passwords in sync, if desired. This may be especially helpful in a directory service environment, where you are more likely to change account passwords externally rather through Mac OS X’s built-in means. It also provides an opt-out capability for those who specifically want different login and keychain passwords.
  • If the computer is bound to a directory service, and a directory service-based login account was compromised, there is a chance that the password for the default keychain in that account is also compromised. Changing the password for the login account in the directory service will protect the login account. However, that will not necessarily protect the keychain stored within the account’s home directory on disk. Whether or not the keychain password was the same as the former password for the login account — the keychain’s password should probably also be changed.
  • The long-term use of the same password for a keychain can be a risk; as it gets stale, it lessens the protection on each key in the keychain.
  • There is currently no policy enforcement mechanism, akin to pwpolicy, for keychain passwords.

Matching UIDs between Exchange meeting updates and existing events in Microsoft Entourage

When Microsoft Entourage is handling meeting updates or cancellations between Exchange users, it’s critically important that it can match up the change to an event with the correct event in a calendar. It does so by comparing unique identifiers between the update/cancellation message and existing calendar events. As nearly as I can tell, this depends on having the right UID, which makes sense.

How can you see the UID for events? Drag the event from an Entourage calendar to a Finder window, where it will become an vCalendar/iCalendar-formatted file with a .ics extension. You can open the resulting file in a text editor. Here’s a sample from a repeating event in an Entourage calendar, with the UID highlighted:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Microsoft Corporation//Entourage Mac 11.0 MIMEDIR//EN
METHOD:PUBLISH
BEGIN:VTIMEZONE
TZID:GMT -0500 (Standard) / GMT -0400 (Daylight)
X-ENTOURAGE-CFTIMEZONE:
BEGIN:STANDARD
TZNAME:Standard Time
RRULE:FREQ=YEARLY;WKST=MO;INTERVAL=1;BYMONTH=11;BYDAY=1SU
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
DTSTART:16010101T020000
END:STANDARD
BEGIN:DAYLIGHT
RRULE:FREQ=YEARLY;WKST=MO;INTERVAL=1;BYMONTH=3;BYDAY=2SU
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
DTSTART:16010101T020000
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
UID:040000008200E00074C5B7101A82E00800000000D0172C9CE47DC801000000000000000010000000C131C7E01B61184999585795596672E0
X-ENTOURAGE_UUID:37656211-A6ED-4E37-99D2-BAD25704AF63
DTSTAMP:20080304T160415Z
DTSTART;TZID="GMT -0500 (Standard) / GMT -0400 (Daylight)":20080305T100000
DTEND;TZID="GMT -0500 (Standard) / GMT -0400 (Daylight)":20080305T113000
LAST-MODIFIED:20080304T160433Z
… [snip]

Similarly, you can drag and drop a proposed change to an event, which is received by e-mail (because Exchange is largely an e-mail based — rather than a network-based — calendar system), to the Finder. This will save the message as the text source of the e-mail, with a .eml extension. That file can also be opened in a text editor, where you can see that it contains a siginficant amount of vCalendar/iCalendar data. Here’s a sample cancellation notice that removes one instance of the recurring event from Entourage — for reference, not the instance shown above, but another one from the same sequence:

BEGIN:VCALENDAR
METHOD:CANCEL
PRODID:Microsoft CDO for Microsoft Exchange
VERSION:2.0
BEGIN:VTIMEZONE
TZID:GMT -0500 (Standard) / GMT -0400 (Daylight)
BEGIN:STANDARD
DTSTART:16010101T020000
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
RRULE:FREQ=YEARLY;WKST=MO;INTERVAL=1;BYMONTH=11;BYDAY=1SU
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
RRULE:FREQ=YEARLY;WKST=MO;INTERVAL=1;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
X-MICROSOFT-CDO-MODPROPS;X-MODPARAM=1:attendee,BEGIN,class,created,description,dtend,dtstamp,dtstart,duration,END,last-modified,location,organizer,priority,recurrence-id,sequence,status,summary,transp,uid,x-microsoft-cdo-alldayevent,x-microsoft-cdo-apptsequence,x-microsoft-cdo-attendee-critical-change,x-microsoft-cdo-busystatus,x-microsoft-cdo-importance,x-microsoft-cdo-insttype,x-microsoft-cdo-intendedstatus,x-microsoft-cdo-owner-critical-change,x-microsoft-cdo-ownerapptid
DTSTAMP:20080422T194244Z
DTSTART;TZID="GMT -0500 (Standard) / GMT -0400 (Daylight)":20080423T100000
UID:040000008200E00074C5B7101A82E00800000000D0172C9CE47DC801000000000000000010000000C131C7E01B61184999585795596672E0
… [snip]

Note that the UID properties match.

If you are troubleshooting a problem where an update/cancellation does not modify an existing event as you expected, and the UIDs of each do not match, that’s probably the source of your problem. Of course, you still have to figure out why the UIDs don’t match. In that case, I’d personally start by looking at what other synchronization software (besides Entourage’s own Exchange sync) and devices are involved.

Packages consulted by Repair Permissions in Jaguar, Panther, Tiger, and Leopard

Here is a table that combines each unique package whose Bill of Materials (BOM) file is checked by Disk Utility when running the Repair Permissions routine. These package names are directly listed in the DiskManagementTool (Panther, Tiger, and Leopard) or DiskFirstAid (Jaguar) executables.

For context, please see the previous post, Will the real Repair Permissions please stand up?

Jaguar Panther Tiger (Intel) Leopard
AdditionalApplications.pkg x
AdditionalAsianFonts.pkg x x x x
AdditionalEssentials.pkg x x
AdditionalFonts.pkg x x x x
AddressBook.pkg x x
ApplicationsServer.pkg x x
Automator.pkg x x
BaseSystem.pkg x x x x
BrotherPrinterDrivers.pkg x x
BSD.pkg x x x x
BSDSDK.pkg x x x x
CanonPrinterDrivers.pkg x x x
CommonAccessCard.pkg x x
CommonCriteriaTools.pkg x x
DevDocumentation.pkg x x x x
DeveloperTools.pkg x x x
DevExamples.pkg x x x x
DevFatLibraries.pkg x x x
DevInternal.pkg x x x
DevPBWO.pkg x
DevSDK.pkg x x x x
DevTools.pkg x
ElectronicsForImagingPrinterDrivers.pkg x x
EpsonPrinterDrivers.pkg x x x
Essentials.pkg x x x x
FatLibraries.pkg x x x
GimpPrintPrinterDrivers.pkg x x x
HewlettPackardPrinterDrivers.pkg x x x
iCal.pkg x x x
iChat.pkg x x
iMovie.pkg x
Internal.pkg x x x
iPhoto.pkg x
iSync.pkg x
iTunes.pkg x x x
iTunesX.pkg x
Java.pkg x x
LexmarkPrinterDrivers.pkg x x x
Mail.pkg x x
MicrosoftIE.pkg x x x
MigrationAssistant.pkg x x
OxfordDictionaries.pkg x x
QuickTimeStreamingServer.pkg x x x
RicohPrinterDrivers.pkg x x
Safari.pkg x x
ServerAdministrationSoftware.pkg x
ServerAdminTools.pkg x x
ServerEssentials.pkg x x x
ServerFatLibraries.pkg x x x
ServerInternal.pkg x x x
ServerSetup.pkg x x
X11SDK.pkg x x x
X11User.pkg x x x
XeroxPrinterDrivers.pkg x x

I would say that “many Bothans died for this,” but that would be overly sensational. I’m not sure anyone really cares besides me, so its importance in the galactic scheme of things is in doubt. Beyond that, it just took some time to generate each of the OS installs via InstaDMG; with the resulting disk images mounted, I just had to read files with the strings utility. I didn’t even need to boot them. (Although, had I wanted to boot them, realize that Mac system administrators, like their developer friends, vote for virtualization.)

How did I construct this table? I got the raw data from the techniques described in the previous post. I saved the results into one text file for each OS version.

I ran the following command to get the the unique package names from my four text files:

$ cat 10200.txt 10390.txt 10411.txt 10500.txt | sed s/^\\/Library\\/Receipts\\/// | sed s/\\/Contents\\/Archive.bom\$// | sort -f | uniq

The sed commands strip the beginning and end of each line, which contain the beginning and ending of the full package and BOM path. This is the same for each package, and thus becomes superfluous in the table. The sort command sorts the results, ignoring case (-f). Then, uniq can determine which of the packages names are unique from the sorted data.

I pasted the output into Excel as a column. I then ran another set of commands to grep for the package names in the original files, reviewed that output, and put an “x” in the right columns to denote which OS versions had which packages listed. Sorry, I didn’t write a script for that and the commands to get there were just:

$ for pkg_file in `cat uniq.txt`; do grep "$pkg_file" *.txt ; done | pbcopy

The output from Excel was then used to make the table.

Will the real Repair Permissions please stand up?

There has long been confusion and misunderstanding about what, exactly, the Repair Permissions routine in Apple’s Disk Utility does. What started as the Repair Privileges Utility available separately for Mac OS X 10.1 has become the subject of some controversy in the intervening years.

“What permissions does it verify?” and “Where does it get the list of permissions it uses for comparison?” are two reasonable questions users and system administrators alike may ask in order to understand the software better.

The answers to these questions are important to prevent the abuse of this technique in the no man’s land of troubleshooting voodoo. Repair Permissions has been debated ad nauseum, but confusion blankets it like a thick fog. It’s 2008 and yet I still come across reasonable people who don’t know what the software is doing — sometimes even holding that it is doing something it isn’t.

Well, there is information available on this topic. First, let’s look at Apple+Knowledgebase+Article+25751:+About+Disk+Utility’s+Repair+Disk+Permissions+feature. As of this writing, it states:

“When you use Disk Utility to verify or repair disk permissions, it reviews each of the .bom files in /Library/Receipts/ and compares its list to the actual permissions on each file listed. If the permissions differ, Disk Utility reports the difference (and corrects them if you use the Repair feature).”

That seems to imply that all of the Bills of Material (BOM) files for every package listed in /Library/Receipts are reviewed. But, the article further explains:

“No [Disk Utility does not check permissions on all files]. Files that aren’t installed as part of an Apple-originated installer package are not listed in a receipt and therefore are not checked. For example, if you install an application using a non-Apple installer application, or by copying it from a disk image, network volume, or other disk instead of installing it via Installer, a receipt file isn’t created. This is expected. Some applications are designed to be installed in one of those ways.”

The article implies that every package receipt is reviewed, but only the files and directories listed in an Apple-originated package are checked and repaired by Repair Permissions. This is somewhat confusing, since it doesn’t really explain how third-party software installed through an Apple installer package relates to this — particularly in the case where a file or directory is in both an Apple-originated package as well as a third-party one.

Macworld magazine’s Dan Frakes tackles the situation with the article Repairing permissions: What you need to know. The author tracks what BOMs Disk Utility is using while running Repair Permissions via fs_usage. He also runs strings on the tool that repairs permissions to find what, if any, BOMs are listed in the executable. Based on that evidence, he finds that a limited subset of BOMs are consulted by Repair Permissions — and all of them are Apple-originated; none are from a third party.

This dovetails nicely with the discussion of Repair Permissions in Michael Bartosh’s excellent Essential Mac OS X Panther Server Administration book from O’Reilly. While Mike is no longer around to explain how he obtained the list of BOMs — printed on page 163 — that are referenced by Disk Utility, it is likely he used one or both of the methods above. It is also likely he had access to information from Apple itself before he made the assertion. He always seemed to prize strong evidence and a deep understanding of the software, and so I trust what he wrote.

Mac OS X Leopard’s online help probably has the best clarification of the matter from an official source. It is succinct and much better than the Apple KB article when it spells out:

“Disk Utility repairs the permissions for files installed by the Mac OS X Installer, Software Update, or an Apple software installer. It doesn’t repair permissions for your documents, your home folder, and third-party applications.

You can verify or repair permissions only on a volume with Mac OS X installed.

It’s best to start up your computer using a disk with the latest version of Mac OS X, including software updates. Software updates may change file permissions to improve security.”

I think we can consider this matter settled; Disk Utility’s Repair Permissions routine is limited in scope. It only uses BOM files from Apple installer packages, and the only packages examined by the utility are those from Apple. Third-party software, even when installed by an Apple package, is not in the mix. The repair process only needs to be run as a troubleshooting technique when you think there is a problem related to the permissions-on-disk.

If you want to do your own sleuthing, the MacWorld article shows how to run strings against the DiskManagementTool executable to find which BOMs it lists. Modifying that a bit to get the information from any given disk you’ve mounted, and placing the output on the pasteboard (so you can paste it in the editor of your choice), you get the following (substituting the volume’s path for “/path/to/startupvolume,” or just take that text out to use your current startup disk):

$ strings /path/to/startupvolume/System/Library/PrivateFrameworks/DiskManagement.framework/Versions/A/Resources/DiskManagementTool | grep Receipts | pbcopy

Neither the DiskManagement framework nor the DiskManagementTool exists in Jaguar, however. I found that the on-disk structure of Disk Utility changed between Jaguar and Panther. After discovering that change, I thought I’d need to substitute “Disk Utility.app/Contents/Resources/Disk Utility Agent” for DiskManagementTool, but even that executable didn’t contain any references to “Receipts.” Instead, I settled upon the following for Jaguar:

$ strings /path/to/jaguarvolume/Applications/Utilities/Disk\ Utility.app/Contents/Resources/DiskFirstAid.bundle/Contents/MacOS/DiskFirstAid | grep Receipts

The result is much shorter: only 12 BOMs are referenced in Jaguar, in contrast to 47 in Leopard. For more, see Packages consulted by Repair Permissions in Jaguar, Panther, Tiger, and Leopard.

Syndicate content