Jaharmi’s Irreality renewable hyperlinks with 0% transfat since 1999

After using Acquia Drupal for a while, I took advantage of a trial subscription to the Acquia Network. The network’s services showed me that I had files present in my install that the agent could not account for.

I suspected this was happening because of the way I manage my Acquia Drupal installation with Mercurial. So, I’ve modified my previous process (and updated my instructions) to extract the downloaded tar archive with the --recursive-unlink option. This option appears to successfully remove the contents of every directory before putting new files back into them.

When the archive is extracted in this way, my repository’s working directory shows modified, unknown, and deleted files. This allows me to treat each category of files individually before I commit the changes for a Drupal update as a revision.

$ hg status

The modified files will be tracked normally because they’ve already been added to the Mercurial repository, so I don’t need to do anything special for them.

The unknown files are ones that are completely new, and have not appeared in the same position in a previous revision. They have yet to be tracked by Mercurial, so I have to add them to the repository. To add just those unknown files, then, I have to pick them out from the status listing:

$ hg status --unknown

In order to operate just on those files to add them to the repository, I run a for loop:

This changes the “?” status to “A,” because the files were successfully being tracked by Mercurial.
I use the “--no-status” flag on the “status” command so that just the file paths are printed; the actual status code is not, which is appropriate for the target of the “add” command in the loop.

I do the same basic steps with deleted files. These are files that were in the previous revisions but have been deleted by the --recursive-unlink option from the tar extraction and not replaced with the extraction of the new Acquia Drupal tar archive. If the deleted files had been replaced by the tar extraction, they would either be unchanged (which would not show up in the “status” output) or marked as modified.

To remove the files that are marked as deleted from the repository’s working directory:

However, that may be the same as simply using the following, which I have to explore further:

$ hg remove --after

So, to follow all of these changes in the repository, I run the loop for the uknown files and the loop for the deleted files. The modified files are already tracked, so I don’t need to do anything additional for them. After that, a “commit” will record all of the changes — modifications, additions, and deletions — in the repo.

These commands are based on my current understanding of Mercurial, and they do work for me right now. There could certainly be another better way to do this in one fell swoop — or at least fewer steps. I would welcome that, so if you’re aware of a way, feel free to comment or contact me.

Update: I found that the “hg addremove” command cleanly replaces all of the shell loops I mentioned above. Therefore, I recommend using it instead of the “for” loops I described.

Here is a sequence of commands and output that show how I keep the Acquia Drupal open source content management system up to date with Mercurial, the open source distributed version control system.

In the example below, my Mercurial repositories for Drupal are located in the “drupal” subdirectory of my “repo” folder. Once I’ve moved into that directory, I download the Acquia Drupal distribution with curl and then extract it into my previously-created Mercurial working directory, “acquia_drupal,” using tar.

(Update: I added the --recursive-unlink option after I noticed that the Acquia Network control panel keeps track of extra — possibly unneeded — files and folders you have in your install. The recursive unlink option seems to avoid having stray files from old versions of modules hanging around in your repository after you install updates.)

After extracting Acquia Drupal my Mercurial working directory, I get the status of the repository. It shows there are changes from the last version I checked in — and this includes new files, denoted by a “?” at the beginning of their line.

Since there are new files, I have to add them so they’ll be tracked by the repository. I only need to add in the parent directory for any changed files, and any new files within it will also be added for tracking.

Excellent; the new files have been added. After this, I just need to accommodate the deleted files that no longer need to be tracked (created when using the “--recursive-unlink” option on tar). For that, see my newer instructions.

Now that the right files are being tracked, I need to commit the changes — modified, added, and deleted files — to the repository. This will create a new revision in the repository’s history, which I’ll tag with the text “Acquia Drupal 1.2.0.”

Once this revision is checked in, I can use it to propagate changes to other repositories. I keep the main Acquia Drupal distribution in its own repository, and then use the “hg fetch” command to pull its changes into one where I track contributed modules. That second repository is then pulled into a third repository which stores just the changes for my production Web site. The use of three repositories in this way modularizes and isolates the updates.

While trying to troubleshoot what I’d done to mess up the Mercurial repositories managing my Drupal installations last weekend, I really would have liked a way to see what files had changes in specific revisions. Each revision to a Mercurial repository affects some files, of course, but it seems awfully hard to figure what files changed in that check-in.

I have since found a way to do that by customizing the output of Mercurial. To customize output, you can create templates on the command line (with --template) or for more powerful reformatting, create an output style file.

I struggled for a while to figure out how to use style files, and eventually came up with something that works for me so far.

Since I’ve installed Mercurial from Lee Cantey’s standard binary package for Mac OS X Leopard, I created the file “map-cmdline.changedfiles” at the “/Library/Python/2.5/site-packages/mercurial/templates” path. (Where you put the file may vary depending on where Mercurial is installed, and I’m sorry but I don’t know where it gets installed on other systems.) The contents of “map-cmdline.changedfiles” are below, along with my possibly inept description of what each line is doing:

# Get all of the files in the selected revision
# and stringify them, whatever that means
# but do not 'tabindent' or wrap them to 68/76 columns
# Without first setting changeset to the list of files
# you won't get output from subsequent lines
changeset = '{files|stringify}'
# List modified files, one per line
# preceded by M to mimic `hg status`
file = 'M {file}\n'
last_file = 'M {file}\n'
# List added files, one per line
# preceded by A to mimic `hg status`
file_add = 'A {file_add}\n'
last_file_add = 'A {file_add}\n'
# List deleted files, one per line
# preceded by ! to mimic `hg status`
file_del = '! {file_del}\n'
last_file_del = '! {file_del}\n'

I don’t know why the “map-cmdline.” portion of the filename is there, but as long as I have it, I can call the style file from the command line with what follows the period. So, I can call the style with “--style changedfiles” — and that tiny bit of voodoo seems reasonable enough to me. (The other styles in the directory above, many of which end in “.tmpl” extensions, seem related to the Mercurial Web server, hgweb. I tried, but I couldn’t use their names at the command line, with or without their extensions. Plus, their contents looked HTML-ish.)

With the “map-cmdline.changedfiles” style file saved in that location, I can call Mercurial’s “log” command:

$ hg log --style changedfiles -r tip

… which gives me a list of the files changed in the “tip” (or latest revision) of the repository. I could substitute in any revision identifier for “tip.”

I haven’t actually seen the “file_add” and “file_del” keywords in action; every time I’ve used this style file in the manner described, I’ve only seen files marked as “M” — even if I’m looking at a revision where new files were first checked into the repo. I’m confused by that, but I’m not going to let it sour my day at this point.

There might have been an easier way to do this but I didn’t find one last weekend. It took me some time to figure even this bit out, and I hope writing this post saves someone new to Mercurial from future frustration.

In my Mercurial-based workflow for updating Drupal sites, there is a sequence of commands I need whenever a new version of Drupal comes out. I have a hard time remembering the options for “tar” in this sequence — and my original source for the instructions differs from what I need to do on my Web host — so I need to help my memory. The tar command, as constructed below, places its output into the specified destination directory.

Here it is, with tar’s “--strip-path=1” and “-C” options:

$ cd path/to/repository/parent/directory
$ curl -O http://ftp.drupal.org/files/projects/drupal-5.12.tar.gz
$ tar --strip-path=1 -C drupal_source -zxv -f drupal-5.12.tar.gz

I’ve been using zsh for a while as my preferred shell. I have a hacked-together zshrc file, and yet really wanted to use it across multiple systems. Some of those systems are running Mac OS X, others Solaris, and still others Linux. Executables are in a different locations and even have different switches across this range of systems, so my cobbled zshrc was not helping me.

As I was about to fall asleep last night, it finally hit me that fixing my zshrc would be a good thing to do. I jotted down some notes about an idea to reorganize it, and did something about it today.

Of course, since I’ve checked my zshrc and other dotfiles into a Mercurial repository, I could experiment without fear.

I created three top-level functions, with one “case” statement in each. Case statements may be evil in some fashion, but they are one of the things I like about shell scripting. These statements do allow the script to make choices based on the host, operating system, or shell that it was running in. (Yeah, it’s a zshrc, but I sometimes do stupid things — like sourcing it in bash on the one Linux system that won’t let me switch to zsh. Site5, I’m looking at you.)

I separated all the important sections of my zshrc into their own individual function calls. Each of those function calls was placed into one of the applicable case statements.

The case statement functions figure out the conditions the zshrc is running in, and then run the other functions to set up my environment.

The changes tested well from first try across the various platforms and hosts I log in to. I did have a minor problem with the `hostname` command, because Solaris doesn’t have a “-s” flag for it. Eventually, I solved that — and the odd “uname: error in setting name: Not owner” error I got, even though I wasn’t directly running `uname` there — by replacing `hostname` with `uname`.

Thankfully, it works for me, and it should be a little easier to manage changes in the future.

When I come across software I might need to add into Mac OS X that requires compilation, I typically want to produce one Universal Binary. Make it a four-way UB and you get both 32- and 64-bit support.

A single binary is ideal for a Radmind transcript (or other package, if you wanted to bundle it into an installer) that can be deployed on both PowerPC and Intel Macs on Leopard.

Since rsync 3.0.2 with some patches is apparently working quite well at preserving Mac OS X data and metadata — passing the Backup Bouncer tests — I thought I'd try my hand at a four-way Universal Binary.

What worked for me, using a Mac Pro 4x2.8 GHz with Mac OS X 10.5.2 and Xcode 3.0, was to start with Mike Bombich's instructions and modify them with some fairly standard Universal Binary build steps. The configure and compile were both less than a minute on this system.

$ ./prepare-source
$ env CFLAGS="-O -g -isysroot /Developer/SDKs/MacOSX10.5.sdk -arch i386 -arch ppc -arch ppc64 -arch x86_64" \
LDFLAGS="-arch i386 -arch ppc -arch ppc64 -arch x86_64" \
./configure --prefix=/usr/local --disable-dependency-tracking
$ make
$ sudo make install

I have seen the use of "-Wl,-syslibroot,/Developer/SDKs/MacOSX10.5.sdk," in the LDFLAGS environment variable when compiling some applications but this did not work for me with rsync; when I removed it, rsync 3.0.2 configured successfully for me.

The result of the above build process appears to be a full four-way UB:

$ lipo -info /usr/local/bin/rsync
Architectures in the fat file: /usr/local/bin/rsync are: i386 ppc7400 ppc64 x86_64

A local transfer on the build system appears to have worked correctly. I did not test with Backup Bouncer, sync with a non-Mac system, or when shuttling data between architectures. So, accept these results with a grain of salt; I’m just happy I got rsync to compile for now.

The Mercurial fetch extension adds the hg fetch command. I’ve heard hg fetch described as “a great Do What I Mean kind of command.” It isn’t enabled by default, but it saves enough steps in the frequently-used push/pull-merge-commit cycle so many people turn it on when they set up Mercurial.

It seems as if the fetch extension is an open secret in the Hg community, but it took me some time to come across it so I thought I’d help raise its visibility. At least when I’m Googling my own site to remember something I’ve forgotten later.