Sync sinks

Synchronizing information between two devices is a tough problem (and doing it among several is harder still). It’s not so bad when the devices are similar, and are using the same software — say, keeping browser bookmarks in sync between two instances of Internet Explorer on two Windows machines. Or keeping two Lotus Notes replicas in sync.

But when you’re trying to synchronize calendars and address books between your primary computer and a mobile device, such as the BlackBerry, well, it can be a nightmare, for a number of reasons.

I used to sync a Palm V with Lotus Notes. I switched to a BlackBerry 957 when they came out, and then to various other BlackBerry “smartphones”. When I left IBM and stopped using Lotus Notes, I started syncing my BlackBerry Curve with the MacOS applications (iCal, Address Book, and Stickies), using PocketMac SyncManager. And now, RIM has come out with its own BlackBerry Desktop Manager for MacOS, released early this month.

In one way or another — generally in multiple ways — they’ve all sucked had serious problems.

When you’re syncing data among instances of the same program, you mostly have to deal with conflicts — situations where a piece of data is

1. changed on multiple instances between synchronizations, or
2. present on one instance and absent on another (was it added or deleted), or
3. confused, in some way, so that the sync software is... well, out of sync.

But when you’re syncing unlike programs on unlike devices, it becomes lots more complicated: you also have to deal with different database formats, which don’t always store the same things. You have to deal with data attributes that don’t match, fields that don’t exist, fields with differing lengths, and so on.

I won’t go into all the details of the problems I had with the various Lotus Notes syncs; I’ll just say that they affected both the devices and Notes itself. I would frequently wind up with corrupted calendar entries in one place or the other, and sometimes both would be corrupted at the same time.

PocketMac would sometimes turn certain address-book entries into sorts of zobmies, entries on my BlackBerry that I could neither change nor delete, and which hid the correct entries with the same names. The only way I could find to get out of this was to clear the entire address-book database, and re-sync.

Nevertheless, it was with a great deal of optimism that I installed RIM’s new BlackBerry Desktop Manager. What was not to be optimistic about?: it was written by the Research In Motion folks themselves, and it syncs with the well known Mac applications, iCal, Address Book, and Mail Notes. Oughta work.

Um. Not so much.

First it has some very basic problems, just with the program, even without considering the synchronization. I start the program and configure it to sync the device calendar to MacOS iCal, setting up some of the options there (sync only certain of my Mac calendars, sync so many days in the past and so many in the future, ...). I configure it to sync the device address book to the Mac’s Address Book, and the device’s notes database to Mail Notes on the Mac. It retains those settings from sync to sync while the program is running, but when I quit the program and re-start it, the settings for the address book and calendar are lost, and I have to set them up again. Oddly, the settings for the notes are retained.

The settings for the notes are retained. But the first sync each time I start the program causes duplication of each one of my notes — Mac Mail and the BlackBerry each get two copies of every note. I have to delete the duplicates and sync again, to make everything OK.

I don’t have that problem with the calendar or the address book, but... both of those applications on the Mac have features that don’t exist on the BlackBerry. For example, the Mac Address Book has a check-box in each entry to say that this entry represents a company, and that changes the interpretation of the “name” field. That’s convenient, when I want to put in my car repair shop, or my fuel-oil company. So I have — well, had — a number of entries with that box checked. They synchronized without incident, the first time.

On the second sync, it turns out that because the BlackBerry doesn’t have a similar indicator, the software considers this to be a synchronization conflict, and asks me to resolve each one — that is, to tell it whether to use the record from the Mac or the record from the BlackBerry. If I tell it to use the one from the Mac, the conflict will simply recur on the next sync. Eventually, I told it to use the records from the BlackBerry. That lost the “company” indicator, but permanently got rid of the conflicts.

I shouldn’t have had to do that. With software that’s explicitly written with knowledge of the two applications, it should be programmed to remember what it did. It should store in the synchronization database the fact that something had to be adjusted to be copied to the device, and it should maintain that adjustment from sync to sync, not bothering me with the details, and not considering it a conflict.

Of course, this is release 1, and it’s possible — dare I say, likely — that they’ll fix these problems. So I can only wait. And, to tell the truth, this software is causing me fewer synchronization problems than I’ve had with any other.

Now, since I’m a standards geek, I’ll note that there’s an attempt at standardizing synchronization, through the SyncML standard developed by the Open Mobile Alliance. I’ve done some work with SyncML in my job, but it’s not a good answer for my BlackBerry issues. It’s meant to sync devices through a SyncML server. The only vendor that supports both the Mac and the BlackBerry is nexthaus (here and here), and they don’t sell a SyncML server, so I’d have to get that from someone else. Or I could use a free service, if I don’t mind having them store (and have access to) all my data. And, of course, there’s no guarantee that nexthaus has no bugs in their implementations either, nor that the servers aren’t also buggy.

Or I could use the open-source OpenSync framework to write my own server (and my own Mac and BlackBerry clients, for that matter)... if I had the time, and if I didn’t mind working through the inevitable bugs in my own code (at least I’d be able to fix them).

I’ll sit tight and see what RIM does. Its bugs are tolerable, and maybe that’s the best I can hope for, because data sync, like math class, is tough!