Halt! Who goes there?

OK, a couple of my colleagues in Internet standards work, Peter St Andre and Dave Cridland, have started on about challenge/response systems (C/R systems, for short). I worry about stepping into it, but I figure that if I can discuss politics and religion in these pages, I can probably take the next step and wade into that swamp. (Next thing you know, I might take on the most socially dangerous subject of all: which text editor is best.)

So first, here’s how a challenge/response system handles an incoming message:

1. Apply various mechanisms to determine whether the message should be accepted or rejected without question. Perhaps the sender is whitelisted (a known correspondent, say) or blacklisted, or the message passes or fails certain content checks.
2. If the message is accepted or rejected, do so, and stop now.
3. If we get to this step, “accept” the message but do not put it in the recipient’s inbox. Store it in a waiting area instead.
4. Send a “challenge” to the sender. This may simply be a message that says “Reply to this if you’re real.” It could also include a CAPTCHA or some other mechanism to try to make it hard on automated systems.
5. If a valid response to the challenge comes back in a reasonable time, deliver the message to the recipient’s inbox. Most systems also whitelist the sender at this point, or provide it as an option.
6. If no valid response is received within a certain time, delete the message.

Instead of rejecting or deleting, the messages can, of course, be put in a “junk” mailbox that the recipient may check.

Proponents often tout C/R systems as “the” solution to the spam problem. They claim no spam and no false positives. Of course, they’ve just defined false positives out of the picture — I would call the challenged messages “false positives”.

The thing is, some of us detest C/R systems passionately. Dave does. I do. Dave explains some of the reasons in his blog entry (linked above), but I’ll repeat them here and add others.

Spammers often send messages with phony sender addresses, often using the address of a real person. That real person gets all the challenges and has to deal with them. The challenges themselves become spam. Dave also explains the difficulty in filtering the bogus challenges, fearing loss of “legitimate” challenge messages.

Of course, challenges could be limited to messages that pass DKIM, SPF, or Sender-ID checks, and that would help with this problem. Only, that would require a decision about the uncertain messages that are left — are they deleted (that might toss real mail, false positives) or delivered (incomplete spam deletion)? And what about messages sent by zombie machines? Such messages can pass these checks and will get challenged, but the challenges will still go to random users who will wonder why they’re getting them. And might they not just reply to them, allowing the spam to be delivered?

Two users of challenge systems might just get into a standoff, where each sends the other a challenge and neither responds. (A worst case of this could be where two badly designed C/R systems actually get into a mail loop, sending challenge messages continuously until one is shut down.)

Spammers can game the system by sending spam that looks like challenge messages.

And there’s the question of whom the challenge is sent to in the first place. Internet email has a number of entities who can be involved. For instance, suppose an executive at a company wants to send a survey to her customers, asking them to do a technical evaluation of the product they recently bought. The exec’s administrative assistant actually sends the message, and replies are directed to the exec’s technical assistant. Delivery-failure notices (“bounces”) go to someone in the sales department for follow-up (to check on the customer and maybe update the contact address).

In mail-header terms, we have something like this:

Return-Path: <sales@example.com>
From: <executive@example.com>
Sender: <admin-asst@example.com>
Reply-To: <tech-asst@example.com>

To which of those do we send the challenge? In this case, any might work, but it’s easy to alter the scenario so that’s not true (perhaps there are contractors involved, who are actually running the survey). A human could probably make a reasonable guess. An algorithm can’t get it right.

A variation on that theme is the case where a C/R user goes to a web site and puts in a trouble report. Mail comes from the customer support system — perhaps an automated reply, or perhaps something from a human support rep — and gets challenged. The result might be that the user does not get the help he needs.

Vary that further, having the challenge annoy someone who’s been asked to do a favour. Helper annoyed, mail ignored, favour not done.

There are legitimate reasons for messages to come from automata — auto-responses for online orders, for example, or things like bank statements and travel itineraries. C/R systems require that the user manually whitelist the senders of these, but the user won’t always know the sending address in advance.

As Dave points out, one reason that challenge/response systems don’t cause more annoyance and damage than they do is that relatively few Internet-mail users use them. St Peter points out that recipients of the challenges don’t seem to mind, but I think that’s because they don’t get that many challenges... and because the ones who would complain might just ignore them and choose not to communicate with him. It simply doesn’t scale, and it if were more widely used it would likely collapse on itself.

Peter also likens the situation to one in the Instant Messaging world, where there’s often some sort of negotiation before people are added to each other’s buddy lists. I think it’s a faulty comparison: email and IM are different environments. They behave differently, and we expect different things from them. Also, the IM challenges are normally handled as instantly as the messages are sent... and yes, I do get annoyed when I want to send an IM to someone and I have to wait a non-trivial amount of time to get the recipient’s approval to do so (because, say, the recipient isn’t at the computer right then).

In the end, Peter might be right about this:

Unfortunately, email is a slum. People take extraordinary measures to keep using the medium despite its slumlike character. These days I think it’s better to use IM, blogs, and well-run discussion lists or web forums. Sad but true.

But whether or not that’s true, if one does use email, and isn’t willing to abandon the email system altogether, I don’t think challenge/response is a good way to try to keep the value in it.