The best feature of Firefox 3

When Firefox 3.0 was released on 17 June, they set it up with a plan to break the record for downloads in a single 24-hour period, and they succeeded:

Thanks to the support of the always amazing Mozilla community, we now hold a Guinness World Record for the most software downloaded in 24 hours. From 18:16 UTC on June 17, 2008 to 18:16 UTC on June 18, 2008, 8,002,530 people downloaded Firefox 3 and are now enjoying a safer, smarter and better Web.

Some of the new features touted for version 3 include better performance; a number of security improvements, including “web site ID”; an improved session manager that’ll automatically restore the tabs you had open in a previous session (this was available before, through extensions); and a convenient “zoom” feature that remembers that you’ve zoomed in or out on a particular site (very handy when a site thinks you’ll appreciate their use of a smaller font).

But here’s my favourite new feature: the smart location bar.

It’s not just an autocompletion thing, like every browser’s had for years. They just match the text you type to URLs. There’s way more than that here. You type something, and it searches for it in your browser history and your bookmarks, matching titles as well as URLs, and also matching user-defined tags. It’s really great! Things that I had to sift through the browser history for, or do fresh Google searches for, now just pop right up.

Suppose I wanted to get back to the article that I read the other day about a judge’s decision about guns at the Atlanta airport, but I hadn’t bookmarked it (and suppose I had’t put it in these pages, either). I can just type something like “airport guns” in the address bar. Or “atlanta airport”, or “atlanta guns”, or whatever... and a short list of things immediately appears under the address bar, one of which is the link to the article I’m looking for. I can’t tell you how much time this feature has saved me, in the eight weeks it’s been available.

If you’re not using Firefox 3 yet, that one feature is reason enough to switch to it!

IETF 72, Dublin

Well, Dublin-ish. The IETF meeting was actually at Citywest, 30 minutes outside of Dublin. I guess that was great for the golfers, but I missed being in the city, and having the option of walking out and getting lunch and dinner. They provided dinner buses into Dublin, but that’s not the same as just walking down the street a block or three. It seems that Trinity College Dublin would have been a fine place to have the conference, and I’m sure there are other facilities that’d have worked as well.

Still, it was a good meeting. The facility we did have was somewhat expensive, but otherwise fine, and, of course, the meetings were productive and useful, as always. It was another full week (I had about two hours of free time the whole week, an hour or so each on Wednesday and Thursday), and managed to schedule the rest of the time, including meals, for work.

It’s too bad, someone pointed out, that it was the 72nd meeting, and not the 33rd — if it were the latter, we’d have heard the local presentations welcome us to the “torty-tord IETF”. As it was, we heard that there were almost torteen hundred attendees (or, one tousand tree hundred), so that was nice.

I’m going to keep the meeting summary on the short side this time, with just a few highlights, but it’ll still be a long post. If you want to read about the vacation days after the meeting, and see photos, look for my other post, to come tomorrow. If you’re really interested in this, click here to see the meeting summary.

apparea — Applications Area general session

Among the general announcements were that NFSv4 will probably be done in the Applications Area, and that there’d be a discussion of email standards status on Tuesday (of which I have a summary below). There was a presentation of two follow-ups from the Applications Area workshop in February — netappvlan and netappmodel — but with no ensuing discussion.

There was a brief discussion about FTP extensions. The points are that, while FTP seems quite mature, and not terribly ripe for extensions, it’s still big business and there are five documents that are out as Internet drafts, looking for standardization. Should these go through as individual submissions? Should we form a working group?

It looks like we’ll form a working group, and John Klensin and I have agreed to co-chair it. We expect it to be a short-lived and non-contentious group, and John and I agree that standards going through the group will require some real support behind them, and existing implementations as evidence that they’re real and worth moving forward.

dkim — Domain Keys Identified Mail

We had a quiet session at this meeting, and my co-chair, Stephen, wrote this succinct summary, which I’ll use here:

DKIM met Monday afternoon, 62 people attended.

The main goal of the meeting was to get the ADSP and Overview documents to the start of working group last call. Each had a couple of open issues at the start of the meeting that were discussed and resolved (modulo confirmation on the mailing list). The result was that a new revision of ADSP will be published this week. Working group last call on both documents will start at that point.

The deployment guide was briefly presented, and work on that continues.

A couple of proposals were made for new work that might require re-chartering. There was some, but not overwhelming, interest in pursuing these, but the group won’t have that discussion until after the two current documents are in the hands of the IESG.

idnabis — Internationalized Domain Names update

We had the usual minor issues discussion, but the most significant by far, taking most of the time in the two sessions, was in the “BiDi” (bi-directional text) document: how to deal with left-to-right text, such as numbers, dots, slashes, and the like, in otherwise right-to-left text, such as Hebrew and Arabic. We had a demonstration of what happens when such text is entered, and it’s not pretty. As someone typed “abc2.[Arabic-text].3def.com”, we watched the “.3” get turned around and moved, so it incorrectly appeared to be attached to the Arabic text. To be sure, this is a user-agent problem — the correct characters will be sent on the wire in the correct order — but it’s a very hard one to solve. There’s no resolution yet, but we have more information, and, as we say, “Our best people are working on it.”

We also had a discussion, which included at least one native speaker of Korean, about a request that we add another set of Korean characters to the list of characters forbidden in domain names.

eai — Email Address Internationalization

We had some discussion of EAI issues during a working dinner on Sunday, and came up with a recommended answer to the question of what to do about List-* header fields. I presented that answer in Monday’s EAI session: for List-ID, note that it is meant to be a machine-readable string, not a human-readable one, and should not be in UTF-8; for others, internationalize them by including multiple URIs, and downgrade by removing the UTF-8 version, leaving the non-UTF-8 versions for use after downgrade.

POP and IMAP are almost done, but there’s an issue about POP and downgrade — the document currently has uncertain language about how to downgrade a UTF-8 message if the client isn’t using UTF-8. In the event that an old client is being used to download the message, but that a newer client (perhaps after a software update) is later used against the same local message store, we agreed, after discussion, that it would be good to have a standard downgrade mechanism used, so there’s some hope that the new client can recover the original message. There was agreement to require the use of the standard EAI downgrade.

We decided to wave our hands around the issue of handling mailto URIs. It will eventually require its own document (a UTF-8-enabled mailto), and for now we’ll point out that mailto does not handle UTF-8 addresses, and that something to deal with that is forthcoming.

There was a report on some very successful experiments with the current EAI specs (including downgrade experiments). Many thanks to the organizations that participated in those. Notwithstanding that, it’s clear that the aggressive schedule that was suggested at IETF 71 isn’t realistic, and we’re setting up a new schedule for re-chartering to put the EAI specs on the standards track.

httpbis — HTTP update

There was a long issue list to get through here, and some issues have been harder than the chairs and authors thought they would be to get consensus on. I’m just including some brief notes here.

The next version of the drafts will include a new registry for messages, to parallel the one for status codes.

The formal grammar is being converted to ABNF, and that’s a lot of work to get right. There’s a particular trickiness with cleaning up LWSP, removing it from where it doesn’t belong, while being sure to leave it where it’s appropriate.

Internationalization is, as always, a sticky issue. Despite standard headers, HTTP implementations have done their own things with respect to internationalization, and it’s making it difficult to decide how to go here. There are questions about support of RFC 2047 encoding, internationalization of filenames in content-disposition headers, and such. There was quite some discussion on this.

There was a discussion of issues with the use of the message header registry in HTTP. The registry is for other uses and is shared by HTTP, and there’s concern about having information in the right place, making sure that implementors read everything necessary to get the implementation right, etc.

For more on internationalization, we came to the question of what character sets to allow in new header fields. The decision was to say that new header fields MUST only use US-ASCII, but that implementations SHOULD accept ISO-8859-1 (including UTF-8 as well, at this point, did not get acceptance).

We finished with a presentation and discussion about security requirements for HTTP, an item specifically out of scope for this first round through the working group, but something that might be added to a re-charter.

morg — Message Organization standards

The MORG BOF’s goal was to review a series of individual Internet drafts and ideas that all relate to searching, sorting, and organizing email messages using IMAP, and to consider forming a working group to bring these documents through the standards process.

The BOF chairs came in with a chunky set of drafts, most of them fairly immature, some of them with significant IMAP-community support already and some that are new items. We went through the drafts, got explanations of each, and discussed them — some in considerable detail.

As a result of the discussion, it seems clear that there are enough document authors, reviewers, and server implementors to get through this lot, and enough energy to easily support a working group. There was some concern, though, first brought up by John Klensin, that this could be another set of IMAP extensions that bloats IMAP, and that really isn’t in significant demand. The problem is that we don’t have much participation by client implementors.

Chris Newman supports the formation of a working group here, and I agree. Chris would like to see the chairs manage the work by assessing the implementation commitments, including commitments from client developers, and dropping items that don’t have commitments to implementation in clients. I also agree with that idea.

The “bloat” discussion led to a suggestion to consider “IMAP5”, which would start by removing features to create a minimal protocol with necessary function, making something that’s easier to implement and get right. Randy Gellens has already started a mailing list for that, and there’s been some initial discussion.

asrg — AntiSpam Research Group

There are two primary documents coming out of this research group so far, both dealing with DNS block-lists. The first, which describes how they work, has been reworked to go on the standards track (as an individual submission, since the IRTF can’t publish standards), rather than being an informational document. The second, recommending best practices in operating and managing a block list, still needs work, and will probably go out as a BCP.

We had some discussion about whether another document should be adopted by the ASRG — the decision was not to adopt it — and that discussion led into some talk about how to focus the ASRG on new research. I will discuss the ASRG at the CEAS conference in late August, and try to get more participation from the academic community into the ASRG (possibly recruiting an ASRG co-chair from that community).

vcarddav — VCard update and CardDAV

We had the usual document status update, and went over some open issues. One that seems simple, but that merited some discussion was that dates must currently be complete (day, month, year), but there are times when one doesn’t want them to be, or doesn’t know the full date (for birthdays, for example, one might just want day and month, but no year). We decided to use existing date syntax from the comparators RFC and adapt it for VCard.

The other significant discussion was on how to handle synchronization issues, especially with collision-detection in situations with multiple copies of the information (one on a host server, one on a laptop, and one on each of two or three mobile devices, for example). Included here is the discussion of unique IDs — whether to have per-property UIDs, whether to require persistence of UIDs, how to handle the situation where a particular device or data store can’t keep the UIDs persistently, and so on.

We also had a small discussion about internationalization questions, in light of EAI. I have a suspicion that we’ll encounter effects from EAI that we haven’t yet anticipated. In the end, I don’t think they’ll be serious, but we’ll have to deal with them.

sieve — Sieve mail filtering language

This was an uneventful session for a working group that’s wrapping up its old work and preparing to re-charter to add some new work — work that is itself close to being finished, actually. We went over document status, and there’s nothing notable to say about that. The most difficult document, which this working group is picking up from lemonade, was discussed there (see below).

We talked a little about possible issues that Sieve will have with EAI — most obviously, that it will likely make scripts more complex and ugly, as a script will have to handle multiple versions of the same email address. We expect to have a sieve-eai document in the new charter.

lemonade — Mail extensions for diverse service environments

We still had a full session of final discussion, despite having thought that we wouldn’t meet here at IETF 72, but this group is wrapping up. We made some decisions on the remaining issues, and agreed to move the last document (imap-sieve) to the Sieve working group.

We finished with an incomplete discussion of imap-sieve (we ran out of time), and we’ll continue that discussion on the Sieve mailing list. Ned Freed had brought up a series of thoughts about the extension, in an email message in May, and I’ve summarized that into eight main issues.

Email standards discussion

After our experience with getting RFC 2821 and RFC 2822 updated and through the process of progressing to Draft Standard, a few people thought it would be good to consider setting up a working group chartered to progress those documents to [full] Standard, and to move other Proposed Standard documents to Draft Standard. This dovetails with my idea of creating a long-standing working group for email standards maintenance, a topic that the IAB and IESG discussed in our joint meeting on Sunday.

We have consensus, energy from participants, and IESG support to create such a working group, and we now have a mailing list to discuss a proposed charter. I suggest that we do a three-stage approach:

• In the first stage, set out a list of Draft Standards to progress to Standard, and then re-charter for the next stage.
• In the second stage, set out a list of Proposed Standards to progress to Draft Standard, and then re-charter for the next stage.
• In the third stage, set out a list of work that needs to be done that might result in new Proposed Standards for email.
• Iterate on those three stages. When there’s no immediate work to do, go dormant and periodically review the situation, waking up and picking up work when there’s work to pick up.

Technical plenary

The technical topic at the plenary this time was operational experience with IPv6. We had a panel of five speakers, who talked about their experience with implementing and operating IPv6, and answered questions from the moderator and the attendees.

Internet Architecture Board

In addition to setting up and running the technical plenary panel, the IAB reported on its work as follows:

• Informational documents:
  • We published RFC 5218, “What Makes for a Successful Protocol”, to help provide guidance for protocol designers.
  • We have “Design Choices When Expanding DNS” almost ready for final submission, giving architectural guidance use those who would use DNS for new purposes.
  • Work is wrapping up on “Principles of Internet Host Configuration”, advising on architectural issues involved in configuring Internet hosts.
  • We have a new document, “On RFC Streams Headers and Boilerplates”, dealing with an administrative issue.
• At our IAB meeting in May, we began work on several Internet architectural projects, and two of them have made some progress since:
  • Evolution of the IP Model, looking at architectural problems caused by layer violations as assumptions leak into higher-layer protocols and applications about how the IP layer works.
  • Peer-to-Peer Architecture, covering architectural questions involved in peer-to-peer protocols and applications
• We have set up a process for assigning liaison shepherds from the IAB, to better manage the liaison relationships between the IETF and other organizations, such as ITU-T, OMA, and W3C.
• We set up some new liaison relationships and other inter-organizational work.
• We have proposed a new organization of the RFC Editor job, splitting it into four functional roles that give the flexibility of contracting and managing different roles separately.

When rules develop lives of their own

Many years ago, I worked on a project that involved developing a computer environment wherein one typed instructions to the computer — this was in the days before graphical interfaces, before widnows and mice and clicking and all that. As it happened, someone had already written a program that would add a desk-calculator function to this sort of environment. A calculator function wasn’t needed, but, well, since it came “for free” (the code was already written), we decided to put it in. What that meant was that in addition to typing instructions about what data files to send where, and such, one could also type “132 / 12” and get the response “11”.

We designed the environment, wrote the code, tested and re-tested, and then gave it to the customer’s technical review team. They, among other things, reviewed all the error messages the program might present to the users, and they had a rule — a generally sensible one — that set out three requirements for each error message:

1. The message must be grammatical and make reasonable sense on its own.
2. There must be an explanation available (in reference material and in response to the user’s typing a question mark) that gives more detail about what happened and why it’s an error.
3. The message must tell the users what they can do about the situation. You can’t just tell someone there’s an error without giving some sort of remedy.

Now, that last one gets a bit tricky. As I said, this is generally a sensible rule... but it’s not one that can be applied slavishly; there are some things for which there simply is no remedy.

And, yet, as you’ve probably guessed, the rule was applied slavishly. In all cases where our error messages didn’t make a remedy clear, they came back to us and demanded that we correct that. We did, for some of them. For some, we hadn’t made things clear enough, and we could, indeed, reword the message and the explanation to tell the users what to do.

For all others, the technical review team simply added a short sentence to the end of the message, “Contact your system administrator,” as though that would be helpful to anyone.

And, so, back in the calculator function, if you typed “132 / 0”, you got this delightful result:

You cannot divide by zero. Contact your system administrator.

Liability for software failures?

Security maven Bruce Schneier suggests, in a column in The Guardian, that a solution to the problem of shoddy software that needs constant patching to fix bugs and security vulnerabilities is to hold vendors legally liable for software failures:

It doesn’t have to be this way. It is possible to write quality software. It is possible to sell software products that work properly, and don’t need to be constantly patched. The problem is that it’s expensive and time consuming. Software vendors won’t do it, of course, because the marketplace won’t reward it.

The key to fixing this is software liabilities. Computers are also the only mass-market consumer item where the vendors accept no liability for faults. The reason automobiles are so well designed is that manufacturers face liabilities if they screw up. A lack of software liability is effectively a vast government subsidy of the computer industry. It allows them to produce more products faster, with less concern about safety, security, and quality.

Mr Schneier particularly points out the problem with web browsers, the single most important type of application program for most users:

A recent study of Internet browsers worldwide discovered that over half — 52% — of Internet Explorer users weren’t using the current version of the software. For other browsers the numbers were better, but not much: 17% of Firefox users, 35% of Safari users, and 44% of Opera users were using an old version.

This is particularly important because browsers are an increasingly common vector for internet attacks, and old versions of browsers don’t have all their security patches up to date. They’re open to attack through vulnerabilities the vendors have already fixed.

Bruce is very often spot on in his analyses, but not this time.

I, too, have often lamented the poor quality of software. If our toasters turned out like our computer software, I often say, we wouldn’t tolerate the situation. Suppose that when you pushed down the lever on your toaster, the machine popped up perfect toast 95% of the time. And suppose that 3% of the time you got your toast burnt to a cinder, and 2% of the time the toaster never heated and never popped, and you had to unplug it, wait 10 seconds, and plug it back in before it would work. You’d never buy that brand again, of course, and if all brands did that you might never buy a toaster again.

And, worse, what if .003% of the time, the toaster burnt your house down? But, no, unlike software, toasters don’t have those sorts of problems.

So, yes, I agree that the quality of software is generally appalling, and I agree that it should be fixed. But, no, the marketplace won’t reward it. The fact still is that new features, not bug fixes, are what will sell the next version, and delaying the release of software until more bugs are fixed just allows the competitors to get in there first.

But the more basic issue is the business model we’ve settled into for a great deal of the software we use, including, notably, web browsers: it’s free.

Of the four browsers that Bruce mentions, Firefox and Opera are completely free, and Internet Explorer and Safari are arguably so, since they’re bundled with every system and freely downloadable, and they’re competing with free “products”. Where’s the financial incentive to spend a lot more money to build a higher-quality freebie? And mightn’t legal threats just lead to the abandonment of risky products that don’t bring in revenue?

Beyond that, who is it who would be held liable for, say, failures in Firefox? It’s an open-source project, with countless, random people contributing to it. It’s not entirely a willy-nilly thing, but it’s hardly tightly controlled. There’s no big corporation, no Microsoft or Apple equivalent, behind it.

It’s tough, in an environment where product is given out for free, to demand quality — the adage that “you get what you pay for” is in full effect. But demanding quality is the only way we have any hope to get it. There’s no way we can look to the legal system, at least not in general.

Now: Who’s willing to stop using web browsers until there’s one available that’s bug-free and guaranteed to have no security holes? Raise your hand.

[Barry looks around, his own hands in his pockets.]

Right, and how many of you are willing to pay, say, $200 per computer for a high-quality, bug-free web browser?

I didn’t think so.

Another look at “reply to all”

In the “Profession” column of the July 2008 issue of IEEE Computer magazine, Anne-Françoise Rutkowski and Michiel van Genuchten talk about “e-mail pollution” in an article titled “No More Reply-to-All” (abstract here; the full article is behind a paywall):

Measuring e-mail pollution

As more people flock to the Internet, external pollution — spam — has received considerable attention and become the target of spam filter countermeasures. We, however, focus on internal pollution. Our experience shows that it takes more time to detect internal pollution than external pollution. Deleting outrageous and misleading offers from parts unknown takes only a few seconds. Reading colleagues’ e-mail messages only to find at its end that you were copied only for their convenience can take minutes. The term e-mail pollution addresses such superfluous e-mail. Adding too many recipients typically causes this pollution, whether they appear on the send-to or cc list.

And, yes, that’s absolutely true. Anyone will tell you that your own eyes and brain can detect spam much more accurately than any automated spam filter can, and the delete key/button/icon is right there to hand. But when I get yet another email message in a discussion thread that has long since become irrelevant to me, but that still has a subject that warrants my attention, I have to read it, often sifting through a lot of forwarded text and other mess, in order to see that it’s really just someone thanking someone else for their input. Oh, great.

The authors go on to suggest a metric wherein someone who fans out more message recipients than what comes in is called a “polluter”, while someone who trims distribution lists relative to what comes in is a “cleaner”. It strikes me that in reality there’s a swath of “neutral“ in between the two, but the point is basically there: we want to encourage “cleaners” among our correspondents and in our businesses, and to discourage “polluters”.

They propose three “rules” to help:

1. No more reply to all. They recommend disabling the mail-program button that performs that function.
2. No more copies than originals. This isn’t explained well, but it seems like they mean to say that you should never add recipients to a reply without taking some others off.
3. No more e-mail fights. If you’re going to get into a battle, do it on the telephone, where the bandwidth is higher and the likelihood of misunderstanding is lower.

I’ve no disagreement with rule 3 — it fits perfectly into my rule that one should use the right communication medium for the task at hand. Rule 2, if I understand it, is very much related to rule 1. So I’ll address my comments to rule 1.

Disabling buttons and making absolute rules is a Very Bad Idea. While it’s certainly true that “reply to all” is abused quite often, there are good reasons to use it. See my extensive comments about that from about three months ago.

I propose that the real chance at solutions to what I’ll rather call “recipient pollution” lies in a different set of three points:

1. Technology: Provide a user interface that makes it easy to make the right choice of recipients, without trying to presuppose what that right choice is. This was what I said in April.
2. Company policy: Make it clear that management expects people to think about the distribution lists on their messages, and that employees who are consistently abusive in that regard will be spoken to.
3. Company culture / social pressure: Encourage an environment where people feel free to complain — politely, but firmly — to their colleagues when they see them stepping over the line. If the response to “Please stop copying me unnecessarily,” is usually “Too bad!”, then the problem will persist. On the other hand, if the typical response to persistent unnecessary copies is, “Hey, Bill, please make sure that when you copy me on a message, it’s legitimate, OK? Remember what our Director said last month,” people will be more likely to stay in line.

The problem is that we’re not doing any of that. Most email clients are no help at all, and corporate policy and culture generally lean toward longer distribution lists.

This isn’t really a technological problem, so let’s be careful about trying to solve it as one.

Technology and the Keystone Cops

Last year, a man was arrested in the New York City for sending abusive and harrassing email messages. Only, he didn’t, a fact that would have been obvious to anyone with any sophistication in things email. The police, though, have no such sophistication, and the man, William Hallowell, went through a great deal of hassle, embarrassment, and tarnished reputation before the charges were dropped. He is now suing the city and the police for their handling of the case.

But what began as an innocent exchange of e-mail messages, Mr. Hallowell said, quickly spiraled into an Internet nightmare, with the librarian mistakenly sending another message meant for him to someone with a similar name, the recipient replying with a crude, abusive response, and the blame falling on Mr. Hallowell.

He was arrested on a harassment charge, interrogated, held in custody for more than 30 hours, and became the subject of local news articles, causing enormous embarrassment, he said in a lawsuit filed on Wednesday in Federal District Court in Manhattan.

In the suit, which names New York City and several police officers as defendants, Mr. Hallowell, 24, says that the officers “deliberately and maliciously ignored a mountain of evidence” that proved that he did not send the offending message.

In an interview, he added that the officers did not even seem to understand how e-mail addresses work.

This is reminiscent of the case of Julie Amero, the substitute teacher in Connecticut who was convicted, and faced 40 years in prison, because police, prosecutors, judges, and jurors do not understand computer technology and the Internet. Ms Amero’s conviction was vacated after three postponements of her sentencing, but only after a huge outcry from people who knew better, and from months of publicity about it. And only after her career was ruined — do we really think anyone will hire her to teach school again?

And so with Mr Hallowell, though, thankfully, it ended before he was convicted of the charges. Still, people know about it, and some are sure to be suspicious. “Was there really something there?”, they will wonder. “After all, it was dropped for lack of evidence. That’s just one of those technicalities that get too many guilty people off.” Will anyone be willing to employ him in a school again?

But one of Mr. Hallowell’s lawyers, Ilann M. Maazel, said the case showed how easy it was for innocent e-mail users to be victimized.

“This could happen to anybody,” he said, “if the police are going to have absolutely no competence when it comes to understanding e-mail or the Internet.”

[...]

William Hallowell said that he vigorously denied sending the lurid e-mail message, and that he invited the officers to review the e-mail messages in his computer. He said that he also showed them the exchange of messages he had earlier had with the librarian.

It was clear, he said, that his account did not contain any with the address linked to the abusive sender.

OK, first, I have to say that since I don’t have any of the evidence, since I have not talked with anyone involved, since I have no information other than what’s in the Times article, I’m just speculating here. But it’s speculation based on an understanding of how poorly the police and the courts generally handle these sorts of things, and how little they understand technology.

The thing is, there’s nothing you can do to protect yourself, and that’s a major difference between this case and the one of Ms Amero. Ms Amero could have turned off the computer, for example, contrary to the instructions she’d been given. But Mr Hallowell’s situation cropped up entirely independent of Mr Hallowell — he wasn’t even there. His boss simply used the wrong email address, and didn’t know that she had... and it became a snowball rolling downhill after that.

The answer to this, in general, is that police departments and courts must have access to expert technological advice on an ongoing basis. It’s not sufficient for the courtroom to be a battleground between expert witnesses for each side — that’s too late, and there are too many axes to grind at that point. The police need help in considering the evidence while they’re investigating. Just as the police can now show a guy to the victim and say, “See, the man you saw was over six feet tall; he’s only 5 foot 8. This isn’t the man you saw,” they have to be able to do equivalent things with Internet-related evidence.

Of course, police officers can’t be expected to know enough to do that, in general, any more than they can be expected to have medical or psychological knowledge. For those, they know they need to get advice, and they have ways to do it. Similar access to advice is needed here. And if the Times article is at all accurate, that would have been enough to turn the whole thing into nothing more than a brief annoyance to Mr Hallowell.

Responses should match the question

I’ve mentioned before, in passing, that when your computer asks you a question, the choices it offers for responses should be parallel to the question. Too often, the folks designing the pop-up box try to give the buttons sensible-seeming responses, in violation of this rule and at the expense of understandability.

Take, for example, this question you might get from Mac OS 10.5 if you use the Finder and drag a file from one folder to another:

An older item named [xxx] already exists in this location. Do you want to replace it with the newer one you are moving?

(Stop)       (Replace)

Now, without the question in the second sentence, “Stop” and “Replace” might be fine choices:

An older item named [xxx] already exists in this location.

(Stop)       (Replace)

...although I might prefer the admittedly longer “Do Not Replace” to “Stop”.

But they’ve added a yes/no question, and the cognitive disconnection in trying to sort out which response means “Yes” and which means “No” (or, alternatively, in trying to push out the question and just dealing with the statement in the first sentence) is jarring. If you ask a yes/no question, the response choices have to be “Yes” and “No” (in whichever order makes sense):

An older item named [xxx] already exists in this location. Do you want to replace it with the newer one you are moving?

(No)       (Yes)

Of course, the larger issue is that these dialogue boxes should be crafted carefully. They must ask straightforward questions that are easy to understand and that users can confidently and reliably answer. For example, this sort of formulation would be horrible:

An item named [xxx] was found in a search of this location in preparation for the move. You can continue, replacing the file, but data might be lost in the process. Do you want to stop?

(No)       (Yes)

Unfortunately, questions not too far from that show up far more often than one would like.

NPR series about email

I was a bit disappointed in the series about email that National Public Radio’s Morning Edition aired last week. It’s not that it was bad, just that I thought it was sort of fluffy, with little real meat. Here’s my executive summary of the series:

• Sunday: Friends and family members send a lot of junk — “jokes”, urban legends, inspirational messages, chain letters. It’s annoying.
• Monday: There’s too much email, and it’s a main factor in information overload. We need strategies to deal with it. And it helps if you don’t contribute to the problem.
• Tuesday: A company has come up with a strategy. Their software makes you assign “importance points” to each message, and you get a limited number of points to assign. Use them wisely.
• Wednesday: Your mail is backed up and archived, and exists forever. Your business mail might be subpoenaed and searched during legal proceedings.
• Thursday: Encryption can keep your mail private, but almost no one uses it. You’re especially exposed if you use public networks. But most people don’t care.
• Friday: Some offices have established mail-free Fridays... and an opinion that that’s just running away from the problem. Also, some talk about stepping away from your BlackBerry.
• Saturday: There's a plethora of ways to communicate (email, IM, text messaging, phone, Skype, social networking...), and senders and recipients have to choose among them, and adapt to each other. Also, a discussion of “friending” on social networking sites.

There’s a bit more stuff on the web site, along with a few links that one might find useful. And, to be fair, it’s likely that I find it a little fluffy because I’ve been intimately involved with email technology for the better part of 30 years. Probably, the average listener found the series interesting. Saturday’s installment was the most interesting to me.

And, too, I’m not sure what I expect them to say. Maybe more advice that’s practical for the average user. Maybe something done in layman’s terms about up-and-coming email or anti-spam technology. Maybe even just a FAQ — despite the ubiquity of all this, I still hear the same questions all the time. Like, “Why is this guy sending me spam?” (A: He’s not; the sender address is faked.), and “How did I get this message? It wasn’t even sent to me.” (A: Yes, it was, on the envelope; it just doesn’t say that in the message.)

Anyway, give the series a listen. And, if you’re so inclined, comment here about what you think of it.

I’m gonna sit right down and write myself a letter
And make believe it came from you
I’m gonna write words, oh, so sweet
They’re gonna knock me off of my feet
A lot of kisses on the bottom
I’ll be glad I got ’em

Peer-to-which-peer?

In the New York Times Bits blog, Brad Stone points us to an interesting University of Washington study that shows how easy it is to fool the organizations that are sending out “takedown notices” that accuse Internet users of illegally downloading copyrighted material (the PDF of the study’s report is here).

Some background and terminology, first, for the less techie readers:

The Digital Millennium Copyright Act of 1998 (DMCA, PDF here) does two main things that are relevant here:

1. It makes it easier to prosecute the use of the Internet to violate copyright protection.
2. It makes it illegal to provide tools to bypass copyright protection, even if the providers of the tools do not themselves violate any copyrights.

There is, of course, disagreement about whether these are good things, but there it is. The second point, for reference, is what was used to take down Napster a few years ago.

Peer-to-peer file sharing (P2P) is a technique wherein individual computers trade files — or, more significantly, fragments of files — with each other, without going through a central server. If you buy a song from iTunes, you’re downloading a file from a server. But if you download a song through a P2P system, the most common of which (right now) is BitTorrent, you’re getting various chunks of the song’s data file from various computers — peers, since they’re just computers belonging to other people like you — all over the Internet.

The P2P method allows you to get different pieces of the file at the same time, often getting the whole thing to you more quickly than from a server. It also allows you to interrupt things and come back later for the missing pieces. It means that no one computer has to have the capacity to provide the entire file. And once you have the first piece or two, you can become the peer that provides those pieces to another user.

But the thing that annoys the copyright owners here is that it’s rather like spies passing around a parcel in an espionage movie: there are random people all over the place that are passing bits back and forth, and it’s difficult for them to have any control over it. The copyright owners can’t stop people from sharing the copyrighted material for free.

What they’ve done instead — where “they” includes organizations such as the Recording Industry Association of America (RIAA) and the Motion Picture Association of America (MPAA) — is to track the P2P users down after the fact, using data collected from other peers on the network (some of which they own) and often enlisting the assistance of colleges and service providers in identifying their users. They then threaten the users, sending takedown notices ordering them to stop and demanding compensation for files the organizations claim were already copied.

Only, now the folks at University of Washington show how flawed the techniques are for tracking the users down. They’ve gotten hundreds of takedown notices sent from the RIAA and MPAA, accusing Internet addresses that they can prove are innocent — in some cases, the addresses belong to printers, devices incapable of downloading music and movies.

I see battles between expert witnesses coming, as these results are used to challenge takedown notices in court. Until now, many college students have been intimidated into paying extortion to the industry organizations, cowed by threats of even more expensive judgments in court if they didn’t take the shortcut offer. There’s now a clear way to cast legal doubt on the whole process.

I do not support the theft of copyrighted material. But neither do I support the strong-arm tactics of these organizations, and, as I’ve said here before, I think the industry has to adjust its attitude and its business model to the way the Internet works and is used.

How not to fight spam

There’s been a discussion on the IETF working group chairs mailing list about some of the criteria that are used to look for spam sent to IETF mailing lists. In particular, the IT department is using these four rules:

1. Check that the HELO hostname has an A or MX record, defer if not.
2. Check that the client IP address maps to a name; defer if not.
3. Check that that name maps back to an IP address; defer if not.
4. Check that the name-to-address mapping matches the client IP; defer if not (that is, check that steps 2 and 3 give consistent results).

There are some current cases where these rules are blocking legitimate posts to the mailing lists, and the discussion is about whether these rules are reasonable. I’ve sent the comments below as my contribution to the discussion, and I’m posting them here too, because they lead up to a significant point about how we fight spam.

The HELO command, in an SMTP email session, serves only to start the session and to identify the sending domain for the log file. The domain that’s given there is used for nothing — nothing at all. It’s easy to think that it’s reasonable to at least make sure that it’s a valid domain for sending mail from (the purpose of test 1, above), but the reality is that it can be bypassed by putting any domain at all there. I can send email and use HELO whitehouse.gov and the receiving mail server will be happy.

There’ve been proposals to validate the sending server’s authority to use that domain (CSV, for example), and to actually use the domain for some purpose, such as checking it against a whitelist or blacklist. But unless it’s both validated and used, there’s no value in doing the DNS check.

When a legitimate user at example.com sends email to the IETF, it will usually be a dedicated email server that makes the connection and transmits the mail. Let’s say that server is email1.example.com, and that its Internet address (IP address) is 2.4.68.1. The IETF mail server will see that the connection came from 2.4.68.1, and step 2 will resolve that to email1.example.com. Step 3 will then resolve the name back to the IP address, and step 4 will confirm that it got that same 2.4.68.1 back. That’s fine.

But if mail is sent illegitimately, by some user within example.com, bypassing the normal mail servers — common if spam is sent by an infected zombie computer — it’s possible that one or both of the DNS resolutions in 2 and 3 will fail.

That’s the theory, anyway. The practice is that the transient IP addresses that are usually assigned to users’ computers often do resolve in DNS... and small domains often don’t have all their computers listed (they aren’t expecting people to want/need to resolve them), so they wind up as false positives and get their mail blocked.

And step 4 just seems odd. If the DNS resolutions work in 2 and 3, a failure in 4 would most likely indicate a configuration error, rather than a “bad guy”. It seems strange to use a check like that to block mail. Can that really be catching any spam?

All of these checks seem wrong to me as currently implemented. And in general, I think it’s a bad idea to use anti-spam techniques that address current spammer tactics that are easily changed once the spammers are aware of what we’re doing. It’s rather like rejecting mail that contains certain keywords: the spammers just start misspelling the words. We long ago learned that that’s a pointless skirmish, and that we need to fight more strategic battles.

A word about “net neutrality”

As the battles about “net neutrality” wage on, I note that I often see the concept mischaracterized, somewhat. So I’d like to make a brief point about what net neutrality is... and isn’t.

Net neutrality doesn’t mean that you, the user, can do anything you want, go anywhere you want, transfer any amount of data you want, without “interference” from your service provider. We all have service agreements, and we all have to live within them. And ISPs have the right to limit your usage to the terms of your service agreement, and to provide fair service to all their customers.

That means that placing limitations on data rates, total data volume, hit rates on hosted web sites, and that sort of thing, are reasonable and expected, and don’t violate net neutrality, per se. If your ISP takes action against you because you’re using BitTorrent and transferring so much data that it exceeds your service agreement and interferes with the service to other customers, they’re within their rights. On the other hand, if they take action simply because they don’t like BitTorrent, that’s bad.

The concept of net neutrality is that your access to the Internet should not be limited by what equipment you choose to use, what software or protocols you choose to use, or what services, servers, or web sites you choose to use... as long as things are working properly and you’re abiding by the terms of your service agreement.

Now, that said, there’s plenty of room for tricky bits there.

Most obviously, service providers have started adjusting their service agreements to give them more leeway in limiting things like BitTorrent. Much of this is related to the following “grey area”: is it ethical, or does it violate net neutrality, to allow you to exceed your service agreement sometimes?

Suppose your service agreement says that you can transfer no more than 20 megabytes per hour at speeds of up to 3 megabits per second. Note, especially, that there’s no minimum speed quoted, and, in fact, most service agreements now state that for residential service, “service availability, speed and uninterrupted service not guaranteed.”[1] Given that service agreement, assume that you use BitTorrent to download videos, transferring as much as 500 megabytes an hour sometimes over a period of at least two weeks. Which of these fall within acceptable net neutrality principles, and which do not?:

1. Your ISP suspends your service.
2. Your ISP takes steps to block your use of peer-to-peer protocols.
3. Your ISP slows your data rate to 100 kilobits per second.
4. Your ISP slows your data rate to 100 kilobits per second, but still gives you 3 megabits per second for normal email and web browsing.
5. When your ISP sees you transferring too much data from certain Internet addresses, it blocks your access to those addresses.
6. I use the same ISP, which provides a for-pay video download service, and I also download 500 megabytes an hour, but from the ISP’s download service. I have the same service agreement as you do, but the ISP takes action against you and not against me.

I put those in roughly increasing order of dubiousness, though others might disagree with the order (and some might switch 2 and 3, in particular).

The main ethical question that’s up in the air is whether it’s OK to sell you restrictive terms of service, and then relax them in “approved” cases. That’s clearly not “neutral”... but would you complain if you were paying for certain service, and were given extra service “for free” sometimes?

It’s clear that we do that all the time in the real, non-Internet world. We take discounts all the time for buying whatever product a store is pushing this week — we call it “a sale”. Far from even making an attempt to be neutral, when a store puts something on sale it’s unabashedly encouraging you to use one product over another.

Is it so wrong to do that on the Internet too? Why couldn’t Verizon make a deal with Borders, say, that encouraged you to buy from them instead of from Barnes and Noble or Amazon? Only, rather than having a coupon to save money, you got better Internet service as your incentive.

The trouble is that that plan takes us to the edge of that slippery slope. It’s too easy for that to lead to de facto censorship, where the differing levels of service effectively block access to certain Internet services. The ISPs, here, are providing something more analogous to roads than to anything else, and it’s their responsibility to keep the roads clear, and not to try to control where you go by artificially limiting access to the roads.

Imagine if your town gave favour to some businesses, at the expense of others, by adjusting speed limits, traffic enforcement, and road construction schedules to make it hard or impossible to get to the disfavoured merchants. That’d be wrong, and you can imagine the lawsuits that would ensue — that do ensue when cases like this become known.
 

---

[1] Quoted from Verizon’s service agreement; others are similar.

Phishing gets more sophisticated

The folks at F-Secure alert us to an interesting phishing technique. Phishing, for the non-techies reading, is the practice of sending email that tries to get you to respond (usually by going to a web site) by giving them personal information, including login credentials, account information, social security numbers, and the like. You’ve seen the email, with dire warnings that your bank account will be frozen unless you click immediately on the link they provide, and log in to clear up some “problem”. What you actually do is give them the information they need to log into your account.

In this variation, the phishers are trying a new ploy to suck you in, and adding a nasty mechanism to install bad software on your computer:

From: Clients support team
Subject: Comerica Bank - Significant information for our clients.

Client authentication using digital certificates in COMERICA BANK®

Dear Customer,

Comerica.com site has requested that you identify yourself with a certificate.
The next step in the transformation of Comerica Online is Digital Certificate (DC) access.
This DC will allow you to access Comerica Bank and other online services through a single sign-on.

All users will be notified and transitioned to the new URL between April 2008 and October 2008.

Please register your DC account and use our servcies safely.

Continue>>

2008 Comerica Bank. All rights reserved.

The link (which is disabled in the text above) takes you to a web page that “explains” what you have to do, and it involves running a “DC Loader Wizard”... which, of course, installs their malware — spyware that records what you do on your computer, including any login credentials that you use later today, or next week, or next month. In other words, it’s a phish that stays around and keeps on stinking.

Now, ignore the awkward phrasing of the message, and raise your hand if you’d give this concept at least a modicum of credibility in your mind — the idea that your bank might switch to certificate-based authentication, in order to improve security. Is your hand up? Mine is.

Now, that doesn’t mean I’d have been caught by this; I wouldn’t, but more on that later. What it does mean is that these phishers have gone away from the scenario that we’ve well learnt not to trust. They aren’t asking for personal information, they’re not asking us to log on.

Instead, they’re asking us to do something that we’ve heard is good for security. Almost. The trick, of course, is getting us to install software — even software that calls itself a “DC Loader Wizard“ — from an untrusted source.

And that’s where they wouldn’t catch me, and shouldn’t catch you. The core bit of advice that’s operative here is that you never, for any reason, visit a “trusted” web site through a link in email. Keep bookmarks in your browser for those things — your banks, your credit cards, your social networking sites, your email accounts — and use your own, trusted and tested links for them, always. Or else type them in by hand. Then you know where you’re going.

And when you get there and you don’t find anything about digital certificates or a DC Loader Wizard, say “Mmm, hmm....”

On replying to email

A couple of weeks ago, David Pogue posted in his New York Times blog a letter from a reader. It’s short, so here:

A Simple E-Mail Design Idea

A good idea from a reader:

David, I had the simplest of ideas for helping people avoid “Reply-All” nightmares (where you humiliate yourself by clicking Reply to All, blasting your response to a huge group, instead of just Reply). E-mail programs like Outlook or Apple Mail should just not put the Reply-All button anywhere near the regular Reply button!

Visually, when the two are so close to each other, it’s easy to mistakenly click the wrong one. But if you had to go down into, say, the bottom-right corner to find the Reply-All button, that would likely jar you enough so that you wouldn’t make an error.

Hope that someone implements this some day soon!

(To which I add: Or how about, at the very least, requiring that you press a key, like Shift, as you click the Reply button to change it to “Reply to All”?)

It sounds simple; it sounds nice; it sounds like it’d solve a lot of problems, doesn’t it?

The trouble with this idea, as with many of these sorts of ideas is that it’s not as simple as that. Oh, yes, people have thought of both the reader’s suggestion and Mr Pogue’s... and some mail programs do one or the other of them. Guess what: it doesn’t fix anything, not really. It only makes things a bit less convenient.

Some of the commenters have the answer too. And so... let’s look at my comments on it, and on what some of the other commenters say.

First, for everyone who says, “They should hide the ‘reply to all’ button, because it’s almost never what you want to do,” you can find at least one person who says the opposite, that “reply to all” is usually right and people don’t use it enough. The real answer is that each is correct sometimes, which is why both options are proffered.

When I send a note to, say, ten people, and you reply, the question really is whether you’re replying to me or to the group... whether everyone ought to see your reply or not. There really isn’t a way to know, in general, which of those is the case. We can infer the former if the group is large enough — it’s less likely that your reply ought to go to the group as the groups size increases — but deciding on the threshold isn’t easy. 100 probably qualifies. But does 10? I have certainly had group discussions by email with ten people at a time.

The consequences of not including the group on your reply are also not benign. It can confuse the discussion greatly when some people miss a sub-thread of the discussion, and it can make coordination difficult when different people are aware of different fragments of the situation. I was recently involved in trying to get four people to meet, when two of the people were replying only to the sender of each message... and it was horrendous.

So the answer isn’t to hide one of the reply options. The answer is to make it easy for the user to make the correct choice for this reply.