Monday, November 20, 2006


Every picture tells a story, don't it?

New Scientist is usually pretty good, if too cursory, but they really get it wrong this week, when they talk about image spam:

Computer security experts are struggling to cope with a new type of spam sweeping the internet. The emails can bypass conventional spam filters because they contain images of messages rather than actual words and sentences.
This stuff isn't "new", in any sense. We (that is, those of us who fight spam for a living) have been watching it grow and working on filtering it for at least a couple of years now. Saying it's a new problem is rather like saying that paying for the war in Iraq is a new problem facing our legislators.

CAPTCHA that says 'ARRRGH!'The article does quote a claim that these sorts of messages constitute 40% of spam now, up from 18% at the beginning of the year, and that "That's a big increase." OK, it sure is, assuming that those figures are accurate (I have no data on that). But given that one estimate says that around 30 billion pieces of spam were sent per day in late 2005, even at 18% that was 5.4 billion image-spam messages every day. No, this is not a new type of spam.

We're also not without mechanisms to block it. The article mentions mechanisms based on routing information, which we've been successfully using on all types of spam. It also talks about optical character recognition techniques as a pie-in-the-sky method that's a futuristic goal. The fact is, though that

  1. we have image recognition techniques that do not actually pull the text out of the image, but that do recognize related images and are successful as filtering tools, and
  2. character recognition is more successful than one might think.

Considering the latter: researchers at Microsoft presented a paper at the 2005 Conference on Email and AntiSpam that showed that in their study of text-based CAPTCHAs, Computers Beat Humans at Single Character Recognition in Reading-Based Human Interaction Proofs (and that's two-year-old research now). That's not to say that we've got the problem solved, but simply that the problem space isn't as straightforward as it seems.

I strongly question the "10 to 30 years away" claim for OCR in the New Scientist article; I think it's lots closer than that — which says not only that we should soon be as good at detecting this sort of spam as we are at purely text-based spam (which we still can't detect 100% of the time, of course), but also that we should not be relying on character-based CAPTCHAs (which I find obnoxious anyway).


The Ridger, FCD said...

I can't believe anybody would call that "new"!

Dr. Momentum said...

I have personally experienced a dramatic uptick in the image spam I get via email. But what gets through nowadays is down at the "annoyance" level which is better than the "disruptive" level that things were at back when I felt I needed to use POPfile.

I read a story somewhere that spammers were using animated gifs in which only part of the text image is contained on each frame of the animation. If they're already attempting to circumvent OCR software, that would imply that OCR is already being used as a means to combat this spam (spammers, I think, are lazy).

It seems to me that OCR has been around for quite a while.