And visions of greengage plums dance in my head

A week ago, New Scientist told us about some new research technology by Toshiba, a system that recognizes fruits and veg at the self-checkout station:

Its system, developed by Susumu Kubota and his team at Toshiba’s research centre in Kawasaki, Japan, uses a webcam, image recognition and machine-learning software to identify loose goods, such as fruit. The company claims the system can tell apart products that look virtually identical, by picking up slight differences in colour and shape, or even faint markings on the surface.

When shoppers want to buy, say, apples at existing self-service checkouts they must choose the right product from a long list of pictures on a screen. Toshiba’s technology, part of which was presented last year at the 11th European Conference on Computer Vision in Chersonissos, Greece, compares the image captured by the webcam against a database of images and detailed information on the item’s appearance. The software uses an algorithm to produce a list of pictures of similar items, with its choice for the closest match at the top. If this choice is the correct one, the checkout user presses a button to confirm the purchase.

The system isn’t quite ready yet, and Toshiba hopes to commercialise the system within three years. They note, Similar ideas designed to identify products without barcodes have never made it to market in the past.

Indeed. Let’s go back to this item from 2003, where USA Today talks about some IBM research, including a system called Veggie Vision:

Researchers at IBM recently assembled several of the high-tech machines for a demonstration at their Industry Solutions Lab in Hawthorne. Among them were the smart shopping cart, a computerized produce scale called Veggie Vision, and a fascinating projection tentatively dubbed the Everything Display.

[...]

There doesn’t seem to be any controversy about Veggie Vision, a scale for fruits and vegetables that is hooked up to a digital camera and a library of hundreds of pictures of produce. When a shopper puts tomatoes on the scale, the machine evaluates their color, texture and shape to determine what they are, then weighs and prices the purchase.

Not only can it tell an apple from a tomato, but unlike some checkout clerks, it can tell a McIntosh apple from a Red Delicious.

Sound familiar? It did to me, because I knew some of the people who worked on Veggie Vision, colleagues at IBM’s T.J. Watson Research Center. And, while the USA Today article is from 2003, the conference papers about Veggie Vision, as well as the patents covering the technology, are from 1996 and 1997 (see this page for the IBM Research description, and links to the papers and the patents). It’s all there, complete with reading through the bag and machine learning.

I remember being impressed with the system (and the cool name), back when my colleagues were working on it and demonstrating it within the research lab. We had a good one, thought I, and according to the IBM Research web page, The system is now ready for prime time, and its developers have signed field test agreements with two scanner manufacturers and one company that makes self-checkout systems.

So, what happened? Why isn’t the IBM system out there at all the self-checkout stations? Why is Toshiba making the science-and-technology news for re-inventing what IBM had ready for market ten years ago? I’d hate to see Toshiba get the credit for what my IBM colleagues did so much earlier.

I have no information about that, alas... only the vague frustration that I often found, where good research projects would never seem to go where we thought they should, after they left the lab.

The thing that hath been, it is that which shall be;
and that which is done is that which shall be done:
and there is no new thing under the sun.

— Ecclesiastes, chapter 1, verse 9 (King James Version)