Friday, October 29, 2010

.

China bound

I get to experience the Great Firewall of China first hand: I’m on my way to three weeks in China — a week at my company’s headquarters in Shenzhen, a week at the IETF meeting in Beijing, and then a week of touring afterward (so I’ll get to experience the Great Wall of China as well).

I don’t know what kind of connectivity to the blog I’ll have while I’m gone, and I’m not sure how much time I’ll have to post to it anyway. We’re assured that IETF attendees will have access to an unfiltered Internet connection during the meeting. Still, it’ll be a busy, full week, as will the weeks before and after.

So don’t expect much, if anything, on these pages until 20 November, when I’m back in New York.

Wednesday, October 27, 2010

.

Free speech and phone calls

Earlier this afternoon, I got a phone call on the house phone, and let it go to the answering machine (I use Google Voice and Skype while I’m working). The machine’s in another room, but I had no trouble hearing the message clearly, for as long as I was willing to listen before I went in to kill it.

The message was of someone SHOUTING, shouting about politics. Shouting about my congressguitarist, and why I shouldn’t vote for him.[1] He did this, he didn’t do that, he eats babies raw, and whatnot.

Shouting, did I say shouting? And, of course, it was a recorded message, the better to save the shouter’s voice for the next round.

This was probably the most egregious violation of my privacy I’ve ever encountered in a phone call that’s protected by the U.S. Supreme Court.

We have a do not call list in the U.S., federally mandated and implemented by the Federal Communications Commission. When Congress set that up, it came with a great deal of controversy about free speech rights, and how it would interfere with free speech. And, so, there are a few (or several, depending upon how you reckon those terms) categories of calls that are exempt, and may be made to anyone, even if her number be on the list:

  1. Calls from organizations with which you have established a business relationship.
  2. Calls for which you have given prior written permission.
  3. Calls which are not commercial or do not include unsolicited advertisements.
  4. Calls by or on behalf of tax-exempt non-profit organizations.

The first two categories come with mechanisms to remove yourself from the organizations’ call lists, and to revoke any permission you’ve given. But those last two categories, put there with free speech in mind, cover religious and political organizations, and such organizations can call you whenever they like, with no requirement that they allow you to opt out.

And, of course, during these last weeks before our elections, as you might imagine, the calls come, as we say, fast and furious. This one was particularly furious.

The battle is long lost, as the Supreme Court has repeatedly set free-speech boundaries well beyond where even I would set them, but this one is a complete mystery to me. I don’t agree that spending money should be equated to speech. I don’t agree that corporations should be given free-speech rights comparable to those of individuals.

But I really don’t see how anyone’s free-speech rights should allow them access to my home. I can’t accept that your right to speak freely includes any right to invade my privacy in order to do it.

I’m sympathetic to the concern over setting up situations where an organization blocks calls without your understanding what they’re blocking. But this isn’t that. I am putting my own numbers on the do-not-call list, and it is my choice not to receive calls from political or religious organizations. I should have that right, and no one should be allowed to force his way, or his electrons, into my home.


[1] Sorry, but I already have: as I noted the other day, I voted last week with an absentee ballot.

Tuesday, October 26, 2010

.

More on Internet cafés and public networks

For my readers who aren’t terribly fond of the entries tagged technology, please stick with this one. It’s important.

Do you log into web sites from public computers, even though I advised against it four years ago? That post only scratched the surface, really: it just talked about using public computers. These days, most people have their laptops with them, and they connect them to the public wireless networks in the cafés.

Most of those networks are unencrypted. That means that you don’t have to enter a key or a password when you access the network. You just select the network name (or let your computer snag it automatically), go to a web page in your browser, and get redirected to some sort of login and/or usage-agreement screen on the network you’ve connected to. Once you click through that, you’re on the Internet.

Suppose there are twenty people in there using that particular network. All twenty of them are sending and receiving stuff through the air. How is it that I only get my stuff, and you only get yours, and we don’t see each other’s, nor the web pages of the other eighteen users? It must be that my web pages are beamed straight to me, and yours to you, right?

No. In fact, everything that everyone sends and receives is out there for all twenty computers to see. But each of our computers is given an IP address, each data packet contains the address that the packet is being sent to... and all of our well behaved computers just look at the addresses and ignore any packets that aren’t meant for them.

Computers do not have to be well behaved. Any computer in the café — or near enough to hear the wireless signals — can see everything that everyone is sending to and receiving from the network. Because the network isn’t encrypted, it’s all out there, in the clear, visible to all who care to be badly behaved.

But we aren’t completely unprotected: we have something called TLS (or SSL, depending upon the version). When the web site’s address, the URL, begins with https, your communication with that web site is encrypted and safe from eavesdropping, even if the network itself isn’t. Perhaps you don’t care who sees you reading the New York Times, but you want to be protected when you visit your bank online. Use http for the Times and https for the bank, and all is well.

And that’s important, because most web authentication just has you send your username and password openly from your browser to the web site. Anyone could snoop your ID and password as you logged in, if your connection to the web site wasn’t encrypted. But that https saves you.

But wait: I have a New York Times account, and I’ve logged into the Times web site (using https). Every time I visit the site, it knows who I am. Even when I just go to http://www.nytimes.com/ ! How does it know that, when I’m not logging in all the time?

Web sites use things called browser cookies to remember stuff about you. A cookie is a short bit of data that the web site sends and asks your browser to attach a name to and keep. Later, when you return, the web site asks if you have a cookie with a particular name, and if you do, your browser sends it. For web sites that you log into, such as your bank and the Times, the login (session) cookie is sent every time your browser touches the web site. Every time I click on another Times article, my Times session cookie is sent again. Every time I go to another page on my bank’s site, my bank’s session cookie is sent again.

My bank is set up securely, as is my credit card site, as is Gmail, as is PayPal: every contact from the login screen until I’m logged out is through https. It’s all encrypted. Not only is my password encrypted when I log in, but the session cookie that the site gives me is encrypted too, every time I send it.

The New York Times, though, doesn’t work that way: only the login itself uses https. Once it gives me the session cookie, everything switches back to http, and there’s no encryption. When I click on an article and my browser sends my cookie again, anyone in the café can grab it.

Now, the cookie doesn’t contain my password, so no one can get my password this way. But as long as I stay logged in, and the cookie is valid, anyone who has that cookie can masquerade as me. If they send my cookie to the New York Times, it will treat them as though they were me, as though they had logged in with my password.

Of course, it’s not just the New York Times that does this. Amazon does it. So do eBay, Twitter, Flickr, Picasa, Blogger, and Facebook. So do many other sites where you can buy and sell things. (All the airline sites I’ve checked do it right, using https after login.) That means that if you use Facebook while you’re at Panera, someone else can borrow your Facebook session cookie and be you, until you log out. If you stop by Starbucks and get on eBay, someone else can use your cookie to make bids from your account.

There’s some protection at some sites. Amazon, for example, will let the cookie thief browse around as you, but will want your password before placing an order... assuming you didn’t enable one-click purchasing. And depending upon the options you have set, eBay might or might not ask for your password when the thief places a bid. But Facebook and Twitter are certainly wide open, here.

To try to increase awareness of this, a guy named Eric Butler has created a Firefox add-on called Firesheep, which will make it trivial for anyone, even someone who knows nothing about the technical details of this stuff, to be a cookie thief and pretend she’s you on Facebook, or Twitter, or Blogger, or the New York Times. Eric isn’t trying to abet unethical or criminal behaviour; he’s trying to push the popular web sites, whose users will be targets of these sorts of attacks, to fix their setups and use https for everything whenever you’re logged in.

So here’s an expanded form of the warning: Don’t do private stuff on public networks, unless you’re absolutely sure your sessions are encrypted. If you don’t know how to be sure, then err on the side of caution.

Monday, October 25, 2010

.

Challenge/response still lives (barely)

Wow; I haven’t gotten one of these in a long time:

ATTENTION!

A message you recently sent to a 0Spam.com user with the subject "[redacted]" was not delivered because they are using the 0Spam.com anti-spam service. Please click the link below to confirm that this is not spam. When you confirm, this message and all future messages you send will automatically be accepted.

I wrote about challenge/response anti-spam systems about three years ago, but probably haven’t seen a challenge message in at least two years. I thought people had given up on them.

Alas, no. But if the last two years is something to judge by, they’ve at least fallen further into disfavour.

Anyway, it’s worth a re-post, then, of my three-year-old item about them. All the problems, all the reasons one shouldn’t use them, are still valid now. So, here’s the link again: head over and read (or re-read) it.

Sunday, October 24, 2010

.

Looking up at New York

Friday, October 22, 2010

.

The Juan Williams saga

Long-time NPR journalist Juan Williams is no longer an NPR journalist, because of a comment he made out of school:

NPR News has terminated the contract of longtime news analyst Juan Williams after remarks he made on the Fox News Channel about Muslims.

Williams appeared Monday on The O’Reilly Factor, and host Bill O’Reilly asked him to comment on the idea that the U.S. is facing a dilemma with Muslims.

[...]

Williams responded: Look, Bill, I’m not a bigot. You know the kind of books I’ve written about the civil rights movement in this country. But when I get on the plane, I got to tell you, if I see people who are in Muslim garb and I think, you know, they are identifying themselves first and foremost as Muslims, I get worried. I get nervous.

Now, we can think about how any sentence that begins with Look, I’m not a bigot, will end. We can note that Mr Williams is more conservative than NPR has a reputation for; some of us knew that before, and some just found out now. We can tut about how that very thing that Mr Williams said is often said by white people about his kind. We can do all those things, but it doesn’t really matter, because the bottom line is that places that practice real journalism[1] — places such as NPR, the New York Times, and the BBC — have rules that make sure their journalists remain publicly neutral.

That means that there’s a difference between people such as Juan Williams and people such as Bill O’Reilly, Keith Olbermann, and Jon Stewart. The other three are supposed to be making political commentary — one might say that they’re paid to be inflammatory. But the folks who report and analyze at NPR and the Times, for instance, are not allowed to support candidates, accept money from people they’re reporting on, and that sort of thing. I have a friend who used to be a researcher for Canada’s CBC news, and I know that they’re not even allowed to have political signs on their front lawns.

So you might imagine that making an inflammatory, politically charged comment about getting the willies when you see Muslims on the plane... well, that clearly crosses a well known line. And it’s no surprise that he was sacked.

Of course, NPR is now taking flak for the sacking. People are complaining that Mr Williams is being censored. People are complaining that NPR is violating Mr Williams’s first-amendment rights.

Damn right he’s being censored. That’s what real news outlets do to make sure that their reporters and analysts do not appear unduly biased. How could anyone take any future item about Muslims by Mr Williams seriously now? He obviously has a non-exclusive contract, and as an analyst he has more flexibility than the reporters do. He was allowed to be on Bill O’Reilly’s show in the first place. But, unlike the guy from New Jersey Transit, he knew the rules. He knew that he was risking his job by saying something that damages his credibility — because, unlike the guy from New Jersey Transit, his credibility is his stock in trade.

But first-amendment rights? Mr Williams absolutely still has his first-amendment rights, and NPR did nothing to take those away — they are not letting me comment on the air, and I’m not making any first-amendment complaints about that. He exercises his rights in his books and his many speaking engagements. NPR can’t stop him from speaking, but they have no responsibility to put him on their payroll or their news feed.

On the other hand, the poor man is not lacking other places to speak: he’s just been given a two million dollar contract by... yes... Fox News.

One might speculate on whether that was in the works before Mr Williams had his tête-à-tête with Bill-O.


[1] Among which I do not include Fox News.

Thursday, October 21, 2010

.

Buy locally... at Walmart?

Gonzo-super-mega-chain Walmart[1] has recently announced that they will be buying more locally grown food and doing other things to support small- and mid-sized local farming:

Wal-Mart Stores announced a program on Thursday that focuses on sustainable agriculture among its suppliers as it tries to reduce its overall environmental impact.

The program is intended to put more locally grown food in Wal-Mart stores in the United States, invest in training and infrastructure for small and medium-size farmers, particularly in emerging markets, and begin to measure how efficiently large suppliers grow and get their produce into stores.

Advocates of environmentally sustainable farming said the announcement was significant because of Wal-Mart’s size and because it would give small farmers a chance at Wal-Mart’s business, but they questioned how local a $405 billion company with two million employees — more than the populations of Alaska, Wyoming and Vermont combined — could be.

Their U.S. goal for selling local food is quite modest: the company plans to double the percentage of locally grown produce it sells to 9 percent. Given how much food they sell, that’s a lot of local food. On the other hand, as the article points out, they’re shooting much higher in other countries, such as Canada.

But the question of how local such a large company can be doesn’t really seem the right one. If they set out to do it, they could easily have stores find their own, local suppliers for meats and produce, and their size would only help them there. Of course, since they define local as within the same state, there’s quite a difference between a store in California or Texas and a store in Rhode Island or Delaware.

Also, a store in California could easily buy a lot of its produce from California growers, but how much local produce is available in Wyoming? Florida stores could buy local citrus, but not apples, with the reverse true in New York.

More to the point, maybe, is the extent to which people buying food at Walmart tend to buy packaged goods and prepared foods, rather than fresh meat, fish, fruits, and vegetables. If most folks aren’t buying fresh foods, then it’s bound to be hard for the company to increase its sales of local produce beyond a certain point.

All that aside, I think this is a good thing. I’d like to see the chain stores buying and selling a lot more local food. It’s silly to go into A&P or Stop & Shop in New York in October, and see apples from Washington. We have wonderful apples grown within a 30-minute drive of my house, yet the supermarkets have them shipped in from 3,000 miles away.

That needs to change, and if Walmart can take some steps in the right direction, that makes me happy.


[1] Or Wal-Mart if you prefer, as the New York Times does. Their store logos have been changed from Wal*Mart"= to "Walmart*, and their web site uses Walmart and Walmart.com everywhere except in the copyright line, which says © 2010 Wal-Mart Stores, Inc.

Wednesday, October 20, 2010

.

Homer: the Catholic role model

Can you imagine? A spokesman for the Vatican has declared cartoon character Homer Simpson a Catholic.

But in an article headlined Homer and Bart are Catholics, the newspaper said: The Simpsons are among the few TV programmes for children in which Christian faith, religion, and questions about God are recurrent themes.

The family recites prayers before meals and, in their own peculiar way, believes in the life thereafter.

Very odd. They’re making reference to a particular episode in which Homer converts, and they’re also, I think, including the character in the same sort of way that President Kennedy included himself in his famous Ich bin ein Berliner. But, really, while the things they mention as recurrent themes are, indeed, there, it’s not really done in a role model kind of way, and the series is, in general, quite irreverent. Religion and religious leaders are mocked regularly, though it’s not as severe as in, say, South Park.

On top of that, Bart is disrespectful to his parents and teachers, and does poorly in school. And Homer spends his time gulping beer and doughnuts, and shirks his work.

You’d think the Vatican could come up with TV programmes for children that are better examples for good Catholic kids. When they have to go to The Simpsons, and use Homer and Bart as role models, you know they’re reaching.

Doh!

Tuesday, October 19, 2010

.

“Albany Governor Debate Verges on Farce”

So says the New York Times headline. The campaign to be governor of New York is an amusing one, looking a bit like a smaller (much, much smaller) version of the California craziness in 2003, when the Governator beat a cast of hundreds that included a porn star or two, and a bit like a British MP election, with a batch of silly, sometimes meaningless parties.

We had a televised debate last night, where the two serious candidates — current Attorney General Andrew Cuomo, the Democrat, and right-wing idiot Carl Paladino, the Republican — were joined by a handful of Fruit Loops that included Kristin Davis (not the actress from Sex and the City, but the former prostitute/madam who participated in the downfall of our former governor, Eliot Spitzer; she’s running in her own Anti-Prohibition Party, and says she will legalize marijuana) and Jimmy McMillan, who has tried to become New York City’s mayor twice before through his self-styled The Rent Is Too Damn High Party.

To round it out were Charles Barron (self-created Democratic Freedom Party; former Black Panther and Brooklyn representative on the New York City Council — he also tried to be mayor in 2005, along with Mr McMillan), Howie Hawkins (Green Party; lost to Hillary Clinton for U.S. Senate in 2006), and Warren Redlich (Libertarian Party; he ran unsuccessfully for U.S. Representative for his district, near Albany).

From all reports, it was not so much a debate as a comedic talk-show panel, where the panelists shouted over the questions and essentially ignored them, and turned it into something of a free-for-all. From the New York Times:

The moderators’ questions were frequently ignored. The candidates barely looked at one another. One wore black gloves and spoke of himself repeatedly in the third person. And Andrew M. Cuomo, the Democratic candidate and the race’s front-runner, at times struggled to suppress laughter.

And from the Wall Street Journal:

New Yorkers watching the seven candidates for governor debate on Monday night heard a former madam articulate a nuanced position on the merits of hydraulic fracturing. They listened to a former Black Panther and a Green Party activist call for massive tax hikes on the rich. And they learned that one candidate is so supportive of marriage equality, he’d let a person marry a shoe.

But for those searching for insights into the platforms of the two major candidates vying to lead a troubled state, Attorney General Andrew Cuomo and Carl Paladino, the first and perhaps last debate before the Nov. 2 election was lacking.

For most of the 90 minutes, the televised debate at Hofstra University on Long Island provoked more laughs and puzzlement than meaningful drama. And while it introduced voters to an obscure cast of characters from the periphery of local politics, it shed little new light on either of the two men who have more than a minuscule chance of winning the race.

I really don’t understand why the organizers of these things, which generally stopped resembling debates years ago, though they’re still called that, don’t exert some control over them. Maybe people have just gotten used to having everyone ignore questions and yell at each other on television now, and maybe that’s what some people want. Is it really what the majority of us want? Wouldn’t we rather see them forced to answer the questions that were asked or have their microphones turned off?

Well, amusing, sad, or silly though it may be, it matters little to me: I’ve already voted on an absentee ballot, because I’ll be travelling on election day. Figuring out whom I voted for is left as an exercise for the reader.

Monday, October 18, 2010

.

Interesting use of context/presence data

MIT has done some interesting research that uses context data from mobile phones — location and usage patterns — to predict broader situations (in this case, illness):

Epidemiologists know that disease outbreaks change mobility patterns, but until now have been unable to track these patterns in any detail. So Madan and colleagues gave cellphones to 70 students in an undergraduate dormitory. The phones came with software that supplied the team with anonymous data on the students’ movements, phone calls and text messages. The students also completed daily surveys on their mental and physical health.

A characteristic signature of illness emerged from the data, which was gathered over a 10-week period in early 2009. Students who came down with a fever or full-blown flu tended to move around less and make fewer calls late at night and early in the morning. When Madan trained software to hunt for this signature in the cellphone data, a daily check correctly identified flu victims 90 per cent of the time.

The technique could be used to monitor the health status of individuals who live alone. Madan is developing a smartphone app that will alert a named contact, perhaps a relative or doctor, when a person’s communication and movement patterns suggest that they are ill.

Public health officials could also use the technique to spot emerging outbreaks of illness ahead of conventional detection systems, which today rely on reports from doctors and virus-testing labs. Similar experiments in larger groups and in different communities will have to be done first though.

This is very much related to work I’d been doing with context data several years ago, and which still interests me a lot. If we can get around the privacy concerns (which is a large part of what I’d been working on), we can synthesize a lot of useful meta-information from the devices that people use. That information can then be used to perform services on the user’s behalf, or do other helpful things.

Alas, the paper is behind a paywall, and I can’t find a loose copy around.

Sunday, October 17, 2010

.

Neighborhood synecdoche watch

A colleague and I recently came across the following sign:

This neighborhood reports suspicious activity.

Ha, ha, my colleague said, I don’t think the neighborhood does any such thing. The people in it may, but not the neighborhood.

Ah, me. Folks who laugh at such things think they’re so clever, don’t they.

Only, they’re not: such things are using a common and acceptable (and rather interesting, if you ask me) figure of speech called synecdoche (pronounced syn-EK-duh-kee) or metonymy (meh-TAH-nim-ee)... you get the challenge of discerning the difference between the two, and deciding which one is operative in this case.

We see these in use all the time. Right lane must turn right. (Yes, it’s the traffic that turns, not the lane itself.) The White House said today that.... (The White House is a building, and says nothing; a spokesperson for the U.S. President is who did the saying.)

If they make you laugh, that’s great: the world needs more joviality. Just don’t be so jovial as to think there’s anything wrong with these locutions.

If you do, I’ll attack you here, because, you know, the pen is mightier than the sword (and bits on the Internet are the mightiest of all).

Saturday, October 16, 2010

.

Why do people carry a cell phone and a BlackBerry?

Back in the early days of BlackBerry devices, when they were basically enhanced pagers that could get your email, they didn’t function as phones. I used to carry a mobile phone and a BlackBerry, using the phone for voice calls and the BlackBerry for email.

But then they put a mobile phone — a good one — into the BlackBerry. My address book and calendar are synchronized with the BlackBerry, and it’s great to have everything on one device. I haven’t had a separate phone since then.

I can’t tell you, though, how often I see someone looking something up in his BlackBerry, and then pulling out his cell phone and calling someone on it. Why? What possible advantage is there to carrying both?

Friday, October 15, 2010

.

New meaning of “touch screen”

Interesting research from Nokia:

Nokia has developed a prototype of its N900 smartphone that lets you feel the texture of icons on the screen — a technology that would add a whole new dimension to touchscreen apps.

This week, Nokia researcher Piers Andrew showed how the technology could give each icon its own feel or add surface texture to photographs. The idea is to have everything on a touchscreen give tactile feedback, Andrew says.

The technology is based on an effect called electrovibration, in which touch receptors in the skin can be fooled into perceiving texture when you swipe a fingertip across an insulating layer above a metal surface carrying an alternating voltage. The higher the frequency of that alternating voltage, the smoother the texture feels.

[...]

The effect is thought to be due to the varying electrostatic attraction between the metal and the deeper, liquid-rich conducting layers of the skin — an effect which changes the perceived friction level.

To mimic this is in a touchscreen phone, Nokia placed two thin layers above the LCD display: the first a transparent conductor, indium tin oxide, and the second a transparent insulator, hafnium oxide. When the user cradles the phone in one hand and touches the screen with the fingers of their other hand, they effectively create a closed circuit. If the indium tin oxide is excited at frequencies between 50 and 200 hertz, the finger above the touchscreen is attracted towards the screen with varying strength, generating the textured effect.

They acknowledge that it’s not ready for production yet, and This is not necessarily the most attractive sensation for some people.

Still, it sounds very interesting.

Wednesday, October 13, 2010

.

On the right to DNA testing: Skinner v. Switzer

Our criminal justice system is sometimes arrogant.

We believe — at least, as the written code tells it — that our juries are infallible, or that their fallibility is an acceptable ill. And we must do so in order to keep the system working, to keep the problems contained. Most of the time, I agree with the acceptable ill attitude. But we often cling to that belief too doggedly, refusing to reconsider convictions when we should.

At no time is our responsibility to reconsider greater than when we decide to execute someone. Before we impose an irreversible sentence, we must take every opportunity we can to correct any possible mistake. We shouldn’t stand on process when someone’s life is at stake.

It would be wrong to refuse to hear a death-row appeal because the paperwork was filed a week late.

It would be wrong to refuse to consider new evidence that had surfaced after the jury made its decision.

And it would be wrong not to allow examination of evidence that existed but that had not been examined.

Yet that last is the concern of a case that the U.S. Supreme Court will hear tomorrow, the case of Hank Skinner (click through, then search for skinner):

The Texas state and federal courts — hearing Skinner’s habeas corpus pleas — refused to allow post-conviction testing of biological evidence, including blood, hair, fingernail clippings and vaginal swabs. The courts held that, under Texas law, a convict must prove, by a preponderance of the evidence, that he or she would not have been prosecuted or convicted had DNA testing been performed. To get DNA testing, a Texas inmate must also demonstrate that his failure to seek such testing at trial was not a strategic decision.

The law shifts the burden onto the defendant, who must show not just that doing the DNA testing would be reasonable, not just that the DNA testing might exonerate him, but that by a preponderance of the evidence, the state would have let him go without even going to trial if they’d done the tests then. And he has to do that just to get the testing done. Then to top that, he also has to explain away the procedural aspects of why this hadn’t been requested earlier.

Now, I’m as curious as the next guy, and I certainly want to ask why his defense team didn’t deal with this before. But I can’t imagine my decision on the testing hinging on that aspect. And I don’t want him to prove anything in order to get the biological evidence tested. The fact is that it’s available and it wasn’t tested, for whatever reasons, and, here: they’re going to kill Mr Skinner; they owe him an assurance that they did everything they could to be sure they’re right about that.

As if that weren’t enough, we have the heads of the justice departments of twenty-two states giving another crazy reason we should deny the request: they say that the states should get to decide this, and the federal government should keep out of it. If they’re short on money and personnel for testing, condemned prisoners will just have to accept their fates.

At least 22 states told the justices that granting Skinner DNA testing through a civil rights suit would undermine their individual statutes, which spell out when an inmate is entitled to it.

To allow this type of procedural legerdemain would both diminish the sovereign interests of the states and at the same time impose a significant burden on the states’ limited law enforcement resources, attorneys general from the 22 states wrote.

That sort of callousness seems enough of a reason, in itself, to demand that they take a step back and think. If it were your child standing accused, how would you want it to be handled? That couldn’t happen? Don’t count on that; sure, it could.

This should never have gotten to the Supreme Court, but now that it has, the court should require the testing. I’m not very confident that it will, though, with Justices Scalia, Thomas, Alito, and Roberts sure to vote against it. The outcome will likely rest on how Justice Kennedy votes, as I suspect this will be a five-to-four decision.

Haven’t there been enough people set free because DNA evidence showed that their convictions had been wrong? Can’t we see that this testing only makes sense from every just perspective? The only reason to refuse such a request is to stand with an arrogance that says, We did everything according to the law, and it’s too bad for you. If you think Mr Skinner is just a low-life who just isn’t worth keeping around, line up on that side.

But if you want to be more certain that the right man is being executed, do the tests.

Tuesday, October 12, 2010

.

Abusive, misleading paper-mail spam

I got something in the mail last week that I found interesting, in a sleazy way. The return address said United Airlines Awards Processing Center, and emblazoned on the envelope was this:

URGENT NOTICE:

Your Mileage Plus® Miles
are expiring. Use by
October 19, 2010.

Looks alarming, with urgent in bold, red letters, no? Well, but the return address used a P.O. box in Utah, and the postage payment area showed a pre-sort permit. Bulk mail.

Inside were the following:

  1. An envelope, pre-addressed to Processing Center (another bulk-mail flag), with the same Utah P.O. box number.
  2. A yellow sheet telling me that I can get faster service by making my redemption online. But not at united.com nor mileageplus.com; the URL is at magsformiles.com, and includes a code that will let them track the specific mailing.
  3. A letter, repeating the URGENT NOTICE, and bearing a date of 24 September — two weeks before I received this.

The letter is, in fact, a solicitation for me to use my miles to buy magazine subscriptions, and does not come from the United Mileage Plus program, but from a vendor (Synapse Group, Inc, in Stamford, CT) that wants my purchase for absolutely no cash cost.

United has a policy that if you have no transactions on your account for 18 months, your miles expire. But as long as you have at least one transaction, however small, within 18 months, you keep your miles forever. This promotion is presented as a way to use small transactions (a few hundred miles) to buy magazine subscriptions, thus keeping tens of thousands of miles from expiring.

The sleazy part is that it’s meant to make me think that my miles will expire next week if I don’t do something quickly. And that’s not true at all: my miles won’t be expiring any time soon, and there’s no reason for me to worry about it (though I did check, just to be sure).

Scumbag business practices are everywhere, and the spam isn’t just online.

Now to go to United’s web page and see if there’s some new we may share your address bit that I haven’t (yet) opted out of.

Monday, October 11, 2010

.

Search engines and their responsibility

A French court has just decided a case that will likely have a great deal of effect on online search engines if the decision is upheld after appeals. A French man had been accused of crimes relating to the corruption of a minor, ultimately resulting in a suspended sentence. He found that Google search results snagged the news items about his case, putting them at the top of search results on his name:

Given extensive press coverage of the alleged crime at the time, querying the man’s name on the popular search engine returns web pages from news publications that suggested he was a rapist, among other non-favorable descriptions.

The man argues that the statements in the online articles still available today adversely characterize him, which puts him in a disadvantageous social position when meeting new people and applying for jobs, among other situations and opportunities.

The man previously contacted Google directly to remove the defamatory articles from its search index, but the company did not do so arguing its proprietary algorithms simply return web pages in its index related to the keywords searched, that is, there is no direct human manipulation of top search results.

The result from the court was this:

The French court sided with the plaintiff, agreeing that those representations were defamatory, and ruled Google could have mitigated costs to the plaintiff by removing the pages.

The ruling ordered Google to pay €100,000, and to reimburse €5,000 in litigation costs incurred by the plaintiff. The ruling also ordered the company to disassociate the man’s name from the defamatory characterizations in Google Suggest, which suggests popular phrases while a person enters search terms in the Google search-box prior to completing a search. Additionally, for every single day the defamatory information remains in the company’s search results, Google would be fined an additional €5,000.

This decision will be disastrous for search engines and other Internet services if it stands. Moreover, it’s just horribly wrong on the surface. It makes no sense to hold indexing services responsible for the information they index, unless it can clearly be shown that they preferentially indexed certain material with a goal of creating a biased view.

Research facilities have, long before the widespread availability of Internet search tools, helped people find news items and other public information that we might rather they didn’t point to, including false information and stories that have since been debunked. We’ve always considered it the responsibility of the researcher to winnow the data.

The difference now, of course, is that the researchers are friends, neighbours, potential romantic partners, and prospective employers... and the information is much more readily available than it ever was. It’s tempting to try to make the search engines let go of obsolete information and only find the current stuff.

The problems with that idea, though, are several. It’s essentially impossible to sort out in any automated way what’s appropriate and what’s not. Even if they prefer legitimate news outlets to other sources of information, and prefer newer articles to older ones, the amount of cross-linking, re-summarizing, and background information will still show searchers plenty of nasty stuff. And who decides what the legitimate news outlets are? The search engines shouldn’t be making those filtering decisions for us.

Any mechanism that isn’t entirely automated doesn’t scale. With the untold millions upon millions of web pages that Google and other search engines have to index every day, there would be no way to respond to individual requests — or demands backed by court mandates — to unlink or otherwise remove specific information.

If this should stand, I can see that Google might have to cease operations in France. If it should spread, it might easily deprive all of us of easy searching on the Internet. That would be a far greater disaster than having a guy in Paris have to explain away unflattering news stories about a false or exaggerated accusation.

Clearing one’s name has always been a difficult challenge, and it’s only been made harder — perhaps, ultimately, impossible — on the Internet. I have a great deal of sympathy for anyone who finds himself relentlessly pursued by his past, especially when that past contains errors that weren’t his.

But this can’t be an answer to that. It just comes with too much collateral damage.

Sunday, October 10, 2010

.

Beer and penitence

I was sitting at a sidewalk table the other day, having a beer. At the next table were three women in their forties, speaking with southern accents. Why is it that southern accents seem so often to mean Christian? As I sipped, I overheard fragments, here and there, of their conversation, and every fragment had something to do with God, praying, or being Christian.

Every day I get up, and I ask God to forgive me for anything I did yesterday.

My first thought on hearing that was to wonder what value there would be — to God or to a real person — in such a series of generic apologies. Whatever I might have done, I’m sorry. No, that doesn’t cut it. Be specific. Acknowledge what you did, and apologize specifically for it. And then don’t do it again.

That ties into this one, of course:

If you’re Christian, even if you make a mistake, every day’s new.

If you’re Jewish or Muslim or Buddhist... or, of course, and especially, atheist... you’re screwed. You make a mistake, and that’s it. Christian, though, well, just get up every morning and tell God you’re sorry, and everything resets.

On the other hand, the Jews, who just went through this process the other week on Yom Kippur, batch it all up for once a year. Spend a day fasting and gathering in prayer, asking generic forgiveness for all the bad things that you’ve all, collectively done over the past year. But feel guilty from day to day; it’s good for the soul.

Later in the conversation, as they talked about their children, one said this:

I pray that I won’t pass down to them all of my dysfunction.

But she is, of course: she’s undoubtedly teaching them her silly superstitions, and showing them how to be dysfunctional and yet start over every day.

10:10:10 10/10/10

Ten.

Just sayin’

Saturday, October 09, 2010

.

70

All we are saying
Is Give peace a chance.

 John Lennon

Holly holy

Barking Pumpkin Records logoThe image to the right was the logo of Barking Pumpkin Records, a record label created by Frank Zappa in the early 1980s. The logo shows a pumpkin barking at a cat, and the cat exclaiming two Chinese syllables in response.

Let’s look at the Chinese characters here: 聖糞

A friend once asked a Chinese-speaking colleague what those two syllables mean, and the colleague hesitated, then responded, somewhat embarrassed, They mean... sacred... dung.

Or, in more idiomatic colloquial English: Holy shit!

The other day, I read a blog post (or perhaps it was a comment to a post), in which the writer typed Holly shit! With two ls. After shaking my head and saying, Moron, I wondered whether the guy might have more company in Morontown than we’d like to think. And so I asked Google...

...and I saw almost 85,000 hits (along with a suggestion for the better way to spell it). 85,000 web references that think holy has two ls. Sample text: HOLLY SHIT!!!! The Hippie movement was created by CIA.

Checking further, I found almost 31,000 references to wholly shit (sample, Wholly Shit They Found A Nuke In Iraq). But take heart: I see only about 7,000 references to holey shit (sample, Holey shit the achievements are so easy to obtain.), so there are limits, after all.

Friday, October 08, 2010

.

What was that you said?

I’ve just had a trip to Washington, DC, for some meetings. I always like visiting DC, and get there at least once or twice a year. I don’t really consider it much in the way of travel, and I generally take Amtrak — not cheap, but no more expensive than flying, and much less hassle.

At the DC end, I also use the trains: the Metro system, their subway. It’s clean, it’s efficient, and it’s nice to use. It’s a bit more expensive than New York’s subway — New York still has a fixed fare, currently $2.25, to go anywhere in the system; DC uses a farecard system that charges your card based on how far you went, and during peak periods a trip downtown from the outskirts can cost $5 or more.

The train operators announce the stops as they go, making their announcements live each time, as the trains go back and forth. You might hear, for example, something like this:

Ness stah Huntuhn lass stahna yell-lie dorsopanna righ.

Elocutions such as that are intelligible only to locals (and I’m sort of a local, as a former resident, though I haven’t lived there for 22 years). Here’s the translation:

Next stop, Huntington, last stop on the Yellow Line. Doors opening on the right.

The thing is, the locals mostly don’t need the announcements, and the visitors have little hope of understanding them. And it’s not because the audio system is bad, but that when human operators have to repeat the same things over and over, they tend to get less than enthusiastic about enunciating it. Also, they may have accents that make it hard to understand them. And we won’t even mention how they tend to pronounce L’Enfant Plaza.

I’ve always wondered why they don’t get someone to record all the regular station announcements, and then just have them play at the appropriate times. The human operator can kick off the playback, or the system can even do that automatically, as happens in many other local transit systems. It seems that it would be clearer and easier for everyone, and would save the operators’ voices.

If the operators really wanted to talk to the passengers, they could certainly do, say, extemporaneous comedy in between the recorded announcements.

Thursday, October 07, 2010

.

Least common denominator

For another comment about something a recent speaker said, we look at the guy yesterday who made a reference to least common denominator, and included a graphic that showed the fraction 9 / 12, then displayed it as 3*3 / 4*3, and concluded with 3 / 4. There are two problems with the graphic.

One is that it’s gratuitous. It has nothing to do with the colloquial meaning of least common denominator, which doesn’t relate to fractions or mathematics at all. In English rhetoric, it refers to a common kernel that can serve or satisfy everyone involved. Alternatively, it can be used disparagingly to refer to someone or something from which every distinguishing and distinguished characteristic has been removed, leaving only a common bit that’s dull and useless.

Some presenters seem to like sticking graphics on every Powerpoint slide they show — sometimes several per slide — whether or not the graphics add anything to the understanding of the slides. Presenters who do that think the graphics make their presentations snazzier.

They don’t.

But the other problem with the graphic is from a mathematical point of view: it’s not illustrating the concept of least common denominator at all. It’s an illustration of greatest common factor. When we reduce a fraction, as in the graphic, we find the greatest common factor of the numerator (the top of the fraction) and the denominator (the bottom) — the largest number we can find that goes evenly into both numbers, that divides both numbers with a remainder of zero. When we cancel that greatest common factor out, what’s left is the fully reduced fraction.

We use the least common denominator to compare (or add or subtract) two or more fractions.

Which is greater?: 5 / 12 ... or ... 9 / 20 ?

To answer that using fractions, we need to convert them into fractions with a common denominator, and we customarily use the least common denominator — the smallest number that is a multiple of both denominators. In this case, 12 = 4 * 3, and 20 = 4 * 5, so the least common denominator would be 4 * 3 * 5 = 60. Multiply both the numerator and denominator by the same amount, and we get 5 / 12 = 25 / 60, and 9 / 20 = 27 / 60. And, so, because 25 is less than 27, 5 / 12 is less than 9 / 20. And the difference between the two is (27 - 25) / 60 = 2 / 60 = 1 / 30 (which we reduced by finding the greatest common factor of 2 and 60).

I have no quibble with the colloquial use of least common denominator as a language idiom, with a meaning that doesn’t relate to the mathematical one (though I do think the usage is trite). But when you bring mathematics into it, please get the maths right.

Wednesday, October 06, 2010

.

Implicit license

A speaker the other day said something curious, and he repeated it. Several times throughout his talk, he said that most of the information he was giving us is available free of charge on our web site. It’s curious, because his need to say it strays from the model we’ve developed of the worldwide web. That he said it — and not once, but several times — made him seem old fashioned, almost as thought he’d told us to send a self-addressed stamped envelope to get a copy of his presentation.

The part that’s out of place is free of charge, because we assume that now. Of course it’s free of charge: it’s on your web site. The web is a place full of free information, and we take that as the default situation.

Not everything there is free, to be sure. Lots of journals put the papers they publish behind paywalls, and some magazines and newspapers demand subscriptions for ready access to their material. But we mention that when it shows up, because paying is now the exception. We assume information on the web is free unless we’re told otherwise.

On the other hand, we do expect that the information remains the property of those we got it from, unless they say otherwise, at least some of us do. We wouldn’t use someone else’s words without attribution, someone else’s research, someone else’s design, without permission. We might even expect to pay for the use, depending upon what it is we’re using, and for what purpose. And we have lots of discussions about fair use in the process.

But what, exactly, is our fair use of material that’s provided free?

It’s a difficult question, and one with no clear answer. The U.S. fair use doctrine isn’t well defined, and courts take it to mean different things in different situations and at different times. Even cases that are obvious might not be so, depending upon who’s making the judgment. Associated Press tried, to derision and laughter from many professional and amateur bloggers, to limit fair use of their material to fewer than five words. They later said that wasn’t what they’d meant, maybe.

Righthaven, on the other hand, decided not to make any definitions. They just made lawsuits:

In a strategic campaign that is attracting growing interest nationwide in legal and media circles, Righthaven — without warning — has sued at least 86 website owners in federal court in Las Vegas since March for copyright infringement.

[...]

But from the get-go, Righthaven hits copyright violators with lawsuits seeking $75,000 in damages and forfeiture of their website domain names.

Righthaven’s legal initiative has critics calling it a frivolous-lawsuit-and-shakedown campaign aimed not at gaining justice for Righthaven, but at putting money in its pockets — charges denied by Righthaven and its entrepreneurial CEO, Las Vegas attorney Steven Gibson.

[...]

Righthaven’s procedure has been to troll to find an infringement of an R-J copyright to a specific story. It then buys the copyright for that story from the R-J’s owner, Stephens Media LLC, and afterward sues the infringer.

Buying the copyright is an important step because it allows Righthaven to seek statutory damages. (Some of the defendants are arguing that Righthaven lacks standing to sue them because Righthaven didn’t own the copyrights at the time of the initial infringement.)

These are clearly predatory tactics: a good-faith approach, if one really thinks a non-commercial user such as a blogger has overstepped the fair-use line, would be to ask them to take down the offending material. Starting with a large lawsuit is clearly just a way to monetize things.

Now the Electronic Frontier Foundation is calling them on it, having filed a countersuit that claims copyright fraud:

The owner of the Las Vegas Review-Journal has for the first time been hit with a counterclaim over its online copyright infringement lawsuit campaign, with attorneys for the Electronic Frontier Foundation accusing the newspaper of entering a sham relationship with the Review-Journal’s copyright enforcement partner Righthaven LLC — and accusing Righthaven of copyright fraud.

[...]

The Electronic Frontier Foundation (EFF) says the lawsuit campaign threatens freedom of speech on the Internet as Righthaven generally sues without first asking that infringing material be removed from websites or be replaced with links as is the standard practice in the U.S. newspaper industry.

A couple of weeks ago, Groklaw published an interesting analysis of one of the cases, where a defendant got a default judgment set aside and will be allowed to plead his case. It’s particularly interesting because this particular defendant has a situation where an entire column from the newspaper was copied — something that clearly ought to go beyond fair use. Yet the judge, in setting aside the default judgment, ruled that the planned defense is sufficiently reasonable to be heard.

The reasonability of it rests on a few points:

  1. The article in question is informational, rather than creative or artistic.
  2. The use was non-commercial.
  3. The article was available for free from the newspaper’s web site, so financial loss to the newspaper could not be great (there might be loss of advertising revenue, resulting from fewer visits to the paper’s web site).
  4. The defense claims that the newspaper gave an implicit license to use the article, by posting it freely on their web site. The judge quotes John S. Sieman, from his paper Using the Implied License to Inject Common Sense into Digital Copyright:
    As the internet has developed into more of an opt-out system, the argument has been made that only the act of sharing information from websites that actively choose to be removed from participating in the system is generally recognized as unacceptable, despite the Copyright Act being an opt-in system.

It’s that last point that’s the most interesting one, and we should follow this case to see where it goes. The defendant has only won the right to argue his case, and he could still lose in the end.

Tuesday, October 05, 2010

.

What’s cooking?

Gourmet dinner menu last night, at a meeting. The theme was Re-imagining American Food.

MUNCHIES
LIQUID OLIVE
BAGELS & LOX
    R.H. COUTIER Ambonnay
Champagne, France N.V.
FLAVORS & TEXTURES
ORGANIZED CAESAR SALAD
OYSTER & UNI
    ABBAZIA DI NOVACELLA Kerner
Alto Adige, Italy 2009
SCALLOP
CIGALAS EN BRIOCHE
    JEAN-CLAUDE BACHELET
Les Macherelles Chassagne
Montrachet, France 2007
PHILLY CHEESESTEAK
VEAL CHEEK TZATZIKI
    ROBERT GROFFIER
Les Hauts Doix, Chambolle Musigny
France 2007
DESSERT
RUM CAKE    EL DORADO Rum 15 yr, Guyana
SWEET SURPRISES
SAFFRON GUMDROP
BACON AND CHOCOLATE
     

I don’t know what the liquid olive was, but it was a soft, squishy ball, olive coloured and tasting like an olive, served in a spoon.

The bagels & lox was a tiny, thin crêpe rolled into a cone filled with dill crème and salmon eggs, served in a small cup full of poppy and sesame seeds.

The organized Caesar salad was romaine, dressing, and shaved cheese rolled sushi-roll style in slices of jicama, served with a crêpe-like crouton with a quail-egg yolk nestled in it.

The scallop was pan-seared and served with a slice of roasted cauliflower and an aioli, and the cigalas en brioche was a piece of langustino in a coating of a flourless brioche, which amounted to an egg-white coating.

The Philly cheesesteak was a very small baguette filled with a tasty brie-like cheese, with slices of cured beef arranged on top.

The veal cheeks, we were told, were braised for 72 hours (um, that’s three days, isn’t it?

The run cake was relatively normal, and the bacon and chocolate was just what you might think: chocolate-covered bacon strips.

A very unusual and interesting meal.

Monday, October 04, 2010

.

A couple of things about Stuxnet

There’s a relatively newly discovered (within the last few months) computer worm called Stuxnet, which exploits several Windows vulnerabilities (some of which were patched some time ago) as it installs itself on people’s computers. It largely replicates through USB memory sticks, and not so much over the Internet (though it can replicate through storage devices shared over networks). And it’s something of an odd bird. Its main target isn’t (at least for now) the computers it’s compromised, and it’s not trying to enslave the computers to send spam, collect credit card numbers, or mount attacks on web sites.

It’s specifically designed to attack one particular industrial automation system by Siemens, and it’s made headlines because of how extensive and sophisticated it is. People suspect it’s the product of a government, aimed at industrial sabotage — very serious stuff.

The folks at F-Secure have a good Q&A blog post about it.

There are two aspects of Stuxnet that I want to talk about here. The first is one of the Windows vulnerabilities that it exploits: a vulnerability in .lnk files that kicks in simply by having an infected Windows shortcut show its icon:

This security update resolves a publicly disclosed vulnerability in Windows Shell. The vulnerability could allow remote code execution if the icon of a specially crafted shortcut is displayed. An attacker who successfully exploited this vulnerability could gain the same user rights as the local user. Users whose accounts are configured to have fewer user rights on the system could be less impacted than users who operate with administrative user rights.

Think about that. You plug in an infected USB stick, and you look at it with Windows Explorer. You don’t click on the icon, you don’t run anything, you don’t try to copy it to your disk... nothing. Simply by looking at the contents of the memory stick (or network drive, or CD, or whatever), as you look at its icon and say, Hm, I wonder what that is. I’d better not click on it, it’s already infecting your computer. And since most Windows users prior to Windows 7 ran with administrator rights, the worm could get access to anything on the system.

You need to make sure this security update is on your Windows systems.

The other aspect is interesting from a security point of view. From the F-Secure Q&A:

Q: Why is Stuxnet considered to be so complex?
A: It uses multiple vulnerabilities and drops its own driver to the system.

Q: How can it install its own driver? Shouldn’t drivers be signed for them to work in Windows?
A: Stuxnet driver was signed with a certificate stolen from Realtek Semiconductor Corp.

Q: Has the stolen certificate been revoked?
A: Yes. Verisign revoked it on 16th of July. A modified variant signed with a certificate stolen from JMicron Technology Corporation was found on 17th of July.

I’ve talked about digital signatures before, at some length. When the private keys are kept private, digital signatures that use current cryptographic suites are, indeed, secure. But...

...anyone who has the private key can create a spoofed signature, and if the private keys are compromised the whole system is compromised. When one gets a signing certificate, the certificate file has both private and public keys in it. Typically, one installs the certificate, then exports a version that only contains the public key, and that certificate is made public. The original certificate, containing the private key, has to be kept close.

But it’s just a file, and anyone with access to it can give it to someone else. Shouldn’t, but can. If you can compromise an employee with the right level of access, you can snag the private key and made unauthorized authorized signatures.

In most cases, it’s far easier to find corruptible (or unsophisticated) people than it is to break the crypto. And if the stakes are high enough, finding corruptible people isn’t hard at all. The Stuxnet people may well have a host of other stolen certs in their pockets.

Sunday, October 03, 2010

.

Quote of the week

From The Tenth Inning, we have Barry Bonds telling reporters that he didn’t care about being booed as he was close to breaking Hank Aaron’s career home-run record:

You gotta have a lot of talent to have 50,000 people shout that you suck.

He does have a point there. They booed him, but they came to see it anyway.

[Paraphrased, because I didn’t write it down soon enough to remember it exactly.]

Saturday, October 02, 2010

.

Who is talking about heart health?

On yesterday’s Brian Lehrer Show, a talk show on the local public radio station:

Joy Behar on Heart Health
Friday, October 01, 2010

Joy Behar, comedienne and co-host of The View, discusses heart health and the Mom’s Second Chance campaign.

That’s nice. But, here: a comedienne/host of a fluffy morning TV talk show... discusses heart health. Serious credentials, there, wot?

How ’bout we get, oh, I don’t know, maybe a cardiologist to come on and discuss heart health? Yes, that sounds better.

Friday, October 01, 2010

.

On compression and sound quality

Actor and filmmaker Adrian Grenier was on a local radio talk show the other day. As an addendum to the show, they posted a brief Q&A on the web, in which he says this:

Q: What are you listening to right now?

A: I just reunited with my record collection. Records sound better than MP3’s. I was just listening to Toots and the Maytals on vinyl.

Now, there’s certainly been a lot of debate about whether analogue sounds better than digital, when it comes to recorded music. When you hear live music, the vibrations reach your ears, your ears pick them up and send them to your brain through your nerves, and you hear exactly what someone sitting at that spot (and with your hearing capabilities) will hear. It’s perfect, in the sense that you can’t get more like really being there than... really being there.

Any recording provides a different experience, and whether that experience is better than the live one depends upon a lot of things, including where you were sitting and where the microphones were, how much extraneous stuff was heard by each (you and the microphones), and so on... along with the social experience and energy of being there to see it.

That said, there have always been those who say that digital recordings sound digital, changing the sound in unpleasant ways. Music is recorded on a CD, for example, by sampling the actual sound at frequent intervals (about 44,000 times a second), and by encoding the sampled sounds as numerical values (16-bit numbers, for CDs). The choice of the frequency of the sampling and the number of values (number of bits) used in the encoding affects the maximum quality of the sound.

I’m not going to get into the debate, here, and I’ll only note that since CDs have given way to other ways of listening, we’ve increased both the sampling rate and the number of bits per sample in some recordings. Whether or not CDs sound good, there’s better digital source material out there.

MP3 files, though, are not original source material: they’re compressed from the original, and their quality can vary greatly. Let’s look at why.

Broadly, there are two kinds of compression: lossless, and lossy. We use lossless compression in computer work all the time, such as when we make ZIP files. To do compression losslessly, we take advantage of redundancies, repetitions, and nearness that show up naturally in data, use alternative representations for common sequences, and that sort of thing. Any lossless compression algorithm works best on the kind of data its designed for, and doesn’t work well on certain other kinds.

Here, for instance, is a lossless algorithm I’m making up as I type this, designed for compressing English text, text consisting of letters, numbers, and a few punctuation marks and symbols:

In normal English text (US-ASCII), each character is represented by one byte. We know that the most common 11 letters in English are, in order, e, t, a, o, i, n, s, h, r, d, and l. So we’re going to represent each of those, plus the space character, with a half-byte instead of a full byte (shown here in binary, for clarity):

0000 = (space), 0001 = e, 0010 = t, 0011 = a, 0100 = o, 0101 = i,
0110 = n, 0111 = s, 1000 = h, 1001 = r, 1010 = d, 1011 = l

That leaves 15 lower-case letters, 26 upper-case letters, and 10 numerals to be represented, and we can introduce one-byte patterns with a half-byte of the form 11xx, unused above. We’ll reserve 1100 0000 for now, and assign the remaining one-byte patterns (11xx xxxx, where the x’s are not all zeroes) arbitrarily to those characters and two the 12 most common punctuation marks:

1101 0000 = b, 1101 0001 = c, 1101 0010 = f, 1101 0011 = g, ...,
1100 1111 = z, 1101 0000 = A, 1101 0001 = B, 1101 0010 = C, ...,
1110 1001 = Z, 1110 1010 = 0, 1110 1011 = 1, 1110 1100 = 2, ...,
1111 0011 = 9, 1111 0100 = (comma), 1111 0101 = (period), .....

Finally, we’ll represent anything else by using our reserved 1100 0000 as an escape, so that the byte immediately following it represents its normal US-ASCII value:

1100 0000 0010 0100 = (dollar), 1100 0000 0010 0101 = (percent),
1100 0000 0010 0110 = (ampersand), ....

Therefore, our system represents the space and the eleven most common letters in half the normal number of bits (4), and the remaining letters and numerals, along with twelve punctuation marks, in the normal number of bits (8)... but takes twice the number of bits (16) to represent anything else. It would be horrible for music files, which are made up of arbitrary binary data and which would get much bigger if put through this compression. But for plain English text, here’s an example, using hexadecimal notation to be more concise:

Original: This is a compressed string.

US-ASCII: 54 68 69 73 20 69 73 20 61 20 63 6F 6D 70 72 65 73 73 65 64 20 73 74 72 69 6E 67 2E

Our algorithm: E3 85 70 57 03 0C 24 C7 C8 91 77 1A 07 29 56 C4 F5

We’ve reduced the string from 28 bytes to 17, and it’s fully reversible (once we’ve dealt with padding needed when we end in the middle of a byte, but that’s easy enough and we don’t need to get into that here).

OK, that was fun to play with, but what about compressing music?

We can’t rely on common byte values for music, because the value of a given sample can be anything — silent, super-loud, or somewhere in between. But we can rely on the fact that in normal music, sounds don’t come in and go out instantaneously, and, therefore, adjacent samples are most likely to be relatively close to one another. If we optimize the algorithm for aspects like that, we can get fairly efficient compression. We can even get lossless compression to a point.

For example, suppose we’re using 32-bit samples, but we say that if we have a 32-bit sample and the next sample is within 15 bits of that one, we can instead use a 16-bit value that represents plus (first bit 0) or minus (first bit 1) from the reference sample. The next sample could similarly be coded as a delta from the second, and so on. We’d have to do some futzing around to signal that we’d gone back to a full 32-bit sample again, and we’d probably want to do that periodically, whether we need to or not, to set up resync points in case something goes wrong with the data streaming. But this is not an ideal nor complete mechanism... just the beginnings of an example.

But for the high levels of compression that we need to use to turn music or video into tolerably sized files, we need to go for lossy compression methods. That means that we can’t turn the compressed file back into the full version, because some information has been lost in the process. And information loss means quality loss — the compressed file is no longer a faithful copy of the original, and any playback is measurably different from the original.

But is it noticeably different from the original?

That, of course, depends upon how sensitive one is. Still, while the difference between CDs and vinyl records could be (and was) hotly debated, this one’s pretty straightforward: the difference is, in general, discernible on good equipment. There are a lot of compromises on the way from ten megabytes or more per minute to one megabyte or less per minute. That heavy compression is what makes it possible for us to put entire music collections in our pockets, so it’s worth it to many.

And when we’re putting the music in our pockets and listening to it with ear buds as we ride the subway and walk down noisy streets, we’re not noticing the digital and compression artifacts, the reduced frequency range, the lower sound quality. It’s entertaining us and keeping the world of chatty commuters and jackhammers and car horns way.

With docking stations, though, we’ve brought that model into our living rooms, and we’re often listening to MP3 files at home, through loudspeakers the size of a paperback book. Where we used to show off audiophile equipment stacked up on shelves and massive speakers that dominated the room and rattled the walls with great sound, we’re now boasting about how compact it all is.

It’s compact, though, at the cost of sound quality, and Adrian Grenier is going back to vinyl.