Are you smarter than a high school senior?

According to a survey commissioned by a group called “Common Core”, 17-year-old American kids don’t know much about history or literature:

Fewer than half of American teenagers who were asked basic history and literature questions in a phone survey knew when the Civil War was fought, and one in four said Columbus sailed to the New World some time after 1750, not in 1492.

The survey results, released on Tuesday, demonstrate that a significant proportion of teenagers live in “stunning ignorance” of history and literature, said the group that commissioned it, Common Core.

[...]

In the survey, 1,200 17-year-olds were called in January and asked to answer 33 multiple-choice questions about history and literature that were read aloud to them. The questions were drawn from a test that the federal government administered in 1986.

About a quarter of the teenagers were unable to correctly identify Hitler as Germany’s chancellor in World War II, instead identifying him as a munitions maker, an Austrian premier and the German kaiser.

On literature, the teenagers fared even worse. Four in 10 could pick the name of Ralph Ellison’s novel about a young man’s growing up in the South and moving to Harlem, “Invisible Man,” from a list of titles. About half knew that in the Bible Job is known for his patience in suffering. About as many said he was known for his skill as a builder, his prowess in battle or his prophetic abilities.

Pretty sad, eh?

Well, but...

Dates have always been a problem, and rote memorization of dates is perhaps the dullest thing one can do in trying to learn history. This isn’t anything new: when I was young, we knew the Columbus year from a mnemonic rhyme, “In fourteen hundred ninety two / Columbus sailed the ocean blue.” Maybe it’s easy to substitute “eighteen hundred”, and get it wrong. In any case, my guess is that a good many of my classmates couldn’t have told you much about the Civil War either, and certainly nothing involving dates. Lincoln, slavery, that’d have been about it. A few buffs might have come up with names like Sherman and Grant.

The Hitler question shows the traps that can come in multiple-choice questions — especially done over the phone, where one can’t look the answers over. Is it really surprising that the kids might not know the difference between the positions of chancellor and kaiser? Might it not be that some who picked “Austrian premier” knew that, in fact, Hitler was born in Austria and lived most of his life there?

And what’s the deal with the “Job” thing? Were we supposed to learn that in high school? I must have missed the Bible classes in my school.

To take it a little farther, we see that these kids do know some things:

The history question that proved easiest asked the respondents to identify the man who declared, “I have a dream.” Ninety-seven percent correctly picked the Rev. Dr. Martin Luther King Jr.

About 8 in 10, a higher percentage than on any other literature question, knew that Harper Lee’s “To Kill a Mockingbird” is about two children affected by the conflict in their community when their father defends a black man in court.

All right! They are paying attention now and then.

Anyway, to what does Common Core attribute this “stunning ignorance”? NCLB, of course:

The group says President Bush’s education law, No Child Left Behind, has impoverished public school curriculums by holding schools accountable for student scores on annual tests in reading and mathematics, but in no other subjects.

[...]

In a joint introduction to their report, Ms. Cortese and Dr. Ravitch did not directly blame the No Child law for the dismal results but said it had led schools to focus too narrowly on reading and math, crowding time out of the school day for history, literature and other subjects.

It’s a different spin from the usual NCLB complaint: they don’t have a problem with testing, but with the fact that the tests don’t cover history and literature. By implication, they seem to want more tests.

OK, now, anyone who reads these pages regularly knows that I’m no fan of NCLB. I think it’s the wrong plan, going in the wrong direction, implemented in the wrong way. In short, it sucks.

But this ridiculously flawed survey is no indictment of NCLB, nor of the students surveyed, nor of their teachers. It reflects badly only on whoever designed the survey, at least judging from what information the Times article gives us.

First, I’ll point out that NCLB is six years old and tests students in grades 3 through 8. Even if we assume that curriculum changes aimed at the NCLB tests were implemented immediately, for the 2002/2003 school year, that’d mean that high school seniors today were affected by it only in 7th and 8th grades, and have had three full high-school years since then to pick up the history and literature lessons.

Second, in order to claim that the kids are doing badly in history because their focus is on math, one would need a survey that also asks them math questions, at least.

Third, in order to blame any deficiency on NCLB, one would need a baseline survey to do a “before” and “after” comparison. If we found that, say, 60% of the kids did well on both the math and the history questions before, and now 80% do well on math and only 40% do well on history, then we might be able to make such a guess at the cause.

Finally, no evaluation like this that’s done over the phone can give really useful information. Telephone polling for opinions and preferences is one thing, but to check knowledge... no, it doesn’t work. It’s too rushed. And if the multiple-choice answers are too similar, there’s no chance to look them over for selection; if they’re too different, the correct answer is too obvious (suppose the choices for Hitler, for example were “German chancellor”, “Egyptian president”, and “Chinese war minister”).

Maybe high school seniors today are more ignorant than we were in my day. But if you want to claim that, don’t just give me a silly publicity stunt for your organization. Show me a properly designed study that can really back it up and can help show us how to fix it.