Tuesday, April 03, 2007


The superprogrammer

Early in my career — at the very start, actually, at a time when new hires were put through a training class — I was given a copy of a book by Ed Yourdon, “Techniques of Program Structure and Design”, now long out of print. In one chapter, he talks about the qualities of a good programmer, and what qualities a “superprogrammer” might possess, and brings it into focus with an amusing story. It's a well-travelled story and most of my readers may have heard it before, but here it is anyway, some 30 years later:

During the past several years, I have had the opportunity to teach advanced programming courses to thousands of students in several different countries around the world. The students, in general, have been experienced programmers in banks, insurance companies, government agencies, manufacturing organizations, scientific installations, universities, and every other conceivable background. More for my own amusement than anything else, I have often begun each course with the question, “What are the qualities of a good programmer?” The answers have been as varied as the students' backgrounds, and some of them are worth repeating:
  • A good programmer writes good (efficient) (well documented) programs.
  • A good programmer works well with other people.
  • A good programmer communicates well with the users of his program.
  • A good programmer takes a bath at least once a week.
  • A good programmer shows up for work on time.
  • A good programmer never shows up for work on time.
  • A good programmer doesn't cause trouble.
  • A good programmer works well under pressure.
  • A good programmer likes classical music.

An argument often breaks out about the first answer to the question. A good programmer is anyone who writes good programs. It is pointed out that managers are usually required to evaluate the “goodness” of a programmer, yet they seem to be singularly incapable of determining the quality of a good program. We will see during this chapter that none of us is really in a position to quantitatively determine the quality of a program, so a manager should not be blamed too much in this difficult situation.

Nevertheless, it seems that some programmers have a reputation in their organization for being “superprogrammers”: The word will spread that Tom can turn out a large complex program in a single day, or that Alice always manages to debug her programs with a maximum of one test shot. Given this natural situation, I have occasionally amended my original question and asked my programming classes:

Is a superprogrammer (i.e., someone who can code faster than a speeding bullet, leap over reams and reams of printouts in a single bound, and generally out-program everyone else in the organization) looked upon with favor and respect by the management?

Though there is often a tremendous amount of heated debate and controversy on this point, it has been surprising how many people — especially programming supervisors and managers — have emphatically said “No!” A programmer in a large bank in Montreal put it rather well:

“If my programs are outrageously inefficient, so inefficient that even my manager can tell, then I'm in trouble. Similarly, if I take ten times longer to write a program than other people in my department, I will be in trouble. For the most part, though, my manager is more interested in my ability to function as a human being in a human organization; my relationship with the computer is my own business. My manager wants me to work reasonably regular hours, interface well with other programmers and computer users, and most of all, not to cause trouble.”

The same management attitude seems to be quite prevalent in large insurance companies, government agencies, and banks; less prevalent in medium-sized manufacturing organizations; and generally not true in universities, research organizations, and computer manufacturing companies.

To the extent that this management attitude is true, the subject of advanced programming, and indeed this entire book, may be rather academic. What is the point of writing the most efficient program in the world if you lack other traits that make you an accepted, if not popular, member of your organizations? Fortunately, the situation is generally not quite that extreme. Management would like to see programs with all the “good” qualities that we will be discussing later in this chapter. Still, it is more important to remember that they are often not willing to tolerate the popular image of the “computer bum” for the sake of obtaining highly efficient programs.

Note that there are different reasons for disliking the so-called “superprogrammer.” Some superprogrammers can develop working programs very quickly, or can write extremely efficient programs — but they are undocumented, impossible to understand, maintain, or modify. On the other hand, there are some superprogrammers who turn out truly superlative code — and yet they are unsociable or, in the words of one manager, “a bit like Allen Ginsberg.”

Unfortunately, many programmers seem to write comments as personal messages to themselves, that is, to remind themselves of the purpose of the particular instruction or program statement they used. The personal note though, may be completely indecipherable to anyone else, and even the original programmer may have difficulty understanding the meaning of the comment (as well as the statement that it accompanies, of course) at some later time.

An interesting example of this occurred several years ago when a lone superprogrammer single-handedly developed a FORTRAN II compiler for a well-known computer manufacturer. After he tested the compiler and turned it over to his manager, the superprogrammer disappeared for several days,[1] during which time, the manager discovered that there were some bugs that required immediate attention. The junior programmer assigned to find and fix the bugs discovered, to his horror, that the entire compiler contained only one comment, which accompanied an octal constant in the following manner:

CONST23: 3443 ; R.I.P.L.V.B.

Since the superprogrammer had a reputation for brevity as well as brilliance, the junior programmer began to think that perhaps this single comment would unlock all of the mysteries of the compiler. After several hours of pondering the meaning of the comment, he finally hit on the answer — the number 3443, in octal, is equivalent to the number 1827 in decimal. Being a classical music fan (recall the earlier list of characteristics of a good programmer) and a collector of trivial information, the junior programmer happened to remember that 1827 was the year Beethoven died! As one might imagine, the manager was quite unamused by all of this, and when the superprogrammer reappeared, he was asked to take his inestimable talents elsewhere.

[1] It was later discovered that the superprogrammer had read in his local newspaper that a Harvard undergraduate student had ridden every New York City subway line (including all branches of the IRT, IND, and BMT lines) on a single subway token in a record time (since nobody had ever done it before) of 48 hours.

An MIT graduate himself, the superprogrammer saw a chance for some healthy intercollegiate rivalry; so he wrote a program to calculate the fastest possible path through the subway lines, and organized a team of friends to attack the problem in a scientific fashion.

The program, written in LISP and using some very complex heuristic algorithms to solve what amounted to a variation on the traveling salesman problem, ran out of memory on the night before the team was to travel to New York and attack the subways. Faced with a choice of optimizing an individual subway line, or “suboptimizing” the entire transit system, they chose to partially optimize their entire route.

It was never determined whether the program actually worked, for after 27 hours on the subways, one team member fell asleep at a critical subway stop; as the other team members raced back into the train to retrieve their comrade, their connecting subway left without them — and they ultimately lost to the Harvard student by 5 minutes. Needless to say, his manager was not amused by all of this.


—— Ed Yourdon, circa 1976


scouter573 said...

I was an electrical engineering student in college when I took a computer science course in assembly language programming (for the CDC6000 series). After some early MIX programming (see Knuth), we went into COMPASS, the CDC assembler. It was the main project of the course to develop a instruction-level emulator for a 24-bit 2s-complement stack machine. Those familiar with the CDC architecture will immediately note that Seymour Cray liked 60-bit 1s-complement arithmetic and never saw a stack he liked. Naturally, the challenge was to keep track of all the cross-architectural details as we added instructions and features throughout the project. Being a computer weenie, I greatly enjoyed the project and completed the phases well before each was due. On (my) final day of development, I was in the basement of the CS building when my final - and correct - test run came thru with flying colors. I gave out a yelp of success and started packing up to leave. I was very proud of the fact that each function and macro had a block of comments describing it, and key lines through the listing were individually commented. A nearby classmate looked up with a tired, haggard look and asked if I would help him get thru "the last bug". Of course I would, just show me the listing and describe the failure symptoms. He opened up a thick listing with two columns of text - the instruction op-code and the operands. Completely and utterly free of commentary adornment. In a soft voice with a stunned look on my face, I asked where the comments were. Oh, I add them after I get the program working, was the quick answer. Sorry, man, I can't help without comments. Good luck. And I left.

scouter573 said...

It's tangential to the Yourdon comment, but there's an article at news.com (here) that reviews the 30-year history of the x86 architecture. The 8086 came out roughly about the time I was writing that emulator in COMPASS. Anyway, I often speculate that many, many of the CPU cycles used in the world are spent moving data from one register to another on the x86, or working around some oddity of the x86, and not spent solving the user's problem. Imagine how far along we'd be had someone chosen a more modern, more regular computer architecture for the IBM PC back then. But he or she didn't, and we're not.