SigSoft93 Reviewer Comments
Cox MON Java Perl

Reviewer Comments

Specialization of Labor as a New Foundation for Software Engineering (renamed Market Processes as ... during the review process

Even though the Call for Papers specifically asked for papers about new foundations, this paper received the most vitrioloic reviews I've ever received.

I interpreted the venom to mean that the paper actually hit the nerve I aimed at. And concluded that to make any headway I should aim at the gut, not at the brain. Establishment academia's brains are too well fortified for mere papers to ever make a dent. No doubt that's why so many papers keep restating the same old conventional wisdom.

What do you think? Email your comments to bcox@virtualschool.edu.

University of Washington; SEATTLE, WASHINGTON 98195; Department of Computer Science and Engineering, FR-35 ; (206) 685-798/FAX (206) 543-2969; notkin@cs.washington.edu

June 24, 1993
I am sorry to inform you that your paper, Specialization of Labor as a New Foundation.., has not been accepted for publication in ACM SIGSOFT-93 Symposium on Foundations of Software Engineering. Ninety-three papers were submitted and 18 papers were accepted. With only a few exceptions, each paper was read and evaluated by at least three program committee members; enclosed you will find their ratings and comments.

I'm sorry that we didn't accept your paper this time, but I hope you will consider attending the conference and submitting your work again in the future

If you have any questions, please don't hesitate to let me know.

Sincerely yours; David Notkin; SIGSOFT-93 Program Chair; Associate Professor

Reviewer 1: Brilliant

Brilliant! Too bad the Japanese invented it. Thls paper presents an idea for marketing software that has the effect of encouraging it to be well engineered. The idea is simple: treat software like records, tapes, songs in general. A use fee is paid for every time an item is used. The use of subitems is then paid for by the builder of the superitem. This encourages the distribution of software -- no one tries to protect it. A little box has to be added to machines that collect the data and send reports (via modem) for subsequent billing. This encourages the development of small-sized, well-engineered items that fit into an open system environment. I love it.

Place a number for each item below corresponding to your evaluation. (5=outstanding, 4=high, 3=medium, 2=fair, 1=poor)
Confidence in your evaluation: 4
Interest/Importance to SIGSOFT-93  5
Quality of presentation: 5
Technical contribution of paper: 2
(5=strong accept, 4=accept, 3=maybe, 2=reject, 1=strong reject)
Recommended action for paper: 5
[BJC: Does a little gremlin put one of these in each batch to keep authors guessing?

Reviewer 2: It's a marketing paper.

This paper has nothing to do with the foundations of software engineering. It is a paper on marketing.

Confidence in your evaluation: 5
Interest/Importance to SIGSOFT-93: 0
Quality of presentation: 1
Technical contribution of paper: 0
(5=strong accept, 4=accept, 3=maybe, 2=reject, 1=strong reject)
Recommended action for paper: 1
Not enough equations and formal proofs of correctness, I surmise

Reviewer 3: It's a Crank Paper

This is a superficial paper, bordering on a crank paper. The idea of superdistribution is moderately interesting, but it is a second-hand description of a technology developed by others. The remainder of the paper is extremely technically shallow. The review of Kuhnian philosophy of science is similarly shallow, and irrelevant to the paper. The definitions of primitive vs advanced engineering (top of page 4) will classify manufacture of space shuttles as more primitive than manufacture of sewing machines. This apparently is an off-the-cuff discussion that has not been carefully thought through. Particularly annoying is the repeated assertion that software can be copied in nanoseconds-, which only points up that the author doesn-t know nanoseconds from milliseconds. This is a really bad, bad paper, and publishing it would be an embarassing way to start the FSE series of symposia.

Confidence in your evaluation: 4
Interest/Importance to SIGSOFT-93: 2
Quality of presentation: 2
Technical contribution of paper: 1
Recommended action for paper: 1
Space shuttles and sewing machines are equally advanced as I defined this term in the paper. Both are assembled from parts that other society-members provide. Examples of primitive organizations are hard to find today. The closest I can think of right now is the non-cooperative chaos of a line of refugees after catastrophic cultural breakdowns as in Rwanda. And the stone age organizational structure of software engineering projects.

Did the Ptolemaic astronomers disuade Copernicus by calling him a crank?

Reviewer 4: Read Mary Shaw

  1. The paper would benefit from Knowledge of and reference to Mary Shaw's Towards and Engineering Discipline of Software. In particular, her distinction between commercialization and engineering argues that true engineering requires a scientific basic, not just a market economy for the products that are engineered. I don't see any new scientific basis for the proposed paradigm shift (unlike the Copernican shift).

  2. The argument of the paper relies on the fact that the main impediment to radically improved abilities to build software is
  3. . But anyone who has tried to modify a large Smalltalk application built by someone else will soon realize that this is not the answer to our problems. Besides, how can one distinguish between such reuse of other's labor and the more common use of C libraries, for example?

  4. The claim that currently software is not a marketable commodity because it is so easy to copy appears to be contradicted by the obvious success of companies such as Microsoft.

Confidence in your evaluation: 3
Interest/Importance to SIGSOFT-93: 5
Quality of presentation: 4
Technical contribution of paper: 2
(5-strong accept, 4=accept, 3=maybe, 2=reject, 1=strong reject)
Recommended action for paper: 3
This comment that convinced me that these reviewers are immovable in their established paradigm, and that nothing I can do or say could ever move them. Shaws paper was printed adjacent to my Planning the Software Industrial Revolution paper in IEEE Software. I was one of her reviewers and objected to her faith in science then.

Perhaps I made a tactical error by not challenging her paper publically. But it is precisely her claim that needs to be attacked. Scientific principles cannot possibly applyl to a software world in which everything the experimenter encounters is entirely new, created from first principles by the programmer who built it.

Reviewer 5: Crisis? What Crisis? I don't see any crisis.

Interesting paper, but... This paper calls attention to an important economic aspect of software and is very interesting. However, my rule is always reject a paper that starts by pleading the software crisis. What software crisis? And how does this concept - superdistribution - solve it?

Confidence in your evaluation: 4
Interest/Importance to SIGSOFT-93: 2
Quality of presentation: 2
Technical contribution of paper: 1
Recommended action for paper: 2
I could do a whole paper about "The closing of the software engineering mind. This prig clearly didn't even read it, and was smug enough to say so in writing.

Tabulated Results

5=outstanding, 4=high, 3=medium, 2=fair, 1=poor

5=strong accept, 4=accept, 3=maybe, 2=reject, 1=strong reject


Virtual School Middle of Nowhere Brad Cox