[[ bjc: I attended a followon workshop arranged by Col. Stackhouse of Jet Propulsion Labs at U. Md. that was hosted by Vic Basili. Quite a spirit of consternation and desire for "somebody to do something" was expressed at this workshop. But as far as I know, nothing tangible came of it. ]]
o The Kyoto Prefectural Comprehensive Center for Small & Medium Enterprises
o The Kyoto Municipal Institute of Industrial Research
o The Advanced Software Technology & Mechatronics Research Institute of
Kyoto
o The Japan Institute of Invention and Innovation, Kyoto Branch
The building in which the workshop was held was the public center of KRP. It contains meeting rooms, dining facilities including a cafe and separate restaurant, and a large lecture hall that was used for the workshop sessions. The sessions had simultaneous translation in Japanese and English so participants could use their natural language. (The wireless receivers were German.) Last year's workshop was conducted entirely in English and this proved to be a deterrent to many of the Japanese participants.
The Joint System Development Corporation (JSD) was founded by 19 major information processing companies and about 100 related businesses in 1976. Its objective is to improve technical standards and to strengthen the Japanese information processing industry. It functions somewhat like the U.S. American Federation of Information Processing (AFIP). JSD focuses on
As a scientist who has been developing software throughout my career, I realized after listening for 2 days that the issues addressed here apply to a very different class of problems than those faced by myself and my peers. The best sense of this came to me while listening to the speaker from Mitsubishi, who claimed that his company produces more than 34 million lines of software (code) each year. Most of this is developed to run industrial processes such as manufacturing production lines. The main questions associated with this software are not if the algorithm is clever, efficient, or creative, but rather how to reduce the number of potential errors that large, multiperson software projects can produce. In fact the most important parameter discussed at this workshop was BKLOC, bugs (errors) per thousand lines of code. In this community there appears to be little place for the traditional questions about numerical methods that physicists, chemists, and engineers raise when thinking about computer programs. Thus, my attendance at this workshop was as an educated observer. I attended all the formal presentations but did not attend the working group sessions during the last 2 days. Nevertheless, it was possible to draw some very interesting conclusions.
FIRST, there was general perception that the Japanese (companies) were doing a better job than their U.S. counterparts in reducing error rates. Basili claimed that Japanese BKLOC rates were about one order of magnitude less than those in the United States, down below 1 BKLOC, and that Japanese software productivity was from 2 to 20 times that in the United States. He claimed that a major reason was that Japanese companies used better management techniques and were better at motivating people. (After listening to several talks by representatives of Japanese industry I concurred with this assessment.) Nevertheless, the gap between industry and research was greater in Japan than in the United States. Basili felt that entry level Japanese software developers are less well prepared than their U.S. counterparts, but it's the other way around after about 5 years of work. There was no disagreement to his statement that Japanese companies are doing a good job of training and retraining workers, so perhaps this accounts for part of this effect. (On a personal level I have been amazed to discover courses in Unix, mathematics beyond calculus, statistics, etc. on the educational channel of our home TV during prime time.) Professor Koji Torii (Osaka University), a co- host of the workshop along with Basili, echoed the remarks about the narrow pipeline between Japanese university research and industry. Incidentally, the excellent facilities at the workshop site, KRP, contrast strongly with the mildly threadbare look of some Japanese universities I have seen.
SECOND, I was surprised by the meager attendance and intellectual presence of U.S. industry, even considering the distance to Kyoto. The U.S. industrial participants who gave papers were from Mitre, Hughes Aircraft, NASA, CTA, and MCC. Agresti (Mitre) discussed how to assure quality in software being developed by others. Deutsch (Hughes) discussed an after-the- fact analysis he did of some of Hughes' projects while he was on sabbatical at CMU. McGarry (NASA) discussed work done to a large extent in collaboration with Basili at Maryland. CTA is a small contractor in Rockville, MD, and MCC is an Austin, TX, software think tank. Some other American companies attended but presented no papers. Representatives from AT&T Bell Labs (Columbus, OH, office only) and GenRad (Concord, MA) were there, but mostly to learn more about Quality Function Deployment (QFD), a method developed by Professor Tadashi Yoshizawa of Tsukuba University. QFD is well known and has been promoted by Yoshizawa since the 1970s. In his talk he indicated that a symposium on QFD was held in Novi, MI, in June 1989. Ohba (IBM Japan) gave a talk, but only about modeling the errors that were discovered after software was delivered. (I questioned his use of linear differential equations as being simplistic.)
On the other hand, Japanese industry was well represented. Papers presented by Oki Electric, NEC, Mitsubishi, Fujitsu, Nippon Steel, Sharp, and JSD gave very clear evidence that these companies are trying hard to learn how to improve software productivity. Papers by university researchers from both countries were mostly excellent, but I was struck by the fact that the Japanese are really working at reducing software errors. There was a strong emphasis on user requirements driving the design process (natural in any company that produces a product), a team approach to solving problems, developing methods that assure quality at every step of the software process, and quality control involving all departments including top management. The speaker from Nippon Steel said it very well, "We put stress on human motivation as well as machine's automation."
The paper on the FASET project by scientists from JSD was interesting to me because it represented one on the major projects undertaken by this group. I was disappointed with the progress they have made. Some other listeners seemed to feel similarly, questioning the use of inefficient LISP as an output language, wondering why logic programming (PROLOG) was not used, and why the tools were not better integrated. This work strikes me as being so high level that it suffers from a lack of concrete focus.
Professor Basili's work (UMD) on packaging software experience was very well received. His collaborations with NASA have apparently been jointly productive. Professor John Knight (University of Virginia) is a recognized expert on safety critical software, such as those used in nuclear power facilities, avionics systems, or dangerous medical appliances. He discussed the problems associated with designing software with failure rates of 10-9 per event or per hour. For extremely reliable hardware systems the usual approach is redundancy. This has been shown both by analysis and experiment to be effective. For software the same technique is often used, i.e., having different contractors develop software with identical specs. Unfortunately, Knight's research shows that the usual assumption of independence between failures of redundant systems is often invalid for software, because difficult parts of the design are likely to lead to errors whoever codes them. He concluded very succinctly, that he was "scared to death" when he thinks about some of the places such safety critical software are used.
McGarry's paper (NASA) was particularly relevant to me because he provided comparisons between software projects done using Fortran and Ada. He showed that, surprisingly, the percentage of time spent in the design, test, and code stages of a big project was about the same with either language. Further, the error profiles were also about the same, although Ada programs had fewer interface errors. The NASA approach to improving software quality lies in "understanding" many of the measures of software quality that have been developed over the past 15 years, "analysis" or defining relationships between the software process and software product, and "automation" or the development of software tools to improve quality. Much of the research has been done within the Software Engineering Laboratory (SEL) jointly supported by NASA, the University of Maryland, and Computer Sciences Corp.
Yamazaki (Nippon Steel) represented his company view that employee motivation and technology are like two wheels on the same axle--both must be equal diameter or straight line motion toward the company's goal cannot occur. A similar view was expressed by Masanori Teramoto (NEC). He estimated that NEC has approximately 15,000 people in 2,500 groups associated with software development. They have production design meetings, software quality control meetings, and evaluation and self-help groups. These involve all levels of company employees, including top management, and are very goal oriented. NEC estimates that in 1991 they will write 140 million lines of code and that BKLOC will be less than 0.1. Further, their reuse percentage (code that can be utilized in other applications without rewriting) will go from 50 percent in 1987 to 60 percent in 1991. As a side remark, he noted that Unix has helped increase their productivity because the same system is used across a wide variety of computers and hence less relearning was needed. My own experience is that Unix is a wonderful system for software hackers who love its flexibility and abbreviated command syntax and an awful system for the inexperienced who are easily confused by these same features.
The paper by Hino from Fujitsu seemed particularly useful for me, but did not generate much interest from most of the participants. The NAIVE language he discussed is higher level than C or Fortran but still has the flavor of "programming." As such it does not represent much new when viewed as a technique for assuring program correctness. It was pointed out during the discussion that it was still possible to write programs in NAIVE that were illogical and that it had no mechanism to insure that changes made in one section were propagated to other sections. I see NAIVE as an incremental improvement in the process of writing programs. Nevertheless, these are just the small steps the Japanese have proved to be very good at making.
There also did not seem to be much interest in formal methods, such as proof of correctness. Gerhart's paper struck me as rather defensive and the first question asked was if it was not hopeless to try to apply these techniques to large systems. My own experience is that formal methods have worked only on very small core sections of big programs and never have provided me with any new insights.
Most of the Japanese spoke in their native language; a few spoke in English that was difficult to follow. One or two read their papers but could not answer questions without a translator. All the people who spoke in English should be congratulated on their efforts. Of course, some Japanese spoke excellent English. The papers were entirely in English although a few were crude translations and sometimes text in the figures was not translated. None of the Americans attempted to lecture in Japanese nor did I detect any interest in learning to speak Japanese among the American scientists. CONCLUSION
There is not enough interaction between the sorts of computer scientists who would naturally attend meetings of this type and physical scientists who develop large scale engineering software. Both groups could benefit from closer contacts. On the basis of the talks presented here, some Japanese companies that write a great deal of software are ahead of those in the United States in managing the software development process. Managing and motivating people are important factors as are clear signals from the top that productivity enhancement tools are to be taken seriously.
Basili, V., "The Experience Factory: Packaging Software Experience" (University of Maryland)
Deutsch, M., "Predicting Project Success From the Software Project Management Process: An Exploratory Analysis" (Hughes Aircraft Co.)
Gerhart, S., "Back to Basic Principles--What Do Formal Methods Contribute to Software Construction?" (MCC)
Hino, K., "Software Quality Improvement by the Specification Language NAIVE" (Fujitsu)
Howden, W., "Verification of Complex Systems Using Incremental Operational Specifications" (University of California, San Diego)
Kawasaki, Y., T. Miyoshi, and T. Oishi, "The FASET Project, Construction of the New Software Development Environment" (Joint System Development Corp.) Knight, J., "Software Quality Where it Really Counts, Safety-Critical Systems" (University of Virginia)
Masuda, M., "Software Quality Target Control, Target and Evaluation" (Mitsubishi Electric Corp.)
Matsumoto, Y., "Introducing KyotoDB-1: A Computer-Aided Software Requirements Engineering Environment" (Kyoto University)
McGarry, F., "The Role of Measurement in Software Quality Improvement" (NASA)
Miyamoto, M., "Database Support for Software Quality Improvement" (Sharp Corp.)
Ohba, M., "Applying Software Reliability Growth Models" (IBM Tokyo Research Laboratory)
Sacchi, W., "A Framework for Modeling Software Evolution" (University of Southern California)
Tamai, T., "Japanese-Based Programming as a Means for Enhancing Software Quality" (University of Tsukuba)
Teramoto, M., "Evaluation of Management Process" (NEC)
Tohma, Y., R. Jacoby, Y. Murata, and M. Yamamoto, "Hyper-Geometric Distribution to Estimate the Number of Residual Software Faults" (Tokyo Institute of Technology)
Tokutsu, H., "Software Quality Assurance System" (Oki Electric Industry Co.)
Torii, K., "A Measurement Environment and Some Results at Class Experiments" (Osaka University)
Velez, C., and S. Balin, "An ADA Design and Implementation Toolset Based on Object-Oriented and Functional Programming Paradigms" (CTA Inc.)
Yamazaki, T., "Software Quality Improvement Activities at a User" (Nippon Steel)
Yoshizawa, T., "Quality Function Deployment for Software Development" (University of Tsukuba in Tokyo)
After the report The Second International Workshop on Software Quality Improvement (Kyoto) was written the author read an article in the 19 March Japan Times by John Boyd "Is a hard fall awaiting American soft?" Some of his conclusions dovetail with the remarks about Japanese word processors. Boyd details a speech made by Bill Totten, president of Ashisuto K.K., a Japanese importer and distributor of foreign software. Some exerpts: "the worst mistake (Americans) could make was to underestimate the Japanese software competition. I think the American software industry needs to face up to the fact that things are changing rapidly. And unless they make substantial changes in the way they're handling their business, they're goiing to lose this industry like they lost the television, automobile, and a lot of other industries. The reasons behind the change sound frighteningly familiar: high U.S. costs and prices compared to Japanese offerings; a parochial business focus and an unwillingness of the part of many U.S. software companies to even listen to the needs of overseas customers." The two-byte character code is a good example; virtually all Japanese software uses these two byte codes, which are necessary to represent the 3,000 or more kanji characters. Several years ago IBM came out a new standard to support the two byte code, but few U.S. software companies currently use it because it isn't necessary in the U.S. or Europe. Totten says that "Japanese companies are coming out with a lot of software that is now functionally the same as American software, but is better manufactured, few bugs, more clearly documented source code, and is smaller and faster." Totten believes that "U.S. companies are on the verge of losing the Japanese market, and as Japanese distributors grow and try to expand to Asia and Europe these markets will be lost too."
My own experience so far is that these observations do not yet apply to scientific software. I have seen very few examples of Japanese engineering software in use. In fact, I have repeatedly been told that the main reason that Japanese scientists like Cray rather than NEC, Fujitsu, or Hitachi supercomputers is not because Cray computers are faster but because there are so many well tuned engineering and analysis packages running on Crays. Unless Japanese supercomputers sell well there will not be much motivation for software vendors to create versions that are specific to them. In the PC and workstation world things are quite different. Any computer with a graphical or menu interface is likely to run Japanese software because of the need to represent kanji.
David Kahaner [ONRFE] 22 March 1990. TED AT THE WORKSHOP