Scoring the Completeness of Software Designs

A standardized system for scoring the completeness of software designs produced in experimental settings is proposed. The system produces a complete and multi-faceted expression of a software design, making it ideal for comparing designs generated in different languages, paradigms, and methodologies. The system decomposes a design into a large number of atomic design "features" and thus is able to characterize the different strengths (and weaknesses) that each design possesses, and to do so in a way that is "paradigm neutral", that is, not unfairly biased towards one language, paradigm, or methodology. As a result of the thoroughness of this scoring system, an absolute completeness score for a design may be computed, facilitating the comparisons of designs across studies, across design problems, and across experimental conditions. The scoring system allows for the representation of design alternatives and optional features, recognizing that software design problems are neither uniquely understood nor sufficiently constrained to identify a unique solution. In addition, the scoring system characterizes each component of a design as being specified to a certain level of detail or "refinement". Techniques for scoring designs and generating dependent measures are proposed. Alternative notions of design quality and correctness are described, and it is shown how they can be incorporated into the scoring system. Using the scoring system methodology as the basis for creating a software design problem typology is discussed.