An overview of metrics-based approaches to support software components reusability assessment <s> Facilitating the replication of validation efforts <s> This paper discusses the formalization effort of object-oriented design metrics definitions and presents some concrete examples, developed upon the UML meta-model. The Object Constraint Language (OCL), a part of the Unified Modeling Language (UML) standard, is used in the formalization. The combination of the UML meta-model and OCL allows unambiguous metrics definition, which in turn helps increasing tool support for object-oriented metrics extraction. This formalization renders possible the comparisons among different sets of metrics, as well as it may be used to establish a common vocabulary among different stakeholders. As consequence, the precision of the metrics collection increases, contributing to the overall quality of the Software Engineering process. <s> BIB001 </s> An overview of metrics-based approaches to support software components reusability assessment <s> Facilitating the replication of validation efforts <s> In component-based software development, it is necessary to measure the reusability of components in order to realize the reuse of components effectively. There are some product metrics for measuring the reusability of object-oriented software. However, in application development with reuse, it is difficult to use conventional metrics because the source codes of components cannot be obtained, and these metrics require analysis of source codes. We propose a metrics suite for measuring the reusability of such black-box components based on limited information that can be obtained from the outside of components without any source codes. We define five metrics for measuring a component's understandability, adaptability, and portability, with confidence intervals that were set by statistical analysis of a number of JavaBeans components. Moreover, we provide a reusability metric by combining these metrics based on a reusability model. As a result of evaluation experiments, it is found that our metrics can effectively identify black-box components with high reusability. <s> BIB002 </s> An overview of metrics-based approaches to support software components reusability assessment <s> Facilitating the replication of validation efforts <s> This paper describes an independent validation study for a suite of reusability metrics for component based design published in the literature. The metrics under validation were originally proposed using a semi-formal notation, namely a combination of mathematical formulae with natural language descriptions for elementary parts of those formulae. They were then computed using proprietary tools. By contrast, we present a formalization for the metrics suite that combines the version 2.0 of the UML metamodel with OCL. This technique provides a formal, portable and executable definition of the metrics set that can be used to perform other cross-validations of the metrics suite. A prototype working environment to perform such cross-validation experiments, both with this and other metrics sets was developed and a metrics collection experiment using that environment is discussed. <s> BIB003 </s> An overview of metrics-based approaches to support software components reusability assessment <s> Facilitating the replication of validation efforts <s> This paper describes a technique for formalizing metrics for COTS-based architectures. This technique is built upon the UML 2.0 metamodel and uses OCL as a metrics definition language. As a proof of concept, an example based upon a set of reusability metrics for fine-grained JavaBeans components is presented. <s> BIB004 </s> An overview of metrics-based approaches to support software components reusability assessment <s> Facilitating the replication of validation efforts <s> The objective of this paper is the formal definition of composition assessment metrics for CBSE, using an extension of the CORBA component model metamodel as the ontology for describing component assemblies. The method used is the representation of a component assembly as an instantiation of the extended CORBA component model metamodel. The resulting meta-objects diagram can then be traversed using object constraint language clauses. These clauses are a formal and executable definition of the metrics that can be used to assess quality attributes from the assembly and its constituent components. The result is the formal definition of context-dependent metrics that cover the different composition mechanisms provided by the CORBA component model and can be used to compare alternative component assemblies; a metamodel extension to capture the topology of component assemblies. The conclusion is that providing a formal and executable definition of metrics for CORBA component assemblies is an enabling precondition to allow for independent scrutiny of such metrics which is, in turn, essential to increase practitioners confidence on predictable quality attributes. <s> BIB005 </s> An overview of metrics-based approaches to support software components reusability assessment <s> Facilitating the replication of validation efforts <s> Objective: In this paper, we present a formalization of the definition of metrics to assess quality attributes of CORBA components and assemblies. The focus is on the formalization technique, rather than on the evaluation of the metrics themselves. Method: We represent a component assembly as an instantiation of the CORBA Component Model metamodel. The resulting meta-object diagram can then be traversed using Object Constraint Language clauses. With these clauses, we construct a formal and executable definition of the metrics. Results: We demonstrate the expressiveness of our technique by formally de fining metrics proposed informally by several authors on different aspects of components and assemblies’ quality attributes. Conclusion: Providing a formal and executable definition of metrics for CORBA components and assemblies is an enabling precondition to allow for independent scrutiny of such metrics, which is, in turn, essential to increase practitioners’ confidence on predictable quality attributes. <s> BIB006 </s> An overview of metrics-based approaches to support software components reusability assessment <s> Facilitating the replication of validation efforts <s> One major problem for integrating study results into a common body of knowledge is the heterogeneity of reporting styles: (1) it is difficult to locate relevant information and (2) important information is often missing. Reporting guidelines are expected to support a systematic, standardized presentation of empirical research, thus improving reporting in order to support readers in (1) finding the information they are looking for, (2) understanding how an experiment is conducted, and (3) assessing the validity of its results. The objective of this paper is to survey the most prominent published proposals for reporting guidelines, and to derive a unified standard that which can serve as a starting point for further discussion. We provide detailed guidance on the expected content of the sections and subsections for reporting a specific type of empirical studies, i.e., controlled experiments. Before the guidelines can be evaluated, feedback from the research community is required. For this purpose, we propose to adapt guideline development processes from other disciplines. <s> BIB007 </s> An overview of metrics-based approaches to support software components reusability assessment <s> Facilitating the replication of validation efforts <s> Data collection, both automatic and manual, lies at the heart of all empirical studies. The quality of data collected from software informs decisions on maintenance, testing and wider issues such as the need for system re-engineering. While of the two types stated, automatic data collection is preferable, there are numerous occasions when manual data collection is unavoidable. Yet, very little evidence exists to assess the error-proneness of the latter. Herein, we investigate the extent to which manual data collection for Java software compared with its automatic counterpart for the same data. We investigate three hypotheses relating to the difference between automated and manual data collection. Five Java systems were used to support our investigation. Results showed that, as expected, manual data collection was error-prone, but nowhere near the extent we had initially envisaged. Key indicators of mistakes in manual data collection were found to be poor developer coding style, poor adherence to sound OO coding principles, and the existence of relatively large classes in some systems. Some interesting results were found relating to the collection of public class features and the types of error made during manual data collection. The study thus offers an insight into some of the typical problems associated with collecting data manually; more significantly, it highlights the problems that poorly written systems have on the quality of visually extracted data. <s> BIB008
Automated metrics extraction is fundamental to foster independent validation efforts. Manual collection of metrics has been shown to be error prone and vulnerable, for instance, to the lack of adherence to sound, widely accepted, coding principles BIB008 . Furthermore, the effort required for large scale manual metrics extraction is prohibitive. Unfortunately, to the best of our knowledge, most of the proposed metrics were only tested by their authors, using proprietary or experimental, non-publicly available, tool support, therefore limiting Experiments replication. This limits knowledge sharing, both in the research and practitioner's communities, hampering results comparison. We have proposed elsewhere an approach to mitigate this problem BIB003  BIB005 BIB006 , which relies on the usage of a metamodel to formally define the concepts we aim to measure, and Object Constraint Language (OCL) expressions to define metrics over that metamodel. It can be summarized as follows: • Selecting or producing an adequate metamodel describing the domain concepts (for instance, for CB systems, we could use the CORBA Components Metamodel  , or an extract of the UML 2 metamodel corresponding to component diagrams   . • Specifying the metrics using OCL  upon the previous metamodel. Notice that the latter is specified as a UML class diagram that can be traversed using OCL expressions. • Instantiating the metamodel, with meta-objects and meta-links corresponding to the target software piece (e.g. code, or model element) that we want to measure. • Collecting the metrics using an OCL-enabled tool that evaluates the OCL expressions upon the previously mentioned instantiation. Our proposal for metrics definition and collection combines formality, understandability and collection efficiency due to the usage of OCL. Furthermore, it ensures their portability among OCL-enabled CASE tools. As OCL is part of the new UML standard, an increasing number of UML CASE tools are supporting it. To collect the metrics, we execute their corresponding OCL definition upon the referred metamodel, instantiated with meta-objects representing the component assemblies to be analyzed. Further details about this technique and case studies that illustrate its applicability have been published in the recent years. The idea of using OCL for defining software metrics to evaluate object-oriented design was proposed in  . FLAME, a library of OCL functions to aid in the definition and extraction of software metrics, based on the UML metamodel was presented in BIB001 . BIB004 moves to the evaluation of software components and discusses the formalization of Washizaki's metrics set BIB002 , using the UML 2 metamodel. This formalization uses the new abstractions for software components provided by UML 2. [27] presents a case study to assess Washizaki's metrics set in a quantitative way. Last, but not the least, the adherence to a common set of research reporting guidelines for presenting the results of experimental work in software engineering would certainly increase the comparability of different research efforts. Using research reporting guidelines such as those proposed in BIB007 is a promising path towards a more effective research on metrics-based approaches to software development, in general, and CBD, in particular. Note that while such guidelines are aimed at the description of experimental work (e.g. in a paper describing a controlled experiment, or set of experiments), our framework was developed to facilitate a systematic comparison of proposals found in the literature.