Definitions and approaches to model quality in model-based software development - A review of literature <s> A Classification of Model Quality Goals (RQ1) <s> In model-driven engineering (MDE), models are the prime artifacts, and developing high-quality systems depends on developing high-quality models and performing transformations that preserve quality or even improve it. This paper presents quality goals in MDE and states that the quality of models is affected by the quality of modeling languages, tools, modeling processes, the knowledge and experience of modelers, and the quality assurance techniques applied. The paper further presents related work on these factors and identifies pertinent research challenges. Some quality goals such as well-formedness and precision are especially important in MDE. Research on quality in MDE can promote adoption of MDE for complex system engineering. <s> BIB001 </s> Definitions and approaches to model quality in model-based software development - A review of literature <s> A Classification of Model Quality Goals (RQ1) <s> This paper reviews definitions of model quality before introducing six properties of models that are important for building high-quality models. These are identified to be correctness, completeness, consistency, comprehensibility, confinement and changeability. We have earlier defined a quality model that separates intangible quality goals from tangible qualitycarrying properties and practices that should be in place to support these properties. A part of that work was to define a metamodel for developing quality models with MDE in mind. In this paper we analyze existing literature in order to extract model quality properties and to build a quality model with focus on the quality of models. For this purpose the metamodel is implemented in a tool that allows us to model quality models. The advantage of defining the metamodel is learning how to precisely define quality elements and relations in the quality model, and building models that may be used to generate documentation, guidelines or checklists. The disadvantage is mainly in the research phase where the metamodel is not stable and undergoes changes. <s> BIB002
In BIB002 we have presented previous classifications of model quality goals  such as:  The Lindland et al.'s quality framework has conceptual models in mind (models of specification statements such as requirement models) and classifies model quality into syntax (adhering to language rules), semantic (correct meaning and relations) and pragmatic quality (comprehensibility by the intended users) [P22] .  Additional model quality goals are added by Krogstie and Sølvberg to the Lindland et al. framework; such as organizational quality (whether a model fulfils the goals of modelling and that all the goals of modelling are addressed through the model) and technical pragmatic quality defined as being interpretable by tools [P15] .  Unhelkar classifies model quality goals into syntax (with focus on correctness), semantics or meaning (with focus on completeness, consistency and representing the domain), and aesthetics (with focus on symmetry and consistency in order to improve the look and to help understanding) [P38] . His work is on UML models.  Nelson and Monarchi provide an overview of quality models and discusses that modelling is a transformation from real world to an implementation in multiple steps [P26] . In each step one should assure that the content is persevered. The author defines perceptual, descriptive, semantic, syntactic, pragmatic, and inferential quality as quality types where some definitions match previous work in [P22] and [P15] . Although the above definitions are useful, we do not see them often used in literature on model quality. Besides, the boundary between syntax and semantics is sometimes blurred  and these terms are used inconsistently in literature. Therefore it is useful to have a classification that is close to the concepts used in the literature related to modelling. Based on the results of this review and our earlier work on developing a quality model for MDE as presented in BIB001 [10] BIB002 , we have identified six classes of model quality goals that are introduced in the remainder of this section. We also present related work that point to the origin of the definitions. C1-Correctness. Correctness is defined as: a) Including right elements and correct relations between them, and including correct statements about the domain; b) Not violating rules and conventions; for example adhering to language syntax (well-formedness or syntactic correctness according to [P22] ), style rules, naming guidelines or other rules or conventions. Several quality models define syntactic correctness relative to the modelling language and semantic correctness relative to the domain and our understanding of it. Nelson and Monarchi write that syntactic quality is determined by comparing the representation to the language while the meaning of the elements should be preserved, called as semantic quality [P26] . Including right elements and relations is related to our understanding of the domain and is called semantic validity in the framework of Lindland et al. [P22] . Berenbach calls a UML model semantically correct if it includes correct relations and is complaint with good practices and corporate standards [P3] . Unhelkar rather talks of errors and mentions that CASE tools keep language syntax errors to a minimum while the semantic aspect requires that the diagrams faithfully represent the underlying reality [P38] . C2-Completeness. Completeness is defined as having all the necessary information that is relevant [P22] and being detailed enough according to the purpose of modelling.  The terms quality goals, characteristics, attributes, properties etc. are all used in literature and in different quality models. We use the term "quality goal" here to cover them all. Berenbach defines requirement and analysis models as complete when they specify all the black-box behaviour of the modelled entity [P3] [P4] . Complete models can then be used to define test cases and create project tasks. Others do not define completeness bur rather discuss incompleteness as missing elements in models [P27] . Mitchell writes that one should discover key problem domain concepts and make sure that these are modelled in software; from system analysis models, through design and into code C3-Consistency. Consistency is defined as no contradictions in the model. It covers consistency between views or diagrams that belong to the same level of abstraction or development phase (horizontal consistency), and between models or diagrams that represent the same aspect, but at different levels of straction or in different development phases (vertical consistency). It also covers semantic consistency between models; i.e., the same element does not have multiple meanings in different diagrams or models. Lange writes that the multi-diagram approach of UML entails the risk for contradictions between diagrams, so called inconsistencies. Besides, consistency defects can occur not only within a model between different diagrams, but also between models at different abstraction levels [P19] . Consistency between diagrams or models of a system is important for correct interpretation of them. Ambler emphasizes consistency when modelling a system in the sense that common elements have common names to avoid co agrams [P3] . Several other studies discuss inconsistency problems and how to avoid them as discussed later in Section 5. C4-Comprehensibility. Comprehensibility is defined as being und her human users or tools. Lindland calls this pragmatic quality [P22] , which is the term used by Krogstie and Sølvberg [P15] [P25] , and finally using the correct type of diagram for the intended audience. For example, Berenbach writes that UML is flexible regarding the choice of diagrams for defining a process. Sequence, collaboration, activity and state diagrams can all be used. However he recommends using sequence diagrams which is his experience are easiest to read for nontechnical reviewers [P3] . Ambler has a set of guidelines to improve readability of diagrams which are both related to aesthetics (such as avoiding crossing lines) and organization of elements on a diagram [P1]. Unhelkar writes that once the syntax and the semantics are correct, we need to consider the a e model [P38] . Aesthetics is simply style or "look and feel" which has a bearing on the models readability or comprehensibility. Focus of most literature is on comprehensibility by humans. For tools, having a precise ogstie and Sølveberg define "technical pragmatic quality" as to what extent tools can be constructed to understand the models [P15] . C5-Confinement. Confinement is defined as being in agreement with the purpose of modelling and the type of system; such as including relevant diagrams and being at the right abs scription from which detail has been removed intentionally. A confined model does not have unnecessary information and is not more complex or detailed than necessary. Ambler writes that models should be kept simple and one must avoid details not necessary for the purpose of modelling [P1] , where the motivation is to improve readability. Other motivations may be to avoid keeping several models in sync and reducing the effort spent on mode t adding details to a high-level diagram is different from adding design details [P25] . The first makes an imprecise model more precise while the second adds unnecessary in Developing the right model for the type of system or purpose also depend on selecting the right modelling language. However, our focus here is on mo C6-Changeability. Changeability is defined as supporting changes or improvements so that models can be changed or evolved rapidly and continuously. Changeability is not mentioned in previous work as a separate quality goal although the need for updating models is obvious and mentioned in several studies. It is required since both the domain and our understanding of it or requirements of the system evolve with time. Mitchell writes that a system must resemble the problem so that it may be changed when the problem domain changes [P25] . Jonkers et al. write that the modelling discipline should be close to the problem domain which enables simpler m e supported by modelling languages and modelling tools as well. For example Kr lations to other elements involved in modelling. Compared with the Lindland et al.'s fram of modelling) to the fram aintenance and evolution of models [P13] . Berenbach recommends avoiding early packaging since it implies partitioning and may result in frequent reorganizations [P3] . Changeability should b ogstie and Sølvberg have identified modifiability of language as a practice that helps achieving validity and completeness [P15] . 6C goals. We call the above quality goals collectively for the 6C (model quality) goals. Figure II  shows Figure III is another view of the 6C goals that shows when in the development process they are important. The figure is inspired by the idea of viewing modelling as a set of transformations [P26] . A model is a representation of a system and should be complete relative to the system it wants to represent and according to the modelling goals defined by the organization. It should also contain correct relations between elements and correct meanings. All these properties depend on the perception of the modeller of the domain and the purpose of modelling. The developed models are required to be correct relative to the language and modelling rules or conventions, and be comprehensible for interpretation by humans or by tools for the purpose of generation, simulation or analysis. Of course precise definition of quality goals dep rder to support reusability of models. The 6C goals are identified based on the analysis of literature covered in this review and the list may therefore be modified or extended if new requirements are detected. ends on the context and the purpose of modelling, especially whether models are used for implementation, testing and maintenance of systems. Finally, we mean that other quality goals discussed in literature can be satisfied if the 6C goals are in place. For example a model that is correct, complete and consistent does not allow multiple interpretations and all of the above goals are important in o In the next section we present the proposed approaches for improving the quality of models which are referred to as "practices" in our quality model, together with the reported empirical evidence.