A systematic literature review of open source software quality assessment models <s> Principal findings <s> There are a wide variety of open source security tools available for deployment within the enterprise. Despite the success of many security-based open source software (OSS) projects, large numbers of these projects become inactive and are eventually abandoned. The purpose of this research is to develop an empirical study to determine the relative efficiency of security-based OSS projects. A security-based OSS projects utilize two key resources in their development, the number of software developers and the number of users submitting bugs found when using OSS code. This research develops a model to measure the relative efficiency of each project and determines the number of inefficient projects benchmarking each efficient project. The result of this research is a model that can be used by project developers to evaluate the relative efficiency of their projects. Our empirical study will be based on the analysis of publicly available data from a repository of OSS project at sourceforge.net. <s> BIB001 </s> A systematic literature review of open source software quality assessment models <s> Principal findings <s> Free Open Source Software (FIOSS) for Customer relationship management (CRM) represents a great opportunity for small and medium enterprises as they can significantly impact their competitiveness. These software solutions can be adopted with success, whatever the size, and offer customized solutions for clients, even with few people working of it. This paper presents an analysis of the quality of CRM FIOSS systems using a predefined quality model. The analysis focuses on the main relevant aspect of both open source domain and application domain proposing a trade off assessment among these. <s> BIB002 </s> A systematic literature review of open source software quality assessment models <s> Principal findings <s> User satisfaction has always been a major factor in the success of software, regardless of whether it is closed proprietary or open source software (OSS). In open source projects, usability aspects cannot be improved unless there are ways to test and measure them. Hence, the increasing popularity of open source projects among novice and non-technical users necessitates a usability evaluation methodology. Consequently, this paper presents a usability maturity model specifically aimed at usability-related issues for open source projects. In particular, the model examines the degree of coordination between open source projects and their usability aspects. The measuring instrument of the model contains factors that have been selected from four of our empirical studies, which examine the perspectives of OSS users, developers, contributors and the industry. In addition to presenting the usability maturity model, this paper discusses assessment questionnaires, a rating methodology and two case studies. <s> BIB003 </s> A systematic literature review of open source software quality assessment models <s> Principal findings <s> Abstract Context The selection and adoption of open source software can significantly influence the competitiveness of organisations. Open source software solutions offer great opportunities for cost reduction and quality improvement, especially for small and medium enterprises that typically have to address major difficulties due to the limited resources available for selecting and adopting a new software system. Objective This paper aims to provide support for selecting the open source software that is most suitable to the specific needs of an enterprise from among the options offering equivalent or overlapping functionality. Method This paper proposes a framework for evaluating the quality and functionality of open source software systems. The name of the framework is EFFORT (Evaluation Framework for Free/Open souRce projecTs). It supports the evaluation of product quality, community trustworthiness and product attractiveness. The framework needs to be customised to the analysis of software systems for a specific context. Results The paper presents the customisation of EFFORT for evaluating Enterprise Resource Planning (ERP) open source software systems. The customised framework was applied to the evaluation and comparison of five ERP open source software systems. The results obtained permitted both the refinement of the measurement framework and the identification of the ERP open source software system that achieved the highest score for each chosen characteristic. Conclusion EFFORT is a useful tool for evaluating and selecting an open source software system. It may significantly reduce the amount of negotiation conducted among an enterprise’s members and reduce the time and cost required for gathering and interpreting data. The EFFORT framework also considers the users’ opinions by introducing relevance markers associated with the metrics and questions in the data aggregation process. <s> BIB004
• From the existing OSS quality models considered in this study, 20% of the models only measure a single quality attribute. Models in this category include: QSOS (which measures maintainability)  , Wray and Mathieu BIB001 (which measures efficiency), OS-UMM (which measures usability) BIB003 ) and Sudhaman and Thangavel model (which measures efficiency) (Sudhaman and Thangavel 2015). Furthermore, 50% of the existing models do not measure community related quality characteristics even though community is what distinguishes OSS from their proprietary counterpart. Models in this category include: QSOS (Atos 2006), Sung et al. model (2007) , Raffoul et al. model (2008) , Alfonzo et al. model (2008) , Wray and Mathieu model BIB001 Wray 2007), Chirila et al. model (2011) , OS-UMM BIB003 , Sudhaman and Thangavel model (2015) and Sarrab and Rehman model (2014) . In addition, 35% of the models touch on all categories. They include: OSMM  , Open BRR  , SQO-OSS (Spinellis et al. 2009 ), EFFORT BIB004 ), Müller model (2011 ) and Sohn et al. model (2015 . Among these models a number of them have been applied to selection scenarios and reported in the literature. A notable example is the EFFORT model, which has been applied to evaluate OSS in the customer relationship management (CRM) domain BIB002 as well as in the enterprise resourceplanning (ERP) domain BIB004 . • From the existing OSS quality models, it is observed that in the aspect of product quality as defined by ISO 25010, maintainability is the most significant quality characteristic; Usability is the most significant quality in use characteristic in the existing OSS quality models while Maintenance capacity is the most significant community related characteristic in the OSS quality assessment models. Also worthy of note is that satisfaction and safety attributes of quality in use are never considered in the OSS quality models. • The model approach is the most adopted selection method in the existing OSS quality models. The least considered are the tool-based and data mining selection approaches. However, as newer publications emerge we expect to see other approaches and data mining gaining more ground. • Majority (47%) of the existing models do not specify any domain of application. As for those with specific domain of application, a greater percentage focus of data-dominant software especially enterprise resource planning software. Computation-dominant software is the least considered in this regard. Software in this category includes: operations research, information management and manipulation, artistic creativity, scientific software and artificial intelligence software. • From the this study, we also observed that none of the existing models evaluate all the criteria that we laid out, in terms of every quality characteristic under product quality, quality in use, and community related quality characteristics.