Software development needs continuous quality control for a timely detection and removal of quality problems. This includes frequent quality assessments, which need to be automated as far as possible to be feasible. One way of automation in assessing the security of software are application scanners that test an executing software for vulnerabilities. At present, common quality assessments do not integrate such scanners for giving an overall quality statement. This paper presents an integration of application scanners into a general quality assessment method based on explicit quality models and Bayesian nets. Its applicability and the detection capabilities of common scanners are investigated in a case study with two open-source web shops.
There is a wide variety of quality models. @cite_23 differentiate between quality definition models and quality assessment models. The former is a specification of what constitutes quality in a software system, the latter describes how a software system's quality can be assessed according to specific rules. In the area of software security, security pattern collections are an example of quality definition models, e.g., @cite_2 .
@cite_23 @cite_2
2095696918 2131441032
Assessing and predicting the complex concept of software quality is still challenging in practice as well as research. Activity-based quality models break down this complex concept into more concrete definitions, more precisely facts about the system, process and environment and their impact on activities performed on and with the system. However, these models lack an operationalisation that allows to use them in assessment and prediction of quality. Bayesian Networks (BN) have been shown to be a viable means for assessment and prediction incorporating variables with uncertainty. This paper describes how activity-based quality models can be used to derive BN models for quality assessment and prediction. The proposed approach is demonstrated in a proof of concept using publicly available data. With the growing need for quality assessment of entire software systems in the industry, new issues are emerging. First, because most software quality metrics are defined at the level of individual software components, there is a need for aggregation methods to summarize the results at the system level. Second, because a software evaluation requires the use of different metrics, with possibly widely varying output ranges, there is a need to combine these results into a unified quality assessment. In this paper we derive, from our experience on real industrial cases and from the scientific literature, requirements for an aggregation method. We then present a solution through the Squale model for metric aggregation, a model specifically designed to address the needs of practitioners. We empirically validate the adequacy of Squale through experiments on Eclipse. Additionally, we compare the Squale model to both traditional aggregation techniques (e.g., the arithmetic mean), and to econometric inequality indices (e.g., the Gini or the Theil indices), recently applied to aggregation of software metrics. Keywords: software metrics; software quality; aggregation; inequality indices Due to their pivotal role in software engineering, considerable effort is spent on the quality assurance of software requirements specifications. As they are mainly described in natural language, relatively few means of automated quality assessment exist. However, we found that clone detection, a technique widely applied to source code, is promising to assess one important quality aspect in an automated way, namely redundancy that stems from copy & paste operations. This paper describes a large-scale case study that applied clone detection to 28 requirements specifications with a total of 8,667 pages. We report on the amount of redundancy found in real-world specifications, discuss its nature as well as its consequences and evaluate in how far existing code clone detection approaches can be applied to assess the quality of requirements specifications in practice. Software practitioners lack a consistent approach to assessing and predicting quality within their products. This research proposes a software quality model that accounts for the influences of development team skill experience, process maturity, and problem complexity throughout the software engineering life cycle. The model is structured using Bayesian Belief Networks and, unlike previous efforts, uses widely-accepted software engineering standards and in-use industry techniques to quantify the indicators and measures of software quality. Data from 28 software engineering projects was acquired for this study, and was used for validation and comparison of the presented software quality models. Three Bayesian model structures are explored and the structure with the highest performance in terms of accuracy of fit and predictive validity is reported. In addition, the Bayesian Belief Networks are compared to both Least Squares Regression and Neural Networks in order to identify the technique is best suited to modeling software product quality. The results indicate that Bayesian Belief Networks outperform both Least Squares Regression and Neural Networks in terms of producing modeled software quality variables that fit the distribution of actual software quality values, and in accurately forecasting 25 different indicators of software quality. Between the Bayesian model structures, the simplest structure, which relates software quality variables to their correlated causal factors, was found to be the most effective in modeling software quality. In addition, the results reveal that the collective skill and experience of the development team, over process maturity or problem complexity, has the most significant impact on the quality of software products. Software quality models are a well-accepted means to support quality management of software systems. Over the last 30 years, a multitude of quality models have been proposed and applied with varying degrees of success. Despite successes and standardisation efforts, quality models are still being criticised, as their application in practice exhibits various problems. To some extent, this criticism is caused by an unclear definition of what quality models are and which purposes they serve. Beyond this, there is a lack of explicitly stated requirements for quality models with respect to their intended mode of application. To remedy this, this paper describes purposes and usage scenarios of quality models and, based on the literature and experiences from the authors, collects critique of existing models. From this, general requirements for quality models are derived. The requirements can be used to support the evaluation of existing quality models for a given context or to guide further quality model development. Maintainability is a key quality attribute of successful software systems. However, its management in practice is still problematic. Currently, there is no comprehensive basis for assessing and improving the maintainability of software systems. Quality models have been proposed to solve this problem. Nevertheless, existing approaches do not explicitly take into account the maintenance activities, that largely determine the software maintenance effort. This paper proposes a 2-dimensional model of maintainability that explicitly associates system properties with the activities carried out during maintenance. The separation of activities and properties facilitates the identification of sound quality criteria and allows to reason about their interdependencies. This transforms the quality model into a structured and comprehensive quality knowledge base that is usable in industrial project environments. For example, review guidelines can be generated from it. The model is based on an explicit quality metamodel that supports its systematic construction and fosters preciseness as well as completeness. An industrial case study demonstrates the applicability of the model for the evaluation of the maintainability of Matlab Simulink models that are frequently used in model-based development of embedded systems. The benefits of a software system are assessable only relative to the business goals the system has been developed to serve. In turn, these benefits result from interactions between the system's functionality and its quality attributes (such as performance, reliabilty and security). Its quality attributes are, in most cases, dictated by its architectural design decisions. Therefore, we argue in this paper that the software architecture is the crucial artifact to study in making design tradeoffs and in performing cost-benefit analyses. A substantial part of such an analysis is in determining the level of uncertainty with which we estimate both costs and benefits. In this paper we offer an architecture-centric approach to the economic modeling of software design decision making called CBAM (Cost Benefit Analysis Method), in which costs and benefits are traded off with system quality attributes. We present the CBAM, the early results from applying this method in a large-scale case study, and discuss the application of more sophisticated economic models to software decision making. There is an ample debate over the quality of Free Libre Open Source Software FLOSS with mixed research results. The authors show that a reason for these mixed results is that quality is being defined, measured, and evaluated differently. They report the most popular approaches including software structure measures, process measures, and maturity assessment models. The way researchers have built their samples has also contributed to the mixed results with different project properties being considered and ignored. Because FLOSS projects evolve with each release, their quality does too, and it must be measured using metrics that take into account their communities' commitment to quality rather than just the structure of the resulting code. Challenges exist in defining what constitutes a defect or bug, and the role of modularity in affecting FLOSS quality. The authors suggest three considerations for future research on FLOSS quality models: 1 defect resolution rate, 2 kind of software product, and 3 modularity-both technical and organizational. The last decade marked the first real attempt to turn software development into engineering through the concepts of Component-Based Software Development (CBSD) and Commercial Off-The-Shelf (COTS) components, with the goal of creating high-quality parts that could be joined together to form a functioning system. One of the most critical processes in CBSD is the selection of a set of software components from in-house or external repositories that fulfil some architectural and user-defined requirements. However, there is a lack of quality models and metrics that can help evaluate the quality characteristics of software components during this selection process. This paper presents a set of measures to assess the Usability of software components, and describes the method followed to obtain and validate them. Such a method can be re-used as a pattern for defining and validating measures for further quality characteristics. Assessing the quality of user generated content is an important problem for many web forums. While quality is currently assessed manually, we propose an algorithm to assess the quality of forum posts automatically and test it on data provided by Nabble.com. We use state-of-the-art classification techniques and experiment with five feature classes: Surface, Lexical, Syntactic, Forum specific and Similarity features. We achieve an accuracy of 89 on the task of automatically assessing post quality in the software domain using forum specific features. Without forum specific features, we achieve an accuracy of 82 . The paper presents extensions to Bell Canada source code quality assessment suite (DATRIX tm) for handling Java language systems. Such extensions are based on source code object metrics, including Java interface metrics, which are presented and explained in detail. The assessment suite helps to evaluate the quality of medium-large software systems identifying parts of the system which have unusual characteristics. The paper also studies and reports the occurrence of clones in medium-large Java software systems. Clone presence affects quality since it increases a system size and often leads to higher maintenance costs. The clone identification process uses Java specific metrics to determine similarities between methods throughout a system. The results obtained from experiments with software evaluation and clone detection techniques, on over 500 KLOC of Java source code, are presented. In the last few years, a plethora of research has addressed security testing issues. Several commercial tools have emerged to provide security testing services. Software security testing goes beyond functional testing to reveal flaws and vulnerabilities in software design and behavior. Access control is a major pillar in computer security. This chapter pursues the goal of describing the landscape in the research area of access control testing. We provide an outline of the different existing research over the literature according to the taxonomy reflecting the different phases of common software testing processes (generation, selection, prioritization, quality assessment, regression). We also provide an outline of some existing initiatives that support usage control besides access control by testing obligation policies. Finally, we point out future research directions that emerge from the current research study. Through this work, we aim at providing useful guidelines for software testers to improve the current trends in access control testing. Assessing the quality of software design is difficult, as "design" is expressed through guidelines and heuristics, not rigorous rules. One successful approach to assess design quality is based on detection strategies, which are metrics-based composed logical conditions, by which design fragments with specific properties are detected in the source code. Such detection strategies, when executed on large software systems usually return large sets of artifacts, which potentially exhibit one or more "design disharmonies", which are then inspected manually, a cumbersome activity. In this article we present disharmony maps, a visualization-based approach to locate such flawed software artifacts in large systems. We display the whole system using a 3D visualization technique based on a city metaphor. We enrich such visualizations with the results returned by a number of detection strategies, and thus render both the static structure and the design problems that affect a subject system. We evaluate our approach on a number of open-source Java systems and report on our findings. Published software quality models either provide abstract quality attributes or concrete quality assessments. There are no models that seamlessly integrate both aspects. In the project Quamoco, we built a comprehensive approach with the aim to close this gap. For this, we developed in several iterations a meta quality model specifying general concepts, a quality base model covering the most important quality factors and a quality assessment approach. The meta model introduces the new concept of a product factor, which bridges the gap between concrete measurements and abstract quality aspects. Product factors have measures and instruments to operationalise quality by measurements from manual inspection and tool analysis. The base model uses the ISO 25010 quality attributes, which we refine by 200 factors and 600 measures for Java and C# systems. We found in several empirical validations that the assessment results fit to the expectations of experts for the corresponding systems. The empirical analyses also showed that several of the correlations are statistically significant and that the maintainability part of the base model has the highest correlation, which fits to the fact that this part is the most comprehensive. Although we still see room for extending and improving the base model, it shows a high correspondence with expert opinions and hence is able to form the basis for repeatable and understandable quality assessments in practice. Known security vulnerabilities can be introduced in software systems as a result of being dependent upon third-party components. These documented software weaknesses are “hiding in plain sight” and represent low hanging fruit for attackers. In this paper we present the Vulnerability Alert Service (VAS), a tool-based process to track known vulnerabilities in software systems throughout their life cycle. We studied its usefulness in the context of external software product quality monitoring provided by the Software Improvement Group, a software advisory company based in Amsterdam, the Netherlands. Besides empirically assessing the usefulness of the VAS, we have also leveraged it to gain insight and report on the prevalence of third-party components with known security vulnerabilities in proprietary applications. The quality of source code is a key factor for any software product and its continuous monitoring is an indispensable task for a software development project. We have developed a method for systematic assessing and improving the code quality of ongoing projects by using the results of various static code analysis tools. With different approaches for monitoring the quality (a trend-based one and a benchmarking-based one) and an according tool support we are able to manage the large amount of data that is generated by these static analyses. First experiences when applying the method with software projects in practice have shown the feasibility of our method. Software architecture has many definitions. One widely accepted definition of software architecture is that it is a composition of a set of architectural design decisions. Hence, designing a software architecture is a decision-making process. Agile methods drastically changed the way of designing a software architecture. In projects using agile methods (e.g. Scrum), making architectural design decisions is not the responsibility of a single person, but rather the whole development team. Despite the popularity of such methods in the industry, little research exists on how to make these decisions from the perspective of a group effectively. Current techniques usually focus on the identification of quality attributes and design alternatives, not addressing the whole decision-making process. The quality of decisions directly reflects the quality of the software architecture. Therefore poor decisions lead to bad software architectures. In this paper, we discuss current research on group decision-making in software architecture and the proposal of a combination of concepts from two architecture definition methods into a single approach that can be used in agile projects and addresses the most critical concerns of group decision-making. This proposal is part of a master's research project. Quality rules are used to capture important implementation and design decisions embedded in a software system's architecture. They can automatically analyze software and assign quality grades to its components. To provide a meaningful evaluation of quality, rules have to stay up-to-date with the continuously evolving system that they describe. However one would encounter unexpected anomalies during a historical overview because the notion of quality is always changing, while the qualitative evolution analysis requires it to remain constant. To understand the anomalies in a quality history of a real-world software system we use an immersive visualization that lays out the quality fluctuations in three dimensions based on two co-evolving properties: quality rules and source code. This helps us to identify and separate the impact caused by the changes of each property, and allows us to detect significant mistakes that happened during the development process. Automatically assessing the subjective quality of a photo is a challenging area in visual computing. Previous works study the aesthetic quality assessment on a general set of photos regardless of the photo's content and mainly use features extracted from the entire image. In this work, we focus on a specific genre of photos: consumer photos with faces. This group of photos constitutes an important part of consumer photo collections. We first conduct an online study on Mechanical Turk to collect ground-truth and subjective opinions for a database of consumer photos with faces. We then extract technical features, perceptual features, and social relationship features to represent the aesthetic quality of a photo, by focusing on face-related regions. Experiments show that our features perform well for categorizing or predicting the aesthetic quality. Code metric analysis is a well-known approach for assessing the quality of a software system. However, current tools and techniques do not take the system architecture (e.g., MVC, Android) into account. This means that all classes are assessed similarly, regardless of their specific responsibilities. In this paper, we propose SATT (Software Architecture Tailored Thresholds), an approach that detects whether an architectural role is considerably different from others in the system in terms of code metrics, and provides a specific threshold for that role. We evaluated our approach on 2 different architectures (MVC and Android) in more than 400 projects. We also interviewed 6 experts in order to explain why some architectural roles are different from others. Our results shows that SATT can overcome issues that traditional approaches have, especially when some architectural role presents very different metric values than others.