In software engineering, relying on experience can render maintainability expertise into prejudice over time. For example, based on their own experience, some consider JavaScript as inelegant language and hence of lowest maintainability. Such prejudice should not guide decisions without prior empirical validation.
Static analysis is an important part of today's quality assurance process. It can be performed manually, by means of code reviews, or automatically, by automated static analysis tools (ASATs). However, there is still much unknown about the state of static analysis. This includes hard data on how prevalent static analysis is among projects. And while there have been studies on how projects use code reviews, current research has not investigated how developers configure the ASATs that they use and how these configurations evolve. In this thesis, we answer these questions by means of a large scale analysis of open source software. We found that both code reviews and ASATs are common, but not ubiquitous. Many projects do not perform code reviews for the changes of core developers and do not enforce a strict use of ASATs. Regarding the use of ASATs, developers both use and avoid maintainability defects to a greater extent than functional defects. Most configurations of developers deviate from the default and hardly contain custom rules. However, there are few default rules that are changed by a significant percentage of developers. Finally, most configuration files never change. And if they do, the changes are small, occur over the lifetime of the project, and are not triggered by ASAT version updates. Traditionally, research on quality attributes was either kept under wraps within the organization that performed it, or carried out by outsiders using narrow, black-box techniques. The emergence of open source software has changed this picture allowing us to evaluate both software products and the processes that yield them. Thus, the software source code and the associated data stored in the version control system, the bug tracking databases, the mailing lists, and the wikis allow us to evaluate quality in a transparent way. Even better, the large number of (often competing) open source projects makes it possible to contrast the quality of comparable systems serving the same domain. Furthermore, by combining historical source code snapshots with significant events, such as bug discoveries and fixes, we can further dig into the causes and effects of problems. Here we present motivating examples, tools, and techniques that can be used to evaluate the quality of open source (and by extension also proprietary) software. There is an ample debate over the quality of Free Libre Open Source Software FLOSS with mixed research results. The authors show that a reason for these mixed results is that quality is being defined, measured, and evaluated differently. They report the most popular approaches including software structure measures, process measures, and maturity assessment models. The way researchers have built their samples has also contributed to the mixed results with different project properties being considered and ignored. Because FLOSS projects evolve with each release, their quality does too, and it must be measured using metrics that take into account their communities' commitment to quality rather than just the structure of the resulting code. Challenges exist in defining what constitutes a defect or bug, and the role of modularity in affecting FLOSS quality. The authors suggest three considerations for future research on FLOSS quality models: 1 defect resolution rate, 2 kind of software product, and 3 modularity-both technical and organizational.