Software systems are not static, they have to undergo frequent changes to stay fit for purpose, and in the process of doing so, their complexity increases. It has been observed that this process often leads to the erosion of the systems design and architecture and with it, the decline of many desirable quality attributes, such as maintainability. This process can be captured in terms of antipatterns-atomic violations of widely accepted design principles. We present a visualisation that exposes the design of evolving Java programs, highlighting instances of selected antipatterns including their emergence and cancerous growth. This visualisation assists software engineers and architects in assessing, tracing and therefore combating design erosion. We evaluated the effectiveness of the visualisation in four case studies with ten participants.
There exist many approaches to visualise software evolution @cite_16 . Most visualisations want to provide an improved understanding of the development activities by visualising structural changes, e.g. by using added and removed lines as metrics @cite_43 @cite_3 @cite_46 @cite_5 @cite_25 @cite_45 @cite_28 or by providing highly aggregated information @cite_13 @cite_32 @cite_4 . Our use case requires the visualisation of the structural evolution of the system and the antipattern instances at the same time. We are not aware of any evolution visualisation that supports this. There exist evolution visualisations of call graphs @cite_38 @cite_19 @cite_36 . However, they do not provide any structural information.
@cite_38 @cite_4 @cite_28 @cite_36 @cite_32 @cite_3 @cite_43 @cite_19 @cite_45 @cite_5 @cite_46 @cite_16 @cite_13 @cite_25
2153150125 2163308664 2018844270 2087053547 2052955308 2128939007 1987706161 2040811166 2108400985 137863291 2104074028 1886625064 2119158444 2762085884 1983592716 2178937556 1600348603 2017303659 2613448825 2566045746
A key issue in software evolution analysis is the identification of particular changes that occur across several versions of a program. We present change distilling, a tree differencing algorithm for fine-grained source code change extraction. For that, we have improved the existing algorithm by for extracting changes in hierarchically structured data. Our algorithm extracts changes by finding both a match between the nodes of the compared two abstract syntax trees and a minimum edit script that can transform one tree into the other given the computed matching. As a result, we can identify fine-grained change types between program versions according to our taxonomy of source code changes. We evaluated our change distilling algorithm with a benchmark that we developed, which consists of 1,064 manually classified changes in 219 revisions of eight methods from three different open source projects. We achieved significant improvements in extracting types of source code changes: Our algorithm approximates the minimum edit script 45 percent better than the original change extraction approach by We are able to find all occurring changes and almost reach the minimum conforming edit script, that is, we reach a mean absolute percentage error of 34 percent, compared to the 79 percent reached by the original algorithm. The paper describes both our change distilling algorithm and the results of our evolution. Large software systems have a rich development history. Mining certain aspects of this rich history can reveal interesting insights into the system and its structure. Previous approaches to visualize the evolution of software systems provide static views. These static views often do not fully capture the dynamic nature of evolution. We introduce the Evolution Storyboard, a visualization which provides dynamic views of the evolution of a softwares structure. Our tool implementation takes as input a series of software graphs, e.g., call graphs or co-change graphs, and automatically generates an evolution storyboard. To illustrate the concept, we present a storyboard for PostgreSQL, as a representative example for large open source systems. Evolution storyboards help to understand a systems structure and to reveal its possible decay over time. The storyboard highlights important changes in the structure during the lifetime of a software system, and how artifacts changed their dependencies over time. During software evolution a developer must investigate source code to locate then understand the entities that must be modified to complete a change task. To help developers in this task, proposed text summarization based approaches to the automatic generation of class and method summaries, and via a study of four developers, they evaluated source code summaries generated using their techniques. In this paper we propose a new topic modeling based approach to source code summarization, and via a study of 14 developers, we evaluate source code summaries generated using the proposed technique. Our study partially replicates the original study by in that it uses the objects, the instruments, and a subset of the summaries from the original study, but it also expands the original study in that it includes more subjects and new summaries. The results of our study both support the findings of the original and provide new insights into the processes and criteria that developers use to evaluate source code summaries. Based on our results, we suggest future directions for research on source code summarization. Versioning systems such as CVS or Subversion exhibit a large potential to investigate the evolution of software systems. They are used to record the development steps of software systems as they make it possible to reconstruct the whole evolution of single files. However, they provide no good means to understand how much a certain file has been changed over time and by whom. In this paper we present an approach to visualize files using fractal figures, which: (1) convey the overall development effort; (2) illustrate the distribution of the effort among various developers; and (3) allow files to be categorized in terms of the distribution of the effort following gestah principles. Our approach allows us to discover files of high development efforts in terms of team size and effort intensity of individual developers. The visualizations allow an analyst or a project manager to get first insights into team structures and code ownership principles. We have analyzed Mozilla as a case study and we show some of the recovered team development patterns in this paper as a validation of our approach Software evolution is one of the most important topics in modern software engineering research. It deals with complex information and large amounts of data. Software visualization can be helpful in this scenario, helping to summarize, analyze and understand software evolution data. This paper presents SourceMiner Evolution (SME), a software tool that uses an interactive differential and temporal approach to visualize software evolution. The tool is implemented as an Eclipse plug-in and has four views that are assembled directly from the IDE AST. The views portray the software from different perspectives. One view shows how metrics of a chosen software entity evolves over time. The other three views show differential comparisons of any two versions of a system structure, dependency and inheritance properties. We exploit recent advances in analysis of graph topology to better understand software evolution, and to construct predictors that facilitate software development and maintenance. Managing an evolving, collaborative software system is a complex and expensive process, which still cannot ensure software reliability. Emerging techniques in graph mining have revolutionized the modeling of many complex systems and processes. We show how we can use a graph-based characterization of a software system to capture its evolution and facilitate development, by helping us estimate bug severity, prioritize refactoring efforts, and predict defect-prone releases. Our work consists of three main thrusts. First, we construct graphs that capture software structure at two different levels: (a) the product, i.e., source code and module level, and (b) the process, i.e., developer collaboration level. We identify a set of graph metrics that capture interesting properties of these graphs. Second, we study the evolution of eleven open source programs, including Firefox, Eclipse, MySQL, over the lifespan of the programs, typically a decade or more. Third, we show how our graph metrics can be used to construct predictors for bug severity, high-maintenance software parts, and failure-prone releases. Our work strongly suggests that using graph topology analysis concepts can open many actionable avenues in software engineering research and practice. This paper presents a technique for visualizing the interactions between developers in software project evolution. The goal is to produce a visualization that shows more detail than animated software histories, like code_swarm [15], but keeps the same focus on aesthetics and presentation. Our software evolution storylines technique draws inspiration from XKCD's "Movie Narrative Charts" and the aesthetic design of metro maps. We provide the algorithm, design choices, and examine the results of using the storylines technique. Our conclusion is that the it is able to show more details when compared to animated software project history videos. However, it does not scale to the largest projects, such as Eclipse and Mozilla. Supporting database schema evolution represents a long-standing challenge of practical and theoretical importance for modern information systems. In this paper, we describe techniques and systems for automating the critical tasks of migrating the database and rewriting the legacy applications. In addition to labor saving, the benefits delivered by these advances are many and include reliable prediction of outcome, minimization of downtime, system-produced documentation, and support for archiving, historical queries, and provenance. The PRISM PRISM++ system delivers these benefits, by solving the difficult problem of automating the migration of databases and the rewriting of queries and updates. In this paper, we present the PRISM PRISM++ system and the novel technology that made it possible. In particular, we focus on the difficult and previously unsolved problem of supporting legacy queries and updates under schema and integrity constraints evolution. The PRISM PRISM++ approach consists in providing the users with a set of SQL-based Schema Modification Operators (SMOs), which describe how the tables in the old schema are modified into those in the new schema. In order to support updates, SMOs are extended with integrity constraints modification operators. By using recent results on schema mapping, the paper (i) characterizes the impact on integrity constraints of structural schema changes, (ii) devises representations that enable the rewriting of updates, and (iii) develop a unified approach for query and update rewriting under constraints. We complement the system with two novel tools: the first automatically collects and provides statistics on schema evolution histories, whereas the second derives equivalent sequences of SMOs from the migration scripts that were used for schema upgrades. These tools were used to produce an extensive testbed containing 15 evolution histories of scientific databases and web information systems, providing over 100 years of aggregate evolution histories and almost 2,000 schema evolution steps. Modern software has to evolve to meet the needs of stakeholders; but the nature and scope of this evolution is difficult to anticipate and manage. In this paper, we examine techniques which can discover interesting patterns of evolution in large object-oriented systems. To locate patterns, we use clustering to group together classes which change in the same manner at the same time. Then, we use dynamic time warping to find if a group of classes is similar to another when we ignore the exact moment when changes occur. Groups that exhibit distinctive evolution properties are potential candidates for new evolution patterns. Finally, in a study of two industrial open-source libraries, we identified four new types of change patterns whose usefulness is determined by perusal of the release notes and the architecture. Many real-world domains can be represented as large node-link graphs: backbone Internet routers connect with 70,000 other hosts, mid-sized Web servers handle between 20,000 and 200,000 hyperlinked documents, and dictionaries contain millions of words defined in terms of each other. Computational manipulation of such large graphs is common, but previous tools for graph visualization have been limited to datasets of a few thousand nodes. Visual depictions of graphs and networks are external representations that exploit human visual processing to reduce the cognitive load of many tasks that require understanding of global or local structure. We assert that the two key advantages of computer-based systems for information visualization over traditional paper-based visual exposition are interactivity and scalability. We also argue that designing visualization software by taking the characteristics of a target user's task domain into account leads to systems that are more effective and scale to larger datasets than previous work. This thesis contains a detailed analysis of three specialized systems for the interactive exploration of large graphs, relating the intended tasks to the spatial layout and visual encoding choices. We present two novel algorithms for specialized layout and drawing that use quite different visual metaphors. The H3 system for visualizing the hyperlink structures of web sites scales to datasets of over 100,000 nodes by using a carefully chosen spanning tree as the layout backbone, 3D hyperbolic geometry for a Focus+Context view, and provides a fluid interactive experience through guaranteed frame rate drawing. The Constellation system features a highly specialized 2D layout intended to spatially encode domain-specific information for computational linguists checking the plausibility of a large semantic network created from dictionaries. The Planet Multicast system for displaying the tunnel topology of the Internet's multicast backbone provides a literal 3D geographic layout of arcs on a globe to help MBone maintainers find misconfigured long-distance tunnels. Each of these three systems provides a very different view of the graph structure, and we evaluate their efficacy for the intended task. We generalize these findings in our analysis of the importance of interactivity and specialization for graph visualization systems that are effective and scalable. The paper presents an adaptive approach to perform impact analysis from a given change request to source code. Given a textual change request (e.g., a bug report), a single snapshot (release) of source code, indexed using Latent Semantic Indexing, is used to estimate the impact set. Should additional contextual information be available, the approach configures the best-fit combination to produce an improved impact set. Contextual information includes the execution trace and an initial source code entity verified for change. Combinations of information retrieval, dynamic analysis, and data mining of past source code commits are considered. The research hypothesis is that these combinations help counter the precision or recall deficit of individual techniques and improve the overall accuracy. The tandem operation of the three techniques sets it apart from other related solutions. Automation along with the effective utilization of two key sources of developer knowledge, which are often overlooked in impact analysis at the change request level, is achieved. To validate our approach, we conducted an empirical evaluation on four open source software systems. A benchmark consisting of a number of maintenance issues, such as feature requests and bug fixes, and their associated source code changes was established by manual examination of these systems and their change history. Our results indicate that there are combinations formed from the augmented developer contextual information that show statistically significant improvement over stand-alone approaches. SUMMARY Substantial research in software engineering focuses on understanding the dynamic nature of software systems in order to improve software maintenance and program comprehension. This research typically makes use of automated instrumentation and profiling techniques after the fact, that is, without considering domain knowledge. In this paper, we examine another source of dynamic information that is generated from statements that have been inserted into the code base during development to draw the system administrators' attention to important run-time phenomena. We call this source communicated information (CI). Examples of CI include execution logs and system events. The availability of CI has sparked the development of an ecosystem of Log Processing Apps (LPAs) that surround the software system under analysis to monitor and document various run-time constraints. The dependence of LPAs on the timeliness, accuracy and granularity of the CI means that it is important to understand the nature of CI and how it evolves over time, both qualitatively and quantitatively. Yet, to our knowledge, little empirical analysis has been performed on CI and its evolution. In a case study on two large open source and one industrial software systems, we explore the evolution of CI by mining the execution logs of these systems and the logging statements in the source code. Our study illustrates the need for better traceability between CI and the LPAs that analyze the CI. In particular, we find that the CI changes at a high rate across versions, which could lead to fragile LPAs. We found that up to 70 of these changes could have been avoided and the impact of 15 to 80 of the changes can be controlled through the use of robust analysis techniques by LPAs. We also found that LPAs that track implementation-level CI (e.g. performance analysis) and the LPAs that monitor error messages (system health monitoring) are more fragile than LPAs that track domain-level CI (e.g. workload modelling), because the latter CI tends to be long-lived. Copyright © 2013 John Wiley & Sons, Ltd. A key problem in software engineering is changing the code. We present a sequence of visualizations and visual metaphors designed to help engineers understand and manage the software change process. The principal metaphors are matrix views, cityscapes, bar and pie charts, data sheets and networks. Linked by selection mechanisms, multiple views are combined to form perspectives that both enable discovery of high-level structure in software change data and allow effective access to details of those data. Use of the views and perspectives is illustrated in two important contexts: understanding software change by exploration of software change data and management of software development. Our approach complements existing visualizations of software structure and software execution. Sufficient training examples are the fundamental requirement for most of the learning tasks. However, collecting well-labelled training examples is costly. Inspired by Zero-shot Learning (ZSL) that can make use of visual attributes or natural language semantics as an intermediate level clue to associate low-level features with high-level classes, in a novel extension of this idea, we aim to synthesise training data for novel classes using only semantic attributes. Despite the simplicity of this idea, there are several challenges. First, how to prevent the synthesised data from over-fitting to training classes? Second, how to guarantee the synthesised data is discriminative for ZSL tasks? Third, we observe that only a few dimensions of the learnt features gain high variances whereas most of the remaining dimensions are not informative. Thus, the question is how to make the concentrated information diffuse to most of the dimensions of synthesised data. To address the above issues, we propose a novel embedding algorithm named Unseen Visual Data Synthesis (UVDS) that projects semantic features to the high-dimensional visual feature space. Two main techniques are introduced in our proposed algorithm. (1) We introduce a latent embedding space which aims to reconcile the structural difference between the visual and semantic spaces, meanwhile preserve the local structure. (2) We propose a novel Diffusion Regularisation (DR) that explicitly forces the variances to diffuse over most dimensions of the synthesised data. By an orthogonal rotation (more precisely, an orthogonal transformation), DR can remove the redundant correlated attributes and further alleviate the over-fitting problem. On four benchmark datasets, we demonstrate the benefit of using synthesised unseen data for zero-shot learning. Extensive experimental results suggest that our proposed approach significantly outperforms the state-of-the-art methods. We present an Evolutionary Visual Exploration (EVE) system that combines visual analytics with stochastic optimisation to aid the exploration of multidimensional datasets characterised by a large number of possible views or projections. Starting from dimensions whose values are automatically calculated by a PCA, an interactive evolutionary algorithm progressively builds (or evolves) non-trivial viewpoints in the form of linear and non-linear dimension combinations, to help users discover new interesting views and relationships in their data. The criteria for evolving new dimensions is not known a priori and are partially specified by the user via an interactive interface: (i) The user selects views with meaningful or interesting visual patterns and provides a satisfaction score. (ii) The system calibrates a fitness function (optimised by the evolutionary algorithm) to take into account the user input, and then calculates new views. Our method leverages automatic tools to detect interesting visual features and human interpretation to derive meaning, validate the findings and guide the exploration without having to grasp advanced statistical concepts. To validate our method, we built a prototype tool (EvoGraphDice) as an extension of an existing scatterplot matrix inspection tool, and conducted an observational study with five domain experts. Our results show that EvoGraphDice can help users quantify qualitative hypotheses and try out different scenarios to dynamically transform their data. Importantly, it allowed our experts to think laterally, better formulate their research questions and build new hypotheses for further investigation. We evaluate and analyse a framework for evolutionary visual exploration EVE that guides users in exploring large search spaces. EVE uses an interactive evolutionary algorithm to steer the exploration of multidimensional data sets toward two-dimensional projections that are interesting to the analyst. Our method smoothly combines automatically calculated metrics and user input in order to propose pertinent views to the user. In this article, we revisit this framework and a prototype application that was developed as a demonstrator, and summarise our previous study with domain experts and its main findings. We then report on results from a new user study with a clearly predefined task, which examines how users leverage the system and how the system evolves to match their needs. While we previously showed that using EVE, domain experts were able to formulate interesting hypotheses and reach new insights when exploring freely, our new findings indicate that users, guided by the interactive evolutionary algorithm, are able to converge quickly to an interesting view of their data when a clear task is specified. We provide a detailed analysis of how users interact with an evolutionary algorithm and how the system responds to their exploration strategies and evaluation patterns. Our work aims at building a bridge between the domains of visual analytics and interactive evolution. The benefits are numerous, in particular for evaluating interactive evolutionary computation IEC techniques based on user study methodologies. References.- Technical Foundations.- 1 Introduction.- 2 Graphs and Their Representation.- 3 Graph Planarity and Embeddings.- 4 Graph Drawing Methods.- References.- WilmaScope - A 3D Graph Visualization System.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- Pajek - Analysis and Visualization of Large Networks.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- Tulip - A Huge Graph Visualization Framework.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- Graphviz and Dynagraph - Static and Dynamic Graph Drawing Tools.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- AGD - A Library of Algorithms for Graph Drawing.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- yFiles - Visualization and Automatic Layout of Graphs.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- GDS - A Graph Drawing Server on the Internet.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- BioPath - Exploration and Visualization of Biochemical Pathways.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- DBdraw - Automatic Layout of Relational Database Schemas.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- GoVisual - A Diagramming Software for UML Class Diagrams.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- CrocoCosmos - 3D Visualization of Large Object-oriented Programs.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- ViSta - Visualizing Statecharts.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- visone - Analysis and Visualization of Social Networks.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References.- Polyphemus and Hermes - Exploration and Visualization of Computer Networks.- 1 Introduction.- 2 Applications.- 3 Algorithms.- 4 Implementation.- 5 Examples.- 6 Software.- References. “Evolution behaves like a tinkerer” (Francois Jacob, Science, 1977). Software systems provide a singular opportunity to understand biological processes using concepts from network theory. The Debian GNU Linux operating system allows us to explore the evolution of a complex network in a unique way. The modular design detected during its growth is based on the reuse of existing code in order to minimize costs during programming. The increase of modularity experienced by the system over time has not counterbalanced the increase in incompatibilities between software packages within modules. This negative effect is far from being a failure of design. A random process of package installation shows that the higher the modularity, the larger the fraction of packages working properly in a local computer. The decrease in the relative number of conflicts between packages from different modules avoids a failure in the functionality of one package spreading throughout the entire system. Some potential analogies with the evolutionary and ecological processes determining the structure of ecological networks of interacting species are discussed. Evolution in scientific software is often according to a specific pattern of software changes: professional scientists, who are not professional software developers, need rapid, dynamic, and domain-specific changes of the software they work this. To address unanticipated software evolution in this field, our objective is to enable these end-users (here: biologists) to change software from the user interface. An approach is presented that integrates technological and methodological solutions. We explain why these solutions are complementary, and how they can be integrated and co-evolved from software design to actual use. Software visualization can be very useful for answering complex questions that arise in the software development process. Although modern visualization engines offer expressive APIs for building such visualizations, developers often have difficulties to (1) identify a suitable visualization technique to answer their particular development question, and to (2) implement that visualization using the existing APIs. Examples that illustrate the usage of an engine to build concrete visualizations offer a good starting point, but developers may have to traverse long lists of categories and analyze examples one-by-one to find a suitable one. We propose MetaVis, a tool that fills the gap between existing visualization techniques and their practical applications during software development. We classify questions frequently formulated by software developers and for each, based on our expertise, identify suitable visualizations. MetaVis uses tags mined from these questions to offer a tag-iconic cloud-based visualization. Each tag links to suitable visualizations that developers can explore, modify and try out. We present initial results of an implementation of MetaVis in the Pharo programming environment. The tool visualizes 76 developers' questions assigned to 49 visualization examples.