Automated builds are integral to the Continuous Integration (CI) software development practice. In CI, developers are encouraged to integrate early and often. However, long build times can be an issue when integrations are frequent. This research focuses on finding a balance between integrating often and keeping developers productive. We propose and analyze models that can predict the build time of a job. Such models can help developers to better manage their time and tasks. Also, project managers can explore different factors to determine the best setup for a build job that will keep the build wait time to an acceptable level. Software organizations transitioning to CI practices can use the predictive models to anticipate build times before CI is implemented. The research community can modify our predictive models to further understand the factors and relationships affecting build times.
Brooks @cite_7 suggested that a build time of 2 minutes was optimal; build times under 10 minutes were considered acceptable. However, the suggested build times were based on experience reports from the same company. In summary, there is a lack of empirical quantitative research to address optimal build times in a CI environment @cite_8 .
@cite_7 @cite_8
2065181971 2099805222 1965854491 2963778169
In this position paper, we present and demonstrate the idea of using an interdisciplinary literature review to accelerate the research on continuous integration practice. A common suggestion has been that build waiting time in continuous integration cycle should be less than 10 minutes. This guideline is based on practitioners' opinion and has not been further investigated. The objective of this study is to understand the effects of build waiting time in software engineering and to get input from waiting time research in other disciplines. The objective is met by performing two literature reviews, first on build waiting time and second on waiting times in the contexts of service operation, web use and computer use. The found effects of build waiting time were categorized into continuous integration specific, cognitive and emotional. Two minute build waiting time was considered optimal, but under 10 minutes was considered acceptable. Insight from other waiting time research suggests that the perceptions of waiting time are important and the perceptions can be lowered by providing feedback and giving developers other activities during the integration. Slow builds can be extremely costly; for example, one of our customers loses forty percent of their developers' productivity to waiting on builds. Build avoidance - building only what must be built to integrate a change - solves the problem but makes the build unreliable unless the build script captures all dependences of build outputs on sources. Therefore, in practice, integrating even a minor change often requires rebuilding a large software system from scratch.Our solution can be summarized as "trust but verify". Exploiting the fact that large software systems tend to be composed of loosely coupled parts, we semi-automatically divide a large, all-or-nothing build into many small "mini-builds", without changing the original build. Each mini-build declares its dependences, which are trusted while scheduling mini-builds to integrate a change but verified by executing each mini-build in a sandbox in which only its declared dependences are available.We implemented these ideas in a tool suite called Grexmk and applied Grexmk to two large all-or-nothing builds. On the first build, we demonstrated linear speedups to six build machines. On the second build, we sped up the average time for an incremental build by a factor of 1.2. Building is an integral part of the software development process. However, little is known about the compiler errors that occur in this process. In this paper, we present an empirical study of 26.6 million builds produced during a period of nine months by thousands of developers. We describe the workflow through which those builds are generated, and we analyze failure frequency, compiler error types, and resolution efforts to fix those compiler errors. The results provide insights on how a large organization build process works, and pinpoints errors for which further developer support would be most effective. We explore efficient neural architecture search methods and present a simple yet powerful evolutionary algorithm that can discover new architectures achieving state of the art results. Our approach combines a novel hierarchical genetic representation scheme that imitates the modularized design pattern commonly adopted by human experts, and an expressive search space that supports complex topologies. Our algorithm efficiently discovers architectures that outperform a large number of manually designed models for image classification, obtaining top-1 error of 3.6 on CIFAR-10 and 20.3 when transferred to ImageNet, which is competitive with the best existing neural architecture search approaches and represents the new state of the art for evolutionary strategies on this task. We also present results using random search, achieving 0.3 less top-1 accuracy on CIFAR-10 and 0.1 less on ImageNet whilst reducing the architecture search time from 36 hours down to 1 hour.