Continuous Experimentation and the Cyber-Physical Systems challenge. An overview in literature and the industrial perspective <s> Format of the case study <s> Abstract Context: Development of software-intensive products and services increasingly occurs by continuously deploying product or service increments, such as new features and enhancements, to customers. Product and service developers must continuously find out what customers want by direct customer feedback and usage behaviour observation. Objective: This paper examines the preconditions for setting up an experimentation system for continuous customer experiments. It describes the RIGHT model for Continuous Experimentation (Rapid Iterative value creation Gained through High-frequency Testing), illustrating the building blocks required for such a system. Method: An initial model for continuous experimentation is analytically derived from prior work. The model is matched against empirical case study findings from two startup companies and further developed. Results: Building blocks for a continuous experimentation system and infrastructure are presented. Conclusions: A suitable experimentation system requires at least the ability to release minimum viable products or features with suitable instrumentation, design and manage experiment plans, link experiment results with a product roadmap, and manage a flexible business strategy. The main challenges are proper, rapid design of experiments, advanced instrumentation of software to collect, analyse, and store relevant data, and the integration of experiment results in both the product development cycle and the software development process. <s> BIB001 </s> Continuous Experimentation and the Cyber-Physical Systems challenge. An overview in literature and the industrial perspective <s> Format of the case study <s> Recently, an increasingly growing number of companies is focusing on achieving self-driving systems towards SAE level 3 and higher. Such systems will have much more complex capabilities than today's advanced driver assistance systems (ADAS) like adaptive cruise control and lane-keeping assistance. For complex software systems in the Web-application domain, the logical successor for Continuous Integration and Deployment (CI/CD) is known as Continuous Experimentation (CE), where product owners jointly with engineers systematically run A/B experiments on possible new features to get quantifiable data about a feature's adoption from the users. While this methodology is increasingly adopted in software-intensive companies, our study is set out to explore advantages and challenges when applying CE during the development and roll-out of functionalities required for self-driving vehicles. This paper reports about the design and results from a multiple case study that was conducted at four companies including two automotive OEMs with a long history of developing vehicles, a Tier-1 supplier, and a start-up company within the area of automated driving systems. Unanimously, all expect higher quality and fast roll-out cycles to the fleet; as major challenges, however, safety concerns next to organizational structures are mentioned. <s> BIB002
The case studies were conducted in a workshop format, each of them lasting between 1.5 and 2 hours, depending on the number of participants. During the workshops, one of the authors would lead it through its different phases, while the other authors would assist and take notes. The format was structured in four phases as follows: Phase I: The workshop would begin with a presentation having the goal of establishing a common understanding and vocabulary of the Continuous practices, i.e. Continuous Integration, Continuous Delivery/Deployment, and Continuous Experimentation. This phase would last around 20 minutes; Phase II: After the initial presentation, the participants were asked the two working questions about Continuous Experimentation, i.e. WQ1 and WQ2. This phase would last around 30 minutes, during which the participants would individually write their answers, each different idea on a different note; Phase III: The participants were asked to go through their notes to explain and clarify the meaning and reasoning behind each of them. Each note would then be placed next to others expressing similar ideas on a whiteboard, thus creating clusters around common ideas. This phase would last around 40 minutes; Phase IV: An infrastructure model for Continuous Experimentation devised for companies with web-based products BIB001 was introduced to the participants. The aim was to start a discussion about the model and its criticalities if it had to be applied to the automotive industry. This phase would last around 15 minutes. The format of these case studies was based on open questions focused on a structured topic, categorizing them as a series of semi-structured case studies  ). This approach was chosen since it fits the exploratory and explanatory goal of the case study by promoting the participants to provide original feedback. Two automotive companies were chosen to run the described case study. Company A is manufacturing heavy vehicles for various commercial operations. From this company 3 representatives joined the case study, 1 manager, 1 team leader and 1 engineer. Company B is producing consumer vehicles. From this company 15 employee took part in our study, where 1 of them was a manager, 7 team leaders and 7 engineers. The overall variety of roles is considered a strengthening factor due to the increased diversity in points of view and resulting perspectives and discussions. Due to the strong connection between the present multiple case study and the previously reported one BIB002 , some details about the composition of the latter will follow. The previous series of case studies involved four companies, adding to the two aforementioned novel cases. 