Empirical analysis in software process simulation modeling
Abstract Software process simulation modeling is increasingly being used to address a variety of issues from the strategic management of software development, to supporting process improvements, to software project management training. The scope of software process simulation applications ranges from narrow focused portions of the life cycle to longer-term product evolutionary models with broad organizational impacts. This paper discusses some of the important empirical issues that arise in software process simulation modeling. We first address issues concerning real-world data used to (1) establish input parameters to a software process simulation model, and (2) establish actual organizational results against which the model’s results (i.e., outputs) will be compared. On the input side, the challenges include small sample sizes, considerable variability and outliers, lack of desired data, loosely defined metrics, and so forth. On the output side, the paper addresses (1) verification and validation of the model, and (2) quantitative approaches to evaluating model outputs in support of managerial decision making including financial performance using Net Present Value (NPV), multi-criteria utility functions, and Data Envelopment Analysis (DEA). The paper focuses on the stochastic modeling using Monte Carlo simulation. The paper is grounded in the authors’ practical application experiences, and major points are illuminated by examples drawn from that field work.