they were assessed most likely to produce variation in the dependent variables.

Sec. REF  also mentioned the Future Estimate Probability \(p_e\) —the probability that agents will exchange estimates of their future design projections instead of their current design values [1]}, [2]}. Variable \(p_e\)  was included to demonstrate how deviating from assumptions can reveal more and less optimal strategies for CESDPs contingent upon other variables (see Sec. REF ). Combined, the independent variable sets consisted of 13,552 unique combinations, each of which ran 100 times for a total of 1,355,200 executions of CESIUM.
The authors first quantified the main and interaction effects by utilizing multivariate ordinary least squares (OLS) regressions with robust standard errors [3]}, [4]}, as described by Models II.1–7 in Table REF  for system performance \(F\) , and by Models III.1–6 in Table REF  for the number of design cycles \(N\) . To start, Models II.1 & III.1 found main effects for the five independent variables through OLS regression of the full dataset (Num. Obs. \(=\)  1,355,200) with objective function dummy variables capturing the objective function fixed effects and the Absolute Sum function represented in the constant. Model II.1 accounted for 61.8% of the System Performance variance and Model III.1 for 72.4% of the variance in the Number of Design Cycles. These two models suggested that CESDP outcomes significantly depend on design space objective functions, system size (Num. Nodes.), development process stopping conditions (Conv. Thresh.), and even the type of information exchanged within development processes (Fut. Est. Prob.). Still, the large residual standard errors indicated significant variation remained unaccounted for which could reside in interaction or more complex relationships.
Models II.2 (R-squared: 96.9%) & III.2 (R-squared: 81.3%) included interaction effects, moderately improving the residuals over Models II.1 & III.1 while attributing significant variation to most of the interaction terms. The significant differences between the objective function interaction coefficients supported that an organization's ability to develop an optimal CES significantly depends on the objective functions corresponding to the design spaces of its constituent artifacts. Models II.3–6 and III.3–6 provided further support for this hypothesis by isolating the interaction terms and residual standard errors wthin the objective function data subsets (Num. Obs. \(=\)  338,800 each). The Levy function absorbed the largest residuals, leaving more-proportionate residuals for the other functions while measuring a variety of interaction effect amplitudes within the functions. The Sphere function System Performance (Model II.4) remained an outlier among the regressions, though, only explaining 12.5% of the variance before log transformations of the System Performance, Number of Nodes, and Convergence Threshold (Model II.7, R-squared: 66.3%).
According to the models, nearly all of the system outcomes depend strongly on the Number of Nodes, the Convergence Threshold, or both, though rarely through interaction effects. The Number of Nodes tended to increase the Number of Design Cycles, while the Convergence Threshold consistently revealed an inverse relationship between System Performance and the Number of Design Cycles, regardless of the objective function selected. In contrast, the main and interaction effects of the Triangle and Future Estimate Probabilities proved scattered with only select cases generating significant effects. For example, Models II.3 (Absolute Sum), II.4 (Sphere), & II.5 (Ackley) identified statistically significant effects between the probability of triangle formation in the system network and the probability of sharing future estimates (Tri. Prob. \(\times \)  Fut. Est. Prob.) with magnitudes comparable to their respective constants, an effect that was limited in the Ackley and absent from the Levy function.
To further support the previous regressions, and capture any non-linear or higher-order effects of each variable, a feature importance analysis using a random forest regressor was applied, shown in Fig. REF . Random forests fit an ensemble of decisions trees to predict the outcome variable, and feature importances describe which variables best reduce errors in outcome predictions when trees split the data on that variable [5]}. Random forests were fit using the default parameters of sci-kit learn v0.24.1 with a maximum depth of four. The results proved largely consistent with the regressions with a few noteworthy exceptions. Again, the Number of Nodes and Convergence Thresholds proved most important. However, the Triangle and Future Estimate Probabilities only substantially affected the Sphere function performance, suggesting their effects may be limited to particular topologies. This somewhat contradicts the regression results which additionally identified an interaction effect for the Absolute Sum and Ackley functions, suggesting that the regressions for those functions may have missed nonlinear and/or higher-order effects, or excluded uncaptured variation in either the network structure or simulated annealing searches. That said, the contributions of the Triangle and Future Estimate Probabilities remained significant for the Sphere function.
Validation
Because CESIUM models organizational processes, this work draws from organization science for its definitions of micro- and macrovalidation. Microvalidation ensures that the behaviors and mechanisms coded into the model sufficiently represent, or are “grounded” in, their real-world analogs. Macrovalidation is divided into face validation—whether results are superficially distinguishable from real-world conceptual phenomena—and empirical validation—whether results are statistically calibrated to match real-world data [6]}, [7]}.
To satisfy the conditions of microvalidation, Sec.  grounds each construction decision of CESIUM in either established research or simplified abstractions of real-world practices. To satisfy face and empirical macrovalidation, simulation results would need to correspond to outcomes one might encounter in practice. While face validation occasionally proves possible, Sec. REF  cites how insufficient data exists about CESDPs to perform empirical validation. Indeed, CESIUM is designed to address the absence of sufficient data by reconstructing the underlying mechanisms governing development processes and generating data instead.
Consequently, it becomes beneficial to view modeling frameworks like CESIUM from a different perspective. Carley refers to computational models as “hypothesis generating machine[s]” [6]}, in CESIUM's case generating data with which to form hypotheses about causal variables in CESDPs. With sufficient evidence from multiple sources of knowledge, researchers can validate those hypotheses and they become established systems engineering theories. Thereafter, systems engineers can devise methods for improving the outcomes resulting from those relationships.
Contributing to that process, the following section will demonstrate both the creation of hypotheses and formation of theory by validating the results of the sensitivity analysis.

Theory Building Demonstration
Consider the results of the sensitivity analysis for each independent variable:

The Number of Design Cycles tended to increase proportional to the Number of Nodes, while System Performance varied depending on the objective function.

Both system outcomes varied depending on the Objective Functions, via both main and interaction effects.

Triangle Probability had small or statistically insignificant effects on system outcomes for all but the Sphere function, where System Performance varied as a function of Triangle Probability, Future Estimate Probability, and interaction between the two.

Increasing the size of Convergence Threshold tended to decrease the Number of Design Cycles while degrading the System Performance.

Increasing the Future Estimate Probability had small or statistically insignificant effects on system outcomes for all but the Sphere function, where System Performance varied through interaction with the Triangle Probability.

Beginning with Result REF , this result suggests that even if System Performance varies differently for different objective functions, (larger) systems with more artifacts will tend to take longer to converge on system designs. This observation can be posed as a hypothesis for further examination.
Hypothesis 1 
The time it takes an organization to converge on a system design increases as a function of system size.

Scholars can now examine this hypothesis against existing and novel studies to determine whether it merits conversion into a theory.
Next, note that Results REF -REF  & REF  involve variation in system outcomes depending on the objective functions of the agents. This, too, could easily form a hypothesis. However, the finding is consistent with extensive research from Game Theory, Decision Science, and forms the underlying premise of Value-Driven Design [9]}, thereby providing at least face validation for the hypothesis. With abundant evidence from multiple disciplines, the claim is more accurately stated as an established piece of knowledge in the form of a theory. Recall that CESIUM assumes that the utilities of the agents designing system artifacts are synonymous with the objective functions of the artifacts. Then:
Postulate 1 
The outcomes of a complex engineered system development process depend on the objective functions of the agents who develop system artifacts.

Variation in the other regression coefficients coincident with the objective functions leaves question as to the relationship between even the most significant of the remaining variables and system outcomes. Nevertheless, CES outcomes certainly depended on the selected function giving this first postulate in combination with existing evidence.
The results lend themselves to two further hypotheses meriting examination. Result REF  suggests:
Hypothesis 2 Given a system objective function in negative null form, system performance varies inversely proportionately to system convergence time as a function of maximum uncertainty required in system performance.

Finally, recall that the Triangle Probability is a proxy for modularity or clustering in the technical network of the system architecture, and that the Future Estimate Probability examines the likelihood that an agent will communicate information about either current design statuses or projected design outcomes gives the final hypothesis. Combining these abstractions with Results REF  & REF  gives:
Hypothesis 3 System performance varies as a function of interactions between modularity in the system architecture and the temporal references of information exchanged within the organization.


Future Directions
As a framework, CESIUM's conceptualization of CESDPs presents numerous opportunities for further exploration including system architectures, system development processes, artifact design, patterns and contents of interaction, and objective functions. To facilitate exploration, this section begins by generalizing the framework (Sec. REF ). Then, it outlines recommended directions for future inquiry based on the framework to facilitate the development of theory (Sec. REF ).
Framework Generalization
The base instance of CESIUM makes simplifying assumptions about CES and CESDPs including a scale-free degree distribution of artifacts, no exogenous variables, one agent per artifact via the mirroring hypothesis, simulated annealing artifact design, a design cycle development process, and a simple sum of artifact performances as representative of CES performance. The following paragraphs relax those assumptions and, in so doing, opens opportunities for exploration.
Assume that a CES is composed of \(n_a\)  artifacts interacting in a technical network and system architecture described by adjacency matrix \(A_{ij}\) . The \(n_b\)  members of an organization (engineers and otherwise) can be represented by agents in an agent-based model which exchange information with one another in a communication network described by adjacency matrix \(B_{qr}\) . Each of the \(n_b\)  agents acts upon or “works on” some set of the \(n_a\)  artifacts according to matrix \(C_{iq}\) . Summing over \(q\)  then yields the number of agents that contribute to the design of each artifact \(\lambda _i=\sum _q C_{iq}\) .
Agents apply design decisions to each artifact they work on, influenced by numerous specific and contextual factors. Describe the decisions made by agent \(w\in \lbrace 1,\ldots ,\lambda _i\rbrace \)  about the \(i\)th artifact with a decision variable vector \(\mathbf {v}_{iw}=[v_{iw1},v_{iw2},\ldots ]\)  of contributions made by each agent working on the artifact. The contributions of any variables exogenous to the system can be described by \(\mathbf {u}_{i}=[u_{i1},u_{i2},\ldots ]\) . As before, these variables can be representatively parameterized so that each artifact is modeled by a single decision variable \(x_i(\mathbf {v}_{i1},\ldots ,\mathbf {v}_{i\lambda _i},\mathbf {u}_i)\)  (but this time of vectors \(\mathbf {v}_{iw}\)  and \(\mathbf {u}_i\) ), thereby mapping a set of complex factors to a more manageable form.
This parameterized specification for the design process of each artifact makes it possible to describe the performance for each as \(y_i\)  which is a function of the artifact's own design \(x_i\) , the vector of designs \(\mathbf {x}_j\)  corresponding to the \(k_i\)  interacting artifacts, and a vector of any exogenous variables \(\mathbf {z}_i\)  that influence performance. The combined notation \(\mathbf {x}_i=[x_i,\mathbf {x}_j,\mathbf {z}_i]\)  then allows the artifact's performance to be described in terms of an objective function \(y_i=f_i(\mathbf {x}_i)\) . Over time, agents can individually act toward the improvement of their own objective functions, CES objective functions, or both. Hence, given one function for each artifact and another vector of exogenous contributions to the system \(\mathbf {\xi }\) , multi-objective functions can be formed quantifying overall outcomes for the CESDP including system performance at different points in time, \(F(t, f_1(\mathbf {x}_1),\ldots ,f_i(\mathbf {x}_i),\ldots ,f_n(\mathbf {x}_n),\mathbf {\xi })\) .
While abstract in this form, these generalizations enable researchers to vary assumptions within the CESIUM framework, from which it becomes possible to explore variable dependencies, pose hypotheses, and form theories about CESDPs. The following section enumerates some such options.

Recommended Investigations
Real technical network distributions vary, so exploring bow-tie structures generated by HOT (Sec. REF ) and complete graphs (in which every node is connected to every other node) could yield novel insights about how artifact interaction networks shape system outcomes. Again, the clustering parameter of the Holme-Kim algorithm leaves it well-suited to study subsystem formation and interaction, closely connecting it to current practices. Of course, artifact networks are rarely static in practice, so incorporating network evolution into CESIUM could shed light on how temporal variation affects both agent-level and system-level outcomes. And because even CES vary dramatically in size, exploring different scales may reveal how outcomes manifest as a result of hierarchical decomposition of systems and system of systems interactions.
Development processes themselves also merit exploration including different permutations of the relationships between artifacts, variable composition, agent contribution, and design space exploration. Artifacts undergo different development processes depending on the industry, design process, designer identity, and designer context [10]}, each of which could be studied through CESIUM. Likewise, many CESDPs employ processes that are more involved than the turn-based Shewhart & Deming Cycle (Sec. REF ), including the Systems V, agile development, scrum, etc. The ability to explore such varied development processes may prove one of the greatest opportunities afforded by CESIUM.
The framework also opens possibilities for studying the contents of interactions between contributors. While researchers have begun to investigate communication frequency in other contexts, the contents of communication and interpretation have received limited treatment to date, thereby lending themselves to questions of feedback, coordination between designers [11]}, and different mediums of information exchange.
Different objective constructions could prove extremely informative at the agent, subsystem, system, and system of systems levels. The functions selected for this work began to explore concepts such as linear relationships between variable contributions (Absolute Sum), parabolic relationships (Sphere), local optima (Ackley), and disproportionate influence by collaborators (Levy), though innumerable topologies remain possible at each level. System performance certainly takes forms other than sums of its parts, creating opportunities to study questions of collective innovation, high-reliability organizations, and evolving landscapes, from completely abstract to precisely representative of real-world systems. Hopefully these suggestions will contribute to the beginning of theory building for CES and CESDPs.

Conclusion
Complex engineered systems play essential roles in society. However, decades of increasing system complexity and development process inadequacies have garnered substantial concern about the capabilities of established systems engineering processes. Leading experts from government, academic, and industry organizations now call for the development of theories to guide solutions to these growing needs. But to date, the notorious difficulty of gathering data from real-world complex engineered system development processes has stymied efforts to form and validate generalizable hypotheses about how systems develop.
This work responds to that challenge by uniting recent advances from systems engineering, complex systems, engineering design, and organization science to form the ComplEx System Integrated Utilities Model. Beginning from a review of terminology, the piece identifies the abstracted elements composing development processes and integrates techniques for simulating each aspect into a modeling framework. It then demonstrates the ability of CESIUM to generate sizeable datasets representing the outcomes of system development processes from which researchers can form theory. The underlying flexibility of the framework leaves numerous opportunities for researchers to explore abstracted phenomena and real-world examples alike toward the construction of systems engineering theory.

Acknowledgment
Thanks to all of our colleagues and friends who reviewed this work and supported us, including Abigail Jacobs, Vanessa Rojano, Mojtaba Arezoomand, Lynette Shaw, and Maciej Kurant.
