<jupyter_start><jupyter_text>Identifying hot spots by instability metrics with Software Analytics  Question Which functional components are especially instable or change critical?  Data Sources * Java structures of the Shopizer system scanned by jQAssistant and available in Neo4j* Git history of the Shopizer system scanned by jQAssistant and available in Neo4j* identification of functional components in the source code is required (see 01)* matching between change frequency and modules possible  Heuristics * none  Validation * graphical overview of the instability and abstractness per functional component  Implementation * Calculation of the complexity metrics as defined by <NAME> on the level of components   * Efferent Coupling (Ce)      * number of outgoing dependencies (Fan-Out) of a component   * Afferent Coupling (Ca)      * number of incoming dependencies (Fan-In) of a component   * Instability (I) = Ce / (Ce + Ca)      * stability of a component against changes to other components (smaller = more stable)      * but: the more stable, the more difficult a component is to change due to many dependent components   * Abstractness (A) = Na / Nc      * dercentage of abstract types in the component   * Distance (D) = |A + I - 1|      * distance to the optimal relation between abstractness and instability (larger = worse) * Zone of Pain  * stable (small I) and concrete (small A)    * Changes to these components lead to many changes in dependent components* Zone of Uselessnes  * instable (large I ) und abstract (large A)    * Provided components have no usage<jupyter_code>%%cypher
// Identifying all Shopizer nodes (duplicates query from notebook 00)
MATCH (artifact:Main:Artifact{group: "com.shopizer"})
SET   artifact:Shopizer
WITH  artifact
MATCH (artifact)-[:CONTAINS]->(c)
SET   c:Shopizer
%%cypher
// Creating a node per functional component (duplicates query from notebook 01)
MATCH    (p:Package:Shopizer)-[:CONTAINS]->(bC:Package:Shopizer)
WHERE    p.fqn = "com.salesmanager.core.business.services"
WITH     collect(DISTINCT bC.name) AS boundedContexts
UNWIND   boundedContexts AS boundedContext
MERGE    (bC:BoundedContext {name: boundedContext})
%%cypher
// Assigning all types to their respective bounded contexts (duplicates query from notebook 01) 
MATCH    (bC:BoundedContext),
         (p:Package:Shopizer)-[:CONTAINS*]->(t:Type:Shopizer)
WHERE    p.name = bC.name
MERGE    (bC)-[:CONTAINS]->(t)
# Calculation of the module instability
module_instability_query = %cypher MATCH    (bC1:BoundedContext) \
                                   WITH     bC1 \
                                   MATCH    (bC1)-[:CONTAINS]->(t:Type:Shopizer)-[:DEPENDS_ON]->(d:Type:Shopizer)<-[:CONTAINS]-(bC2:BoundedContext) \
                                   WHERE    bC1 <> bC2 \
                                   WITH     bC1,\
                                            count(d) AS EfferentCoupling \
                                   MATCH    (bC1)-[:CONTAINS]->(t:Type:Shopizer)<-[:DEPENDS_ON]-(d:Type:Shopizer)<-[:CONTAINS]-(bC2:BoundedContext) \
                                   WHERE    bC1 <> bC2 \
                                   WITH     bC1, \
                                            EfferentCoupling, count(d) AS AfferentCoupling \
                                   WITH     bC1, \
                                            toFloat(EfferentCoupling) / (EfferentCoupling + AfferentCoupling) AS Instability \
                                   RETURN   bC1.name AS Name, Instability \
                                   ORDER BY Instability DESC
module_instability_df = module_instability_query.get_dataframe()
# Calculation of the module abstractness
module_abstractness_query = %cypher MATCH    (bC:BoundedContext)-[:CONTAINS]->(t:Type:Shopizer) \
                                    WITH     bC, \
                                             count(t) AS Total \
                                    OPTIONAL MATCH (bC)-[:CONTAINS]->(t:Type:Shopizer) \
                                    WHERE    t:Interface OR exists(t.abstract) \
                                    WITH     bC, \
                                             toFloat(count(t)) / Total AS Abstractness \
                                    RETURN   bC.name AS Name, Abstractness \
                                    ORDER BY Abstractness DESC 
module_abstractness_df = module_abstractness_query.get_dataframe()
# Calculation of the module distance
module_distance_df = pd.merge(module_instability_df, module_abstractness_df, how='outer', on = ['Name'])
module_distance_df = module_distance_df.fillna(0)

module_distance_doc = []
for _id in module_distance_df.T:
    data = module_distance_df.T[_id]
    values = {'value': (data.Abstractness, data.Instability), 'label': data.Name}
    module_distance_doc.append(values)<jupyter_output><empty_output><jupyter_text>Results<jupyter_code>xy_module_chart = pg.XY(stroke=False, x_title='Abstractness', y_title='Instability')
xy_module_chart.title = '<NAME>'
xy_module_chart.add('Abstractness to Instability', module_distance_doc)
xy_module_chart.add('Optimum', [(0, 1), (1, 0)], stroke=True)
xy_module_chart.add('Zone of Pain', [(0, 0.3), (0.3, 0)], stroke=True)
xy_module_chart.add('Zone of Uselesness', [(1, 0.7), (0.7, 1)], stroke=True)
display(HTML(base_html.format(rendered_chart=xy_module_chart.render(is_unicode=True))))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09116073076213324, 'fraction_numerical': 0.007750507473703636, 'mean_word_length': 2.510362694300518, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 22, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 28, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '395883', 'license': 'Apache-2.0', 'n_tokens_mistral': 1575, 'n_tokens_neox': 1471, 'n_words': 432, 'path': 'Software Analytics - 04 - Instability.ipynb', 'repo_name': 'StephanPirnbaum/software-analytics-starter', 'size': '5448'}