<jupyter_start><jupyter_text>Analysis of Q1 on the Maven Dependency Dataset (MDD)This document is the first chapter of a set of notebooks that accompany the paper "Breaking Bad? Semantic Versioning and Impact of Breaking Changes in Maven Central". In this chapter, we investigate Q1 and its corresponding null hypothesis.**Q1**: How are semantic versioning principles applied in the Maven Central Repository in terms of breaking changes?**H$_1$**: BCs are widespread without regard for semantic versioning principles. --- Table of Contents                        Setup                            Dataset                                                Load Dataset                                                    Clean Dataset                                                    Finalize Dataset                                                        Exploratory Data Analysis (EDA)                                                Dataset Summary                                                    Descriptive Statistics                                                    Java Versions                                                    Semantic Versioning Levels                                                    Interval Between Releases                                                    Age                                                        Results                                                Breaking Upgrades                                                    Number of Breaking Changes                                                    Breaking Change Types                                                        EOF             --- Setup<jupyter_code># Import required libraries
library(ggplot2)
library(tidyverse)
library(ggthemes)

# Set theme
theme_set(theme_stata())<jupyter_output>Warning message:
“package ‘ggplot2’ was built under R version 3.5.2”
Warning message:
“package ‘tidyverse’ was built under R version 3.5.2”
── [1mAttaching packages[22m ─────────────────────────────────────── tidyverse 1.3.0 ──

[32m✔[39m [34mtibble [39m 3.1.2     [32m✔[39m [34mdplyr  [39m 1.0.6
[32m✔[39m [34mtidyr  [39m 1.0.0     [32m✔[39m [34mstringr[39m 1.4.0
[32m✔[39m [34mreadr  [39m 1.3.1     [32m✔[39m [34mforcats[39m 0.4.0
[32m✔[39m [34mpurrr  [39m 0.3.3     

Warning message:
“package ‘tidyr’ was built under R version 3.5.2”
Warning message:
“package ‘purrr’ was built under R version 3.5.2”
Warning message:
“package ‘stringr’ was built under R version 3.5.2”
Warning message:
“package ‘forcats’ was built under R version 3.5.2”
── [1mConflicts[22m ────────────────────────────────────────── tidyverse_conflicts() ──
[31m✖[39m [34mdplyr[39m::[32mfilter()[39m masks [34mstats[39m::filter()
[31m✖[39m [34mdplyr[39m::[32mlag()[39m    masks [3[...]<jupyter_text>--- Dataset  Load DatasetFirst, we load the `deltas-raemaekers.csv` dataset which contains information about breaking changes computed by Maracas on the selected upgrades of the MDD.<jupyter_code>deltas <- read.csv("../code/cypher-queries/data/gen/deltas-raemaekers.csv", stringsAsFactors=FALSE, colClasses=c("level"="factor", "year"="factor", "java_version_v1"="factor", "java_version_v2"="factor", "expected_level"="factor"))
sprintf("Successfully loaded %d deltas", nrow(deltas))
head(deltas)<jupyter_output><empty_output><jupyter_text>Clean DatasetHere, we clean the dataset to discard the upgrades and deltas not complying with our requirements.<jupyter_code># What issues did we encounter when attempting to compute the deltas? (Java version >= 8, JAR not produced from Java code, JAR not found on Maven Central, etc.)
table(deltas$exception)
# Which languages did we find?
table(deltas$language)
# Discard all deltas that raised an exception
deltas <- subset(deltas, exception == -1)

# Discard all deltas for upgrades that do not contain any code
sprintf("%d deltas correspond to JARs not containing any code", nrow(subset(deltas, declarations_v1 == 0 & declarations_v2 == 0)))
deltas <- subset(deltas, declarations_v1 > 0 | declarations_v2 > 0)

# Discard all deltas where v1 was released _after_ v2
sprintf("%d where v1 was released after v2", nrow(subset(deltas, age_diff < 0)))
deltas <- subset(deltas, age_diff >= 0)

sprintf("%d remaining deltas after cleaning", nrow(deltas))
# Remove cases where dates are used as versions (e.g. 20081010.0.1, 1.20081010.1, 1.0.20081010)
deltas <- deltas[grep("^[0-9]{0,7}[.][0-9]{0,7}([.][0-9]{0,7})?$", deltas$v1),]
deltas <- deltas[grep("^[0-9]{0,7}[.][0-9]{0,7}([.][0-9]{0,7})?$", deltas$v2),]

deltas <- deltas[!grepl("^2[0-9]{3}[.][0-9]{2}([.][0-9]{2})?$", deltas$v1),]
deltas <- deltas[!grepl("^[0-9]{2}[.][0-9]{2}[.]2[0-9]{3}$", deltas$v1),]
deltas <- deltas[!grepl("^[0-9]{2}[.]2[0-9]{3}$", deltas$v1),]

deltas <- deltas[!grepl("^2[0-9]{3}[.][0-9]{2}([.][0-9]{2})?$", deltas$v2),]
deltas <- deltas[!grepl("^[0-9]{2}[.][0-9]{2}[.]2[0-9]{3}$", deltas$v2),]
deltas <- deltas[!grepl("^[0-9]{2}[.]2[0-9]{3}$", deltas$v2),]

sprintf("%d deltas remaining after removing the ones with dates as versions", nrow(deltas))
# Keep only pairs released before 2012 to match the original study's corpus
deltas <- subset(deltas, as.numeric(as.character(year)) <= 2011)
sprintf("%d deltas remaining after removing the ones with release date >= 2012", nrow(deltas))<jupyter_output><empty_output><jupyter_text>Finalize DatasetHere, we incorporate additional derived information into the dataset.<jupyter_code># Add column with all BCs excluding the ones related to: 
# - annotationDeprecatedAdded: not a BC
# - methodAddedToPublicClass: not a BC
# - classNowCheckedException: not binary incompatible
# - methodNowThrowsCheckedException: not binary incompatible
# - fieldStaticAndOverridesStatic: lack of alignment with JLS
# - superclassModifiedIncompatible: lack of alignment with JLS
# - methodIsStaticAndOverridesNotStatic: lack of alignment with JLS
# - methodAbstractAddedInSuperclass: covered by other BC
# - methodAbstractAddedInImplementedInterface: covered by other BC
# - methodLessAccessibleThanInSuperclass: covered by other BC
# - fieldLessAccessibleThanInSuperclass: covered by other BC
# - fieldRemovedInSuperclass: covered by other BC
# - methodRemovedInSuperclass: covered by other BC

deltas$bcs_clean = deltas$bcs -
    deltas$annotationDeprecatedAdded -
    deltas$methodAddedToPublicClass - 
    deltas$classNowCheckedException - 
    deltas$methodNowThrowsCheckedException -
    deltas$fieldStaticAndOverridesStatic -
    deltas$superclassModifiedIncompatible - 
    deltas$methodIsStaticAndOverridesNotStatic -
    deltas$methodAbstractAddedInSuperclass -
    deltas$methodAbstractAddedInImplementedInterface -
    deltas$methodLessAccessibleThanInSuperclass - 
    deltas$fieldLessAccessibleThanInSuperclass -
    deltas$fieldRemovedInSuperclass -
    deltas$methodRemovedInSuperclass

# Same thing for the stable part of the API
deltas$bcs_clean_stable = deltas$bcs_stable -
    deltas$annotationDeprecatedAdded_stable -
    deltas$methodAddedToPublicClass_stable - 
    deltas$classNowCheckedException_stable - 
    deltas$methodNowThrowsCheckedException_stable -
    deltas$fieldStaticAndOverridesStatic_stable -
    deltas$superclassModifiedIncompatible_stable - 
    deltas$methodIsStaticAndOverridesNotStatic_stable -
    deltas$methodAbstractAddedInSuperclass_stable -
    deltas$methodAbstractAddedInImplementedInterface_stable -
    deltas$methodLessAccessibleThanInSuperclass_stable - 
    deltas$fieldLessAccessibleThanInSuperclass_stable -
    deltas$fieldRemovedInSuperclass_stable -
    deltas$methodRemovedInSuperclass_stable
# Add columns with BCs ratios (i.e. BCs / V1 declarations)
deltas$bcs_ratio_clean = deltas$bcs_clean / deltas$declarations_v1
deltas$bcs_ratio_clean_stable = deltas$bcs_clean_stable / deltas$declarations_v1
# Assign the 'DEV' semver level to versions of the form 0.x.x
levels(deltas$level) <- c(levels(deltas$level), "DEV")
deltas[grepl("^0[.]", deltas$v1),]$level = "DEV"<jupyter_output><empty_output><jupyter_text>--- Exploratory Data Analysis (EDA)  Dataset Summary<jupyter_code>sprintf("Final size of the dataset: %s deltas", nrow(deltas))
summary(deltas)<jupyter_output><empty_output><jupyter_text>Descriptive StatisticsThis section presents some descriptive statistics on the dataset (number of clients, semver levels, popularity, releases, Java versions, etc.).<jupyter_code># Computes a density plot
# dataset: data frame with input data
# dim: vector with input data (column of the data frame)
# dim_label: name of the dimension 
# x_label: label of the x axis
# Returns a ggplot2 density plot
plot_dens_hist <- function(dataset, dim, dim_label, x_label, binw=30) {
    plot <- ggplot(dataset, aes(x=dim, y=..density..)) + 
        labs(title=sprintf("%s Density Histogram", dim_label),
             x=x_label,
             y="Density") +
        geom_histogram(binwidth=30) +
        geom_vline(aes(xintercept=median(dim)), 
                   linetype="dashed",
                   size=0.6)
    
    return(plot)
}
# Computes a density plot
# dataset: data frame with input data
# dim: vector with input data (column of the data frame)
# dim_label: name of the dimension 
# x_label: label of the x axis
# Returns a ggplot2 density plot
plot_dens_bar <- function(dataset, dim, dim_label, x_label) {
    plot <- ggplot(dataset, aes(x=dim, y=..prop.., group=1)) + 
        labs(title=sprintf("%s Density Histogram", dim_label),
             x=x_label,
             y="Density") +
        geom_bar() +
        ylim(0, 1) +
        theme_light(base_size=18)
    
    return(plot)
}<jupyter_output><empty_output><jupyter_text>Java Versions<jupyter_code>p <- plot_dens_bar(deltas, deltas$java_version_v1, "Java Versions", "Java version")
p
ggsave("figures/mdd-java-versions.pdf", p)<jupyter_output>Saving 7 x 7 in image<jupyter_text>Semantic Versioning Levels<jupyter_code>p <- plot_dens_bar(deltas, deltas$level, "Semver Level", "Semver level")
p
ggsave("figures/mdd-semver-levels.pdf", p)<jupyter_output>Saving 7 x 7 in image<jupyter_text>Interval Between Releases The interval between releases is given in months.<jupyter_code>plot_dens_bar(deltas, deltas$age_diff, "Age Difference", "Age diff")<jupyter_output><empty_output><jupyter_text>AgeThe age of the upgrade is relative to the first release of the library (in months).<jupyter_code>plot_dens_bar(deltas, deltas$age, "Age", "Age")<jupyter_output><empty_output><jupyter_text>--- Results  Breaking UpgradesFirst, we study the number/ratio of breaking upgrades per semantic versioning level.<jupyter_code># Get datasets per semver level
major <- subset(deltas, level == "MAJOR")
minor <- subset(deltas, level == "MINOR")
patch <- subset(deltas, level == "PATCH")
devs <- subset(deltas, level == "DEV")
# Computes the number of breaking upgrades
number_breaking_upgrades <- function(dataset) {
    return (nrow(subset(dataset, bcs_clean_stable > 0)))
}

# Computes the percentage of breaking upgrades
perc_breaking_upgrades <- function(dataset) {
    return (nrow(subset(dataset, bcs_clean_stable > 0)) / nrow(dataset))
}

# Computes the percentage of breaking upgrades given a list of datasets
perc_breaking_upgrades_multiple <- function(datasets) {
    total <- 0
    cases <- 0
    
    for (d in datasets) {
        cases <- cases + nrow(subset(d, bcs_clean_stable > 0))
        total <- total + nrow(d)
    }
    return (cases / total)
}
# Get compound datasets
nrow_nonmajor <- nrow(minor) + nrow(patch)
nrow_all <- nrow_nonmajor + nrow(major) + nrow(devs)

breaking_upgrades <- data.frame(
    level = c("MAJOR", "MINOR", "PATCH", "DEV", "NON-MAJOR", "ALL"),
    number = c(nrow(major), nrow(minor), nrow(patch), nrow(devs), nrow_nonmajor, nrow_all),
    ratio = c(nrow(major) / nrow(deltas), nrow(minor) / nrow(deltas), nrow(patch) / nrow(deltas), nrow(devs) / nrow(deltas), nrow_nonmajor / nrow(deltas), nrow_all / nrow(deltas)),
    broken = c(number_breaking_upgrades(major),
               number_breaking_upgrades(minor),
               number_breaking_upgrades(patch),
               number_breaking_upgrades(devs),
               number_breaking_upgrades(minor) + number_breaking_upgrades(patch),
               number_breaking_upgrades(major) + number_breaking_upgrades(minor) + number_breaking_upgrades(patch) + number_breaking_upgrades(devs)),
    broken_ratio = c(perc_breaking_upgrades(major),
             perc_breaking_upgrades(minor),
             perc_breaking_upgrades(patch),
             perc_breaking_upgrades(devs),
             perc_breaking_upgrades_multiple(list(minor, patch)),
             perc_breaking_upgrades_multiple(list(major, minor, patch, devs)))
)

breaking_upgrades
ggplot(breaking_upgrades, aes(x=level, y=broken_ratio)) + 
        labs(title="Ratio of Breaking Upgrades per Semver Level",
             x="Semver level",
             y="Ratio of breaking upgrades") +
        geom_bar(stat="identity")<jupyter_output><empty_output><jupyter_text>Number of Breaking Changes<jupyter_code># Now we are only interested in breaking upgrades, so filter out the other ones
dt <- subset(deltas, bcs_clean_stable > 0)
summary(dt$bcs_clean_stable)
summary(subset(dt, level == "MAJOR")$bcs_clean_stable)
summary(subset(dt, level == "MINOR")$bcs_clean_stable)
summary(subset(dt, level == "PATCH")$bcs_clean_stable)
summary(subset(dt, level == "DEV")$bcs_clean_stable)
# Compute median value
med_fun <- function(x) {
    return (data.frame(y=median(x), label = round(10^median(x), digits=2)))
}

# Compute value of the first quantile
fstq_fun <- function(x) {
    return (data.frame(y=quantile(x)[2], label = round(10^(quantile(x)[2]), digits=2)))
}

# Compute value of the third quantile
thrq_fun <- function(x) {
    return (data.frame(y=quantile(x)[4], label = round(10^(quantile(x)[4]), digits=2)))
}

# Number of breaking changes per broken release per semantic versioning level
p <- ggplot(dt, aes(x=level, y=bcs_clean_stable)) +
    labs(x="Semver level", y="Number of BCs (log10 scale)") +
    geom_violin() +
    geom_boxplot(width=0.3, outlier.shape = NA, coef=0) +
    stat_summary(fun.data = med_fun, geom = "text", vjust=-0.5) +
    stat_summary(fun.data = fstq_fun, geom = "text", vjust=-0.5) +
    stat_summary(fun.data = thrq_fun, geom = "text", vjust=+1.5) +
    scale_y_continuous(trans='log10') +
    theme_light(base_size=18)

p
ggsave("figures/mdd-bc-number.pdf", p)<jupyter_output>Saving 7 x 7 in image<jupyter_text>Breaking Change Types<jupyter_code># Exclude BCs that are not considered in the study
# which(colnames(deltas)=="classNowCheckedException") #41
# which(colnames(deltas)=="methodNowThrowsCheckedException") #61
# which(colnames(deltas)=="fieldStaticAndOverridesStatic") #67
# which(colnames(deltas)=="methodAbstractAddedInSuperclass") #63
# which(colnames(deltas)=="methodAbstractAddedInImplementedInterface") #64
# which(colnames(deltas)=="methodLessAccessibleThanInSuperclass") #51
# which(colnames(deltas)=="fieldLessAccessibleThanInSuperclass") #68
# which(colnames(deltas)=="fieldRemovedInSuperclass") #74
# which(colnames(deltas)=="methodRemovedInSuperclass") #49
# which(colnames(deltas)=="superclassModifiedIncompatible") #45
# which(colnames(deltas)=="methodIsStaticAndOverridesNotStatic") #53

# Get dataframe with semver level and BC types
f <- deltas[c(5, 36:40, 42:44, 46:48, 50, 52, 54:59, 62, 65:66, 69:73, 75:78)]
# Heatmap of occurences of breaking changes per semver level
formatted <- gather(f, "kind", "count", 2:32)

aggregated <- formatted %>%
    group_by(level, kind) %>%
    summarise(log_count=log10(sum(count)))

p <- ggplot(aggregated, aes(x=level, y=reorder(kind, log_count), fill=log_count)) +
    labs(x="Semver level", y="", fill="log10 count") +
    geom_tile() +
    scale_fill_gradientn(colours = c("white", "white", "lightgray", "black"), values = c(-Inf, 0, 0.1, 1)) +
    theme(axis.text.y = element_text(angle=0, size=12), axis.text.x = element_text(size=12), 
          legend.text = element_text(size=10), plot.background = element_rect(fill = "white"))

p
ggsave("figures/mdd-heatmap.pdf", p)
# Histogram with number of breaking changes per semver level
formatted <- gather(f, "kind", "count", 2:32)

aggregated <- formatted %>%
    group_by(level, kind) %>%
    summarise(total=sum(count))

normalized <- aggregated %>%
    group_by(level) %>%
    summarise(kind = kind, ratio = total / sum(total)) %>%
    arrange(across(ratio, desc))

p <- ggplot(normalized, aes(x = reorder(kind,-ratio), y = ratio, fill = level)) + 
    labs(x = "BC kind", y = "Ratio", fill="Semver level") +
    geom_bar(stat = 'identity', position = 'dodge') +
    scale_fill_manual(values = c("grey0","grey30","grey50","grey80")) +
    ylim(0.0, 0.5) +
    theme_bw() + 
    theme(axis.text.x = element_text(angle = 66, vjust = 1, hjust = 1), aspect.ratio=0.75)

p
ggsave("figures/mdd-bc-kind-count.pdf")<jupyter_output>`summarise()` has grouped output by 'level'. You can override using the `.groups` argument.

`summarise()` has grouped output by 'level'. You can override using the `.groups` argument.

Saving 7 x 7 in image
{'fraction_non_alphanumeric': 0.1300736928295707, 'fraction_numerical': 0.025909463095098843, 'mean_word_length': 3.9735311227457824, 'pattern_counts': {'":': 0, '<': 97, '<?xml version=': 0, '>': 64, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 32, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '20587', 'license': 'MIT', 'n_tokens_mistral': 6045, 'n_tokens_neox': 5625, 'n_words': 1314, 'path': 'notebooks/Q1-MDD.ipynb', 'repo_name': 'tdegueul/maven-api-dataset', 'size': '17098'}