After constructing the initial codebook, the two authors iteratively coded sessions, discussing disagreements after each iteration and revising the codebook. Instead of coding a single session, we choose several representative sessions with differing codebases, programming languages, and development tools. Through these iterations, the two authors coded 20 distinct episodes and 166 activities. The last iteration yielded a Chen's Kappa inter-rater agreement [1]} of 75% for episodes and 84% for activities, reflecting substantial and almost perfect agreement, respectively.
Using the final codebook, the first author then coded the entire dataset using observe.dev-online (Section ).
The entire dataset, including the videos and codes, is publicly availablehttps://bit.ly/3kkbL2W.
