researchers to study and improve collaborative editing support in crowd-shared platforms.
 We discuss the implications below.
Forum Designers. The quality assurance of shared content is paramount for the usefulness and popularity of a crowd-shared knowledge-sharing platform like SO. While the editing of content allows users to suggest improvements in quality, the lack of proper guidance to the user can lead to unnecessary rejections of the suggested edits. SO can use our developed machine learning model and the EditEx tool to offer on-demand and context-aware edit fix recommendations to the users. As we observed in Section REF , the SO users that used EditEx had significantly less number of rejections compared to the users that did not use EditEx tool. The SO edit assessment queue is set up to ensure novice SO users (with less than 2K reputation) can not make bad edits. Many users in SO fall under this novice category, but their inputs to edits are equally as important as the non-novice users. However, as we noted, the SO edit queue can be often congested with many suggested edits that it could take a disproportionate time for an edit to get reviewed. Even after the review, many suggestions can be rejected due to trivial edits like undesired text formatting. A tool like EditEx can help SO users by reducing such trivial edits, which then can ultimately help SO and its expert edit reviewers with less rejection, which in turn can reduce the workload on the edit queue.
Forum Users. Interactive browser plug-ins like EditEx can warn forum users of potential edits that could be rejected. Thus EditEx can improve the confidence of SO users during the editing. Indeed, as we reported in Section REF , the study participants were more confident while using EditEx than using SO editing system (average confidence of 4 while using EditEx vs 3.1 while using SO). The SO users had only 16% of their edits rejected while using EditEx, while the rejection rate was 65% while using SO. While a rejection ratio of 65% for SO could be biased due to our choice of editing task, the much lower rejection rate while using EditEx does indeed highlight that SO users can benefit from a simple tool like EditEx. EditEx is very easy to install. Given that EditEx is simply a browser plug-in that SO users can easily install, we hope that EditEx will be accepted by the wider SO community.
Indeed, our developed classifiers can be used to detect rejected edit reasons in SO automatically. It can extend current tools and techniques that predominantly use contents from suggested edits to recommend editing suggestions (e.g., see the works of Chen et al. [1]}, [2]}). The tool EditEx, with further modification (as suggested by our study participants in Section REF ), can be influential in the reduction of rejected edit reasons in SO and can improve the overall satisfaction of SO users. In the long term, the tool can promote better content because users will be more motivated. Such high quality contents then can offer better content and recommendation support for tools and techniques that focus on the quality of contents shared in SO [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, the suite of tools and techniques developed to detect and recommend quality posts [9]}, [10]}, [11]}, [12]}, [13]}, [14]}.
Researchers. The quality of knowledge shared in SO is important because developers worldwide now rely on this shared knowledge. Indeed, knowledge shared in SO can support diverse activities like bug fixing, feature enhancement, API selection, and documentation [15]}, [16]}, [17]}, [18]}, [19]}, [20]}, [21]}, [22]}. This sharing of knowledge is important, because official software documentation can often be lacking [23]}, [24]}. However, the editing of content is a voluntary activity. SO users can be demotivated to produce quality edits if they become frustrated due to unnecessary/unwanted rejection of their suggested edits. Tools like EditEx or proactive policy assurance by Chen et al. [25]}, [1]} can help SO users with suggestions to improve their edits. The positive survey responses of our tool EditEx show the potential of deploying the tool in SO. Future research can contribute by including more features into EditEx and by conducting more studies to learn how SO users can further benefit from such tools. Such findings can promote quality contents, which then can support all development tasks that rely on SO, as noted above.
Threats to Validity
Threats to internal validity relate to experimental errors and biases [27]}. We asked participants to suggest edits using EditEx (treatment group) and SO's standard edit system (control group). The suggested edits were either accepted or rejected by the expert review. However, the accepted edits could be rejected later by a rollback that might affect the edit rejection ratio. We thus further analyzed how many accepted edits were rejected by rollbacks in our dataset. However, such a statistic was less than 1% in our dataset. Thus, it might not affect our results significantly.
Threats to external validity relate to the generalizability of a technique. We agree that there might be rejection reasons that we could not identify. However, we analyze statistically significant samples of rejected edits by rollbacks from questions and answers. We thus believe that our manual investigation exposes all the main rejection reasons. Still, there is scope to analyze more samples to explore additional rejection reasons. However, suggested edits can be rejected by either rollback or expert review. Unfortunately, we could not collect samples of those edits rejected by expert reviews. We could not find any convenient way to collect such samples since their information is not readily available in the SO data dump. Thus, similar to existing literature, we consider the rejected edits by rollbacks [28]}. However, suggested edits are reviewed in SO by users with at least a 2K reputation score. In our manually analyzed dataset, 90.6% (692 out of 764) of users who rolled back edits have a reputation score \(\ge \)  2K. The remaining were self-rollback (i.e., rolled back by post owner). Therefore, our intuition is that the main reasons for rejections by expert reviews would be similar to our identified reasons.
Our survey participants range from novice to experienced, mainly software developers and academicians (Table REF ). Such diversity in the survey participants offers validity and applicability to the survey findings. Furthermore, we ensure that control and treatment groups have participants with different professions and experience levels to mitigate individual bias.
We set a task list T1–T7 (Table REF ) and asked participants (control & treatment group) to suggest edits to posts based on that list. However, tasks T1–T6 were set based on our identified rejection reasons. Thus, EditEx alerts the treatment group while suggesting T1–T6, but the control group does not get such alerts. As a result, the treatment group receives favor from EditEx, which might reduce their rejections. To mitigate this bias, we asked participants to suggest free-form edits (T7). From T7, we attempt to see the effectiveness of EditEx in assisting users in suggesting regular edits besides preventing common rejections. However, EditEx can support the SO edit system to prevent 12% rejections. Such a finding confirms the effectiveness of EditEx in preventing not only common rejections but also rejections from regular edits.

Deviations From Registered Report
This section discusses the deviations of this study from our registered report [29]} and explains them.
Rollback Reasons & Predictors.
In our registered report, we wanted to see whether the addition/deletion of emotion influences edit rejection using EmoTxt [30]}. Our primary analysis found that emotion has almost no effect on edit rejection/acceptance. Furthermore, the overall accuracy of our rejected edit classifier improved only about 1% if we consider emotion as a predictor. However, integrating a complex model to capture emotion and its deployment is costly. Therefore, it can affect the performance of our online tool EditEx. We thus discard emotion from this study.
On the other hand, we added reputation as a predictor (Table REF ). The reputation score estimates how much the community trusts a user [31]}, [32]}. Therefore, we added Community Trust as a rollback reason (which was absent in the registered report) (Table REF ) that was estimated by reputation score. Moreover, users with lower reputation scores might violate the editing guidelines more than those with higher reputation scores. Note that violating edit guidelines is one of the causes of edit rejection. We thus consider reputation as a predictor that significantly improves the performance of our rejected edit classifier.
Another discrepancy to the registered report is that we included the Introducing Spam rollback reason under Other.
Manually Investigated Sample Size.
In the registered report, we manually investigated 777 rollback edits (382 questions + 395 answers). The statistically significant sample size of rollback edits for both question and answer is 382. However, due to a programming problem, we randomly selected 395 samples from rejected answer revisions. We kept 395 samples in the registered report because - (1) we completed our analysis using 395 samples, and (2) 395 is more than the statistically significant sample size. However, we later randomly selected 382 (among 395) samples (Section REF ) to equalize it with the statistically significant sample size and analyze them.
Recruitment of Participants.
We planned to recruit 30 participants (15 for treatment + 15 for the control group) who edited at least 100 posts. After deploying our tool EditEx, we realized that EditEx could be helpful to both expert and novice SO users. Therefore, we relaxed our constraints. We recruit participants who edited any SO post to ensure familiarity with SO editing. However, we struggled to recruit 30 participants due to COVID-19 and the extensive nature of this user study. The study was extensive because each user had to do multiple edits of SO posts. Primarily, we planned to recruit participants using a snowball approach. To recruit more participants, we then extend our approach. Besides snowball, we attempt to recruit participants using an open circular. However, finally, we recruited 20 participants (10 for treatment + 10 for the control group) with different experience levels and diverse professions (Section REF ).
Number of Edits Per User.
We planned to ask each participant to suggest ten edits. However, several participants (especially the control group) could not edit ten posts for three main challenges, as we discussed in Section REF .
EditEx's Functionality of Highlighting Texts.
We planned to include EditEx's functionality to highlight texts that may cause rejection. However, the current version of EditEx cannot highlight texts. EditEx predicts the edit decisions (accepted/rejected) and alerts users with the potential rejection reasons if rejected. While highlighting texts could be helpful, we found that the EditEx tool with the basic features was usable and effective. Therefore, we leave the highlighting of texts in EditEx as a future extension.
NASA TLX Workload.
We planned to estimate TLX effort as a task load by combining all the ratings provided by a participant in five dimensions in the TLX metrics. However, most existing studies estimate subjective workload by combining all the ratings in six dimensions [33]}, [34]}, [35]}, [36]}, [37]}. Therefore, we also take ratings from each participant on six dimensions (mental demand, physical demand, temporal demand, effort, performance, and frustration). Finally, we estimate the cognitive workload (Fig. REF ). However, we consider a scale of ten with one step size [38]} to take ratings conveniently from participants for each dimension. It slightly sacrifices granularity in comparison to scale 100 with step size five. However, the results should not be affected much.
Unidentified Reasons.
Our rejection reason classifier cannot identify a couple of rejection reasons, such as partial acceptance and incorrect text/code change. We can extract added or removed text/code using appropriate HTML tags. However, partly accepted text/code cannot be separated from added text/code. Furthermore, it requires analyzing the future revisions to check partial acceptance, which is impractical. We did not find any patterns that can identify incorrect changes in code/texts. Identification of such reasons demands manual efforts.

Related Work
We developed our tool EditEx to recommend fixes to suggested edits in SO so that SO users can avoid committing undesired edits that may lead to the rejection of the edits. As such, our research in this paper belongs to a broader area called `collaborative editing in social forums'. Major related work can broadly be divided into Studies of collaborative editing systems in crowd-sourced forums (see Section REF ) and Techniques to suggest improvements to the editing system (see Section REF ). In addition, SO data are used extensively in SE research for various tasks (see Section REF ), all of which could be potentially impacted by having low-quality data due to erroneous/inefficient edits.
Studies of Collaborative Editing Systems
Editing of content can improve the content quality. As such, it is intuitive that the social Q&A forums offer to edit the post contents. Since social forums can be accessed by many users simultaneously, it is a cost-effective measure for the forums to support collaborative editing by allowing their users to do the editing. Indeed, studies
show that collaborating editing in social forums and online collaborative knowledge-sharing portals (e.g., Wikipedia) can positively impact towards the improvement of shared contents [39]}, [40]}. The nature of the collaborative editing can be similar across the social forums (e.g., Q&A site) and knowledge portals (e.g., Wikipedia). The research of Li et al. [39]} looked at the adoption of Wikipedia-style collaborative editing into a Q&A site like SO. They found that users with good edits are rewarded with positive votes by other users. They analyzed five years of historical editing data from SO and found that substantive edits from other users can increase the number of positive votes by 18% for the questions and 119% for answers. This reward can be beneficial for a user who does the edit because the edit may only offer at most 5% improvement over the original post (i.e., the user can be rewarded with mindful but low-cost editing efforts).
Indeed, the SO reward system can serve as an added influence to the users to suggest edits. A recent study by Wang et al. [28]} in SO found that users are motivated to edit more when they are closer to getting
a badge.
Overall, both Wang et al. [28]} and Li et al. [39]} conclude that offering incentives as reputation scores is useful to improve post quality within a collaborative editing platform like SO.
This finding was also observed in other collaborative editing platforms like webcasts [45]} and Wikipedia [40]}.
Munteanu et al. [45]} tested the effectiveness of engaged users in collaborating in a wiki-like webcast platform to edit/correct transcripts that are produced from webcasts through
an automated speech recognition system. Collaborative editing can be a cost-effective but useful means to improve the quality of the ASR (Automated Speech Recognition) system in webcasts because ASR systems can have an average error rate of 45% - above the accepted threshold of 25%. The field study carried out by the authors in a real lecture environment found that using students to edit the webcast transcript was useful in reducing the error rate. The editing was supported via a webcast extension that engages users to collaborate in a wiki-like manner.
Kittur et al. [40]} find that the increase in the number of editors does not guarantee the quality of the articles on Wikipedia.
The quality of the question is important to get an answer: lack of clarity, relatedness, and reproducibility of the problem, as well as the too short question, could dissuade developers from answering the question [49]}, [8]}. The reputation
and past activity of an asker could also factor into the likelihood of a question getting resolved [6]}. As such
factors of good questions are investigated, e.g., code to text ratio, etc. [14]}, [53]}.
However, depending on the platforms and user characteristics, these factors can vary [5]}.
As such, it is important to detect content quality automatically [9]}, [56]}, [11]}. Wang et al. [28]} found that users who make more edits in a short time are likely to get more edits rejected. Thus bad edits can harm the content quality.
Our research on SO rollback edits initially started in 2019 to better understand the edit rejection reasons as reported by Wang et al. [28]}. Through our qualitative analysis of SO posts, we also found all the edit rejection reasons reported by Wang et al. [28]}. In addition, we found four more edit rejection reasons. We report the edit rejection reasons in Section  of this paper. While the above papers, including Wang et al. [28]}, focus on analyzing editing mechanisms in collaborative platforms based on empirical studies, our paper focuses on developing techniques to automatically suggest fixes to suggested edits so that the edits will not be rejected upon submission. As such, our developed tool EditEx can further contribute to supporting the content quality in social forums by assisting users with guidance on improving the quality of their suggested contents. Thus, our paper offers complementary viewpoints to the above studies by offering tools and techniques that can facilitate improved edit content in a social Q&A site like SO.

Techniques to Develop to Improve Collaborative Editing Systems
Collaborative editing systems are common in Wikipedia  [39]}, [40]},
GitHub code editing [64]}, webcasts [45]},
scientific contents [66]}, [67]}, and so on. Compared to substantial research on conducting studies on existing collaborative editing systems, we are not aware of much research that focuses on developing tools and techniques to improve the systems. This is perhaps due to the fact that currently available collaborative platforms like Wikipedia seem to work well and are hugely popular. In all these platforms, the focus of collaborative editing is to improve the quality of the shared content based on
user engagement [7]}.
Chen et al. [1]} observed that most of the edits in SO are small sentence edits. While developing their SOTorrent database, Baltes et al. [70]} also observed that majority of edits in SO are relatively small. In a follow-up study,
Chen et al. [25]} predicted whether a post needs to be edited. Their approach is based on the concept of `proactive policy assurance', which assures that a modification to a suggested edit will satisfy the current `reactive policy assurance' in SO, which accepts/rejects based on the matching of exiting editing policy after an edit is submitted (i.e., reactive).
They developed a deep-learning-based policy assurance tool to recommend post owners or other users' potential mid-level edits to given post content. The deep learning model is a CNN (Convolutional Neural Network). In a large-scale experiment, they find that the tool offers good precision, recall, and F1-score (at least 0.7) while suggesting mid-level edits.
As we noted in Section REF , our research of this paper started in 2019 to gain hands-on experience on the edit rejection reasons observed by Wang et al. [28]}. Our initial exploration led to an expansion of the edit rejection reasons and to the submission of a registered protocol report in 2020 [29]}. In the registered protocol report, we outlined our vision of this paper by offering to develop machine learning model to automatically detect the edit rejection reasons and to build our EditEx tool that can offer proactive guidance to fix suggested edits. While working on this paper, we observed that some edit rejection reasons could be present both in accepted and rejected edits, resulting in inconsistencies in the editing acceptance/rejection process. We reported a catalog of such inconsistencies in our MSR 2021 paper [74]}. In our MSR 2021 paper, we also report several rule-based tools that we developed to detect inconsistencies in SO edits automatically. While developing our EditEx tool in this paper, we purposefully did not consider those inconsistencies, given those were not outlined in our 2020 registered protocol report [29]}. We note that an immediate extension of EditEx could investigate whether and how the inclusion of the inconsistencies into the rejection prediction models and the EditEx tool could make the overall editing process more effective for the SO users. We leave it as our immediate future work.

Other SE Research using SO Data
Several studies have been conducted to study developer discussions on different crowd-shared developer platforms, including SO. Seaman et al. studied developer discussion on inspection meetings [76]}. Reiner et al. used content analysis to study developer discussions on software processes [77]}. Gottipati et al. study relevant answers in 3 software forums, Dzone, Tips, and Oracle forums [78]}. Several studies has focused on discussions on microblogs, such as, Twitter [79]}, [80]}, [81]}, and chat communities, such as, HipChat [82]}, IRC messages [83]}, [84]}, and Slack [85]}. Recently, SO Q&A forums have been subject to a number of papers to study various aspects of software development, such as what developers are discussing in general [86]},
or about a particular aspect, e.g., concurrency [87]}, big data [88]}, chatbot development [89]}.
Several studies has been conducted to study developer sentiments on online discussions (e.g., SO data) [90]}, [91]}, [92]}, [93]}, [19]}, [15]}, [16]}, [18]}, [22]}, [21]}, [20]}, [101]}. Guzman et. al. applied sentiment analysis on code comments [90]}. Islam and Zibran study emotional variations in commit messages [103]}. Garcia et al.  [104]} study the emotions of developers in the Gentoo community. Guzman and Bruegge studied developer sentiments on mailing lists [105]}. Novielli et al. conduct sentiment analysis on SO and Github discussions [106]}. Many of these studies use automated sentiment analysis tools, which are found to provide contradictory results in software engineering research [107]}.
All the above research works using SO data could be benefited from improved data quality offered by collaborative editing in SO. As such, our tool EditEx, once adopted by the SO users, can help the SO users as well as the SO-based research community with better quality data.

Conclusion
SO has become an essential online resource with millions of programming-related problems and solutions. However, the quality of the shared knowledge is vital for the growth and success of SO. To promote quality, SO introduces an edit system so that users can suggest an improvement to posts. Unfortunately, numerous suggested edits are rejected due to either undesired changes of posts or violating edit guidelines. Such a scenario not only hurts the quality of content but also frustrates and demotivates users. We conducted a qualitative analysis of 764 (382 questions + 382 answers) rejected edits by rollbacks and identified 19 rejection reasons. We then extract 15 texts and user-based features to automatically capture those reasons and develop four machine learning models using them. Our best-performing model can predict rejected edits with about 70% accuracy, and the rejection reason classifier can identify the potential rejection reasons with 67% accuracy.
We also introduced an online tool named EditEx that can be integrated with the SO edit system. It analyzes the edits, predicts whether they will be rejected, and suggests users with the potential rejection reasons.
We conduct a survey to assess EditEx and SO edit system. According to survey results, the participants find reasons for rejection identified by EditEx influential. Moreover, EditEx can support the SO edit system to prevent 49% rejections, including the commonly rejected reasons. Such a statistic is 12% when users suggest regular free-form edits. Moreover, our tool significantly decreases the subjective workload and increases participants' confidence in suggesting edits.
This research is supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery grants, and by an NSERC Collaborative Research and Training Experience (CREATE) grant, and by two Canada First Research Excellence Fund (CFREF) grants coordinated by the Global Institute for Food Security (GIFS) and the Global Institute for Water Security (GIWS).
