Learning rate.
 To tune these hyparparameters, we use standard Grid search technique [1]}. We select a set of batch sizes (i.e., 8, 16, and 32), epochs (i.e., 2, and 3), and learning rates (i.e., 1e-5, 2e-5, and 3e-5) as suggested in hugging face. We run the models on the dataset for each set of hyperparameters. Then we choose the best hyperparameter set for each model based on their performances. For example, we find the best performance of BERT for batch size of 32, epoch of 3, and learning rate of 2e-5.
In Table REF , we show the values of selected hyperparameters in our deep learning model after tuning.
Our replication package details the hyperparameter values with source code.
Data Processing. We prepare our dataset with only textual discussion. However, there may be some urls. These unwanted texts may hinder the model's performance. Thus, we check for URLs and remove them using regular expressions. For implementation, we use Python'regex' library. In addition to this, we remove all stop words. As the importance of stop words is low and the frequency of these words is high, these words can obscure other significant words. Moreover, recent studies related to sentiment analysis found that stop words removal is an effective step in improving a model's performance [2]}. Therefore, we remove all stop words using `nltk' [3]} library.
Model Features. We used the pre-trained embedding in BERT models, which learns
additional contextual information for a given domain (e.g., in our case, security in developer discussions) during the training phase. The combination of pre-trained and domain-specific learning into such pre-trained models offers more contextual information during classification tasks than traditional DL models. We add `EOS' and `CLS' to each sentence and then tokenize all sentences using a pretrained BERT-Tokenizer. We
padded each sentences to 100 length with `zero' padding.
Performance Metrics
We analyze the performance of the classification models for security aspect detection using five standard metrics in
information retrieval [4]}: Precision (P), Recall (R), F1-score (F1), Accuracy (Acc), and Matthews Correlation Coefficient (MCC).

Precision (P) is the fraction of sentences labeled as `HasSecurity' (i.e., the sentence contains security discussions) out of all the sentences labeled as `HasSecurity'.
\(P = \frac{TP}{TP+FP}\) 
Here TP = Correctly classified a sentence as containing discussion about security, FP = Incorrectly classified as containing discussion about security,

Recall (R) is the fraction of all `HasSecurity' sentences that are successfully identified out of all sentences that contain security discussions.
\(R = \frac{TP}{TP+FN}\) 
Here FN = Incorrectly classified as not containing discussion about security.

F1-score is the harmonic mean of precision (P) and Recall (R).
\(F1 = 2*\frac{P*R}{P+R}\) 

Accuracy (Acc) is the fraction of all correctly classified out of all records.
\(Acc = \frac{TP + TN}{TP + TN + FP + FN}\) 
Here TN = Correctly classified as not containing discussion about security.

Matthews Correlation Coefficient (MCC) is the linear correlation between all `HasSecurity' labeled sentences and the sentences that contain security discussion. It takes into account all possible scenarios- i.e., correctly identified security-related sentences, incorrectly identified security-related sentences, correctly identified non-security-related sentences, and incorrectly identified non-security-related sentences.
\(MCC = \frac{TP * TN - FP * FN}{\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}}\) 

Area Under the Curve (AUC) is the area under the Receiver Operating Characteristics (ROC) curve. An ROC curve is a graphical plot to illustrate the ability of a binary classifier (such as ours).
\( TPR = \frac{TP}{TP + FN}, ~FPR = \frac{FP}{FN + TP}\) 
\(AUC = \int _{x = 0}^{1} {TPR}({FPR}^{-1}(x))dx\) 

F1-score is preferred over metrics like accuracy (Acc) when evaluation dataset is imbalanced. For example,
our IoT benchmark dataset is imbalanced with lower number of sentences labeled as 1 (i.e., `HasSecurity' = 1) than sentences labeled as 0.
Following standard practices in Literature [5]}, [6]},
we use F1-score to determine and report the best model.

Empirical Study Results
In this section, we answer four research questions (RQ) using SO data dump of September 2019. We compare the results against those found in SO data dump of January 2022 in Section .

How do the deep learning pre-trained transformer models perform to detect IoT security aspects in the benchmark? (Section REF )

What are the error categories in the misclassified cases of the best performing model? (Section REF )

What are topics in the IoT security discussions collected from the entire IoT SO dataset using the best performing model? (Section REF )

How do the IoT security topics evolve in SO? (Section REF )

RQ\(_1\)  How do the deep learning pre-trained transformer models perform to detect IoT security aspects in the benchmark?
Motivation
Pretrained transformer models have already shown dominating performances over shallow and deep models in software engineering [5]}. Moreover, recent studies on software engineering have found the effectiveness of domain-specific embedding (e.g., BERT, RoBERTa, XLNet, etc.) than a generic one. For example, Tabassum et al. [8]} shows that the BERTOverflow model that is pre-trained on Stack Overflow dumps beats the performance of efficient transformers like RoBERTa on developer discussions such as Github and Stack Overflow. Therefore, we investigate the possibility of improving the baseline performance to detect security aspects in IoT discussions using studied Transformer Models, e.g., BERT.

Approach
<FIGURE>We first use the newly created IoT Security dataset
from Section REF  to get the best performing transformer model. For experimentation, similar to Uddin and Khomh [6]}, we use a stratified k-fold cross-validation ( k = 10). Each fold divides dataset \(D\)  into \(k\)  disjoint folds, each of
approximately equal size.
\(D_i\cap D_j=\phi ~~~\forall ~i,j\in k~and ~i\ne j \)  and
\(D_1\cup D_2\cup ...\cup D_k = D\) . To avoid overfitting: we train the models \(k\)  times each time we hold one of the \(k\)  folds for validation, and we use the rest \(k-1\)  folds for training. The value of \(k\)  is a hyperparameter. In our experiments, we set K as 10. In Figure REF , we show 10-Fold cross validation techniques. We run the model 10 times. In each iteration, we train the model on nine different folds and test the trained model on the last fold. Then, we calculate the average performance of these 10 iterations. For example, according to the Figure REF , at the first iteration, we pick `Fold-2', `Fold-3', `Fold-4', `Fold-5', `Fold-6', `Fold-7', `Fold-8', `Fold-9', and `Fold-10' to train the model, and we test this model on the `Fold-1'. Therefore, in our case, we fine tune our model during each training (i.e., k times) and then test the tuned model on the test fold.
To tune the hyperparameter, we follow grid search [10]} tuning techniques. For example, we set learning rate as 1e-5, 2e-5, and 3e-5 and select the one that has highest performance for each model. For the two baselines (Logit and SVM), we hyper tuned all the parameters used by Uddin and Khomh [6]}, [12]}. We find the performance of transformer models on benchmark dataset and select the best performing model based on the F1-Score. Then, we test the trained model on our Judgmental dataset to check either the model learn to generalize the security discussion in IoT domain.
<TABLE>
Results
In Table REF , we report the performance of two baselines and the five advanced language-based pre-trained transformer models on the IoT security dataset. For each model,
we report six performance metrics from Section REF .
Among the two shallow learning baselines, SVM offers the best performance for all six metrics, outperforming Logits. SVM shows an F1-score of 0.866, while Logits shows an F1-score
of 0.864. For the other five performance metrics - Accuracy, Precision, Recall, AUC, and MCC, the improvements are around .01%, .01%, .02%, .02%, and .04% respectively. We note that the performance for both models is high. The higher performance of these models on our dataset is due to the following reasons: Classifications can often be more straightforward for security aspects, i.e., parsing of words could suffice (e.g., secure, attack, and hacking). So, non-contextual models like SVM and Logits can detect security aspects with high precision and recall.
All five transformer models show better performances in terms of all metrics than the baseline models from Uddin [12]} (SVM and Logit). The last two rows in Table REF  show the performance change (in percentages) we observed in the best performing transformer model (RoBERTa) compared to the two baselines. For example, the best
performing transformer model is RoBERTa with an F1-score of 0.935, which outperforms the baselines SVM by 8.0%, and Logits by 8.2%. Higher performances mean that transformers are precisely (i.e., around 6% higher precision) discovering (i.e., around 10% higher recall) security-specific sentences. It is because these models can understand the contextual meaning in a sentence. For example, the following non-security related sentence “My goal is to create a sort of password-like system where if you push the right buttons in the right order then a button will light up.” The comments describe the procedure of a system using the references `password like' that have no relation with security. The baselines model erroneously labeled it as 0 due to the misleading keyword- `password like' where the transformer models discerned the context. Hence, it found no security threats in this sentence. This superiority of these models over baselines is not surprising given that the transformer models are generally found to have offered better performance than state-of-the-art classifiers for multiple other text classification tasks in software engineering (e.g., sentiment detection [5]}).
Among the transformers, RoBERTa shows the best performance with an F1-Score of 0.935, while DistilBERT is the lowest with an F1-Score of 0.914. F1-Score for the other three models- e.g., BERT, BERTOverflow, and XLNet are 0.934, 0.933, and 0.921 respectively. There is a tiny gap between the performance of RoBERTa and BERT. It is because the architecture and training procedures of these two models are almost similar. The modification, dynamic changes of the masking pattern (in RoBERTa) instead of static masking (in BERT) during the training phase, could be the reason behind the subtle improvement in performance.
XLNet improves the performances by increasing the training to handle the dependency between the words and capture long-distance information. In our dataset, each sentence is simple and small in size as well. So, large-scale training on the dataset causes overfitting and lower performance than BERT. The performance drop of DistilBERT is understandable as the distillation of the BERT model is obviously lower performing than its original model.
Although BERTOverflow has the same architecture as BERT and is trained on the StackOverflow dumps, the performance of this model is lower than RoBERTa and BERT. RoBERTa and BERT model beat BERTOverflow by 0.21% and 0.11% in terms of F1-Score. However, BERTOverflow has the best precision of 0.938 among the transformer models. This result indicates that BERTOverflow predicts security-specific sentences precisely (i.e., better precision) but lacks in dicoverability (i.e., lower recall and F1-Score) in comparison to RoBERTa and BERT. We look into the cause of BERTOverflow's performance drop. Our finding in this scenario is that BERTOverflow pretrained on StackOverflow dumps containing
code snippets, so the model faced some security-related words (e.g., 'private`,
'protected`, 'bug`, 'break`, 'lock`, 'key`) in more programming contexts than
security contexts. The model fails to learn the correct embedding for these
words during the fine-tuning or training phase. For example, consider the
following security-specific sentence “It will be helpful if anyone states which
are keys to lock so that in addition in my python script for locking phone
programmatically.” BERTOverflow failed to embed security context for this
sentence and labeled it as 0- i.e., non-security.
<TABLE>Our intuitions indicate that RoBERTa is the best security-discussion detector, which is also supported by the experimental results. However, RoBERTa has only a 0.1% and 0.2% lead over the second and third best models, which is not a significant difference. Thus, we perform a statistical analysis to understand the difference in results between RoBERTa and the other two models. We calculate the Spearman Rank Correlation Coefficient [15]} for the results of RoBERTa and BERT, and RoBERTa and BERTOverflow. We show the result in Table REF . For both pairs, the correlation scores are as high as 0.94. However, the results of RoBERTa and BERT are more correlated (0.1% higher) than the results of RoBERTa and BERTOverflow. Following Leclezio et al. [16]}, the scores indicate that the results of RoBERTa are strongly correlated with the results of BERT and BERTOverflow. Nevertheless, the score also indicates that there is a difference between the results of RoBERTa and other models, as the score is far less than 1.
<FIGURE>From these experiments and analysis of the results, we find RoBERTa is the best performing deep learning model. We further test this trained RoBERTa model on our judgemental dataset. We find that the model performs similarly on the judgemental dataset. RoBERTa has a precision of 0.95, a recall of 0.85, and a F1-Score of 0.9. We report the performance in the Figure REF . Compared to the performance on the benchmark dataset, model has better precision but lower recall, which results in a slightly lower F1-Score on the judgemental dataset. This result indicates that the model has learned to generalize security discussion in IoT domain. Thus, our RoBERTa model is reliable enough to carry on further experiments. From now on, we refer to this best-performing transformer model, RoBERTa, as SecBot in the remainder of the paper.

RQ\(_2\)  What are the error categories in the misclassified cases of the best performing model?
Motivation
Our observation in the previous research question (RQ\(_1\) ) is that the most robust generic model Secbot (RoBERTa) outperforms domain-specific BERTOverflow. Although improvement is subtle, the statistical significance in Table REF  suggests there are enough differences in both model's predictions. Our intuition behind this RQ is that lack of domain knowledge in Secbot (RoBERTa) should have some impact on its performance. Thus, we perform a qualitative study of the misclassifications of the best-performing model (i.e., SecBot) to detect security aspects in IoT discussions. Proper reasoning of misclassification will strengthen our claim that the model performance is not arbitrary. Moreover, this gives us leverage to confine our model's vulnerabilities in detecting security aspects in IoT domains. In the future, this will provide a scope for the researcher to learn about the possible avenues for improvement of the model.

Approach
In our benchmark of 5,919 samples, SecBot was wrong in 139 cases (false positive or false negative). We analyzed these 139 samples from our IoT security dataset as follows. We applied 10-fold cross-validation to our entire
dataset following steps similar to Section REF .
As such, we were able to use the entire 5,919 sentences for training and testing. At the end of the 10th iterations, we find a total of 139 misclassifications in the IoT dataset by the RoBERTa model used for SecBot. Then we manually analyze all the 139 sentences to determine the reason for misclassification. The manual analysis process consists of three stages. Both of the authors consulted together (over Skype) on all 139 sentences to give a textual label that could explain the reason for misclassification. For example, the following sentence is erroneously labeled as 1 (i.e., contains security discussion) by SecBot “I have attempted to set it so that it loads as the root user each time with no luck.”
After careful comparison of this sentence with other sentences in our IoT samples, we determine that the misclassification is due to the ambiguity in context, which arose due to the words- `attempted to set' and `root user' in the sentence. We label the misclassification reason as `Ambiguous Context';
<FIGURE>
Results
In Figure REF , we show the misclassification categories in all 139 sentences collected from the IoT Security dataset.
Overall, we find six error categories:
Ambiguous Keywords,

Ambiguous Context,

Implicit Context,

Infrequent/Unknown Keywords,

Text Parsing Issues, and

Code Parsing Issues.
 The most observed error category is `Ambiguous Keywords', which we observe in 37.4% of all 139 sentences (i.e., 58 sentences).
The category `Ambiguous Context' is observed the second most times (20.1% sentences), followed by the category `Implicit Context' (17.3% sentences), `Infrequent/Unknown Keywords' (12.2% sentences), `Text Parsing Issues' (7.2% sentences) and
Code Parsing Issues (5.8% sentences), respectively. We discuss the error categories below.
[(1)]

Ambiguous Keywords. This type of error occurred due to the ambiguity of a word in the dataset. We found two scenarios where this type of error occurred. When a sentence offers insufficient context to detect security concerns, thus, prediction mostly depends on the keywords present in that sentence. In the worst-case scenario, these keywords are found in both security and non-security contexts during training. The model hardly guessed correctly when this intriguing scenario occurred. For example, SecBot erroneously labeled the following sentence as 1, due to the ambiguous presence of keywords like `signed'- “I'm not sure how to handle signed bytes.” It got confused as `signed' word associated with various security-related terms (e.g., `signed device', `self-signed cert', `signed aws') and also non-security specific sentences (e.g. `signed bit', `signed number') as well. So, the model erroneously predicted this programming-related non-security sentence as a security-specific sentence. Secondly, this error occurred when a keyword was predominantly associated with security-specific sentences during training and pretraining phases. But in the erroneous cases, the keyword was found in a non-security specific scenario or vice versa. Consider the sentence- “I am trying to add the option to change the wi-fi password, but that part of the form does not seem to be available to the code. ” SecBot labeled this sentence as 1 (i.e., security), but the underlying scenario is about the configuration of wifi. SecBot was confused due to the keywords `password' and `change' –which are predominantly found in the security-specific sentences during training.

Ambiguous Context. These errors occur when the underlying context is too ambiguous to understand, i.e., security-specific. Such ambiguity occurred because more information from surrounding sentences was needed. Consider the following sentence: “I hope that someone here can help me, I want to control my arduino uno by sending it commands from a c++ program that performs some basic face recognition.” SecBot labeled the sentence as 1, while it is 0 in the manual label. It is because the context is about controlling an IoT device that performs face recognition using some commands. While the manual labeling phase considered the sentences non-security, the model found it a possible security breach due to the context of controlling a security device.

Implicit Context. SecBot was unable to discern the underlying security context when it was implicit. Consider the following sentence: “The main difference from software development standpoint of ios private api that such apis aren't declared in header files.” Here, the context is about ios private API which can only accessed and used by Apple developers. The elementary context is that developers can not use any API without importing, but implicitly it means that this API is restricted for non-Apple developers. As such, SecBot labels it as 0, whereas the benchmark manual label is 1.
Sentences with implicit security contexts are difficult to capture for a fully automated system as domain knowledge experts can only bisect them from other sentences.

Infrequent/Unknown Keywords. This type of error occurred when the model encountered a word that was not present during the training or was present so sporadically that it could not learn the word correctly. But that word had profound significance in understanding the sentence. For example, the following sentence is labeled as 0 by SecBot: “The user should have a iam access.”. Here, `iam access'– an AWS access token—is the keyword to classify the sentence correctly. SecBot encountered this unknown keyword during the testing phase and misclassified the sentence. Consider another instance: “See the hid usage tables, v1.12 document for more information about hid usage values.” The sentence contains a less frequent word, `hid' is present. SecBot could not learn the weights for this keyword correctly, and so it labeled the sentence as 0, where manual labeling considered it as 1.

Text Parsing Issues. This error occurred due to the incompetence of the model to parse the textual contents correctly in a sentence. For example, SecBot erroneously labels the following sentence as 0, even though it is about the enablement of secure socket layer (SSL) in the Raspbian Stretch device: “The version of Openssl on Raspbian Stretch is 1.0.2, but according to Openssl website, these versions should be binary compatible (https://www.openssl.org/policies/releasestrat.html).” While SecBot knew from training that SSL is related to the security aspect, SSL in this sentence is denoted as Openssl, which is unknown to SecBot.

Code Parsing Issues. These errors are very few compared to the other categories. This error occurred due to the presence of code snippets and inline code in the samples. For example, SecBot is unable to determine security threats related to input injection noted in the code block that is provided as text in this sentence: “In Package tag add ...In Capabilities add ...<rescap:Capability Name="inputInjection" /><rescap:Capability Name="inputInjectionBrokered" />”. This type of parsing error is inevitable for automated models like SecBot because security-specific context can be buried anywhere in a code snippet.
RQ\(_3\)  What are topics in the IoT security discussions collected from the entire IoT SO dataset using the best performing model?
We automatically detect security topics in the sentences labeled as security-specific by SecBot in our IoT dataset.
Motivation
The rapid growth and usage of IoT devices, tools, and technologies have become prominent threats to data protection, security, and privacy. IoT developers encounter various security challenges while using these devices and tools. It is vital to understand those challenges in order to improve and design more secure devices and tools. So, the understanding of topics in the SO IoT security discussion can be helpful to infer those challenges.

Approach
First, we apply SecBot on the entire 672,678 sentences that we collected from our 53K
IoT posts (Section REF ). SecBot returned 30,595 sentences
as 1, i.e., those containing security discussions. Second, we preprocess this dataset as
follows. We removed stopwords, extreme words (e.g., words with less than 20 frequencies), most frequent words (e.g., words with more than 2000 frequencies), short tokens, and letter accents. The stopwords are collected from
NLTK [3]}. We applied lemmatization to sentences to get root form
of each word. Third, we fed the preprocessed sentences into the topic modeling algorithm Latent Dirichlet Allocation (LDA) [18]}
available via Gensim [19]}. LDA takes a
set of texts as input and outputs a list of topics to group the texts into K
topics. This K is a hyperparameter of the algorithm. To get the optimal value of K,
we used standard practice as originally proposed by Arun at
el[20]}. This approach suggests that the optimal number
of topics has the highest coherence among them, i.e., the more coherent the
topics, the better they can encapsulate the underlying concepts. To determine
this coherence metric, we use the standard c_v metric as originally
proposed by Röder et al. [21]}. We use the topic coherence metric as available in
Gensim package [19]}. To determine our optimal number of topics, we use grid search technique. We fist
select a set of values for K. Then we run our LDA
algorithm for each value of K and measured the coherence metric. Finally, we
choose the value of K that is associated with the highest coherence score. In our
experiment, we set K = 1,4,8,9, 10,11, 12, 16,20,24,28,32, 36, 40 and got maximum coherence for K = 9. Thus, we picked 9 as our optimal number of topics. LDA also uses two hyperparameters, \(\alpha \)  and \(\beta \)  to control the
distribution of words in topics and the distribution of topics in posts following Biggers et al.
 [23]}.
Following previous work [24]}, [25]}, [26]}, [23]}, [28]}, [29]}, we thus use standard
values \(50/K\)  and 0.01 for the two hyper parameters. Fourth, we label each topic a name by manually
analyzing a random sample of at least 50 sentences per topic. We do not limit ourselves to a finite number of sentences per topic during the labeling to ensure that the assigned label is reliable. The first and second authors virtually meet to label the topic manually. First, both authors review a number of sentences for a topic and come up with possible topic labels. Then both authors share their thoughts on the possible topic labels. If both authors have the same thoughts, they label the topic with a suitable title; otherwise, they go over the sentences again and label the topics together. For example, the following sentence is about the certificate validation for different users, “...Found the answer as I was trying to import the certificate with the Root and was testing with a different user.” Another sentence assigned to the same topic is “ ...The commands where I input the SSID and Password are the ones causing trouble.”. We thus label the topic as `User/Device Authorization'.
Fifth, we assign each topic a category, where a
category denotes a higher order grouping of the topics. For example, we assign the topics `User/Device Authorization' and `Crypto/Encryption Support' to the category `Software', because
the topics contain discussion about the support of software in IoT devices to ensure security. The topic `Secure Transaction' is assigned to the category `Network, because
it contains discussions about the securing of transactions/communications between IoT devices/services over the network.
<FIGURE><TABLE>
Results
We found 9 IoT security-related topics, which are grouped into three high-level categories: Software, Network, and Hardware. Figure REF  shows the distribution of these three high-level categories. The Software category contains the greatest number of topics (5) and covers more than 57% of all sentences. In Table REF , we show the 9 topics based on the distribution of sentences in IoT security dataset. The distribution column indicates the percentage of sentences in our IoT security dataset that cover this topic. The topic `IoT Device Config' from the Hardware category covers the lowest number of sentences among all topics (2402 sentences, around 7.9%). It contains discussions of programming and configuration challenges IoT developers face while using Bluetooth/BLE/Microchip interfaces in their IoT devices to setup a service (e.g., setting up a barcode scanner in rpi2 to verify coupon \(Q_{30721018}\)). \(Q_i\)  denotes a question with an ID \(i\)  in SO. The topic `Storage/Database Management' from the Software category covers the highest number of sentences (4463 sentences, around 14.6%). This topic contains discussions about developers' problems to database set up, configuration (e.g. \(Q_{30073381}\)), and attack like SQL Injection (e.g. \(Q_{6826176}\)), Weak Authentication(e.g. \(Q_{5503812}\)), Privilege abuse(e.g. \(Q_{39232652}\)), and etc.
The other four topics in the software category
are:
Mitigation techniques of
vulnerabilities/attacks in IoT devices (3808 sentences, around 12.4%),

Discussions about the vulnerability of specific approaches in frameworks (3169 sentences, around 10.4%),

Authorization of users and IoT devices (3101 sentences, around 10.1%), and

Documentation and usages of Cryptography/Encryption support (3004 sentences, around 9.8%).
The Network category has the second highest number of topics (3) and it covers 34.8% of all sentences. Two of the three topics under the Network category are also among the top four topics (each with around 12% of sentences). The first two topics are `Secure Transaction' and `Secure Connection Configuration', which contain discussions about ensuring security over secure payments and secure copy/transfer of digital files via/among IoT devices. The last network topic is 'IoT Hub Federation'. Questions in this topic concern the cloud-based setup of IoT devices, e.g., using Azure/AWS/Google cloud. An IoT hub is a managed service in the cloud that acts as a central messaging service to allow bi-directional communication between an IoT application and the smart devices it manages (e.g., connect to MQTT with authenticated Cognito credential \(Q_{51093154}\)).
RQ\(_4\)  How do the IoT security topics evolve in SO? 
Motivation
We investigate how the popularity of the three IoT security topic categories has changed over time by determining the total number of new sentences created per topic category over time. A new sentence can only be created if a new post is created. Therefore, the evolution of the topics over time offers us information about the relative popularity of the categories and whether a category is discussed more over time than the other categories.

Approach
For each of the 30K security-related sentences in our dataset, we identify the ID of the SO post from where the sentence is detected. We consider the creation time of the SO post to be the sentence creation time. We group the 30K sentences by the three topic categories. For each category, we compute the total number of sentences created every six months based on the creation time. Following Wan et al. [30]}, we report the evolution of IoT security topics using the metric Topic Absolute Impact.
Topic absolute impact is calculated by determining the total number of sentences created per the 9 IoT topics every six months since 2008. We do this by first assigning each sentence to its most dominant topic, i.e., the topic with which the sentence shows the highest correlation score. We then assign the creation time to a sentence as the creation time of the post where the sentence is found. Finally, for each topic, we put the sentences into 
time-buckets, where each bucket contains all sentences created within a time window (e.g., from Jan 2009 till June 2009, etc.). We further analyze the absolute growth to correlate the growth in real world contexts for each topic. It is possible that developers might be influenced by the release of new IoT APIs, tools, or devices. As such, we figure out the impact of those real-world contexts in our developer discussion in SO. We first identify some major releases in the IoT domain during the period from 2009 to 2019. Then, we find all the spikes in our absolute growth chart for each topic. Next, we collect all time frames of six months (e.g., January 2010 to July 2010) in which the spikes took place. To identify related real-world events (i.e., major updates and releases), we identify all major updates and releases during that time frame for each topic. Finally, we manually check the availability of discussions for all identified events in our IoT security dataset during that time frame. Based on that, we either count an event as a striking contributor to IoT discussions or discard the event. For example, during the time interval between January 2014 and July 2014, both `vue.js' and `Riot.js' were released. However, we find that our IoT dataset has enough discussion for `Riot.js' only during January 2014 and July 2014. As a result, we discard `vue.js' and count `Riot.js' as a striking event.
<FIGURE>
Results
Figure REF  shows the absolute growth of the three IoT security topic categories over time by showing the number of new sentences per category every six months. While the software category has the most new sentences among the three categories, the network category started to catch up starting from 2017. The number of discussions under Software shows an upswing between January 2011 and July 2011, when Django 1.2.6, a free open-source web framework that encourages rapid development, was launched. Some web security tools (e.g., `XSS', `memory-cache-backed session' and `CSRF btoken') were first introduced in this release. Many discussions in 2014 in our dataset are related to Riot.js. Similarly, the release of Kali Linux in 2006 and Django 1.10.8 gave a spike in software related security discussions. Overall, the periods between June 2011 and June 2014 showcase the rapid adoption of IoT-based SDKs/framekwors (e.g., rpi2 versions). IoT security concerns are raised at the same time due to the emergence of ransomware like CryptoLokcer and banking Trojans.
At the beginning of 2010, the network topic category got high priority due to internet connectivity and cloud storage. A high spike was found at that time. Firebase, a real-time cloud database suited for IoT applications, was released in May 2012. As such, we found many related security discussions across the Network category during this time. Similarly, we found the release of IoT hubs like AWS IoT in late 2015 and IBM Watson IoT in 2017 contributed to many network-related discussions during the respective periods. Recently, the emergence of online transactions and IoT cloud storage solutions has also contributed to the increased popularity of the network category (#new sentences). 
Hardware related discussions showed continuous growth over the years. Many hardware supported security modules like fingerprint locks, smart doors, biometric devices, etc., have emerged during our study period. We found the release of popular open-source IoT hardware has increased the rate of discussion about that hardware. For example, the release of Intel Galelio back in mid-2012 produced a spike between June 2013 and December 2013. Similarly, after the release of `Rasberry Pi 2' in February 2016 and `Arduino Uno Wifi rev 2' in February 2017, spikes in the curve were visible. 
In conclusion, the evolution of IoT security topic categories and subsequent topics can provide insight into new, emerging, or existing threats.

Discussion
For our experiments, we use SO data dumps up to September 2019. However, we also
investigate the most recent data dumps of SO. We use the online Stack Exchange data
archive to get the most recent
SO data dump, which was for January 2022. We collect IoT datasets from this data dump by
following all the steps we described in Section REF .
We found that many SO posts related to IoT were deleted by SO in the January 2022 data dump that were previously
available in the September 2019 data dump. As such, we first analyze the extent of
such deleted posts in the January 2022 dump (in Section REF ).
We then compare the results of our empirical studies for RQ3 (recall Section REF ) and RQ4 (recall Section REF ) between the two data dumps, i.e., Sept 2019 and Jan 2022 in
Sections REF , REF .
The “Case” of Post Deletion by Stack Overflow
As per the data dumps of September 2019, there are around 53K IoT related posts,
whereas the most recent data dumps (January 2022) have only 45K IoT related
posts. This indicates that most recent data dumps have fewer IoT-related posts
even though our trend charts from Figure REF  based on Sept 2019 data dump showed 
that IoT-related discussions have an upward trend in recent times. We
investigate this drop in the number of posts in Jan 2022 data dumps. We identified
all the missing posts that are available in the September 2019 data dumps. Then,
we check the recent activity and status of these missing posts. We find that
these posts were deleted in between September 2019 and Jan 2022. As a
result, these posts appear in the 2019 data dumps but do not appear in recent
data dumps. For example, `ESP32' tagged SO question \(Q_{56098708}\) was created in
May 2019, but later it was deleted. We further apply Secbot to the new IoT data
and find only 6K security-related sentences. Compared to the 30K
security-related discussion in RQ\(_3\) (Section REF ), these results are
quite low. This is because the size of the dataset is smaller and most of the
posts related to the security discussions are removed. For example, an
encryption-related post, \(Q_{11501569}\), that contains multiple security-related
discussions, has been removed in recent data dumps.
<FIGURE>
Prevalence of IoT Security Topics After 2019
Following the RQREF , we apply topic modeling on 6k sentences we got
from the Jan 2022 data dump. We find that the optimal number of topics for this
dataset is 6. Then we manually label these six topics. The topics are: `IoT
Device Config/Set-up', `Secure Connection', `Secure Transaction',
`Framework/SDK-based Security', `Authentication Management', and
`Crypto/Encryption Support'. We further investigate how these topics differ from
previously found topics in RQ\(_3\)  (Section REF ). Although five topics are
present in our previous findings, `Authentication Management' is different from
what we found previously in `User/Device Authentication'. `Authentication
Management' is comprised of discussion related to authentication mechanisms,
devices, techniques, protocols, implementation supports, etc., whereas
`User/Device Authentication' only consists of user or device level
authentication. On the contrary, we didn't find any discussion related to
`Storage/Database Management', `Vulnerability/Attacks', or `IoT Hub Federation'.
This happens because of the removed posts in recent data dumps. We further
categorize these topics into Software, Hardware, and Network group. We find 3
software-related topics (i.e., `Framework/SDK-based Security',
`Crypto/Encryption Support', and `Authentication Management'), 2 network-related
topics (i.e., `Secure Connection' and `Secure Transaction' ), and a hardware
related topic (i.e., `IoT Device Config/Set-up'). There are 22.3% hardware,
37.2% network, and 40.5% software related security-discussions in the February
2022 data dumps. Finally, we compare the topic distribution in the most recent
data dumps (i.e., February 2022) and the September 2019 data dumps. We show the
comparison in Figure REF . The relative percentage of
hardware-related discussions is higher in February 2022 data dumps than in
September 2019 data dumps. Although the network topic holds similar percentages
in both data dumps, the software topic has a lower value in the February 2022
data dumps. This is because most of the software-security related IoT posts were
removed in the February 2022 data dumps, and in recent times, developers are
discussing hardware-security more than they did in past times.
<FIGURE>
Evolution of IoT Security Topics
We analyze how the topics found in the Jan 2022 data dump evolved over the time period. We first calculate the
absolute growth of the topics following RQ\(_4\)  (Section REF ). Then we
generate a trend chart showing how the three topics (i.e., Hardware, Software,
and Network) evolve over the time period between 2011 and 2022. We report the
trend chart in Figure REF . From this chart, we find
that security related discussions follow an upward trend for all three topics
(i.e., hardware, software, and network) before 2019, similar to our previous
findings in Section REF . However, there is a downward slope after 2019
for all three topics. One of the possible reasons for this downward swing could
be the lack of releases of any software, hardware, and network related devices,
tools, technologies, etc. during this period. Another contradictory finding is
that network and software related discussions have almost similar numbers, but
our previous finding says there are more software-related discussions than
network ones. This could be because of the removed posts in the February 2022
dumps.

Discoverability of IoT security sentences
<FIGURE>Our study data is collected from SO, by identifying questions in SO labeled as one of the 75 IoT related tags. As we noted in Section REF , the 75 tags were previously used to learn IoT developer discussions in SO by Uddin et al. [31]}, who found that the most popular and most frequently discussed topics were related to the adoption of security measures in IoT devices (e.g., secure messaging). In addition, the same dataset was also used by Uddin [32]} to determine the prevalence of security-related discussions at the sentence level. Both studies show that the dataset contains diverse discussions about security for IoT. Note that not all the 75 IoT tags have explicit reference to any security measures, but we observed that many of the questions (and their answers) labeled as the tags contained security discussions. In this section, we offer quantitative evidence on the extent of security discussions in questions labeled as non-security tags as follows.
 We investigate whether SO security-specific tags cover all IoT security sentences. Thus, we pick 10 frequently used security-related tags, i.e., `security', `ssh', `ssl', `passwords', `authentication', `authorization', `encryption', `cryptography', and `hash' in SO. Then, we collect all posts from our 53K IoT posts with at least one of the tags. In the collected posts, we only manage to find 8% of 30K sentences, i.e., 92% of our IoT security sentences are not discoverable using the 10 tags (see Figure REF ). We also investigate the missing posts in the latest data dumps (January 2022) using these 10 security tags. The result shows the numbers are similar, only 10% posts can be discovered with these 10 tags in 2022 data dumps. This indicates that SO security tags do not cover all security discussions. However, our security discussion includes sentences that are tagged with security tags.

Implications of Findings
The findings from our study can guide the following major stakeholders in the rapidly emerging IoT ecosystems:

IoT Security Enthusiasts to learn about IoT security aspects from IoT developer discussions,

IoT Vendors to offer tools and techniques to support security in IoT devices,

IoT Developers to determine the current trends in IoT security based on developer discussions and to use the information to guide future development needs,

IoT Educators to develop tutorials and documentation to educate security principles to IoT practitioners, and

IoT Researchers to improve the detection of IoT security aspect in developer discussions and to study how to properly address the difficulty in the adoption of security practices for IoT devices and techniques.

We discuss the implications below.
<FIGURE>IoT Security Enthusiasts. IoT enthusiasts can be IoT practitioners or
simply IoT market watchers, but they nevertheless form a crucial part of the IoT
ecosystem in various ways (e.g., buying/analyzing IoT devices/solutions, etc.).
The development and adoption of IoT-based solutions by developers is facilitated
by the exponential growth of IoT devices, software, and platforms. Our
increasingly interconnected digital world relies on smart devices built using
the IoT, which means security for IoT devices is paramount and it is the job of
IoT developers to adopt proper security measures for their solutions. Indeed, if
we compare the number of new security sentences per month to all sentences in
the month in our 53K SO posts, we observe a slow but relative growth of IoT
security-related discussions, especially after 2016 (see Figure REF ). This means that interest in IoT security is
increasing among IoT developers in SO. Therefore, IoT security enthusiasts can
benefit from the security discussions by IoT developers in SO. As we noted in
Section   (Figure REF ), security
discussions in SO IoT posts can be buried inside general or non-security IoT
discussions. Therefore, to offer security-specific insights from SO IoT
discussions, we need tools to automatically detect those. The high performance
of our developed tool, SecBot, is suited to meet the needs.
<FIGURE>Tradeoff between popularity and difficulty of IoT security topics in our IoT dataset


IoT Vendors. IoT vendors need to support IoT developers with proper and usable secure IoT techniques. To determine what is appropriate and what is working, though, the vendors need to know the problems faced by IoT developers. Forums like SO have become a go-to place to look for solutions to their technical problems. Thus it has become essential for the IoT vendors to keep an eye on this open discussion forum. Our research can help them in two ways. First, the high precision and recall of our tool SecBot will help IoT vendors reliably find security-related issues in SO. They can analyse those discussions and take a decision accordingly. Second, our findings from RQ\(_4\)  (Section REF ) show that IoT developers are aware of major releases and immediately discuss them more frequently. Thus, we can say the vendors can get quick feedback from the developers about their most recent release of any IoT device, tools, techniques, or software. This could play a vital role in their upcoming releases. For example, if there are multiple bugs or issues, they can fix those bugs in their next release or offer a quick release fixing those bugs. Overall, such insights can be crucial for vendors to improve their offerings and to compare the solutions of their competitors. Hence, they could be well ahead of their competitors.
In Figure , we show a bubble chart to show the tradeoff between the popualrity vs difficulty of IoT security posts. The chart is
constructed as follows. Out of the 30K IoT security sentences in our dataset, we pick the sentences that we find in the SO question posts.
For each of the 9 IoT security topics, we pick the sentences that we find in the SO question posts. For the total number of distinct questions thus
found for a given IoT security topic, we analyze two metrics as follows:
[(1)]
Popularity analysis based on the average view counts of the associated questions to the topic.

Difficulty analysis to get an accepted answer to a question by computing the percentage of question per topic that remain without an accepted answer.
 Both of the metrics have been used previously in literature to determine the popularity and difficulty of topics generated from SO posts.
The x-axis in Figure  shows the difficulty score of each IoT security topic, and the y-axis shows the popularity score of the IoT security posts. The size of each topic bubble is based on the number of distinct questions assigned to the topic. Therefore, the more right a topic in the chart, the more difficult it is in SO. Topics like `Framework/SDK Based Security', and `Storage/Database Management', and `IoT Device Config' are among the most difficult. The topic `IoT Device Config' is among the most difficult as well as the most popular IoT topics. Therefore, IoT vendors can accelerate the adoption of security in IoT Microschip and Bluetooth (e.g., BLE) devices, which will then increase the chance of getting answers to the problems.
IoT Developers. The security of IoT devices and solutions is of paramount importance. As the hacker community is looking for the single opportunity to breach the IoT system or device, the developers should not consider ignoring any corner of security vulnerability which can be exploited. We thus need IoT developers to stay aware of the IoT security trends and to adopt prevalent IoT tools. The developer community is one of the best places that could help them keep pace with the cutting edge technologies that can't be breached, at least for that time frame. Such insights can benefit from such precise, specific, but automatically mined security knowledge to stay aware of the trends related to IoT security. In addition, such knowledge can also help them make better decisions, like picking a popular IoT security technique over another. For example, in Figure , we find that the topic `IoT Hub Fedearation' is the least difficult, while also not very popular. This means that IoT developers leveraging cloud infrastructure to implement security methods for their IoT devices are not finding enough support from SO developers. The topic `Secure Transaction' between IoT devices or between IoT and non-IoT devices contain a large number of questions, which are also popular. This means that IoT developers can devote time to learn and implement secure transactions using their IoT devices, and they can also inquire with other IoT developers in SO about the problems they face.
The developer community cannot rely on new technologies every time, as new technologies can be challenging, complex, or incompatible with the current environment. For example, a developer asks how he can create a digital signature function PDF from Hardware Security Modules (HSM) using C# in SO question \(Q_{54922093}\). However, the discussion section suggests that he can't use HSM directly in C# instead he needs to go through `CAPI' or `CNG' api or `ncryptoki'. Developers could face these challenges in the middle of any project, which could cost more time or, at worst, they could end up switching to another framework or tool. This scenario can be avoided if developers gather the necessary information about the required apis, devices, or software. Developer forum like SO is the ideal place to collect those information. Our research provides information about IoT security topics. Further mining of their discussed topics such as which features they are discussing, what are the sentiment of those discussion, etc. will be helpful for everyone to learn about an api, tools, frameworks, software, or devices. From this informative feedback, the developer community can decide whether they adopt the new technology or not. Thus, they can select their developing tools and frameworks wisely and minimize the risk of a project.
IoT Security Educators. Our research findings are more promising for IoT educators in many ways. From RQ\(_4\)  (Section REF ), we learn that IoT topics are getting more attention and thus the responsibility of the IoT educators is also getting escalated. As the IoT security demands are increasing, the educator must prepare quality contents to motivate the newcomers and to enlight current practitioners.  Thus, to create content, they can follow the topics we find in RQ\(_3\)  (Section REF ). They should also be quite selective in selecting the most recent techniques and tools over the old ones. Otherwise, the learners will lack the new tools, which will eventually end up going backward instead of forward. SO can be a helping hand here for them. As SO contains IoT security discussions about all those 9 topics, they can collect that data to analyse and prepare their lessons. Moreover, as we previously presented, one can find enough discussions related to the most recent releases of any IoT security tools, software, and devices. The educator can also provide these insights regarding the most recent IoT tools, software, and devices into their lesson. This will also be beneficial for the learner.
IoT security educators can use the bubble chart in Figure  to prioritize their efforts to develop security tutorials and documentation for IoT developers and practitioners. For example, the topic `Secure Connection Config' has the highest number of questions. It is also among the most popular topics, while also less difficult than most topics in terms of getting an accepted answer. Therefore, IoT security educators can analyze the questions related to user/device authorization in SO to develop comprehensive tutorials for developers. The topic `Crypto/Encryption Support' has the second most questions, while it is the most popular yet one of the most difficult topics. Therefore, the IoT security educators can analyze the questions in SO and consult IoT resources to ensure that IoT developers can learn about the risks associated with leaving IoT ports open and how they can configure them properly to ensure security.
IoT Researchers
This research creates a vast space for the researcher to go forward in multiple dimensions. Security aspect detection is a major task that we accomplish in this paper. However, this task requires more attention in the future. We find our Secbot has a maximum performance of 0.935, which seems like a good result considering no prior security detector has achieved that far. But when we figure out the errors made by our Secbot, we find multiple implicit contexts, ambiguous contexts, and ambiguous keywords related errors. These errors can further be researched to extract informative information like the identification of key phrases or words that cause ambiguity. Individual research can be conducted on how these key words or phases can be detected automatically. In another way, these error categories give us a way of improving the performance of Secbot. The researcher can approach this in three ways. First, they can add IoT contexts to Secbot. For example, they can use some rules based patterns to extract information and then feed it to the Secbot. Second, they can add more samples to reduce the sparsity of security-aspects in the dataset. For this, one may consider only adding the samples that are close to those error category types. They can even try to make a balanced dataset to see how the performance changes. Third, they can try a deep model that can only discern samples related to those error categories, and then they can apply ensemble Secbot and the deep model.
Besides this, we find that domain-specific BERTOverflow fails to perform as well as the generic model Secbot. Additionally, the correlation coefficient score from Table REF  suggests that the results of BERTOverflow vary from the results of Secbot. This could be an interesting topic to delve deeply into. The researcher can explore the misclassification made by BERTOverflow to compare the results between Secbot and BERTOverflow. If there is a significant difference in error categories between these two models, they can consider developing an ensemble model. Another direction of research exploration could be developing a purely domain-based security aspect detector. Although BERTOverflow has knowledge about StackOverflow, compared to its overall knowledge, SO knowledge is not sufficient to represent it as an intelligent SO expert. Thus, an entirely SO knowledge-based model may be more suitable in security-aspect detection.
IoT researchers can analyze the security discussions to learn about the specific challenges that IoT developers are facing based on their real-world experience. Such insights can be useful for researchers to invent new techniques and tools for IoT security. It is important that research in IoT security can be influenced by the emerging trends in IoT security. As we observed in Section REF  (Figure REF ), we see an almost equal number of sentences per the the two topic categories (Software and Network) starting from January 2017. This means that security research in the IoT needs to put equal emphasis on both software and network security. Similarly, discussions about secured hardware for IoT devices are also increasing over time, although not as much as the software and network topics. IoT security researchers need to design and develop innovative techniques to secure IoT software and networks. One of the 9 topics is `Vulnerability/Attack Concerns' in IoT devices, which points to the issues IoT developers are facing with regards to addressing specific vulnerabilities in their IoT-based solutions. As we find in Figure , this topic is also considerably popular and difficult. Therefore, IoT software and hardware security researchers can use the discussions to develop tools and techniques. Data management and storage are subject to the topic `Secure Data Management', which can benefit from research in database security.
Additionally, the research community can conduct rigorous research on IoT security topics such as what types of questions developers asked in each topic category, how the new release affects the developers, what factors developers discussed most about any new release, etc.
Threats to Validity
Internal validity threats relate to the authors' bias while conducting the analysis. We mitigated the bias in our benchmark creation process and topic labeling processes by computing agreements (security sentences) and labeling together (topic modeling). During the topic labeling, the authors communicated over Skype and using Google drive to reduce individual biases. The agreement between the two coders is above 95% all the time. We use a standard random sampling technique to reduce locality biases in our benchmark dataset. The machine learning models are trained, tested, and reported using standard practices. There was no common data between the training and test sets. We shuffle the training and testing datasets in each iteration to introduce a new scenario every time. We perform cross-validation on the entire dataset to make sure our assessment is not biased towards any subsets. We follow standard hyperparameter tuning to minimize the effects of hyperparameters on the final results.
Construct validity threats relate to the selection and creation of IoT security dataset. As we include all tags related to IoT tags, security-related tags are also present there. However, there may be some security tags that have not been included in the dataset creation process. In the future, SO may include more IoT and security tags, which could challenge our dataset. However, as we discussed in Section REF , we observed that IoT developers in SO did not constrain their security-related discussions in SO to only questions labeled as security-related tags. In fact, more than 90% of our security-related sentences are not covered by the security-specific tags in our dataset. As such, in Section , we discussed the implications of our automated tool to detect security-related sentences in SO, which could find such sentences even in SO IoT tags that do not explicitly refer to any security issues. Beside this, construction validity threats relate to the difficulty of finding data to create our IoT security-related sentences. Our benchmark creation process was exhaustive, as we processed more than 53K posts from SO. The evolution of IoT topic categories considers post-creation time as sentence creation time. A post can be edited after its creation with new sentences, but we observed less than 1% of such cases in our dataset. We find security-specific 30K sentences out of 672K sentences in the IoT dataset using the best performing SecBot model. During the topic modeling phase, we observed only a few non-security related sentences. Further investigation can be carried on. This may give us more insights about the model, which we leave as our future task. In addition, we select a topic for each sentence based on the higher coherence score of the topics. During the topic labeling step, we label each topic by analyzing the topic top 30 words and posts associated with the label. This approach is consistent with the related work that also analyzed topics in SO posts [33]}, [34]}.
External validity threats relate to the generalizability of our findings. Our developed model, SecBot, demonstrates high accuracy. As such, the model is expected to offer good performance for other developer forums. There is a possibility that IoT developers may discuss different types of security discussions in other forums. For example, we found that most of the IoT developers discuss Database/Storage related security in SO. It is possible that in other forums, they discuss more about encryption. A detailed evaluation of SecBot in other forum data is our future work. In summary, our model exhibits good performance in this research direction, but the results should not be taken as an automatic implication of the same result in general. An extensive analysis of the diverse nature of challenges and characteristics can validate the transposition of the results to other domains. Besides this, SecBot is designed to work on textual datasets. However, SO contains code-snippets, logging, and urls which are filtered out during our dataset creation. Therefore, if SecBot is applied to SO posts containing source codes, logging, or urls, the model's performance may drop.

Related Work
Related work can broadly be divided into Studies to understand and Techniques to detect/mitigate IoT security issues.
Studies. Literature in IoT so far has focused on
surveys of IoT techniques and
architectures [35]}, [36]},
the underlying middleware solutions (e.g.,
Hub) [37]}, the use of big data analytics
to make smarter devices [38]}, the
design of secure protocols and
techniques [36]}, [40]}, [41]}
and their applications on diverse domains (e.g.,
eHealth [42]}), the Industrial adoption
of
IoT [43]},
and the evolution and visions related to IoT
technologies [44]}, [45]}.
The unauthorized inference of sensitive information from/among IoT devices
is a prevalent concern[46]}.
We are aware of no previous research that focused on understanding
IoT security discussions in SO.
In SE, topic modeling is used to learn aspects like software logging [47]},
feature location [48]}, [49]},
traceability linking [50]}, [51]},
software and source code evolution [52]}, [53]}, [54]},
source code categorization [55]}, code refactoring [56]},
defect analysis [57]}, and various software maintenance
tasks [58]}, [59]}.
The SO posts are subject to topic modeling to understand concurrency [26]}, big
data [25]} and chatbot issues [34]}.
Yang et al. [29]} studied security-topics in SO. They used SO tags to identify mobile security which might miss many security discussions which were not tagged as any security related tags. Our studies resolve this issue by following Uddin et al. [31]} to collect all IoT posts. Next, they applied topic modeling to the security posts. As the recent studies have found that larger documents may have multiple topics, LDA topic modeling is unstable for such cases [65]}. We thus use sentence level topic modeling. We develop a precise security detector, SecBot, to identify security related sentences from SO posts and apply LDA topic modeling on our security dataset. Unlike Yang et al. [29]}, we focus on IoT security topics.
Techniques. IoT devices
can be easy target for cyber threats [41]}, [68]}. As such, significant research efforts are underway to improve IoT security. Automated IoT security and safety measures are
studied in Soteria [69]},
IoTGuard [70]}. Encryption and
hashing technologies make communication more secure and certified [71]}. Many authorization
techniques for IoT are proposed like SmartAuth [72]}. For smart home security, IoT security techniques are proposed like Piano [73]}, smart authentication [74]}, and cross-App Interference threat mitigation [75]}. Session management and token verification are used in web security to
prevent intruder getting information. Attacks on Zigbee, an IEEE specification used to support interoperability can make IoT devices vulnerable [76]}. Smart gateway for IoT is proposed to tackle
malicious attack  [77]}. To the best of our knowledge, our developed SecBot+ is the first DL model that can be used to
automatically detect IoT security-related developers discussions. IoT researchers can gain insights to
offer increased support/security for a problematic IoT device as observed in the developer discussions.

Conclusions
The rapid adoption of IoT-based solutions has raised concerns about the security of IoT devices and communications. As such, it is important to understand the problems developers discuss when discussing their usage of IoT security tools and techniques in online technical forums like SO. With a view to automatically detecting developers' security discussions at the granularity level of sentences, in this paper, we have investigated a total of five advanced pre-trained language-based deep learning techniques (e.g., BERT, RoBERTa). The best-performing model, SectBot, based on RoBERTa, offers an F1-score of 0.935 to detect IoT security discussions. We use SecBot to automatically mine all the 30K IoT security-related sentences from the 53K IoT posts from SO. We apply topic modeling to the 30K sentences to find IoT security topics in developer discussions. We observed 9 topics that are grouped into three categories: Software, Network, and Hardware. Our developed tools and study findings can guide automated collection and analysis of IoT security problems in developer discussions.
