Difficulty analysis to get an accepted answer to a question by computing the percentage of question per topic that remain without an accepted answer.
 Both of the metrics have been used previously in literature to determine the popularity and difficulty of topics generated from SO posts.
The x-axis in Figure  shows the difficulty score of each IoT security topic, and the y-axis shows the popularity score of the IoT security posts. The size of each topic bubble is based on the number of distinct questions assigned to the topic. Therefore, the more right a topic in the chart, the more difficult it is in SO. Topics like `Framework/SDK Based Security', and `Storage/Database Management', and `IoT Device Config' are among the most difficult. The topic `IoT Device Config' is among the most difficult as well as the most popular IoT topics. Therefore, IoT vendors can accelerate the adoption of security in IoT Microschip and Bluetooth (e.g., BLE) devices, which will then increase the chance of getting answers to the problems.
IoT Developers. The security of IoT devices and solutions is of paramount importance. As the hacker community is looking for the single opportunity to breach the IoT system or device, the developers should not consider ignoring any corner of security vulnerability which can be exploited. We thus need IoT developers to stay aware of the IoT security trends and to adopt prevalent IoT tools. The developer community is one of the best places that could help them keep pace with the cutting edge technologies that can't be breached, at least for that time frame. Such insights can benefit from such precise, specific, but automatically mined security knowledge to stay aware of the trends related to IoT security. In addition, such knowledge can also help them make better decisions, like picking a popular IoT security technique over another. For example, in Figure , we find that the topic `IoT Hub Fedearation' is the least difficult, while also not very popular. This means that IoT developers leveraging cloud infrastructure to implement security methods for their IoT devices are not finding enough support from SO developers. The topic `Secure Transaction' between IoT devices or between IoT and non-IoT devices contain a large number of questions, which are also popular. This means that IoT developers can devote time to learn and implement secure transactions using their IoT devices, and they can also inquire with other IoT developers in SO about the problems they face.
The developer community cannot rely on new technologies every time, as new technologies can be challenging, complex, or incompatible with the current environment. For example, a developer asks how he can create a digital signature function PDF from Hardware Security Modules (HSM) using C# in SO question \(Q_{54922093}\). However, the discussion section suggests that he can't use HSM directly in C# instead he needs to go through `CAPI' or `CNG' api or `ncryptoki'. Developers could face these challenges in the middle of any project, which could cost more time or, at worst, they could end up switching to another framework or tool. This scenario can be avoided if developers gather the necessary information about the required apis, devices, or software. Developer forum like SO is the ideal place to collect those information. Our research provides information about IoT security topics. Further mining of their discussed topics such as which features they are discussing, what are the sentiment of those discussion, etc. will be helpful for everyone to learn about an api, tools, frameworks, software, or devices. From this informative feedback, the developer community can decide whether they adopt the new technology or not. Thus, they can select their developing tools and frameworks wisely and minimize the risk of a project.
IoT Security Educators. Our research findings are more promising for IoT educators in many ways. From RQ\(_4\)  (Section REF ), we learn that IoT topics are getting more attention and thus the responsibility of the IoT educators is also getting escalated. As the IoT security demands are increasing, the educator must prepare quality contents to motivate the newcomers and to enlight current practitioners.  Thus, to create content, they can follow the topics we find in RQ\(_3\)  (Section REF ). They should also be quite selective in selecting the most recent techniques and tools over the old ones. Otherwise, the learners will lack the new tools, which will eventually end up going backward instead of forward. SO can be a helping hand here for them. As SO contains IoT security discussions about all those 9 topics, they can collect that data to analyse and prepare their lessons. Moreover, as we previously presented, one can find enough discussions related to the most recent releases of any IoT security tools, software, and devices. The educator can also provide these insights regarding the most recent IoT tools, software, and devices into their lesson. This will also be beneficial for the learner.
IoT security educators can use the bubble chart in Figure  to prioritize their efforts to develop security tutorials and documentation for IoT developers and practitioners. For example, the topic `Secure Connection Config' has the highest number of questions. It is also among the most popular topics, while also less difficult than most topics in terms of getting an accepted answer. Therefore, IoT security educators can analyze the questions related to user/device authorization in SO to develop comprehensive tutorials for developers. The topic `Crypto/Encryption Support' has the second most questions, while it is the most popular yet one of the most difficult topics. Therefore, the IoT security educators can analyze the questions in SO and consult IoT resources to ensure that IoT developers can learn about the risks associated with leaving IoT ports open and how they can configure them properly to ensure security.
IoT Researchers
This research creates a vast space for the researcher to go forward in multiple dimensions. Security aspect detection is a major task that we accomplish in this paper. However, this task requires more attention in the future. We find our Secbot has a maximum performance of 0.935, which seems like a good result considering no prior security detector has achieved that far. But when we figure out the errors made by our Secbot, we find multiple implicit contexts, ambiguous contexts, and ambiguous keywords related errors. These errors can further be researched to extract informative information like the identification of key phrases or words that cause ambiguity. Individual research can be conducted on how these key words or phases can be detected automatically. In another way, these error categories give us a way of improving the performance of Secbot. The researcher can approach this in three ways. First, they can add IoT contexts to Secbot. For example, they can use some rules based patterns to extract information and then feed it to the Secbot. Second, they can add more samples to reduce the sparsity of security-aspects in the dataset. For this, one may consider only adding the samples that are close to those error category types. They can even try to make a balanced dataset to see how the performance changes. Third, they can try a deep model that can only discern samples related to those error categories, and then they can apply ensemble Secbot and the deep model.
Besides this, we find that domain-specific BERTOverflow fails to perform as well as the generic model Secbot. Additionally, the correlation coefficient score from Table REF  suggests that the results of BERTOverflow vary from the results of Secbot. This could be an interesting topic to delve deeply into. The researcher can explore the misclassification made by BERTOverflow to compare the results between Secbot and BERTOverflow. If there is a significant difference in error categories between these two models, they can consider developing an ensemble model. Another direction of research exploration could be developing a purely domain-based security aspect detector. Although BERTOverflow has knowledge about StackOverflow, compared to its overall knowledge, SO knowledge is not sufficient to represent it as an intelligent SO expert. Thus, an entirely SO knowledge-based model may be more suitable in security-aspect detection.
IoT researchers can analyze the security discussions to learn about the specific challenges that IoT developers are facing based on their real-world experience. Such insights can be useful for researchers to invent new techniques and tools for IoT security. It is important that research in IoT security can be influenced by the emerging trends in IoT security. As we observed in Section REF  (Figure REF ), we see an almost equal number of sentences per the the two topic categories (Software and Network) starting from January 2017. This means that security research in the IoT needs to put equal emphasis on both software and network security. Similarly, discussions about secured hardware for IoT devices are also increasing over time, although not as much as the software and network topics. IoT security researchers need to design and develop innovative techniques to secure IoT software and networks. One of the 9 topics is `Vulnerability/Attack Concerns' in IoT devices, which points to the issues IoT developers are facing with regards to addressing specific vulnerabilities in their IoT-based solutions. As we find in Figure , this topic is also considerably popular and difficult. Therefore, IoT software and hardware security researchers can use the discussions to develop tools and techniques. Data management and storage are subject to the topic `Secure Data Management', which can benefit from research in database security.
Additionally, the research community can conduct rigorous research on IoT security topics such as what types of questions developers asked in each topic category, how the new release affects the developers, what factors developers discussed most about any new release, etc.
Threats to Validity
Internal validity threats relate to the authors' bias while conducting the analysis. We mitigated the bias in our benchmark creation process and topic labeling processes by computing agreements (security sentences) and labeling together (topic modeling). During the topic labeling, the authors communicated over Skype and using Google drive to reduce individual biases. The agreement between the two coders is above 95% all the time. We use a standard random sampling technique to reduce locality biases in our benchmark dataset. The machine learning models are trained, tested, and reported using standard practices. There was no common data between the training and test sets. We shuffle the training and testing datasets in each iteration to introduce a new scenario every time. We perform cross-validation on the entire dataset to make sure our assessment is not biased towards any subsets. We follow standard hyperparameter tuning to minimize the effects of hyperparameters on the final results.
Construct validity threats relate to the selection and creation of IoT security dataset. As we include all tags related to IoT tags, security-related tags are also present there. However, there may be some security tags that have not been included in the dataset creation process. In the future, SO may include more IoT and security tags, which could challenge our dataset. However, as we discussed in Section REF , we observed that IoT developers in SO did not constrain their security-related discussions in SO to only questions labeled as security-related tags. In fact, more than 90% of our security-related sentences are not covered by the security-specific tags in our dataset. As such, in Section , we discussed the implications of our automated tool to detect security-related sentences in SO, which could find such sentences even in SO IoT tags that do not explicitly refer to any security issues. Beside this, construction validity threats relate to the difficulty of finding data to create our IoT security-related sentences. Our benchmark creation process was exhaustive, as we processed more than 53K posts from SO. The evolution of IoT topic categories considers post-creation time as sentence creation time. A post can be edited after its creation with new sentences, but we observed less than 1% of such cases in our dataset. We find security-specific 30K sentences out of 672K sentences in the IoT dataset using the best performing SecBot model. During the topic modeling phase, we observed only a few non-security related sentences. Further investigation can be carried on. This may give us more insights about the model, which we leave as our future task. In addition, we select a topic for each sentence based on the higher coherence score of the topics. During the topic labeling step, we label each topic by analyzing the topic top 30 words and posts associated with the label. This approach is consistent with the related work that also analyzed topics in SO posts [1]}, [2]}.
External validity threats relate to the generalizability of our findings. Our developed model, SecBot, demonstrates high accuracy. As such, the model is expected to offer good performance for other developer forums. There is a possibility that IoT developers may discuss different types of security discussions in other forums. For example, we found that most of the IoT developers discuss Database/Storage related security in SO. It is possible that in other forums, they discuss more about encryption. A detailed evaluation of SecBot in other forum data is our future work. In summary, our model exhibits good performance in this research direction, but the results should not be taken as an automatic implication of the same result in general. An extensive analysis of the diverse nature of challenges and characteristics can validate the transposition of the results to other domains. Besides this, SecBot is designed to work on textual datasets. However, SO contains code-snippets, logging, and urls which are filtered out during our dataset creation. Therefore, if SecBot is applied to SO posts containing source codes, logging, or urls, the model's performance may drop.

Related Work
Related work can broadly be divided into Studies to understand and Techniques to detect/mitigate IoT security issues.
Studies. Literature in IoT so far has focused on
surveys of IoT techniques and
architectures [3]}, [4]},
the underlying middleware solutions (e.g.,
Hub) [5]}, the use of big data analytics
to make smarter devices [6]}, the
design of secure protocols and
techniques [4]}, [8]}, [9]}
and their applications on diverse domains (e.g.,
eHealth [10]}), the Industrial adoption
of
IoT [11]},
and the evolution and visions related to IoT
technologies [12]}, [13]}.
The unauthorized inference of sensitive information from/among IoT devices
is a prevalent concern[14]}.
We are aware of no previous research that focused on understanding
IoT security discussions in SO.
In SE, topic modeling is used to learn aspects like software logging [15]},
feature location [16]}, [17]},
traceability linking [18]}, [19]},
software and source code evolution [20]}, [21]}, [22]},
source code categorization [23]}, code refactoring [24]},
defect analysis [25]}, and various software maintenance
tasks [26]}, [27]}.
The SO posts are subject to topic modeling to understand concurrency [28]}, big
data [29]} and chatbot issues [2]}.
Yang et al. [31]} studied security-topics in SO. They used SO tags to identify mobile security which might miss many security discussions which were not tagged as any security related tags. Our studies resolve this issue by following Uddin et al. [32]} to collect all IoT posts. Next, they applied topic modeling to the security posts. As the recent studies have found that larger documents may have multiple topics, LDA topic modeling is unstable for such cases [33]}. We thus use sentence level topic modeling. We develop a precise security detector, SecBot, to identify security related sentences from SO posts and apply LDA topic modeling on our security dataset. Unlike Yang et al. [31]}, we focus on IoT security topics.
Techniques. IoT devices
can be easy target for cyber threats [9]}, [36]}. As such, significant research efforts are underway to improve IoT security. Automated IoT security and safety measures are
studied in Soteria [37]},
IoTGuard [38]}. Encryption and
hashing technologies make communication more secure and certified [39]}. Many authorization
techniques for IoT are proposed like SmartAuth [40]}. For smart home security, IoT security techniques are proposed like Piano [41]}, smart authentication [42]}, and cross-App Interference threat mitigation [43]}. Session management and token verification are used in web security to
prevent intruder getting information. Attacks on Zigbee, an IEEE specification used to support interoperability can make IoT devices vulnerable [44]}. Smart gateway for IoT is proposed to tackle
malicious attack  [45]}. To the best of our knowledge, our developed SecBot+ is the first DL model that can be used to
automatically detect IoT security-related developers discussions. IoT researchers can gain insights to
offer increased support/security for a problematic IoT device as observed in the developer discussions.

Conclusions
The rapid adoption of IoT-based solutions has raised concerns about the security of IoT devices and communications. As such, it is important to understand the problems developers discuss when discussing their usage of IoT security tools and techniques in online technical forums like SO. With a view to automatically detecting developers' security discussions at the granularity level of sentences, in this paper, we have investigated a total of five advanced pre-trained language-based deep learning techniques (e.g., BERT, RoBERTa). The best-performing model, SectBot, based on RoBERTa, offers an F1-score of 0.935 to detect IoT security discussions. We use SecBot to automatically mine all the 30K IoT security-related sentences from the 53K IoT posts from SO. We apply topic modeling to the 30K sentences to find IoT security topics in developer discussions. We observed 9 topics that are grouped into three categories: Software, Network, and Hardware. Our developed tools and study findings can guide automated collection and analysis of IoT security problems in developer discussions.
