Though it is quite new, <cite>BERT</cite> has been tried in many different domains than the one proposed in <cite>Devlin et al. (2018)</cite> . However, almost all of these studies have two things in common: they don't start training <cite>BERT</cite> from scratch and the target domain contains very limited data (Zhu et al., 2018; Yang et al., 2019; Alberti et al., 2019) . In this study, on the other hand, we address (1) the performance of <cite>BERT</cite> by comparing its domain specific pre-trained and fine-tuned performances, and (2) in the setting where the target domain has extensively more data.