Despite the outstanding performance of deep neural networks in different applications, they remain computationally expensive and require a great amount of memory. This motivates more research on reducing the resources needed for implementing such networks. An efficient approach addressed for this purpose is matrix factorization, which has been shown to be effective on different networks. In this paper, we design a training algorithm which utilizes binary matrix factorization and show its efficiency in reducing the required resources in deep neural networks. In effect, this technique leads to the fast and practical implementation of such networks.