Deep neural nets ( DNN ) have led to a revolution in the areas of machine learning , audio analysis , and computer vision . Many state - of - the - art results have been achieved using these architectures . In this work we study the properties of these architectures with random weights . We prove that DNN preserve the distances in the data along their layers and that this property allows stably recovering the original data from the features calculated by the network . Our results provide insights into the outstanding empirically observed performance of DNN and the size of the training data .