This work has introduced a new family of loss functions for the direct minimization of the top - k error ( that is , without the need for fine - tuning ) . We have empirically shown that non - sparsity is essential for loss functions to work well with deep neural networks . Thanks to a connection to polynomial algebra and a novel approximation , we have presented efficient algorithms to compute the smooth loss and its gradient . The experimental results have demonstrated that our smooth top-5 loss function is more robust to noise and overfitting than cross - entropy when the amount of training data is limited .