ReLU. Linear approximation of the ReLU function is firstly proposed in the work [1]} by offering a pair of bounds to ReLU function instead of its piecewise feature. Fast-Lin [2]} gives parallel linear bounds to ReLU function. State-of-the-art linear approximation of ReLU function is proposed in DeepPoly [3]}, which has the highest precision among all relevant works by producing the smallest over-approximation region of ReLU function.
