np.extract(1 - np.eye(5), M)
array([False, False, False, False, False, False, False, False, False, False,
    False, False, False, False, False, False, False, False, False, False],
    dtype=bool)
Acces all off diagonal elements of boolean numpy matrix