You can get to the old town by using the stairway at the rear of the cathedral.
Back outside, from the Episcopal Garden (Jardin de l'Evache), at the rear of the cathedral, a stairway takes you down to the old town.