1. A listening device adapted for being located in or at an ear of a user and comprising an input transducer for converting an input sound to an electric input signal; and an output transducer for converting a processed electric signal to an output sound; a forward signal path being defined there between and comprising a signal processing unit for processing the electric input signal or a signal derived therefrom and providing a processed output signal; a manually operable user interface located on the listening device allowing a user to control a function of the listening device; a feedback estimation system for estimating a feedback path from the output transducer to the input transducer, the feedback estimation system comprising an adaptive filter, the adaptive filter comprising a variable filter part and an algorithm part comprising an adaptive algorithm, the variable filter part being adapted for providing a transfer function to a filter input signal and providing a filtered output signal, the transfer function being controlled by filter coefficients determined in the algorithm part and transferred to the variable filter part, the feedback estimation system further comprising an update control unit adapted for controlling the adaptive algorithm including the transfer of filter coefficients to the variable filter part, wherein the update control unit is adapted to monitor the manually operable user interface and to provide that an activation of the manually operable user interface is used for influencing the control of the adaptive algorithm, including the timing of the calculation of filter coefficients and/or the transfer of filter coefficients to the variable filter part.