
The major considerations on the gradient of mutual information were focused on the conjunction between estimation and information theories. However, in this paper we show the central role of the gradient of mutual information to interpret the behavior of the stochastic systems with feedback.

We introduce the notion of mutual information variation in Theorem REF  and Corollary REF , which hold for any stochastic systems with continuous system input and system output.

We take a functional approach to represent the relationship between the input and output of a stochastic system in general. We suppose that the system model can be defined through known functions. Our system model imposes no constraints on the noise model and also the system input can be dependent on the system noise. Moreover, the system input is allowed to be a function of the system output and hence the system parameters, which may occur for example in the presence of feedback. Comparing with the statistical description of the input-output relation of the systems, reported in the literature, our functional approach gives more insights into the interactions of the system's elements (see Section REF ).

For the aforementioned general system model, we obtain the gradient of mutual information with respect to the system parameters in Theorem REF  and the impact of these parameters on the gradient is considered.

Particularizing the obtained results for MAC system with feedback as an important and practical system, the capability of the gradient of mutual information to interpret the behavior of this system is shown in Theorem REF . Corollaries REF  and REF  are devoted to the Gaussian case, where the extensions of I-MMSE formula are derived for the additive Gaussian noise MAC with feedback. As a special case of the obtained results, Corollary REF  considers the additive Gaussian noise MAC without feedback, where the extension of I-MMSE formula reported in [1]} is recovered. Based on Corollaries REF  and REF  the results of [2]} and [3]} are recovered, respectively.

Using the proposed approach in this paper, we show how feedback compensates the negative effect of the interference on the achievable rates of the users and increases the capacity of MAC with feedback comparing with MAC without feedbacks in Section REF  figures REF  and REF .

Based on our functional approach to the system representation, in Theorem REF  we show that the result of [4]} on the gradient of mutual information with respect to the system parameter holds iff the noise is additive in the system model.

