Functional derivative

In the calculus of variations, a field of mathematical analysis, the functional derivative (or variational derivative) relates a change in a Functional (a functional in this sense is a function that acts on functions) to a change in a function on which the functional depends.
In the calculus of variations, functionals are usually expressed in terms of an integral of functions, their arguments, and their derivatives. In an integral of a functional, if a function is varied by adding to it another function that is arbitrarily small, and the resulting integrand is expanded in powers of , the coefficient of in the first order term is called the functional derivative.
For example, consider the functional
where . If is varied by adding to it a function , and the resulting integrand is expanded in powers of , then the change in the value of to first order in can be expressed as follows:
where the variation in the derivative, was rewritten as the derivative of the variation , and integration by parts was used.
Definition.
In this section, the functional derivative is defined. Then the functional differential is defined in terms of the functional derivative.
Functional derivative.
Given a manifold representing (continuous/smooth) functions (with certain boundary conditions etc.), and a functional defined as 
the functional derivative of "ρ"], denoted , is defined through
where formula_5 is an arbitrary function. The quantity formula_6 is called the variation of . 
In other words,
is a linear functional, so one may apply the Riesz–Markov–Kakutani representation theorem to represent this functional as integration against some measure.
Then is defined to be the Radon–Nikodym derivative of this measure.
One thinks of the function as the gradient of at the point and 
as the directional derivative at point in the direction of . Then analogous to vector calculus, the inner product with the gradient gives the directional derivative.
Functional differential.
The differential (or variation or first variation) of the functional formula_9 is 
Heuristically, formula_5 is the change in formula_12, so we 'formally' have formula_13, and then
this is similar in form to the total differential of a function formula_14,
where formula_16 are independent variables. 
Comparing the last two equations, the functional derivative formula_17 has a role similar to that of the partial derivative formula_18, where the variable of integration formula_19 is like a continuous version of the summation index formula_20.
Properties.
Like the derivative of a function, the functional derivative satisfies the following properties, where ["ρ"] and ["ρ"] are functionals:
where are constants.
Determining functional derivatives.
A formula to determine functional derivatives for a common class of functionals can be written as the integral of a function and its derivatives. This is a generalization of the Euler–Lagrange equation: indeed, the functional derivative was introduced in physics within the derivation of the Lagrange equation of the second kind from the principle of least action in Lagrangian mechanics (18th century). The first three examples below are taken from density functional theory (20th century), the fourth from statistical mechanics (19th century).
Formula.
Given a functional 
and a function () that vanishes on the boundary of the region of integration, from a previous section Definition,
The second line is obtained using the total derivative, where "ρ" is a derivative of a scalar with respect to a vector.
The third line was obtained by use of a product rule for divergence. The fourth line was obtained using the divergence theorem and the condition that on the boundary of the region of integration. Since is also an arbitrary function, applying the fundamental lemma of calculus of variations to the last line, the functional derivative is
where "ρ" = "ρ"() and , "ρ", ∇"ρ"). This formula is for the case of the functional form given by ["ρ"] at the beginning of this section. For other functional forms, the definition of the functional derivative can be used as the starting point for its determination. (See the example Coulomb potential energy functional.)
The above equation for the functional derivative can be generalized to the case that includes higher dimensions and higher order derivatives. The functional would be,
where the vector , and is a tensor whose components are partial derivative operators of order , 
An analogous application of the definition of the functional derivative yields
In the last two equations, the components of the tensor formula_31 are partial derivatives of with respect to partial derivatives of "ρ",
and the tensor scalar product is,
Examples.
Thomas–Fermi kinetic energy functional.
The Thomas–Fermi model of 1927 used a kinetic energy functional for a noninteracting uniform electron gas in a first attempt of density-functional theory of electronic structure:
Since the integrand of ["ρ"] does not involve derivatives of "ρ", the functional derivative of ["ρ"] is,
Coulomb potential energy functional.
For the electron-nucleus potential, Thomas and Fermi employed the Coulomb potential energy functional
Applying the definition of functional derivative,
So,
For the classical part of the electron-electron interaction, Thomas and Fermi employed the Coulomb potential energy functional
From the definition of the functional derivative, 
The first and second terms on the right hand side of the last equation are equal, since and in the second term can be interchanged without changing the value of the integral. Therefore,
and the functional derivative of the electron-electron coulomb potential energy functional ["ρ"] is,
The second functional derivative is
Weizsäcker kinetic energy functional.
In 1935 von Weizsäcker proposed to add a gradient correction to the Thomas-Fermi kinetic energy functional to make it suit better a molecular electron cloud:
where
Using a previously derived formula for the functional derivative,
and the result is,
Entropy.
The entropy of a discrete random variable is a functional of the probability mass function.
Thus,
Thus,
Exponential.
Let
Using the delta function as a test function,
Thus,
This is particularly useful in calculating the correlation functions from the partition function in quantum field theory.
Functional derivative of a function.
A function can be written in the form of an integral like a functional. For example,
Since the integrand does not depend on derivatives of "ρ", the functional derivative of "ρ" is,
Functional derivative of iterated function.
The functional derivative of the iterated function formula_56 is given by:
and 
In general:
Putting in N=0 gives:
Using the delta function as a test function.
In physics, it is common to use the Dirac delta function formula_61 in place of a generic test function formula_62, for yielding the functional derivative at the point formula_63 (this is a point of the whole functional derivative as a partial derivative is a component of the gradient):
This works in cases when formula_65 formally can be expanded as a series (or at least up to first order) in formula_66. The formula is however not mathematically rigorous, since formula_67 is usually not even defined.
The definition given in a previous section is based on a relationship that holds for all test functions , so one might think that it should hold also when is chosen to be a specific function such as the delta function. However, the latter is not a valid test function (it is not even a proper function).
In the definition, the functional derivative describes how the functional formula_68 changes as a result of a small change in the entire function formula_69. The particular form of the change in formula_69 is not specified, but it should stretch over the whole interval on which formula_19 is defined. Employing the particular form of the perturbation given by the delta function has the meaning that formula_69 is varied only in the point formula_63. Except for this point, there is no variation in formula_69.