Polynomial interpolation

In numerical analysis, polynomial interpolation is the interpolation of a given data set by the polynomial of lowest possible degree that passes through the points of the dataset.
Applications.
Polynomials can be used to approximate complicated curves, for example, the shapes of letters in typography, given a few points. A relevant application is the evaluation of the natural logarithm and trigonometric functions: pick a few known data points, create a lookup table, and interpolate between those data points. This results in significantly faster computations. Polynomial interpolation also forms the basis for algorithms in numerical quadrature and numerical ordinary differential equations and Secure Multi Party Computation, Secret Sharing schemes.
Polynomial interpolation is also essential to perform sub-quadratic multiplication and squaring such as Karatsuba multiplication and Toom–Cook multiplication, where an interpolation through points on a polynomial which defines the product yields the product itself. For example, given "a" = "f"("x") = "a"0"x"0 + "a"1"x"1 + ... and "b" = "g"("x") = "b"0"x"0 + "b"1"x"1 + ..., the product "ab" is equivalent to "W"("x") = "f"("x")"g"("x"). Finding points along "W"("x") by substituting "x" for small values in "f"("x") and "g"("x") yields points on the curve. Interpolation based on those points will yield the terms of "W"("x") and subsequently the product "ab". In the case of Karatsuba multiplication this technique is substantially faster than quadratic multiplication, even for modest-sized inputs. This is especially true when implemented in parallel hardware.
Definition.
Given a set of data points where no two are the same, a polynomial formula_1 is said to "interpolate" the data if formula_2 for each formula_3.
Interpolation theorem.
Given formula_4 distinct points formula_5 and corresponding values formula_6, there exists a unique polynomial of degree at most formula_7 that interpolates the data formula_8.
Proof.
Consider the Lagrange basis functions given by
Notice that formula_10 is a polynomial of degree formula_7. Furthermore, for each formula_12 we have formula_13, where formula_14 is the Kronecker delta. It follows that the linear combination
is an interpolating polynomial of degree formula_7.
To prove uniqueness, assume that there exists another interpolating polynomial formula_17 of degree at most formula_7. Since formula_19 for all formula_20, it follows that the polynomial formula_21 has formula_4 distinct zeros. However, formula_21 is of degree at most formula_7 and, by the fundamental theorem of algebra, can have at most formula_7 zeros; therefore, formula_26.
Corollary.
An interesting corollary to the interpolation theorem is that if formula_27 is a polynomial of degree at most formula_7, then the interpolating polynomial of formula_27 at formula_4 distinct points is formula_27 itself.
Unisolvence theorem.
Given a set of data points where no two are the same, one is looking for a polynomial of degree at most with the property
The unisolvence theorem states that such a polynomial "p" exists and is unique, and can be proved by the Vandermonde matrix, as described below.
The theorem states that for interpolation nodes , polynomial interpolation defines a linear bijection
where Π"n" is the vector space of polynomials (defined on any interval containing the nodes) of degree at most .
Constructing the interpolation polynomial.
Suppose that the interpolation polynomial is in the form
The statement that "p" interpolates the data points means that
If we substitute equation (1) in here, we get a system of linear equations in the coefficients . The system in matrix-vector form reads the following multiplication:
We have to solve this system for to construct the interpolant "p"("x"). The matrix on the left is commonly referred to as a Vandermonde matrix.
The condition number of the Vandermonde matrix may be large, causing large errors when computing the coefficients if the system of equations is solved using Gaussian elimination.
Several authors have therefore proposed algorithms which exploit the structure of the Vandermonde matrix to compute numerically stable solutions in O("n"2) operations instead of the O("n"3) required by Gaussian elimination. These methods rely on constructing first a Newton interpolation of the polynomial and then converting it to the monomial form above.
Alternatively, we may write down the polynomial immediately in terms of Lagrange polynomials:
For matrix arguments, this formula is called Sylvester's formula and the matrix-valued Lagrange polynomials are the Frobenius covariants.
Uniqueness of the interpolating polynomial.
Proof 1.
Suppose we interpolate through data points with an at-most degree polynomial "p"("x") (we need at least datapoints or else the polynomial cannot be fully solved for). Suppose also another polynomial exists also of degree at most that also interpolates the points; call it "q"("x").
Consider formula_38. We know,
But "r"("x") is a polynomial of degree . It has one root too many. Formally, if "r"("x") is any non-zero polynomial, it must be writable as formula_40, for some constant "A". By distributivity, the "x"'s multiply together to give leading term formula_41, i.e. one degree higher than the maximum we set. So the only way "r"("x") can exist is if , or equivalently, .
So "q"("x") (which could be any polynomial, so long as it interpolates the points) is identical with "p"("x"), and "q"("x") is unique.
Proof 2.
Given the Vandermonde matrix used above to construct the interpolant, we can set up the system
To prove that V is nonsingular we use the Vandermonde determinant formula:
since the points are distinct, the determinant can't be zero as formula_45 is never zero, therefore "V" is nonsingular and the system has a unique solution.
Either way this means that no matter what method we use to do our interpolation: direct, Lagrange etc., (assuming we can do all our calculations perfectly) we will always get the same polynomial.
Non-Vandermonde solutions.
We are trying to construct our unique interpolation polynomial in the vector space Π"n" of polynomials of degree . When using a monomial basis for Π"n" we have to solve the Vandermonde matrix to construct the coefficients for the interpolation polynomial. This can be a very costly operation (as counted in clock cycles of a computer trying to do the job). By choosing another basis for Π"n" we can simplify the calculation of the coefficients but then we have to do additional calculations when we want to express the interpolation polynomial in terms of a monomial basis.
One method is to write the interpolation polynomial in the Newton form and use the method of divided differences to construct the coefficients, e.g. Neville's algorithm. The cost is O("n"2) operations, while Gaussian elimination costs O("n"3) operations. Furthermore, you only need to do O("n") extra work if an extra point is added to the data set, while for the other methods, you have to redo the whole computation.
Another method is to use the Lagrange form of the interpolation polynomial. The resulting formula immediately shows that the interpolation polynomial exists under the conditions stated in the above theorem. Lagrange formula is to be preferred to Vandermonde formula when we are not interested in computing the coefficients of the polynomial, but in computing the value of "p"("x") in a given "x" not in the original data set. In this case, we can reduce complexity to O("n"2).
The Bernstein form was used in a constructive proof of the Weierstrass approximation theorem by Bernstein and has gained great importance in computer graphics in the form of Bézier curves.
Linear combination of the given values.
The Lagrange form of the interpolating polynomial is a linear combination of the given values. In many scenarios, an efficient and convenient polynomial interpolation is a linear combination of the given values, using previously known coefficients. Given a set of formula_46 data points formula_47 where each data point is a (position, value) pair and where no two positions formula_48 are the same, the interpolation polynomial in the Lagrange form is a linear combination
of the given values formula_50 with each coefficient formula_51 given by evaluating the corresponding Lagrange basis polynomial using the given formula_46 positions formula_48.
Each coefficient formula_51 in the linear combination depends on the given positions formula_48 and the desired position formula_57, but not on the given values formula_50. For each coefficient, inserting the values of the given positions formula_48 and simplifying yields an expression formula_51, which depends only on formula_57. Thus the same coefficient expressions formula_62 can be used in a polynomial interpolation of a given second set of formula_46 data points formula_64 at the same given positions formula_48, where the second given values formula_66 differ from the first given values formula_50. Using the same coefficient expressions formula_62 as for the first set of data points, the interpolation polynomial of the second set of data points is the linear combination
For each coefficient formula_51 in the linear combination, the expression resulting from the Lagrange basis polynomial formula_71 only depends on the relative spaces between the given positions, not on the individual value of any position. Thus the same coefficient expressions formula_62 can be used in a polynomial interpolation of a given third set of formula_46 data points
where each position formula_75 is related to the corresponding position formula_48 in the first set by formula_77 and the desired positions are related by formula_78, for a constant scaling factor "a" and a constant shift "b" for all positions. Using the same coefficient expressions formula_79 as for the first set of data points, the interpolation polynomial of the third set of data points is the linear combination
In many applications of polynomial interpolation, the given set of formula_46 data points is at equally spaced positions. In this case, it can be convenient to define the "x"-axis of the positions such that formula_82. For example, a given set of 3 equally-spaced data points formula_83 is then formula_84.
The interpolation polynomial in the Lagrange form is the linear combination
This quadratic interpolation is valid for any position "x", near or far from the given positions. So, given 3 equally-spaced data points at formula_86 defining a quadratic polynomial, at an example desired position formula_87, the interpolated value after simplification is given by formula_88
This is a quadratic interpolation typically used in the Multigrid method. Again given 3 equally-spaced data points at formula_86 defining a quadratic polynomial, at the next equally spaced position formula_90, the interpolated value after simplification is given by
In the above polynomial interpolations using a linear combination of the given values, the coefficients were determined using the Lagrange method. In some scenarios, the coefficients can be more easily determined using other methods. Examples follow.
According to the method of finite differences, for any polynomial of degree "d" or less, any sequence of formula_92 values at equally spaced positions has a formula_93th difference exactly equal to 0. The element "s""d+1" of the Binomial transform is such a formula_93th difference. This area is surveyed here. The binomial transform, "T", of a sequence of values {"v""n"}, is the sequence {"s""n"} defined by
Ignoring the sign term formula_96, the formula_4 coefficients of the element "s""n" are the respective formula_4 elements of the row "n" of Pascal's Triangle. The triangle of binomial transform coefficients is like Pascal's triangle. The entry in the "n"th row and "k"th column of the BTC triangle is formula_99 for any non-negative integer "n" and any integer "k" between 0 and "n". This results in the following example rows "n" = 0 through "n" = 7, top to bottom, for the BTC triangle:
For convenience, each row "n" of the above example BTC triangle also has a label formula_100. Thus for any polynomial of degree "d" or less, any sequence of formula_92 values at equally spaced positions has a linear combination result of 0, when using the formula_92 elements of row "d" as the corresponding linear coefficients.
For example, 4 equally spaced data points of a quadratic polynomial obey the linear equation given by row formula_103 of the BTC triangle. formula_104 This is the same linear equation as obtained above using the Lagrange method.
The BTC triangle can also be used to derive other polynomial interpolations. For example, the above quadratic interpolation
can be derived in 3 simple steps as follows. The equally spaced points of a quadratic polynomial obey the rows of the BTC triangle with formula_103 or higher. First, the row formula_107 spans the given and desired data points formula_108 with the linear equation
Second, the unwanted data point formula_110 is replaced by an expression in terms of wanted data points. The row formula_103 provides a linear equation with a term formula_112, which results in a term formula_113 by multiplying both sides of the linear equation by 4. formula_114 Third, the above two linear equations are added to yield a linear equation equivalent to the above quadratic interpolation for formula_115. formula_116
Similar to other uses of linear equations, the above derivation scales and adds vectors of coefficients. In polynomial interpolation as a linear combination of values, the elements of a vector correspond to a contiguous sequence of regularly spaced positions. The "p" non-zero elements of a vector are the "p" coefficients in a linear equation obeyed by any sequence of "p" data points from any degree "d" polynomial on any regularly spaced grid, where "d" is noted by the subscript of the vector. For any vector of coefficients, the subscript obeys formula_117. When adding vectors with various subscript values, the lowest subscript applies for the resulting vector. So, starting with the vector of row formula_107 and the vector of row formula_103 of the BTC triangle, the above quadratic interpolation for formula_115 is derived by the vector calculation
Similarly, the cubic interpolation typical in the Multigrid method,
can be derived by a vector calculation starting with the vector of row formula_123 and the vector of row formula_107 of the BTC triangle.
Interpolation error.
When interpolating a given function "f" by a polynomial of degree at the nodes "x"0...,"x""n" we get the error
where
is the notation for divided differences.
If "f" is times continuously differentiable on a closed interval "I" and formula_128 is a polynomial of degree at most that interpolates "f" at distinct points {"x""i"} ("i"=0,1...,n) in that interval, then for each x in the interval there exists in that interval such that
The above error bound suggests choosing the interpolation points such that the product formula_130 is as small as possible. The Chebyshev nodes achieve this.
Proof.
Set the error term as
and set up an auxiliary function:
where
Since are roots of formula_134 and formula_135, we have , which means has at least roots. From Rolle's theorem, formula_136 has at least roots, then formula_137 has at least one root , where is in the interval .
So we can get
Since formula_128 is a polynomial of degree at most , then
Thus
Since is the root of formula_137, so
Therefore,
Thus the remainder term in the Lagrange form of the Taylor theorem is a special case of interpolation error when all interpolation nodes are identical. Note that the error will be zero when formula_145 for any "i". Thus, the maximum error will occur at some point in the interval between two successive nodes.
For equally spaced intervals.
In the case of equally spaced interpolation nodes where formula_146, for formula_147 and where formula_148 the product term in the interpolation error formula can be bound as
Thus the error bound can be given as
However, this assumes that formula_151 is dominated by formula_152, i.e. formula_153. In several cases, this is not true and the error actually increases as (see Runge's phenomenon). That question is treated in the section "Convergence properties".
Lebesgue constants.
We fix the interpolation nodes "x"0, ..., "x""n" and an interval ["a", "b"] containing all the interpolation nodes. The process of interpolation maps the function "f" to a polynomial "p". This defines a mapping "X" from the space "C"(["a", "b"]) of all continuous functions on ["a", "b"] to itself. The map "X" is linear and it is a projection on the subspace Π"n" of polynomials of degree "n" or less.
The Lebesgue constant "L" is defined as the operator norm of "X". One has (a special case of Lebesgue's lemma):
In other words, the interpolation polynomial is at most a factor ("L" + 1) worse than the best possible approximation. This suggests that we look for a set of interpolation nodes that makes "L" small. In particular, we have for Chebyshev nodes:
We conclude again that Chebyshev nodes are a very good choice for polynomial interpolation, as the growth in "n" is exponential for equidistant nodes. However, those nodes are not optimal.
Convergence properties.
It is natural to ask, for which classes of functions and for which interpolation nodes the sequence of interpolating polynomials converges to the interpolated function as ? Convergence may be understood in different ways, e.g. pointwise, uniform or in some integral norm.
The situation is rather bad for equidistant nodes, in that uniform convergence is not even guaranteed for infinitely differentiable functions. One classical example, due to Carl Runge, is the function "f"("x") = 1 / (1 + "x"2) on the interval . The interpolation error grows without bound as . Another example is the function "f"("x") = |"x"| on the interval , for which the interpolating polynomials do not even converge pointwise except at the three points "x" = ±1, 0.
One might think that better convergence properties may be obtained by choosing different interpolation nodes. The following result seems to give a rather encouraging answer:
Proof. It's clear that the sequence of polynomials of best approximation formula_157 converges to "f"("x") uniformly (due to Weierstrass approximation theorem). Now we have only to show that each formula_157 may be obtained by means of interpolation on certain nodes. But this is true due to a special property of polynomials of best approximation known from the equioscillation theorem. Specifically, we know that such polynomials should intersect "f"("x") at least times. Choosing the points of intersection as interpolation nodes we obtain the interpolating polynomial coinciding with the best approximation polynomial.
The defect of this method, however, is that interpolation nodes should be calculated anew for each new function "f"("x"), but the algorithm is hard to be implemented numerically. Does there exist a single table of nodes for which the sequence of interpolating polynomials converge to any continuous function "f"("x")? The answer is unfortunately negative:
The proof essentially uses the lower bound estimation of the Lebesgue constant, which we defined above to be the operator norm of "X""n" (where "X""n" is the projection operator on Π"n"). Now we seek a table of nodes for which
Due to the Banach–Steinhaus theorem, this is only possible when norms of "X""n" are uniformly bounded, which cannot be true since we know that
For example, if equidistant points are chosen as interpolation nodes, the function from Runge's phenomenon demonstrates divergence of such interpolation. Note that this function is not only continuous but even infinitely differentiable on . For better Chebyshev nodes, however, such an example is much harder to find due to the following result:
Related concepts.
Runge's phenomenon shows that for high values of , the interpolation polynomial may oscillate wildly between the data points. This problem is commonly resolved by the use of spline interpolation. Here, the interpolant is not a polynomial but a spline: a chain of several polynomials of a lower degree.
Interpolation of periodic functions by harmonic functions is accomplished by Fourier transform. This can be seen as a form of polynomial interpolation with harmonic base functions, see trigonometric interpolation and trigonometric polynomial.
Hermite interpolation problems are those where not only the values of the polynomial "p" at the nodes are given, but also all derivatives up to a given order. This turns out to be equivalent to a system of simultaneous polynomial congruences, and may be solved by means of the Chinese remainder theorem for polynomials. Birkhoff interpolation is a further generalization where only derivatives of some orders are prescribed, not necessarily all orders from 0 to a "k".
Collocation methods for the solution of differential and integral equations are based on polynomial interpolation.
The technique of rational function modeling is a generalization that considers ratios of polynomial functions.
At last, multivariate interpolation for higher dimensions.