Modeling methods are known generally in the state of the art, for example from the following articles:
HOSKING J. M. R. et al. “A statistical perspective on data mining” FUTURE GENERATION COMPUTER SYSTEMS, November 1997, ELSEVIER, Vol. 13, No. 2-3, pages 117-134, which describes the use of statistical methods for processing large volumes of data;
ROSSIUS R. et al. “A short note about the application of polynomial kernels with fractional degree in support vector learning” MACHINE LEARNING ECML-98. 10th European Conference on Machine Learning. Proceedings CHEMNITZ Apr. 21-23, 1998, pages 143-148, pertaining to the application of classification and forecasting methods.
The learning problem can be considered as a problem of finding dependencies using a limited number of observations. Thus, it is a question of choosing from a given set of functions f(x,α), α∈A, where A is a set of parameters, the one which best approximates the output.
If L(y,f(x,α) is a measure of the deviation between the real output y and the output predicted by the model f(x,α), it is thus necessary to minimize the effective risk:R(α)=∫L(y,f(x,α))dF(x,y)  (Eq. 1)while knowing that the joint probability distribution function F(x,y) is unknown and that the only available information is contained in the k observations (x1, y1) . . . ,(xk, yk) from the learning set.
Classically, one determines the function that minimizes the empirical risk calculated on the basis of the learning set:                     Remp        =                              ∑                          i              =              1                        k                    ⁢                      L            ⁡                          (                                                y                  i                                ,                                  f                  ⁡                                      (                                                                  x                        i                                            ,                      α                                        )                                                              )                                                          (                  Eq          .                                           ⁢          2                )            
One then postulates that this function would be the best approximation of the function that minimizes the effective risk given by (Eq. 1).
The problem posed is to know the extent to which a system constructed on the empirical risk minimization inductive principle (Eq. 2) is generalizable, i.e., enables minimizing the effective risk (Eq. 1) including data that have not been learned.
Mathematically, a problem is said to be well-posed when it allows a unique solution and this solution is stable, i.e., that a small deviation in the initial conditions can only modify in an infinitesimal manner the form of the solutions. Problems that do not satisfy these conditions are referred to as ill-posed problems.
It occurs frequently that the problem of finding f satisfying the equality A,f=u is ill-posed: even if there exists a unique solution to this equation, a small deviation of the right-hand side of this equation can cause large deviations in the solution.
And thus if the right-hand member is not exact (uε instead of u with ∥u−uε∥≦ε), the functions that minimize the empirical risk R(ƒ)=∥Aƒ−u∈∥2 are not necessarily good approximations of the desired solution, even if ∈ tends to 0.
An improvement in solving such problems consists in minimizing another so-called regularized functional of the form:R(ƒ)=R(ƒ)+λ(∈)Ω(ƒ)  (Eq. 3)where:                Ω(ƒ) is some functional belonging to a special type of operators referred to as regularizing;        λ(∈) is an appropriately chosen constant depending on the level of noise existing on the data.        
One then obtains a sequence of solutions that converge to the desired one as ∈ tends to 0. Minimizing the regularized risk rather than the empirical risk allows obtaining from a limited number of observations a solution that is generalizable to any case.
Introduction of the regularizing term makes it feasible to provide with certainty a unique solution to an ill-posed problem. This solution can be slightly less accurate than the classic one, but it possesses the fundamental property of being stable, thus endowing the results with greater robustness.
The methods for solving ill-posed problems demonstrate that there exist other inductive principles that enable obtaining a better regularization capacity than the principle consisting in minimizing the error made on the learning set.
Therefore, the main objective of theoretical analysis is to find the principles making it feasible to control the generalization capacity of learning systems and to construct algorithms that implement these principles.
Vapnik's theory is the tool that establishes the necessary and sufficient conditions for a learning process based on the empirical risk minimization principle to be generalizable, leading to a new inductive principle referred to as the structural risk minimization principle. It can be demonstrated that the effective risk satisfies an inequality of the form:R(α)<Remp(α)+F(h,k)  (Eq. 4)where:                h is the Vapnik-Chervonenkis dimension of the set of functions f(x,α) among which the solution is sought;        k is the number of observations available for constructing the model;        F is an increasing function of h and a decreasing function of k.        
It can be seen immediately that, since the number k of available observations is finite, the fact of minimizing the empirical error is not sufficient for minimizing the effective error. The general idea of the structural risk minimization principle is to take into account the two terms of the right-hand member of the equation (Eq. 4), rather than only the empirical risk. This implies constraining the structure of the set of the functions f(x, α) among which the solution is sought so as to limit or even control the parameter h.
According to this principle, the development of new algorithms enabling control of the robustness of learning processes would be advantageous.