The finite element methods are important and powerful tools for solving partial differential equations on complex geometries, but their high-order generalizations pose significant challenges in terms of robustness and mesh generation. In this paper, we introduce a high-order finite element method, which is insensitive to element quality and hence is more robust, and which requires only linear elements even for curved geometries and hence significantly simplifies meshing. Based on our recent work on Adaptive Extended Stencil FEM, or AES-FEM (Int. J. Num. Meth. Engrg., 2016, DOI:10.1002 nme.5246), the proposed method replaces the traditional interpolatory FEM basis functions with generalized Lagrange polynomial basis functions, constructed via local weighted least-squares approximations. In this work, we show that AES-FEM can achieve high-order accuracy using linear elements to discretize the geometry and using the standard FEM hat functions as test functions even for curved geometries. This is in contrast to most high-order generalizations of FEM, which require high-order curved elements for curved geometries. We describe the selection of stencils and the treatment of boundary conditions for high-order AES-FEM, and show that even-degree polynomial basis functions are preferred for even-order PDEs. We present numerical results in 2D and 3D for the Poisson equation and the convection-diffusion equation and demonstrate up to sixth order accuracy with AES-FEM. Overall, AES-FEM is much easier to use than FEM for curved boundaries, while being more accurate, more stable, and more efficient in terms of runtime versus error over relatively fine meshes.
Consider a partial differential equation on a bounded, simply-connected domain @math , subject to the Dirichlet and Neumann boundary conditions @math denotes a linear differential operator, @math and @math are disjoint sets of the boundary with @math , and @math denotes the outward normal to @math . A finite element method, or more generally a weighted-residual method @cite_38 , introduces a set of test functions (a.k.a. weight functions) @math , and require the residual @math to be orthogonal to @math , i.e., Eq. ) and ) are the and of the PDE, respectively. In finite element methods, @math is typically (weakly) differentiate and has a local support. In addition, @math typically forms a partition of unity, i.e., @math , so that the weighted-residual formulation is global conservative in that @math
@cite_38
1965727018 2885396681 2041509689 1971181547 1994793184 2766371994 2030010167 2083690159 2552723767 140909251 2063399239 1985536181 2949393806 1970063032 2949940189 2950162031 2089135543 1724455341 2035238652 2950933581
In this paper, we propose a method for the approximation of the solution of high- dimensional weakly coercive problems formulated in tensor spaces using low-rank approximation for- mats. The method can be seen as a perturbation of a minimal residual method with a measure of the residual corresponding to the error in a specified solution norm. The residual norm can be de- signed such that the resulting low-rank approximations are optimal with respect to particular norms of interest, thus allowing to take into account a particular objective in the definition of reduced order approximations of high-dimensional problems. We introduce and analyze an iterative algorithm that is able to provide an approximation of the optimal approximation of the solution in a given low-rank subset, without any ap rioriinformation on this solution. We also introduce a weak greedy algorithm which uses this perturbed minimal residual method for the computation of successive greedy correc- tions in small tensor subsets. We prove its convergence under some conditions on the parameters of the algorithm. The proposed numerical method is applied to the solution of a stochastic partial differential equation which is discretized using standard Galerkin methods in tensor product spaces. Mathematics Subject Classification. 15A69, 35J50, 41A63, 65D15, 65N12. We develop fast and memory efficient numerical methods for learning functions of many variables that admit sparse representations in terms of general bounded orthonormal tensor product bases. Such functions appear in many applications including, e.g., various Uncertainty Quantification(UQ) problems involving the solution of parametric PDE that are approximately sparse in Chebyshev or Legendre product bases. We expect that our results provide a starting point for a new line of research on sublinear-time solution techniques for UQ applications of the type above which will eventually be able to scale to significantly higher-dimensional problems than what are currently computationally feasible. More concretely, let @math be a finite Bounded Orthonormal Product Basis (BOPB) of cardinality @math . We will develop methods that approximate any function @math that is sparse in the BOPB, that is, @math of the form @math with @math of cardinality @math . Our method has a runtime of just @math , uses only @math function evaluations on a fixed and nonadaptive grid, and not more than @math bits of memory. For @math , the runtime @math will be less than what is required to simply enumerate the elements of the basis @math ; thus our method is the first approach applicable in a general BOPB framework that falls into the class referred to as "sublinear-time". This and the similarly reduced sample and memory requirements set our algorithm apart from previous works based on standard compressive sensing algorithms such as basis pursuit which typically store and utilize full intermediate basis representations of size @math . The numerical approximation of parametric partial differential equations D(u,y)=0 is a computational challenge when the dimension d of the parameter vector y is large, due to the so-called curse of dimensionality. It was recently shown in [1,2] that, for a certain class of elliptic PDEs with diffusion coefficients depending on the parameters in an affine manner, there exist polynomial approximations to the solution map y↦u(y) with an algebraic convergence rate that is independent of the parametric dimension d. The analysis in [1,2] used, however, the affine parameter dependence of the operator. The present paper proposes a strategy for establishing similar results for classes of parametric PDEs that do not necessarily fall in this category. Our approach is based on building an analytic extension z↦u(z) of the solution map on certain tensor product of ellipses in the complex domain, and using this extension to estimate the Legendre coefficients of u. The varying semi-axes lengths of the ellipses in each coordinate zj reflect the anisotropy of the solution map with respect to the corresponding parametric variables yj. This allows us to derive algebraic convergence rates for tensorized Legendre expansions in the case d=∞. We also show that such rates are preserved when using certain interpolation procedures, which is an instance of a non-intrusive method. As examples of parametric PDEs that are covered by this approach, we consider (i) elliptic diffusion equations with coefficients that depend on the parameter vector y in a not necessarily affine manner, (ii) parabolic diffusion equations with similar dependence of the coefficient on y, (iii) nonlinear, monotone parametric elliptic PDEs, and (iv) elliptic equations set on a domain that is parametrized by the vector y. We give general strategies that allow us to derive the analytic extension in a unified abstract way for all these examples, in particular based on the holomorphic version of the implicit function theorem in Banach spaces. We expect that this approach can be applied to a large variety of parametric PDEs, showing that the curse of dimensionality can be overcome under mild assumptions. A simple way to solve a partial differential equation in a non-rectangular domain @W is to embed the domain in a rectangle B and solve the problem (more easily) in the rectangle. To apply a Fourier spectral method on the rectangle, it is necessary to extend the inhomogeneous term in the PDE, f(x,y), to B in such a way that the extended function g(x,y) is periodic and infinitely differentiable, and yet is equal to f(x,y) everywhere in the irregular domain @W. If the boundary of @W, @[email protected], is defined as the zero isoline of a function @F(x,y), then a suitable extension is g(x,y)=f(x,y)H([email protected](x,y) constant) where H is a smoothed approximation to the step function which is defined in the main text. A new domain decomposition method is presented for the exterior Helmholtz problem. The nonlocal Dirichlet-to-Neumann (DtN) map is used as a nonreflecting condition on the outer computational boundary. The computational domain is divided into nonoverlapping subdomains with Sommerfeld-type conditions on the adjacent subdomain boundaries to ensure uniqueness. An iterative scheme is developed, where independent subdomain boundary-value problems are obtained by applying the DtN operator to values from the previous iteration. The independent problems are then discretized with finite elements and can be solved concurrently. Numerical results are presented for a two-dimensional model problem, and both the solution accuracy and convergence rate are investigated. We consider the problem of learning a one-hidden-layer neural network: we assume the input @math is from Gaussian distribution and the label @math , where @math is a nonnegative vector in @math with @math , @math is a full-rank weight matrix, and @math is a noise vector. We first give an analytic formula for the population risk of the standard squared loss and demonstrate that it implicitly attempts to decompose a sequence of low-rank tensors simultaneously. Inspired by the formula, we design a non-convex objective function @math whose landscape is guaranteed to have the following properties: 1. All local minima of @math are also global minima. 2. All global minima of @math correspond to the ground truth parameters. 3. The value and gradient of @math can be estimated using samples. With these properties, stochastic gradient descent on @math provably converges to the global minimum and learn the ground-truth parameters. We also prove finite sample complexity result and validate the results by simulations. Making use of a perturbed Lagrangian formulation, a finite element procedure for contact problems is developed for the general case in which node-to-node contact no longer holds. The proposed procedure leads naturally to a discretization of the contact interface into contact segments. Within the context of a bilinear interpolation for the displacement field, a mixed finite element approximation is introduced by assuming discont naous contact pressure, constant on the contact segment. Because of this piece-wise constant approximation, the gap function enters into the formulation in an ‘average’ sense instead of through a point-wise definition. Numerical examples are presented that illustrate the performance of the proposed procedure. Current finite element formulations for contact problems based on either the classical Lagrange parameter procedure [l-3, 12, 203 or the penalty-function method [4-6,11], are characterized by a point-wise enforcement of the contact-constraint condition, in the sense that penetration of the bodies is established on a nodal basis. Moreover, in this methods the recovery of the contact pressure over the element from the contact nodal forces generally requires an additional procedure. Within the framework of classical Lagrange multiplier methods the contact condition is exactly satisfied by transforming the constrained problem into an unconstrained one with the introduction of additional variables (Lagrange multipliers). These extra variables add computational effort to the solution process which often requires special procedures to handle the presence of zero diagonal terms. Penalty methods, on the other hand, enable one to transform the constrained problem into an unconstrained one without introducing additional variables. The constraint condition is now satisfied only approximately for finite values of the penalty parameter. The main difhculty associated with these methods, however, lies in the poor conditioning of the problem as the penalty is increased to more accurately enforce the constraint condition. This is a well-understood phenomenon, particularly in the context of the incompressible and nearly incompressible problem in solid and fluid mechanics (e.g. see [l&22,25] for a review). Recently, augmented Lagrangian procedures have been proposed as a promising way to partially overcome these difficulties and ‘regularize’ the penalty formulation (e.g. see the survey in [7,8]). Within the framework of Zinearited e u c , it is possible to restrict the finite element formulation of contact problems by assuming that node-to-node contact occurs. This is in fact the case often considered in the literature [l, 2, 4, 6, 12, 20, 211. In the general context of fully We present a novel boundary integral formulation of the Helmholtz transmission problem for bounded composite scatterers (that is, piecewise constant material parameters in "subdomains") that directly lends itself to operator preconditioning via Calderon projectors. The method relies on local traces on subdomains and weak enforcement of transmission conditions. The variational formulation is set in Cartesian products of standard Dirichlet and special Neumann trace spaces for which restriction and extension by zero are well defined. In particular, the Neumann trace spaces over each subdomain boundary are built as piecewise @math -distributions over each associated interface. Through the use of interior Calderon projectors, the problem is cast in variational Galerkin form with an operator matrix whose diagonal is composed of block boundary integral operators associated with the subdomains. We show existence and uniqueness of solutions based on an extension of Lions' projection lemma for non-closed subspaces. We also investigate asymptotic quasi-optimality of conforming boundary element Galerkin discretization. Numerical experiments in 2-D confirm the efficacy of the method and a performance matching that of another widely used boundary element discretization. They also demonstrate its amenability to different types of preconditioning. We introduce the framework of blind regression motivated by matrix completion for recommendation systems: given @math users, @math movies, and a subset of user-movie ratings, the goal is to predict the unobserved user-movie ratings given the data, i.e., to complete the partially observed matrix. Following the framework of non-parametric statistics, we posit that user @math and movie @math have features @math and @math respectively, and their corresponding rating @math is a noisy measurement of @math for some unknown function @math . In contrast with classical regression, the features @math are not observed, making it challenging to apply standard regression methods to predict the unobserved ratings. Inspired by the classical Taylor's expansion for differentiable functions, we provide a prediction algorithm that is consistent for all Lipschitz functions. In fact, the analysis through our framework naturally leads to a variant of collaborative filtering, shedding insight into the widespread success of collaborative filtering in practice. Assuming each entry is sampled independently with probability at least @math with @math , we prove that the expected fraction of our estimates with error greater than @math is less than @math plus a polynomially decaying term, where @math is the variance of the additive entry-wise noise term. Experiments with the MovieLens and Netflix datasets suggest that our algorithm provides principled improvements over basic collaborative filtering and is competitive with matrix factorization methods. Discontinuous Galerkin (DG) methods for solving partial differential equations, developed in the late 1990s, have become popular among computational scientists. This book covers both theory and computation as it focuses on three primal DG methods--the symmetric interior penalty Galerkin, incomplete interior penalty Galerkin, and nonsymmetric interior penalty Galerkin which are variations of interior penalty methods. The author provides the basic tools for analysis and discusses coding issues, including data structure, construction of local matrices, and assembling of the global matrix. Computational examples and applications to important engineering problems are also included. Discontinuous Galerkin Methods for Solving Elliptic and Parabolic Equations: Theory and Implementation is divided into three parts: Part I focuses on the application of DG methods to second order elliptic problems in one dimension and in higher dimensions. Part II presents the time-dependent parabolic problems without and with convection. Part III contains applications of DG methods to solid mechanics (linear elasticity), fluid dynamics (Stokes and Navier Stokes), and porous media flow (two-phase and miscible displacement). Appendices contain proofs and MATLAB code for one-dimensional problems for elliptic equations and routines written in C that correspond to algorithms for the implementation of DG methods in two or three dimensions. Audience: This book is intended for numerical analysts, computational and applied mathematicians interested in numerical methods for partial differential equations or who study the applications discussed in the book, and engineers who work in fluid dynamics and solid mechanics and want to use DG methods for their numerical results. The book is appropriate for graduate courses in finite element methods, numerical methods for partial differential equations, numerical analysis, and scientific computing. Chapter 1 is suitable for a senior undergraduate class in scientific computing. Contents: List of Figures; List of Tables; List of Algorithms; Preface; Part I: Elliptic Problems; Chapter 1: One-dimensional problem; Chapter 2: Higher dimensional problem; Part II: Parabolic Problems; Chaper 3: Purely parabolic problems; Chapter 4: Parabolic problems with convection; Part III: Applications; Chapter 5: Linear elasticity; Chapter 6: Stokes flow; Chapter 7: Navier-Stokes flow; Chapter 8: Flow in porous media; Appendix A: Quadrature rules; Appendix B: DG codes; Appendix C: An approximation result; Bibliography; Index. We introduce a general purpose algorithm for rapidly computing certain types of oscillatory integrals which frequently arise in problems connected to wave propagation, general hyperbolic equations, and curvilinear tomography. The problem is to numerically evaluate a so-called Fourier integral operator (FIO) of the form @math at points given on a Cartesian grid. Here, @math is a frequency variable, @math is the Fourier transform of the input @math , @math is an amplitude, and @math is a phase function, which is typically as large as @math ; hence the integral is highly oscillatory. Because a FIO is a dense matrix, a naive matrix vector product with an input given on a Cartesian grid of size @math by @math would require @math operations. This paper develops a new numerical algorithm which requires @math operations and as low as @math in storage space (the constants in front of these estimates are small). It operates by localizing the integral over polar wedges with small angular aperture in the frequency plane. On each wedge, the algorithm factorizes the kernel @math into two components: (1) a diffeomorphism which is handled by means of a nonuniform FFT and (2) a residual factor which is handled by numerical separation of the spatial and frequency variables. The key to the complexity and accuracy estimates is the fact that the separation rank of the residual kernel is provably independent of the problem size. Several numerical examples demonstrate the numerical accuracy and low computational complexity of the proposed methodology. We also discuss the potential of our ideas for various applications such as reflection seismology. Many conjugate gradient-like methods for solving linear systems @math use recursion formulas for updating residual vectors instead of computing the residuals directly. For such methods it is shown that the difference between the actual residuals and the updated approximate residual vectors generated in finite precision arithmetic depends on the machine precision @math and on the maximum norm of an iterate divided by the norm of the true solution. It is often observed numerically, and can sometimes be proved, that the norms of the updated approximate residual vectors converge to zero or, at least, become orders of magnitude smaller than the machine precision. In such cases, the actual residual norm reaches the level @math times the maximum ratio of the norm of an iterate to that of the true solution. Using exact arithmetic theory to bound the size of the iterates, we give a priori estimates of the size of the final residual for a number of algorithms. We show a new way to round vector solutions of semidefinite programming (SDP) hierarchies into integral solutions, based on a connection between these hierarchies and the spectrum of the input graph. We demonstrate the utility of our method by providing a new SDP-hierarchy based algorithm for constraint satisfaction problems with 2-variable constraints (2-CSP's). More concretely, we show for every 2-CSP instance I a rounding algorithm for r rounds of the Lasserre SDP hierarchy for I that obtains an integral solution that is at most worse than the relaxation's value (normalized to lie in [0,1]), as long as r > k ( ) ( ) ;, where k is the alphabet size of I, @math , and @math denotes the number of eigenvalues larger than @math in the normalized adjacency matrix of the constraint graph of @math . In the case that @math is a instance, the threshold @math is only a polynomial in @math , and is independent of the alphabet size. Also in this case, we can give a non-trivial bound on the number of rounds for instance. In particular our result yields an SDP-hierarchy based algorithm that matches the performance of the recent subexponential algorithm of Arora, Barak and Steurer (FOCS 2010) in the worst case, but runs faster on a natural family of instances, thus further restricting the set of possible hard instances for Khot's Unique Games Conjecture. Our algorithm actually requires less than the @math constraints specified by the @math level of the Lasserre hierarchy, and in some cases @math rounds of our program can be evaluated in time @math . We study Byzantine fault-tolerant distributed optimization of a sum of convex (cost) functions with real-valued scalar input ouput. In particular, the goal is to optimize a global cost function @math , where @math is the set of non-faulty agents, and @math is agent @math 's local cost function, which is initially known only to agent @math . In general, when some of the agents may be Byzantine faulty, the above goal is unachievable, because the identity of the faulty agents is not necessarily known to the non-faulty agents, and the faulty agents may behave arbitrarily. Since the above global cost function cannot be optimized exactly in presence of Byzantine agents, we define a weaker version of the problem. The goal for the weaker problem is to generate an output that is an optimum of a function formed as a convex combination of local cost functions of the non-faulty agents. More precisely, for some choice of weights @math for @math such that @math and @math , the output must be an optimum of the cost function @math . Ideally, we would like @math for all @math -- however, this cannot be guaranteed due to the presence of faulty agents. In fact, we show that the maximum achievable number of nonzero weights ( @math 's) is @math , where @math is the upper bound on the number of Byzantine agents. In addition, we present algorithms that ensure that at least @math agents have weights that are bounded away from 0. We also propose a low-complexity suboptimal algorithm, which ensures that at least @math agents have weights that are bounded away from 0, where @math is the total number of agents, and @math ( @math ) is the actual number of Byzantine agents. Estimation of functions of @math variables is considered using ridge combinations of the form @math where the activation function @math is a function with bounded value and derivative. These include single-hidden layer neural networks, polynomials, and sinusoidal models. From a sample of size @math of possibly noisy values at random sites @math , the minimax mean square error is examined for functions in the closure of the @math hull of ridge functions with activation @math . It is shown to be of order @math to a fractional power (when @math is of smaller order than @math ), and to be of order @math to a fractional power (when @math is of larger order than @math ). Dependence on constraints @math and @math on the @math norms of inner parameter @math and outer parameter @math , respectively, is also examined. Also, lower and upper bounds on the fractional power are given. The heart of the analysis is development of information-theoretic packing numbers for these classes of functions. We investigate the clustering performances of the relaxed @math means in the setting of sub-Gaussian Mixture Model (sGMM) and Stochastic Block Model (SBM). After identifying the appropriate signal-to-noise ratio (SNR), we prove that the misclassification error decay exponentially fast with respect to this SNR. These partial recovery bounds for the relaxed @math means improve upon results currently known in the sGMM setting. In the SBM setting, applying the relaxed @math means SDP allows to handle general connection probabilities whereas other SDPs investigated in the literature are restricted to the assortative case (where within group probabilities are larger than between group probabilities). Again, this partial recovery bound complements the state-of-the-art results. All together, these results put forward the versatility of the relaxed @math means. We give efficient algorithms for volume sampling, i.e., for picking @math -subsets of the rows of any given matrix with probabilities proportional to the squared volumes of the simplices defined by them and the origin (or the squared volumes of the parallelepipeds defined by these subsets of rows). In other words, we can efficiently sample @math -subsets of @math with probabilities proportional to the corresponding @math by @math principal minors of any given @math by @math positive semi definite matrix. This solves an open problem from the monograph on spectral algorithms by Kannan and Vempala (see Section @math of KV , also implicit in BDM, DRVW ). Our first algorithm for volume sampling @math -subsets of rows from an @math -by- @math matrix runs in @math arithmetic operations (where @math is the exponent of matrix multiplication) and a second variant of it for @math -approximate volume sampling runs in @math arithmetic operations, which is almost linear in the size of the input (i.e., the number of entries) for small @math . Our efficient volume sampling algorithms imply the following results for low-rank matrix approximation: (1) Given @math , in @math arithmetic operations we can find @math of its rows such that projecting onto their span gives a @math -approximation to the matrix of rank @math closest to @math under the Frobenius norm. This improves the @math -approximation of Boutsidis, Drineas and Mahoney BDM and matches the lower bound shown in DRVW . The method of conditional expectations gives a algorithm with the same complexity. The running time can be improved to @math at the cost of losing an extra @math in the approximation factor. (2) The same rows and projection as in the previous point give a @math -approximation to the matrix of rank @math closest to @math under the spectral norm. In this paper, we show an almost matching lower bound of @math , even for @math . Let @math be an i.i.d. sequence of random variables in @math , @math . We show that, for any function @math , under regularity conditions, [n^ 1 2 (n^ -1 i=1 ^n (X_i) f ^(X_i) - (x) ,dx ) P 0, ] where @math is the classical kernel estimator of the density of @math . This result is striking because it speeds up traditional rates, in root @math , derived from the central limit theorem when @math . Although this paper highlights some applications, we mainly address theoretical issues related to the later result. We derive upper bounds for the rate of convergence in probability. These bounds depend on the regularity of the functions @math and @math , the dimension @math and the bandwidth of the kernel estimator @math . Moreover, they are shown to be accurate since they are used as renormalizing sequences in two central limit theorems each reflecting different degrees of smoothness of @math . As an application to regression modelling with random design, we provide the asymptotic normality of the estimation of the linear functionals of a regression function. As a consequence of the above result, the asymptotic variance does not depend on the regression function. Finally, we debate the choice of the bandwidth for integral approximation and we highlight the good behavior of our procedure through simulations. We propose and study a domain decomposition method for solving the truncated perfectly matched layer (PML) approximation in bounded domain of Helmholtz scattering problems. The method is based on the decomposition of the domain into nonoverlapping layers and the idea of source transfer which transfers the sources equivalently layer by layer so that the solution in the final layer can be solved using a PML method defined locally outside the last two layers. The convergence of the method is proved for the case of constant wave number based on the analysis of the fundamental solution of the PML equation. The method can be used as an efficient preconditioner in the preconditioned GMRES method for solving discrete Helmholtz equations with constant and heterogeneous wave numbers. Numerical examples are included. We investigate the approximability of several classes of real-valued functions by functions of a small number of variables ( juntas ). Our main results are tight bounds on the number of variables required to approximate a function @math within @math -error @math over the uniform distribution: 1. If @math is submodular, then it is @math -close to a function of @math variables. This is an exponential improvement over previously known results. We note that @math variables are necessary even for linear functions. 2. If @math is fractionally subadditive (XOS) it is @math -close to a function of @math variables. This result holds for all functions with low total @math -influence and is a real-valued analogue of Friedgut's theorem for boolean functions. We show that @math variables are necessary even for XOS functions. As applications of these results, we provide learning algorithms over the uniform distribution. For XOS functions, we give a PAC learning algorithm that runs in time @math . For submodular functions we give an algorithm in the more demanding PMAC learning model (Balcan and Harvey, 2011) which requires a multiplicative @math factor approximation with probability at least @math over the target distribution. Our uniform distribution algorithm runs in time @math . This is the first algorithm in the PMAC model that over the uniform distribution can achieve a constant approximation factor arbitrarily close to 1 for all submodular functions. As follows from the lower bounds in (, 2013) both of these algorithms are close to optimal. We also give applications for proper learning, testing and agnostic learning with value queries of these classes.