In this paper, we propose a deep learning framework based on randomized neural network. In particular, inspired by the principles of Random Vector Functional Link (RVFL) network, we present a deep RVFL network (dRVFL) with stacked layers. The parameters of the hidden layers of the dRVFL are randomly generated within a suitable range and kept fixed while the output weights are computed using the closed form solution as in a standard RVFL network. We also propose an ensemble deep network (edRVFL) that can be regarded as a marriage of ensemble learning with deep learning. Unlike traditional ensembling approaches that require training several models independently from scratch, edRVFL is obtained by training a single dRVFL network once. Both dRVFL and edRVFL frameworks are generic and can be used with any RVFL variant. To illustrate this, we integrate the deep learning networks with a recently proposed sparse-pretrained RVFL (SP-RVFL). Extensive experiments on benchmark datasets from diverse domains show the superior performance of our proposed deep RVFL networks.
Typically, Eq. can be solved via a closed form solution using either ridge regression (i.e. @math ) or Moore-Penrose pseudoinverse (i.e. @math ) @cite_4 . Using Moore-Penrose pseudoinverse, the solution is given by: @math while using the regularized least squares (or ridge regression), the closed form solution is given by:
@cite_4
2024591553 199271301 1813460488 2135921993
We study the approximation of Ef(X-T) by a Monte Carlo algorithm, where X is the solution of a stochastic differential equation and f is a given function. We introduce a new variance reduction method, which can be viewed as a statistical analogue of Romberg extrapolation method. Namely, we use two Euler schemes with steps delta and delta(beta), 0 < beta < 1. This leads to an algorithm which, for a given level of the statistical error, has a complexity significantly lower than the complexity of the standard Monte Carlo method. We analyze the asymptotic error of this algorithm in the context of general (possibly degenerate) diffusions. In order to find the optimal beta (which turns out to be beta = 1 2), we establish a central limit: type theorem, based on a result of Jacod and Protter for the asymptotic distribution of the error in the Euler scheme. We test our method on various examples. In particular, we adapt it to Asian options. In this setting, we have a CLT and, as a by-product, an explicit expansion of the discretization error. We study a decomposition-based scalable approach to performing kernel ridge regression. The method is simple to describe: it randomly partitions a dataset of size N into m subsets of equal size, computes an independent kernel ridge regression estimator for each subset, then averages the local solutions into a global predictor. This partitioning leads to a substantial reduction in computation time versus the standard approach of performing kernel ridge regression on all N samples. Our main theorem establishes that despite the computational speed-up, statistical optimality is retained: if m is not too large, the partition-based estimate achieves optimal rates of convergence for the full sample size N. As concrete examples, our theory guarantees that m may grow polynomially in N for Sobolev spaces, and nearly linearly for finite-rank kernels and Gaussian kernels. We conclude with simulations complementing our theoretical results and exhibiting the computational and statistical benefits of our approach. The ep regression problem takes as input a matrix A ∈ ℝn, a vector b ∈ ℝn, and a number p ∈ [1, ∞), and it returns as output a number Z and a vector xOPT ∈ ℝd such that Z = minx∈ℝd ||Ax - b||p = ||AxOPT - b||p. In this paper, we construct coresets and obtain an efficient two-stage sampling-based approximation algorithm for the very overconstrained (n G d) version of this classical problem, for all p ∈ [1, ∞). The first stage of our algorithm non-uniformly samples r1 = O(36pdmax p 2+1, p +1) rows of A and the corresponding elements of b, and then it solves the lp regression problem on the sample; we prove this is an 8-approximation. The second stage of our algorithm uses the output of the first stage to resample r1 e2 constraints, and then it solves the lp regression problem on the new sample; we prove this is a (1 + e)-approximation. Our algorithm unifies, improves upon, and extends the existing algorithms for special cases of ep regression, namely p = 1,2 [10, 13]. In course of proving our result, we develop two concepts--well-conditioned bases and subspace-preserving sampling--that are of independent interest. We propose a fast algorithm for ridge regression when the number of features is much larger than the number of observations (p ≫ n). The standard way to solve ridge regression in this setting works in the dual space and gives a running time of O(n2p). Our algorithm Subsampled Randomized Hadamard Transform- Dual Ridge Regression (SRHT-DRR) runs in time O(np log(n)) and works by preconditioning the design matrix by a Randomized Walsh-Hadamard Transform with a subsequent subsampling of features. We provide risk bounds for our SRHT-DRR algorithm in the fixed design setting and show experimental results on synthetic and real datasets.