We address the problem of complementing higher-order patterns without repetitions of existential variables. Differently from the first-order case, the complement of a pattern cannot, in general, be described by a pattern, or even by a finite set of patterns. We therefore generalize the simply-typed λ-calculus to include an internal notion of strict function so that we can directly express that a term must depend on a given variable. We show that, in this more expressive calculus, finite sets of patterns without repeated variables are closed under complement and intersection. Our principal application is the transformational approach to negation in higher-order logic programs.
The class of higher-order patterns inherits many properties from first-order terms. However, as we will see, it is closed under complement, but a special subclass is. We call a canonical pattern @math if each occurrence of an existential variable @math under binders @math is applied to some permutation of the variables in @math and @math . Fully applied patterns play an important role in functional logic programming and rewriting @cite_6 , because any fully applied existential variable @math denotes all canonical terms of type @math with parameters from @math . It is this property which makes complementation particularly simple.
@cite_6
1550461844
This paper presents the temporal logic of rewriting @math . Syntactically, @math is a very simple extension of @math which just adds action atoms, in the form of spatial action patterns, to @math . Semantically and pragmatically, however, when used together with rewriting logic as a "tandem" of system specification and property specification logics, it has substantially more expressive power than purely state-based logics like @math , or purely action-based logics like A- @math . Furthermore, it avoids the system property mismatch problem experienced in state-based or action-based logics, which makes many useful properties inexpressible in those frameworks without unnatural changes to a system's specification. The advantages in expresiveness of @math are gained without losing the ability to use existing tools and algorithms to model check its properties: a faithful translation of models and formulas is given that allows verifying @math properties with @math model checkers. We introduce the notion of a canonical tester for a class of properties on distributions, that is, a tester strong and general enough that “a distribution property in the class is testable if and only if the canonical tester tests it.” We construct a canonical tester for the class of properties of one or two distributions that are symmetric and satisfy a certain weak continuity condition. Analyzing the performance of the canonical tester on specific properties resolves two open problems, establishing lower bounds that match known upper bounds: we show that distinguishing between entropy @math on distributions over @math requires @math samples, and distinguishing whether a pair of distributions has statistical distance @math requires @math samples. Our techniques also resolve a conjecture about a property that our canonical tester does not apply to: distinguishing identical distributions from those with statistical distance @math requires @math samples. We consider document listing on string collections, that is, finding in which strings a given pattern appears. In particular, we focus on repetitive collections: a collection of size @math over alphabet @math is composed of @math copies of a string of size @math , and @math edits are applied on ranges of copies. We introduce the first document listing index with size @math , precisely @math bits, and with useful worst-case time guarantees: Given a pattern of length @math , the index reports the @math strings where it appears in time @math , for any constant @math (and tells in time @math if @math ). Our technique is to augment a range data structure that is commonly used on grammar-based indexes, so that instead of retrieving all the pattern occurrences, it computes useful summaries on them. We show that the idea has independent interest: we introduce the first grammar-based index that, on a text @math with a grammar of size @math , uses @math bits and counts the number of occurrences of a pattern @math in time @math , for any constant @math . We also give the first index using @math bits, where @math is parsed by Lempel-Ziv into @math phrases, counting occurrences in time @math . Let @math be a class of probability distributions over the discrete domain @math We show that if @math satisfies a rather general condition -- essentially, that each distribution in @math can be well-approximated by a variable-width histogram with few bins -- then there is a highly efficient (both in terms of running time and sample complexity) algorithm that can learn any mixture of @math unknown distributions from @math We analyze several natural types of distributions over @math , including log-concave, monotone hazard rate and unimodal distributions, and show that they have the required structural property of being well-approximated by a histogram with few bins. Applying our general algorithm, we obtain near-optimally efficient algorithms for all these mixture learning problems. This paper studies problems of inferring order given noisy information. In these problems there is an unknown order (permutation) @math on @math elements denoted by @math . We assume that information is generated in a way correlated with @math . The goal is to find a maximum likelihood @math given the information observed. We will consider two different types of observations: noisy comparisons and noisy orders. The data in Noisy orders are permutations given from an exponential distribution correlated with (this is also called the Mallow's model). The data in Noisy Comparisons is a signal given for each pair of elements which is correlated with their true ordering. In this paper we present polynomial time algorithms for solving both problems with high probability. As part of our proof we show that for both models the maximum likelihood solution @math is close to the original permutation @math . Our results are of interest in applications to ranking, such as ranking in sports, or ranking of search items based on comparisons by experts. Convex and submodular functions play an important role in many applications, and in particular in combinatorial optimization. Here we study two special cases: convexity in one dimension and submodularity in two dimensions. The latter type of functions are equivalent to the well-known Monge matrices. A matrix @math is called a Monge matrix if for every @math and @math we have @math . If inequality holds in the opposite direction, then V is an inverse Monge matrix (supermodular function). Many problems, such as the traveling salesperson problem and various transportation problems, can be solved more efficiently if the input is a Monge matrix. In this work we present testing algorithms for the above properties. A testing algorithm for a predetermined property @math is given query access to an unknown function f and a distance parameter @math . The algorithm should accept f with high probability if it has the property @math and reject it with high probability if more than an @math -fraction of the function values should be modified so that f obtains the property. Our algorithm for testing whether a 1-dimensional function @math is convex (concave) has query complexity and running time of @math . Our algorithm for testing whether an n1 × n2 matrix V is a Monge (inverse Monge) matrix has query complexity and running time of @math . We define the hierarchical singular value decomposition (SVD) for tensors of order @math . This hierarchical SVD has properties like the matrix SVD (and collapses to the SVD in @math ), and we prove these. In particular, one can find low rank (almost) best approximations in a hierarchical format ( @math -Tucker) which requires only @math parameters, where @math is the order of the tensor, @math the size of the modes, and @math the (hierarchical) rank. The @math -Tucker format is a specialization of the Tucker format and it contains as a special case all (canonical) rank @math tensors. Based on this new concept of a hierarchical SVD we present algorithms for hierarchical tensor calculations allowing for a rigorous error analysis. The complexity of the truncation (finding lower rank approximations to hierarchical rank @math tensors) is in @math and the attainable accuracy is just 2-3 digits less than machine precision. We investigate the approximability of several classes of real-valued functions by functions of a small number of variables ( juntas ). Our main results are tight bounds on the number of variables required to approximate a function @math within @math -error @math over the uniform distribution: 1. If @math is submodular, then it is @math -close to a function of @math variables. This is an exponential improvement over previously known results. We note that @math variables are necessary even for linear functions. 2. If @math is fractionally subadditive (XOS) it is @math -close to a function of @math variables. This result holds for all functions with low total @math -influence and is a real-valued analogue of Friedgut's theorem for boolean functions. We show that @math variables are necessary even for XOS functions. As applications of these results, we provide learning algorithms over the uniform distribution. For XOS functions, we give a PAC learning algorithm that runs in time @math . For submodular functions we give an algorithm in the more demanding PMAC learning model (Balcan and Harvey, 2011) which requires a multiplicative @math factor approximation with probability at least @math over the target distribution. Our uniform distribution algorithm runs in time @math . This is the first algorithm in the PMAC model that over the uniform distribution can achieve a constant approximation factor arbitrarily close to 1 for all submodular functions. As follows from the lower bounds in (, 2013) both of these algorithms are close to optimal. We also give applications for proper learning, testing and agnostic learning with value queries of these classes. In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger class of reward functions such as the @math function and nonlinear utility functions. Existing techniques relying on accurate estimations of the means of random variables, such as the upper confidence bound (UCB) technique, do not work directly on these functions. We propose a new algorithm called stochastically dominant confidence bound (SDCB), which estimates the distributions of underlying random variables and their stochastically dominant confidence bounds. We prove that SDCB can achieve @math distribution-dependent regret and @math distribution-independent regret, where @math is the time horizon. We apply our results to the @math -MAX problem and expected utility maximization problems. In particular, for @math -MAX, we provide the first polynomial-time approximation scheme (PTAS) for its offline problem, and give the first @math bound on the @math -approximation regret of its online problem, for any @math . Estimation of functions of @math variables is considered using ridge combinations of the form @math where the activation function @math is a function with bounded value and derivative. These include single-hidden layer neural networks, polynomials, and sinusoidal models. From a sample of size @math of possibly noisy values at random sites @math , the minimax mean square error is examined for functions in the closure of the @math hull of ridge functions with activation @math . It is shown to be of order @math to a fractional power (when @math is of smaller order than @math ), and to be of order @math to a fractional power (when @math is of larger order than @math ). Dependence on constraints @math and @math on the @math norms of inner parameter @math and outer parameter @math , respectively, is also examined. Also, lower and upper bounds on the fractional power are given. The heart of the analysis is development of information-theoretic packing numbers for these classes of functions. Given a directed graph @math and a list @math of terminal pairs, the Directed Steiner Network problem asks for a minimum-cost subgraph of @math that contains a directed @math path for every @math . The special case Directed Steiner Tree (when we ask for paths from a root @math to terminals @math ) is known to be fixed-parameter tractable parameterized by the number of terminals, while the special case Strongly Connected Steiner Subgraph (when we ask for a path from every @math to every other @math ) is known to be W[1]-hard. We systematically explore the complexity landscape of directed Steiner problems to fully understand which other special cases are FPT or W[1]-hard. Formally, if @math is a class of directed graphs, then we look at the special case of Directed Steiner Network where the list @math of requests form a directed graph that is a member of @math . Our main result is a complete characterization of the classes @math resulting in fixed-parameter tractable special cases: we show that if every pattern in @math has the combinatorial property of being "transitively equivalent to a bounded-length caterpillar with a bounded number of extra edges," then the problem is FPT, and it is W[1]-hard for every recursively enumerable @math not having this property. This complete dichotomy unifies and generalizes the known results showing that Directed Steiner Tree is FPT [Dreyfus and Wagner, Networks 1971], @math -Root Steiner Tree is FPT for constant @math [Such 'y, WG 2016], Strongly Connected Steiner Subgraph is W[1]-hard [, SIAM J. Discrete Math. 2011], and Directed Steiner Network is solvable in polynomial-time for constant number of terminals [Feldman and Ruhl, SIAM J. Comput. 2006], and moreover reveals a large continent of tractable cases that were not known before. Indexing highly repetitive texts --- such as genomic databases, software repositories and versioned text collections --- has become an important problem since the turn of the millennium. A relevant compressibility measure for repetitive texts is @math , the number of runs in their Burrows-Wheeler Transform (BWT). One of the earliest indexes for repetitive collections, the Run-Length FM-index, used @math space and was able to efficiently count the number of occurrences of a pattern of length @math in the text (in loglogarithmic time per pattern symbol, with current techniques). However, it was unable to locate the positions of those occurrences efficiently within a space bounded in terms of @math . Since then, a number of other indexes with space bounded by other measures of repetitiveness --- the number of phrases in the Lempel-Ziv parse, the size of the smallest grammar generating the text, the size of the smallest automaton recognizing the text factors --- have been proposed for efficiently locating, but not directly counting, the occurrences of a pattern. In this paper we close this long-standing problem, showing how to extend the Run-Length FM-index so that it can locate the @math occurrences efficiently within @math space (in loglogarithmic time each), and reaching optimal time @math within @math space, on a RAM machine of @math bits. Within @math space, our index can also count in optimal time @math . Raising the space to @math , we support count and locate in @math and @math time, which is optimal in the packed setting and had not been obtained before in compressed space. We also describe a structure using @math space that replaces the text and extracts any text substring of length @math in almost-optimal time @math . (...continues...) In statistics and machine learning, people are often interested in the eigenvectors (or singular vectors) of certain matrices (e.g. covariance matrices, data matrices, etc). However, those matrices are usually perturbed by noises or statistical errors, either from random sampling or structural patterns. One usually employs Davis-Kahan @math theorem to bound the difference between the eigenvectors of a matrix @math and those of a perturbed matrix @math , in terms of @math norm. In this paper, we prove that when @math is a low-rank and incoherent matrix, the @math norm perturbation bound of singular vectors (or eigenvectors in the symmetric case) is smaller by a factor of @math or @math for left and right vectors, where @math and @math are the matrix dimensions. The power of this new perturbation result is shown in robust covariance estimation, particularly when random variables have heavy tails. There, we propose new robust covariance estimators and establish their asymptotic properties using the newly developed perturbation bound. Our theoretical results are verified through extensive numerical experiments. We consider the problem of selecting the best subset of exactly @math columns from an @math matrix @math . We present and analyze a novel two-stage algorithm that runs in @math time and returns as output an @math matrix @math consisting of exactly @math columns of @math . In the first (randomized) stage, the algorithm randomly selects @math columns according to a judiciously-chosen probability distribution that depends on information in the top- @math right singular subspace of @math . In the second (deterministic) stage, the algorithm applies a deterministic column-selection procedure to select and return exactly @math columns from the set of columns selected in the first stage. Let @math be the @math matrix containing those @math columns, let @math denote the projection matrix onto the span of those columns, and let @math denote the best rank- @math approximation to the matrix @math . Then, we prove that, with probability at least 0.8, @math This Frobenius norm bound is only a factor of @math worse than the best previously existing existential result and is roughly @math better than the best previous algorithmic result for the Frobenius norm version of this Column Subset Selection Problem (CSSP). We also prove that, with probability at least 0.8, @math This spectral norm bound is not directly comparable to the best previously existing bounds for the spectral norm version of this CSSP. Our bound depends on @math , whereas previous results depend on @math ; if these two quantities are comparable, then our bound is asymptotically worse by a @math factor. We consider the problem of estimating the conditional mean of a real Gaussian variable @math where the vector of the covariates @math follows a joint Gaussian distribution. This issue often occurs when one aims at estimating the graph or the distribution of a Gaussian graphical model. We introduce a general model selection procedure which is based on the minimization of a penalized least-squares type criterion. It handles a variety of problems such as ordered and complete variable selection, allows to incorporate some prior knowledge on the model and applies when the number of covariates @math is larger than the number of observations @math . Moreover, it is shown to achieve a non-asymptotic oracle inequality independently of the correlation structure of the covariates. We also exhibit various minimax rates of estimation in the considered framework and hence derive adaptiveness properties of our procedure. The convergence of a class of Metropolis-type Markov-chain annealing algorithms for global optimization of a smooth function @math on @math is established. No prior information is assumed as to what bounded region contains a global minimum. The analysis contained herein is based on writing the Metropolis-type algorithm in the form of a recursive stochastic algorithm @math , where @math is a standard white Gaussian sequence, @math are random variables, and @math , @math for k large. Convergence results for @math are then applied from our previous work [SIAM Journal on Control and Optimization, 29 (1991), pp. 999–1018]. Since the analysis of @math is based on the asymptotic behavior of the related Langevin-type Markov diffusion annealing algorithm @math , where @math is a standard Wiener process and $c(t) = ... Understanding the local behaviour of structured multi-dimensional data is a fundamental problem in various areas of computer science. As the amount of data is often huge, it is desirable to obtain sublinear time algorithms, and specifically property testers, to understand local properties of the data. We focus on the natural local problem of testing pattern freeness: given a large @math -dimensional array @math and a fixed @math -dimensional pattern @math over a finite alphabet, we say that @math is @math -free if it does not contain a copy of the forbidden pattern @math as a consecutive subarray. The distance of @math to @math -freeness is the fraction of entries of @math that need to be modified to make it @math -free. For any @math and any large enough pattern @math over any alphabet, other than a very small set of exceptional patterns, we design a tolerant tester that distinguishes between the case that the distance is at least @math and the case that it is at most @math , with query complexity and running time @math , where @math and @math depend only on @math . To analyze the testers we establish several combinatorial results, including the following @math -dimensional modification lemma, which might be of independent interest: for any large enough pattern @math over any alphabet (excluding a small set of exceptional patterns for the binary case), and any array @math containing a copy of @math , one can delete this copy by modifying one of its locations without creating new @math -copies in @math . Our results address an open question of Fischer and Newman, who asked whether there exist efficient testers for properties related to tight substructures in multi-dimensional structured data. They serve as a first step towards a general understanding of local properties of multi-dimensional arrays, as any such property can be characterized by a fixed family of forbidden patterns. This paper investigates the information-theoretic limits of energy-harvesting (EH) channels in the finite blocklength regime. The EH process is characterized by a sequence of i.i.d. random variables with finite variances. We use the save-and-transmit strategy proposed by Ozel and Ulukus (2012) together with Shannon’s non-asymptotic achievability bound to obtain lower bounds on the achievable rates for both additive white Gaussian noise channels and discrete memoryless channels under EH constraints. The first-order terms of the lower bounds of the achievable rates are equal to @math and the second-order (backoff from capacity) terms are proportional to @math , where @math denotes the blocklength and @math denotes the capacity of the EH channel, which is the same as the capacity without the EH constraints. The constant of proportionality of the backoff term is found and qualitative interpretations are provided. We propose a novel geometric structure induced by any given convex polygon @math , called @math , which is an arrangement of @math segments, each of which is parallel to an edge of @math , where @math denotes the number of edges of @math . This structure enjoys six simple yet nontrivial properties, which can be derived from two elementary geometric properties -- convexity and parallelism. We then apply @math to solve the geometric optimization problem of computing the parallelogram(s) in @math with the maximum area. We show that the essential nature of the locally maximal area parallelograms is captured by @math and that computing all these parallelograms reduces to answering @math location queries on @math . We answer each of these queries in @math time, and thus obtain an @math time algorithm for this optimization problem. Our algorithm avoids an explicit construction of @math , which would take @math time. We develop fast and memory efficient numerical methods for learning functions of many variables that admit sparse representations in terms of general bounded orthonormal tensor product bases. Such functions appear in many applications including, e.g., various Uncertainty Quantification(UQ) problems involving the solution of parametric PDE that are approximately sparse in Chebyshev or Legendre product bases. We expect that our results provide a starting point for a new line of research on sublinear-time solution techniques for UQ applications of the type above which will eventually be able to scale to significantly higher-dimensional problems than what are currently computationally feasible. More concretely, let @math be a finite Bounded Orthonormal Product Basis (BOPB) of cardinality @math . We will develop methods that approximate any function @math that is sparse in the BOPB, that is, @math of the form @math with @math of cardinality @math . Our method has a runtime of just @math , uses only @math function evaluations on a fixed and nonadaptive grid, and not more than @math bits of memory. For @math , the runtime @math will be less than what is required to simply enumerate the elements of the basis @math ; thus our method is the first approach applicable in a general BOPB framework that falls into the class referred to as "sublinear-time". This and the similarly reduced sample and memory requirements set our algorithm apart from previous works based on standard compressive sensing algorithms such as basis pursuit which typically store and utilize full intermediate basis representations of size @math .