We study the problem of supervised learning for both binary and multiclass classification from a unified geometric perspective. In particular, we propose a geometric regularization technique to find the submanifold corresponding to a robust estimator of the class probability @math . The regularization term measures the volume of this submanifold, based on the intuition that overfitting produces rapid local oscillations and hence large volume of the estimator. This technique can be applied to regularize any classification function that satisfies two requirements: firstly, an estimator of the class probability can be obtained; secondly, first and second derivatives of the class probability estimator can be calculated. In experiments, we apply our regularization technique to standard loss functions for classification, our RBF-based implementation compares favorably to widely used regularization methods for both binary and multiclass classification.
Our training procedure for finding the optimal graph of a function is, in a general sense, also related to the manifold learning problem @cite_8 @cite_20 @cite_1 @cite_21 @cite_12 @cite_2 . The most closely related work is @cite_21 , which seeks a flat submanifold of Euclidean space that contains a dataset. Again, there are key differences. Since the goal of @cite_21 is dimensionality reduction, their manifold has high codimension, while our functional graph has codimension @math , which may be as low as @math . More importantly, we do not assume that the graph of our target function is a flat (or volume minimizing) submanifold, and we instead flow towards a function whose graph is as flat (or volume minimizing) as possible. In this regard, our work is related to a large body of literature on Morse theory in finite and infinite dimensions, and on mean curvature flow @cite_0 @cite_28 .
@cite_8 @cite_28 @cite_21 @cite_1 @cite_0 @cite_2 @cite_20 @cite_12
2908292748 2036959865 2021889595 2296324964
We inspect Vietoris-Rips complexes @math of certain metric spaces @math using a new generalization of Bestvina-Brady discrete Morse theory. Our main result is a pair of metric criteria on @math , called the Morse Criterion and Link Criterion, that allow us to deduce information about the homotopy types of certain @math . One application is to topological data analysis, specifically persistence of homotopy type for certain Vietoris-Rips complexes. For example we recover some results of Adamaszek-Adams and Hausmann regarding homotopy types of @math . Another application is to geometric group theory; we prove that any group acting geometrically on a metric space satisfying a version of the Link Criterion admits a geometric action on a contractible simplicial complex, which has implications for the finiteness properties of the group. This applies for example to asymptotically @math groups. We also prove that any group with a word metric satisfying the Link Criterion in an appropriate range has a contractible Vietoris-Rips complex, and use combings to exhibit a family of groups with this property. We study the metric properties of finite subsets of L1. The analysis of such metrics is central to a number of important algorithmic problems involving the cut structure of weighted graphs, including the Sparsest Cut Problem, one of the most compelling open problems in the field of approximation algorithms. Additionally, many open questions in geometric non-linear functional analysis involve the properties of finite subsets of L1.We present some new observations concerning the relation of L1 to dimension, topology, and Euclidean distortion. We show that every n-point subset of L1 embeds into L2 with average distortion O(√log n), yielding the first evidence that the conjectured worst-case bound of O(√log n) is valid. We also address the issue of dimension reduction in Lp for p ∈ (1, 2). We resolve a question left open by M. Charikar and A. Sahai [Dimension reduction in the l1 norm, in: Proceedings of the 43rd Annual IEEE Conference on Foundations of Computer Science, ACM, 2002, pp. 251-260] concerning the impossibility of dimension reduction with a linear map in the above cases, and we show that a natural variant of the recent example of Brinkman and Charikar [On the impossibility of dimension reduction in l1, in: Proceedings of the 44th Annual IEEE Conference on Foundations of Computer Science, ACM, 2003, pp. 514-523], cannot be used to prove a lower bound for the non-linear case. This is acomplished by exhibiting constant-distortion embeddings of snowflaked planar metrics into Euclidean space. In this paper, we consider the problem of multifunctional compression with side information. The problem is how we can compress a source X so that the receiver is able to compute some deterministic functions f1(X, Y1), ..., fm(X, Ym), where Yi, 1 ≤ i ≤ m, are available at the receiver as side information. In [1], Wyner and Ziv considered this problem for the special case of m = 1 and f1(X, Y1) = X and derived a rate-distortion function. Yamamoto extended this result in [2] to the case of having one general function f1(X, Y1). Both of these results were in terms of an auxiliary random variable. For the case of zero distortion, in [3], Orlitsky and Roche gave an interpretation of this variable in terms of properties of the characteristic graph which led to a particular coding scheme. This result was extended in [4] by providing an achievable scheme based on colorings of the characteristic graph. In a recent work, reference [5] has considered this problem for a general tree network where intermediate nodes are allowed to perform some computations. These previous works only considered the case where the receiver only wants to compute one function (m=1). Here, we want to consider the case in which the receiver wants to compute several functions with different side information random variables and zero distortion. Our results do not depend on the fact that all functions are desired in one receiver and one can apply them to the case of having several receivers with different desired functions (i.e., functions are separable). We define a new concept named the multi-functional graph entropy which is an extension of the graph entropy defined by Korner in [6]. We show that the minimum achievable rate for this problem is equal to the conditional multi-functional graph entropy of random variable X given side informations. We also propose a coding scheme based on graph colorings to achieve this rate. In this paper, we aim to learn a mapping (or embedding) from images to a compact binary space in which Hamming distances correspond to a ranking measure for the image retrieval task. We make use of a triplet loss because this has been shown to be most effective for ranking problems. However, training in previous works can be prohibitively expensive due to the fact that optimization is directly performed on the triplet space, where the number of possible triplets for training is cubic in the number of training examples. To address this issue, we propose to formulate high-order binary codes learning as a multi-label classification problem by explicitly separating learning into two interleaved stages. To solve the first stage, we design a large-scale high-order binary codes inference algorithm to reduce the high-order objective to a standard binary quadratic problem such that graph cuts can be used to efficiently infer the binary code which serve as the label of each training datum. In the second stage we propose to map the original image to compact binary codes via carefully designed deep convolutional neural networks (CNNs) and the hashing function fitting can be solved by training binary CNN classifiers. An incremental interleaved optimization strategy is proffered to ensure that these two steps are interactive with each other during training for better accuracy. We conduct experiments on several benchmark datasets, which demonstrate both improved training time (by as much as two orders of magnitude) as well as producing state-of-the-art hashing for various retrieval tasks.