The modern data analyst must cope with data encoded in various forms, vectors, matrices, strings, graphs, or more. Consequently, statistical and machine learning models tailored to different data encodings are important. We focus on data encoded as normalized vectors, so that their "direction" is more important than their magnitude. Specifically, we consider high-dimensional vectors that lie either on the surface of the unit hypersphere or on the real projective plane. For such data, we briefly review common mathematical models prevalent in machine learning, while also outlining some technical aspects, software, applications, and open mathematical challenges.
A powerful aspect of diffusion MR imaging is the ability to reconstruct fiber orientations in brain white matter; however, the application of traditional learning algorithms is challenging due to the directional nature of the data. In this paper, we present an algorithmic approach to clustering such spatial and orientation data and apply it to brain white matter supervoxel segmentation. This approach is an extension of the DP-means algorithm to support axial data, and we present its theoretical connection to probabilistic models, including the Gaussian and Watson distributions. We evaluate our method with the analysis of synthetic data and an application to diffusion tensor atlas segmentation. We find our approach to be efficient and effective for the automatic extraction of regions of interest that respect the structure of brain white matter. The resulting supervoxel segmentation could be used to map regional anatomical changes in clinical studies or serve as a domain for more complex modeling. We introduce the Spherical Admixture Model (SAM), a Bayesian topic model for arbitrary l2 normalized data. SAM maintains the same hierarchical structure as Latent Dirichlet Allocation (LDA), but models documents as points on a high-dimensional spherical manifold, allowing a natural likelihood parameterization in terms of cosine distance. Furthermore, SAM can model word absence presence at the document level, and unlike previous models can assign explicit negative weight to topic terms. Performance is evaluated empirically, both through human ratings of topic quality and through diverse classification tasks from natural language processing and computer vision. In these experiments, SAM consistently outperforms existing models. Abstract Understanding the organization of the human brain requires identification of its functional subdivisions. Clustering schemes based on resting-state functional magnetic resonance imaging (fMRI) data are rapidly emerging as non-invasive alternatives to cytoarchitectonic mapping in postmortem brains. Here, we propose a novel spatio-temporal probabilistic parcellation scheme that overcomes major weaknesses of existing approaches by (i) modeling the fMRI time series of a voxel as a von Mises-Fisher distribution, which is widely used for clustering high dimensional data; (ii) modeling the latent cluster labels as a Markov random field, which provides spatial regularization on the cluster labels by penalizing neighboring voxels having different cluster labels; and (iii) introducing a prior on the number of labels, which helps in uncovering the number of clusters automatically from the data. Cluster labels and model parameters are estimated by an iterative expectation maximization procedure wherein, given the data and current estimates of model parameters, the latent cluster labels, are computed using α-expansion, a state of the art graph cut, method. In turn, given the current estimates of cluster labels, model parameters are estimated by maximizing the pseudo log-likelihood. The performance of the proposed method is validated using extensive computer simulations. Using novel stability analysis we examine the sensitivity of our methods to parameter initialization and demonstrate that the method is robust to a wide range of initial parameter values. We demonstrate the application of our methods by parcellating spatially contiguous as well as non-contiguous brain regions at both the individual participant and group levels. Notably, our analyses yield new data on the posterior boundaries of the supplementary motor area and provide new insights into functional organization of the insular cortex. Taken together, our findings suggest that our method is a powerful tool for investigating functional subdivisions in the human brain. The mean shift algorithm, which is a nonparametric density estimator for detecting the modes of a distribution on a Euclidean space, was recently extended to operate on analytic manifolds. The extension is extrinsic in the sense that the inherent optimization is performed on the tangent spaces of these manifolds. This approach specifically requires the use of the exponential map at each iteration. This paper presents an alternative mean shift formulation, which performs the iterative optimization “on” the manifold of interest and intrinsically locates the modes via consecutive evaluations of a mapping. In particular, these evaluations constitute a modified gradient ascent scheme that avoids the computation of the exponential maps for Stiefel and Grassmann manifolds. The performance of our algorithm is evaluated by conducting extensive comparative studies on synthetic data as well as experiments on object categorization and segmentation of multiple motions. SUMMARY The angular central Gaussian distribution is an alternative to the Bingham distribution for modeling antipodal symmetric directional data. In this paper the statistical theory for the angular central Gaussian model is presented. Some topics treated are maximum likelihood estimation of the parameters, testing for uniformity and circularity, and principal components analysis. Comparisons to methods based upon the sample second moments are made via an example. This paper proposes a generative model-based speaker clustering algorithm in the maximum a posteriori adapted Gaussian mixture model (GMM) mean supervector space. The algorithm can be viewed as an extension of the standard expectation maximization algorithm for fitting a mixture model to the data, which iterates between two steps - a sample re-assignment step (E-step) and a model re-estimation step (M-step) - until it converges. The directional scattering patterns of GMM mean supervectors suggest that we employ a mixture of von Mises-Fisher distributions in the model re-estimation step. In the sample re-assignment step, four sample-to-mixture assignment strategies, namely soft, hard, stochastic, and deterministic annealing assignments, are used. Our experiments on the GALE Mandarin dataset show that the use of a mixture of von Mises-Fisher distributions as the underlying model yields significantly higher speaker clustering accuracies than the use of a mixture of Gaussian distributions. It is further shown that deterministic annealing assignment outperforms soft assignment, that soft assignment is comparable to stochastic assignment, and that both soft and stochastic assignments outperform hard assignment. Beyond linear and kernel-based feature extraction, we propose in this paper the generalized feature extraction formulation based on the so-called graph embedding framework. Two novel correlation metric based algorithms are presented based on this formulation. correlation embedding analysis (CEA), which incorporates both correlational mapping and discriminating analysis, boosts the discriminating power by mapping data from a high-dimensional hypersphere onto another low-dimensional hypersphere and preserving the intrinsic neighbor relations with local graph modeling. correlational principal component analysis (CPCA) generalizes the conventional Principal Component Analysis (PCA) algorithm to the case with data distributed on a high-dimensional hypersphere. Their advantages stem from two facts: 1) tailored to normalized data, which are often the outputs from the data preprocessing step, and 2) directly designed with correlation metric, which shows to be generally better than Euclidean distance for classification purpose. Extensive comparisons with existing algorithms on visual classification experiments demonstrate the effectiveness of the proposed algorithms. In this paper, we propose an unsupervised clustering method for axially symmetric directional unit vectors. Our method exploits the Watson distribution and Bregman Divergence within a Model Based Clustering framework. The main objectives of our method are: (a) provide efficient solution to estimate the parameters of a Watson Mixture Model (WMM), (b) generate a set of WMMs and (b) select the optimal model. To this aim, we develop: (a) an efficient soft clustering method, (b) a hierarchical clustering approach in parameter space and (c) a model selection strategy by exploiting information criteria and an evaluation graph. We empirically validate the proposed method using synthetic data. Next, we apply the method for clustering image normals and demonstrate that the proposed method is a potential tool for analyzing the depth image.  High angular resolution diffusion imaging (HARDI) permits the computation of water molecule displacement probabilities over the sphere. This probability is often referred to as the orientation distribution function (ODF). In this paper we present a novel model for representing this diffusion ODF namely, a mixture of von Mises-Fisher (vMF) distributions. Our model is compact in that it requires very few parameters to represent complicated ODF geometries which occur specifically in the presence of heterogeneous nerve fiber orientations. We present a Riemannian geometric framework for computing intrinsic distances (in closed-form) and for performing interpolation between ODFs represented by vMF mixtures. We also present closed-form equations for entropy and variance based anisotropy measures that are then computed and illustrated for real HARDI data from a rat brain. Von Mises-Fisher (vMF) Distribution is one of the most commonly used distributions for fitting directional data. Mixtures of vMF (MovMF) distributions have been used successfully in many applications. One of the important problems in mixture models is the problem of local minima of the objective function. Therefore, approaches to avoid local minima problem is essential in improving the performance. Recently, an algorithm called k-means++ was introduced in the literature and used successfully for finding initial parameters for mixtures of Gaussian (MoG) distributions. In this paper, we adopt this algorithm for finding good initializations for MovMF distributions. We show that MovMF distribution will lead to the same cost function as MoGs and therefore similar guarantee as the case of MoG distributions will also hold here. We also demonstrate the performance of the method on some real datasets. The k-means method is a widely used clustering technique that seeks to minimize the average squared distance between points in the same cluster. Although it offers no accuracy guarantees, its simplicity and speed are very appealing in practice. By augmenting k-means with a very simple, randomized seeding technique, we obtain an algorithm that is Θ(logk)-competitive with the optimal clustering. Preliminary experiments show that our augmentation improves both the speed and the accuracy of k-means, often quite dramatically. Automated unsupervised learning of topic-based clusters is used in various text data mining applications, e.g., document organization in content management, information retrieval and filtering in news aggregation services. Typically batch models are used for this purpose, which perform clustering on the document collection in aggregate. In this paper, we first analyze three batch topic models that have been recently proposed in the machine learning and data mining community – Latent Dirichlet Allocation (LDA), Dirichlet Compound Multinomial (DCM) mixtures and von-Mises Fisher (vMF) mixture models. Our discussion uses a common framework based on the particular assumptions made regarding the conditional distributions corresponding to each component and the topic priors. Our experiments on large real-world document collections demonstrate that though LDA is a good model for finding word-level topics, vMF finds better document-level topic clusters more efficiently, which is often important in text mining applications. In cases where offline clustering on complete document collections is infeasible due to resource constraints, online unsupervised clustering methods that process incoming data incrementally are necessary. To this end, we propose online variants of vMF, EDCM and LDA. Experiments on real-world streaming text illustrate the speed and performance benefits of online vMF. Finally, we propose a practical heuristic for hybrid topic modeling, which learns online topic models on streaming text data and intermittently runs batch topic models on aggregated documents offline. Such a hybrid model is useful for applications (e.g., dynamic topic-based aggregation of consumer-generated content in social networking sites) that need a good tradeoff between the performance of batch offline algorithms and efficiency of incremental online algorithms.  In this paper new expressions for the Spatial Fading Correlation (SFC) functions of Antenna Arrays (AA) in a 3-dimensional (3D) multipath channel are derived. In particular the Uniform Circular Array (UCA) antenna topology is considered. The derivation of the novel SFC function uses a Probability Density Function (PDF) originating from the field of directional statistics, the Von Mises Fisher (VMF) PDF. In particular the novel SFC function is based on the concept of mixture modeling and hence uses a mixture of VMF distributions. Since the SFC function is dependent on the Angle of Arrival (AoA) as well as the power of each cluster, the more appropriate power azimuth colatitude spectrum term has been used. The choice of distribution is validated with the use of Multiple Input Multiple Output (MIMO) experimental data that was obtained in an outdoor drive test campaign in Germany. A mixture can be composed of any number of clusters and this is mainly dependent on the clutter type encountered in the propagation environment. The parameters of the individual clusters within the mixture are derived and an estimation of those parameters is achieved using the soft-Expectation Maximization (EM) algorithm. The results indicate that the proposed model fits well with the MIMO data. We present a method for discovering patterns of selectivity in fMRI data for experiments with multiple stimuli tasks. We introduce a representation of the data as profiles of selectivity using linear regression estimates, and employ mixture model density estimation to identify functional systems with distinct types of selectivity. The method characterizes these systems by their selectivity patterns and spatial maps, both estimated simultaneously via the EM algorithm. We demonstrate a corresponding method for group analysis that avoids the need for spatial correspondence among subjects. Consistency of the selectivity profiles across subjects provides a way to assess the validity of the discovered systems. We validate this model in the context of category selectivity in visual cortex, demonstrating good agreement with the findings based on prior hypothesis-driven methods. Mixture modelling involves explaining some observed evidence using a combination of probability distributions. The crux of the problem is the inference of an optimal number of mixture components and their corresponding parameters. This paper discusses unsupervised learning of mixture models using the Bayesian Minimum Message Length (MML) criterion. To demonstrate the effectiveness of search and inference of mixture parameters using the proposed approach, we select two key probability distributions, each handling fundamentally different types of data: the multivariate Gaussian distribution to address mixture modelling of data distributed in Euclidean space, and the multivariate von Mises-Fisher (vMF) distribution to address mixture modelling of directional data distributed on a unit hypersphere. The key contributions of this paper, in addition to the general search and inference methodology, include the derivation of MML expressions for encoding the data using multivariate Gaussian and von Mises-Fisher distributions, and the analytical derivation of the MML estimates of the parameters of the two distributions. Our approach is tested on simulated and real world data sets. For instance, we infer vMF mixtures that concisely explain experimentally determined three-dimensional protein conformations, providing an effective null model description of protein structures that is central to many inference problems in structural bioinformatics. The experimental results demonstrate that the performance of our proposed search and inference method along with the encoding schemes improve on the state of the art mixture modelling techniques.