For each VQA v1 question, VQA v2 was constructed to also contain an image which is similar to the VQA v1 image, but has a different answer to the same question -effectively reducing the sharpness of question-only priors.

even with this additional 'balancing' there exist significant biases in the dataset.


Since the individual densities follow the parametric model exactly, these extra labeled examples can be used to estimate the density parameters and hence the decision regions.

using a simple union bound it can be shown ( [10]) that the asymptotic rate for convergence of such estimation procedure is O ) .


We can then iterate this procedure to generate a further improved strategy h that is a rollout strategy relative to h . It is easy to prove that after a finite number of such iterations, we would arrive at an optimal strategy [2].

the computation time required grows exponentially in the number of iterations, so this may not be practical.


Figure 2 shows the power of the tests for the above setting with ρ = 1 and growing n. We find that when r = 2, that is, true rank is known, both bootstrap tests perform well under alternative hypothesis, and outperform Asymp-TW, although Boot-ASE has a high type-I error rate.

when an over-estimate of rank is used (r = 4), both bootstrap tests break down -Boot-EPA always rejects while Boot-ASE always accepts -but the performance of Asymp-TW is robust to this parameter change.


People appear to make rational statistical inferences from noisy, uncertain input in a wide variety of perceptual and cognitive domains [1,9].

the computations for such inference, even for relatively small problems, are often intractable.


In these applications, a clearly defined reward function, i.e., whether a game is won or lost in the case of Go or the number of achieved points in the case of Atari games, is optimized by a reinforcement learning agent interacting with the environment.

in many applications it is very difficult to specify a reward function that captures all important aspects.


Given a specific decoy, the problem of estimating in a real-life problem may seem difficult.

as decoys are not new-even though the use we make of them here is-a number of methods [Heath and Chatterjee, 1995] have been designed to estimate the quality of a decoy, which is directly related to , and, with limited work, this parameter may be estimated as well.


In this case, the optimal solution for the relaxation is also an optimal solution for the original model, i.e., r-map ⋆ = map ⋆ .

if a relaxation's optimal assignment sets X i and X j to different states, then it is not a valid assignment for the original model M, as it violates the equivalence constraint and thus has log probability −∞.


The maps in (b) contain blurry boundaries, which indicates that an algorithm with random initialization is unlikely to deblur images effectively.

the maps in (d) show clear edges and result in a sharper and visually more pleasant deblurred image in (e).


Similarly, work on target/conditional shift focuses on matching marginal distributions on labels Y and adjusting conditional distributions P (X|Y ), such that P (X) can be matched across domains.

covariate shift assumes that the support of test P (X) is contained in the support of training P (X), i.e., the training set is richer than the test set.


The solutions proposed above provide attractive intuitive explanations for the cause of adversarial examples, and empirical results seem to suggest that they are effective at eliminating them.

none of the hypotheses on which these methods are based have been formally proven.


Much existing work [3,14] only establishes statistical consistency of global minima and then provides specialized algorithms for obtaining specific local optima that are provably close to global optima.

our results demonstrate that any optimization algorithm converging to a local optimum suffices.


Specifically, Deep Voice 1's segmentation model computes the output of each layer as where h (l) is the output of the l-th layer, W (l) is the convolution filterbank, b (l) is the bias vector, and ⇤ is the convolution operator.

deep Voice 2's segmentation model layers instead compute where BN is batch normalization (Ioffe and Szegedy, 2015).


Existing sensor devices for collecting X-ray images are only sensitive to signal intensities but not the phases.

it is very important to be able to recover the missing phase information as it reveals finer structures of the subjects than using the intensities alone.


Using an RBF kernel [31] gave performances competitive with state-of-the-art methods when MFCCs were combined with learned sparse filtering features (Table 3).

concatenating ICA and sparse coding features with MFCCs resulted in decreased performance when compared to MFCCs alone.


The obvious probabilistic approach to clustering genes would be to simply apply a Dirichlet process mixture (DPM) of Gaussians, but considering the genes (our dimensions) as samples, and our samples as "features" so that the partitioning would be over the genes.

this approach would not achieve the desired result of clustering correlated variables, and would rather cluster together variables close in terms of Euclidean distance.


The proposed algorithm readily applies to general convex sets.

we confine ourselves to specific sets of interest in our problem.


Unlike interior point methods, Random Conic Pursuit does not excel in producing highly exact solutions.

it is more scalable and provides the ability to trade off computation for more approximate solutions.


This process is similar to boosting [11] and active learning [7].

it is commonly known that boosting and active learning are sensitive to outliers and noise, and a few wrongly selected instances can deteriorate the learning performance of the whole model [12,3].


The Internet makes it possible for people around the world to cooperate on the labeling of these datasets.

this makes it unrealistic for individual researchers to obtain the ground truth of each label with absolute certainty.


The strong correlation between the frequency of words and their naming latency has been well documented.

as early as 1973, the Age of Acquisition (AoA) of a word was alleged to be the actual variable of interest, but these studies seem to have been ignored in most of the literature.


Both baselines usually preserve the pose and the layout of the original images.

the methods are likely to generate a new image based on the original layout and do not preserve the contents that are not relevant to the text.


If λ is too small θ u (λ) is too sensitive to statistical fluctuations.

if λ is too large θ u (λ) has too much of a bias towards zero.


In the absence of a smoothness constraint, any inference method can perform arbitrarily poorly [14].

the notion of smoothness is not well-defined in the context of probability distributions.


[37] generalize these results to the case of ordinal regression.

their calibration functions have at best a 1 /k factor if the surrogate is normalized w.r.t.


VGG [26], Inception [28], all propose to promote the network capability by making the network deeper.

the network at this time still cannot be too deep because of the problem of vanishing gradient.


The IBP confirms the intuitive explanation that there is not enough statistical evidence to break (differentiate) the objects into individual features and thus the unitization behavior of the participants is justified.

there is no comparison with the same underlying feature set to when statistical evidence warrants differentiation, so that the individual features should be learned as features.


Theoretical and empirical evidence indicates that the depth of neural networks is crucial for their success.

training becomes more difficult as depth increases, and training of very deep networks remains an open problem.


It takes around 0.5 seconds for the proposed online RPCA to process 10, 000 samples of 100 dimensional, on a PC with Quad CPU with 2.83GHz and RAM of 8GB.


[11,12] have recently suggested that short-term synaptic plasticity arranges for each local dendritic postsynaptic potential, v d i , to (approximately) represent the posterior mean of the corresponding presynaptic membrane potential: Thus, it would be tempting to say that in order to achieve the computational goal of Eq.

2, the way the dendritic tree (together with the soma) should integrate these local potentials, as given by f , should be directly determined by the function that needs to be computed: f = f . it is easy to see that in general this is going to be incorrect: where the l.h.s.


The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective.

this technique does not easily apply to commonly used distributions such as beta or gamma without further approximations, and most practical applications of the reparameterization gradient fit Gaussian distributions.


In more general work, Gaussian processes have been used to model the conditional distributions of a fully visible Bayesian network [12].

these 'Gaussian process networks' cannot deal with multimodal conditional distributions or with large datasets (currently 10 4 points would require further approximation).


Readers who are familiar with DPP might identify the conditional distribution is also a DPP, restricted to the ground set Y t . The conditional probability is defined in such a way that at time t, the subset selected should be diverse among Y t as well as be diverse from previously selected y t−1 .

beyond those two priors, the subset is not constrained by subsets selected in the distant past.


In the worst-case we can always bound X ∞ by 1 since X is an orthonormal basis.

in principle we could hope that a much better bound holds provided that the target subspace U has small coordinates.


If learning only depends on cross-correlations, we expect to see similar performance on these two types of tests.

human performace was significantly different for triplets (true triplets were preferred) and doublets (embedded and mixture doublets were not distinguished) (Fig.


Microarray technology allows researchers to simultaneously measure expression levels associated with thousands or ten thousands of genes in a single experiment (e.g., [7]).

the number of replicates in these experiments is often seriously limited (tipically a few dozen).


While interpretable, these methods seek specific differences (e.g., covariance-rescaled average differences) or operate under stringent assumptions (e.g., log-linear model).

sPARDA (with a positive-definite divergence) aims to find features that characterize a priori unspecified differences between general multivariate distributions.


Although the variants of ADMM with fast convergence rates have been developed under smoothness, strong convexity and other regularity conditions (e.g., the matrix A has full rank), the best iteration complexities of deterministic ADMM and stochastic ADMM for general convex optimization remain O(1/ ) and O(1/ 2 ), respectively.

many studies have reported that the performance of ADMM is very sensitive to the penalty parameter β.


The k-means objective is a simple, intuitive, and widely-cited clustering objective for data in Euclidean space.

although many clustering algorithms have been designed with the k-means objective in mind, very few have approximation guarantees with respect to this objective.


Both forms of uncertainties are postulated to decrease the influence of top-down information on representational inference and increase the rate of learning.

unexpected uncertainty rises whenever there is a global change in the world, such as a context change, while expected uncertainty is a more subtle quantity dependent on internal representations of properties of the world.


By taking M = N and the labels Y = I, we obtain a formulation similar to [7] where we are looking for a latent representation that can recover the identity matrix.

unlike [5,7], our discriminative clustering framework is based on a probabilistic model which may allow natural extensions.


At first thought, applying k-means with k = 2 or a mixture of two Gaussians would allow estimation of the two populations.

there is no evidence showing that similarity distances follow symmetrical or normal distributions.


This yields the same complexity bound ) as SVRG [26].


That is, it places high confidence on its wrong predictions.

eDL represents a good balance between the prediction uncertainty and accuracy.


In some sense, this criterion can be seen as providing a "lower bound" on learnability for a specific distribution.

we are interested in finite-sample convergence rates, and would like those to depend on simple properties of the distribution.


The SMC samplers converge to the exact solution in the limit of infinite particles.

it is not enough to be more efficient per particle, the crucial point is how efficient the algorithm is overall.


The output of Algorithm 1 satisfies, where Theorem 1 indicates that Algorithm 1 has a 1/4 approximation guarantee and 1/ p k rate.

it has good empirical performance as demonstrated by the experiments in Section 5.


Therefore the positive effect could be explained by a negative effect in prior slopes a z as the spatial frequency increases.

we do not have any explanation for the observed constant likelihood width as it is not consistent with the speed width of the stimuli σ V = 1 t z which is decreasing with spatial frequency.


To obtain the prediction on one test sample, Nyström approximation only needs O(md) flops to compute c, and O(m) flops to compute the decision value cβ, so it becomes an effective ways to improve the prediction speed.

nyström approximation usually needs more than 100 landmark points to achieve reasonable good accuracy, which is still expensive for large-scale applications.


|V| = K m ), finding the maximizing state x * t at each update may be hard.

the PCT only requires us to find some state x * t such that w T t v t ≤ 0 and in most cases this can easily be verified.


The entities which have visual properties, i.e., a yellow tennis suit and white tennis shoes in Figure 4a, and a denim shirt in Figure 4b, are correct.

a relatively small object (e.g., a cigarette in Figure 4b) and the entity that requires semantic inference (e.g., a male conductor in Figure 4c) are incorrect.


We find iMedLDA is the most efficient, which benefits from (1) generateing samples outside the iteration loop and uses them for all iterations; and (2) using the MAP estimates to collapse the sample space of (Θ, Φ) to a "single sample" for efficiency.

both gMedLDA and fMedLDA have to iteratively update the variables or variational parameters.


For example, the narrowestover-threshold (NOT) algorithm [1] is more suitable when different segments between the change points have comparable lengths, and the stepwise marginal likelihood (SML) method [5] works better to identify frequent change points.

in practice it is often the case that distances between consecutive change points may vary dramatically, while only those with certain distance gaps are of interest.


DropMax (q = p) has an entropy regularizer to address this issue, but the model obtains suboptimal performance due to the inaccurate posterior estimation.

the gating function of DropMax is optimally regularized to make a crude selection of candidate classes via the proposed variational inference framework, and shows consistent and significant improvements over the baselines across all datasets.


As stated in [7], in analogy to SVD, the rank of a tensor A can be defined as the minimum number r for decomposing the tensor into rank-one components as follows: where • denotes outer product, D ∈ R r×r×•••×r is an N -order diagonal tensor whose jth diagonal element is λ j , and The above decomposition model is called PARAFAC.

this rank definition is a highly nonconvex and discontinuous function on the tensor space.


Another encouraging example is to learn a homogeneous linear separator for data uniformly distributed on the unit sphere of R d . In this case active learning can still give exponential savings in the label complexity [Das05].

there are also very simple problems that active learning does not help at all.


Modern discriminative predictors have been shown to match natural intelligences in specific perceptual tasks in image classification, object and part detection, boundary extraction, etc.

a major advantage that natural intelligences still have is that they work well for all perceptual problems together, solving them efficiently and coherently in an integrated manner.


In particular, box-like tuning functions can be superior to cosine tuning functions.

numerical evaluation of the minimum mean squared error for angular variables is much more difficult than the evaluation of the minimal discrimination error proposed here, and the above claim has only been verified up to N = 20 neurons.


A standard metric to evaluate the quality of saliency maps is the localisation accuracy of the saliency map.

it should be noted that saliency is not equivalent to localisation.


Thus, as the size of a population increases, a parametric model rapidly loses flexibility in describing the full spike distribution.

nonparametric models allow flexibility to grow with the amount of data [10,11,12,13,14].


Applying such joint and alternating approaches to the MTA objective (1) leads to a degenerate solution with zero similarity.

the simplicity of MTA enables us to specify the optimal task similarity matrix for T = 2 (see Sec.


C = B) the learning rate of nonlinear MTL is of O(( T n ) 1/4 ), as if the tasks were learned independently.


Being a parametric model, this method can be used to predict past (t * < 0), present (t * = 0) and future (t * > 0) occupancy maps using a fixed number of parameters (M + 1).

in practice, it may not be required to generate future or past maps at every time step.


Such an algorithm may then require on the order of 100, 000 days of experience (trials) to learn.

in Section 4, we show that if a reinforcement learning algorithm is given access to the local rewards, it can be possible to learn in such problems with an exponentially smaller O(log n) sample complexity.


In the covert communication environment, we found that Bob trained with both MADDPG and DDPG out-performs Eve in terms of reconstructing Alice's message.

bob trained with MADDPG achieves a larger relative success rate compared with DDPG (52.4% to 25.1%).


The visual-semantic matching that measures similarities between the visual and class semantic features is a key to address ZSL.

the visual features extracted from the original images and the class semantic features are located in different structural spaces, a simple matching method may not align the semantics well.


In (3c ), the various log probabilities involved are similarly natural products of perceptual or predictive computations.

the calculation of the distribution P θ (L i | V i , S i ) in (3b) is less easily reconciled with biological constraints.


We illustrate epiphany with a single unknown region K = 1, U = U 1 .

we contrast two shapes of U: in one set of experiments we set U = [0.4, 0.6] which contains the decision boundary 0.5.


Intuitively, this suggests that the only favorable initialization schemes (from which convergence to a global maximum is possible) are those in which (1) all initialized centers fall in the neighborhood of exactly one cluster of true centers, (2) the number of centers initialized within each cluster of true centers exactly matches the number of true centers in that cluster.

this observation alone only suffices to guarantee that the success probability is polynomially small in M . In order to demonstrate that the success probability is exponentially small in M , we need to further refine this construction.


Each user i is holding a private input item x i (a point in R d in our case), and the server's goal is to compute some function of the inputs (approximate the k-means in our case).

in this model, the users do not send their data as is to the server.


We discard its smallest singular value and corresponding eigenvector to revert back to an order k eigensystem.

instead of merely discarding we can absorb the smallest singular value and eigenvector into the D component by bounding the remaining outerproduct with a diagonal term.


In the specific, the posterior probability of y ∈ Y given a sample x ∈ X is inductively defined as: The same posterior probabilities with respect to a forest F ⊆ T can be obtained as in (2).

each CS decision tree is trained in an iterative way and, similar to the case of standard decision trees, a decision about whether to branch new nodes or produce a leaf is taken based on a subset of the training samples Z ⊆ D. in contrast to the standard setting, the learning process depends on the order in which nodes of the tree are grown because split functions depend on t (<m) which in turn is affected by the node ordering.


This leads to a training objective similar to that of structural SVMs or conditional random fields (Equation ( 2)).


In the previous section, we introduced a method for transferring an embedding from the source to the target domain.

this only enforces alignment of the global domain statistics with no class specific transfer.


However, since the model represents object's appearance as a rigid template in 3D, its performance has been shown to be inferior to (2D) deformable part-based models (DPMs) [1].

in this paper we extend DPM to reason in 3D.


Source localization Sound source localization is a mature area of signal processing and robotics research [3,35,34,42].

unlike the proposed 360 • spatialization problem, these works rely on microphone arrays using beamforming techniques [43] or binaural audio and HRTF cues similar to those used by humans [18].


In particular, they obtain algorithms with O(log(1/ )) communication complexity.

these works consider efficient algorithms only in the realizable case.


The NV reveals an average process of settling by measuring the convergence of firing across different trials.

it provides little insight into the course of motor planning on a single trial.


These benefits are notable because they will be inherited by our extension of the framework in this paper.

this algorithm's practical performance can still be mediocre on real data, which is often easier to predict than an adversarial setup would have us believe.


Most known results on the exact solution of phase retrieval problems [8,29,10,38,36,35] pertain to generic random matrices, where the entries of A are independent subgaussian random variables.

in practice it is almost impossible to design purely random measurement matrices: in many cases as we mentioned above, the measurement is much more structured -generated by passing a signal through a manually designed filter.


For example, for d = O( −2 log n) and Φ = poly(n), it improves over the "vanilla" bound by a factor of O(log n/ log log n) and is lossier than the bound of [6] by a factor of O(log log n).

compared to the latter, our construction time is nearly linear in n. The comparison is summarized in Table 1.


As a consequence, optimizing this type of loss has an inherent computational cost.

in practice, branch and bound based ILP solvers find the solution of (12) in a reasonable time for a reduced number of labels K. We discuss the computational implications of choosing the Kemeny embedding section 5.2.


Recently, Lee and Bresler [17] proposed an algorithm (ADMiRA) motivated by the orthogonal matching pursuit line of work in compressed sensing and show that for affine constraints with isometry constant δ 4k ≤ 0.04, their algorithm recovers the optimal solution.

their method is not very efficient for large datasets and when the rank of the optimal solution is relatively large.


Researchers have previously attempted to cope with this issue by clustering or randomly downsampling the pool [19,20,21,22]; however, such strategies provide no guarantees as to the potential loss in active selection quality.

when applying our approach for this task, we can consider orders of magnitude fewer points when making the next active label request, yet guarantee selections within a known error of the traditional exhaustive pool-based technique.


Indeed, they are being studied and used in a variety of downstream applications (e.g., document ranking [22], sentiment analysis [14], and question retrieval [17]).

the embeddings also pinpoint sexism implicit in text.


Classic structure-from-motion methods use an explicit affine projection model to extract a 3D point cloud and camera homography matrices [8].

slow Feature Analysis [38] has no model, instead simply penalizing the rate of change in time-independent components and encouraging their decorrelation.


In practice, cross-validation procedures [10] are quite effective.

they are only feasible when the number of hyperparameters is very small, whereas our approach can optimize many hyperparameters.


The main limitation of these methods is that they are theoretically justifiable only for subspaces of low relative dimension d/D.

for applications such as 3D point cloud analysis, two/three-view geometry in computer vision, and system identification, a subspace of dimension D − 1 (high relative dimension) is sought [21,26].


For such speech, only the second filter from 50-200 Hz will resolve the harmonic at 180 Hz.

the first filter from 25-100 Hz will pass low frequency noise; the third filter from 100-400 Hz will pass the first and second harmonics at 180 Hz and 360 Hz, and the fourth filter from 200-800 Hz will pass the second through fourth harmonics at 360, 540, and 720 Hz.


This idea has been explored, in different ways, by a number of studies in the literature [13,19].

previous studies address the problem of breaking filter bubbles by the means of content recommendation.


We note a recent work [5] in robust stochastic optimization which is able to tolerate a constant fraction of corruptions α → 1.

their algorithms operate in the list-decoding model wherein they output not one, but as many as O 1 1−α models, of which one (unknown) model is guaranteed to be correct.


If the imputed firing rates truly reflect the rate functions underlying the observed spikes, then the mean behavior of the imputed firing rates should track the empirical firing rates.

if the latent system were inadequate to describe the activity, we should expect to see dynamical features in the empirical firing that could not be captured by the imputed firing rates.


For example, in a large document corpus, one news article about a hockey game may serve as an excellent representative of hundreds or thousands of other similar pieces about hockey games.

there may only be a few articles about luge, so it is also important to include at least one article about luge.


Besides stand-alone matrix approximation methods, ensemble methods, e.g., DFC [15], LLORMA [12], WEMAREC [5], etc., and mixture models, e.g., MPMA [4], etc., have been proposed to improve the recommendation accuracy and/or scalability by weighing different base models across different users/items.

the above methods do not consider using different ranks to derive different base models.


In [3], we show how quasi-orthogonalization can be achieved in a noise-invariant way through a method based on the fourth-order cumulant tensor.

a direct implementation of that method requires estimating the full fourth-order cumulant tensor, which is computationally challenging even in relatively low dimensions.


Many of them rely on weighing the hand-engineered image features first, based on some measure such as saliency and then performing matching [6,24,25].

this is done in a non-deep framework unlike ours.


Typical approaches to the mixture regression problem employ local search methods such as Expectation Maximization (EM) that are prone to spurious local optima.

a number of recent theoretically-motivated Tensor-based methods either have high sample complexity, or require the knowledge of the input distribution, which is not available in most of practical situations.


As we will see in section 6 , considering uncertainty does not improve the search in our problem.

the choice of acquisition function remains a worthy candidate for further exploration.


when exposed to a new environment, hungry animals may become anxious about the uncertainty in the situation (whether they will be able to find food to survive), which makes them prefer immediate rewards to delayed ones.

it may also be related to the specific reward structure in the model.


Therefore, ¢ 1 is an appropriate evaluation measure of the errors in our experimental setting.

alternative evaluation methods should be devised for general recordings when the test signal is not confined to a finite set of possible directions.


The standard approach for learning the parameters in presence of latent variables is to use an EM type algorithm [9].

since the variational distribution is restricted to have the same system noise covariance (see Appendix A) as the true posterior, the EM algorithm would leave this covariance completely unchanged in the M step and cannot be used for learning this crucial parameter.


One idea is to modify the CD step in [9] to the proximal gradient descent.

it will greatly increase the computation time due to the large number of inner gradient descent steps.


This enables explicit feature learning which aims to minimize/maximize convolutional activation differences between matching/non-matching pixel pairs [5,19].

ground-truth flow fields f * i are still needed for learning the networks, and it predicts the geometric Source Target Feature Extraction Net.


Bubenik chooses a z-test to test for equality between the mean persistence landscapes.

we can test for true equality in distribution.


This idea has been explored in recent work such as [16] which uses the idea of introducing new symmetries or [19] which uses unsupervised learning to reduce the objects in the domain.

in both these approaches, it may turn out that for certain cases, the bias skews the MLN distribution to a large extent.


Perhaps conceptually closest to us is a recent work by [5] where they consider pair-wise queries for clustering.


This objective function works well in applications where signals need to be reconstructed, like coding and denoising.

discriminative methods, such as linear discriminative analysis (LDA), are better suited for classification tasks.


For all real-world datasets, the probability distributions D k are not readily available to the learner.

corollary 4 extends the learning guarantees of our solution to the case where an estimate Dk is used in lieu of the ideal distribution D k . Thus, we used standard density estimation methods to derive an estimate Dk for each k ∈ [p].


Note that, when a few target samples are available, the transfer of samples from a combination of the source tasks using the BAT algorithm is still beneficial.

the performance attainable by BAT is bounded by the transfer error corresponding to the best source task combination (which in this case is large).


That variable is the prototype of the target class c, D c . We thought the lack of an interaction among the category prototypes might be overcome with KFU-Distrib, because with a distributed prototype representation, all of the state variables jointly determine the target category prototype.

our intuition turned out to be incorrect.


In contrast, an algorithm resulting from the CWGD viewpoint (usually called potential booster) may not necessarily be a boosting algorithm in the probability approximately correct (PAC) learning sense.

while it is rather difficult to construct a boosting algorithm based on the first view, the algorithmic frameworks, e.g., AnyBoost [4], resulting from the second viewpoint have proven to be particularly prolific when it comes to developing new boosting algorithms.


Highway networks also contain residual modules and skip connections that bypass them.

the output of each path is attenuated by a gating function t, which has learned parameters and is dependent on its input.


As was pointed out in Section 1, the weights of the neural networks do not seem to converge in the usual sense during training (the norms grow continually).

as depicted in Figure 4a, the sequence of (normalized) margin distributions is itself converging.


m . The so-called "direct" approach fits regression estimates μt of µ t on each dataset {(X i , Y i ) : [33].

direct approaches may not generalize as well as weighting-based approaches [7].


In supervised learning, FS has been widely studied, with most methods falling into two classes: filters, which work independently of the subsequent learning algorithm; wrappers, which use the learning algorithm to evaluate feature subsets [12].

fS has received little attention in clustering, mainly because, without class labels, it is unclear how to assess feature relevance.


This typically generates far more activity than strictly required, a redundancy that can be seen as inefficient.

supervised learning models involving spiking neurons have essentially concentrated on the learning of precise spike sequences [8][9][10].


Approximations of the Ω-return could, in principle, replace the λ-return in the whole family of TD(λ) algorithms.

using the Ω-return for TD(λ) raises several interesting questions that are beyond the scope of this initial work (e.g., is there a linear-time way to estimate the Ω-return?


For example, if they were quadratic and if π was linear then the problem would be a quadratic optimization problem.

except for special cases 3 the action value functions will be more complicated, in which case this step can be expensive.


Let n denotes the input dimension, we show that width-(n + 4) ReLU networks can approximate any Lebesgue integrable function on n-dimensional space with respect to L 1 distance.

except for a zero measure set, all Lebesgue integrable functions cannot be approximated by width-n ReLU networks, which demonstrate a phase transition.


4 Experiments We evaluate the properties of our representation by following the protocol introduced by [34] to measure the disentanglement of a representation.

[34] operates within one domain only, whereas we learn cross-domain representations.


Alternatively, generalization can be achieved by imposing task-dependent policies [27].

this approach might restrict the choice of optimal controls given the assumed structure of control policy.


From the update in Equation ( 13), we observe that Gen-Oja is performing an Oja update but with a controlled Markovian noise.

we would like to highlight that classical techniques in the study of stochastic approximation with Markovian noise (as the Poisson Equation [6,22]) were not enough to provide adequate control on the noise to show convergence.


We suspected that the choice of this distribution would have a significant impact on the accuracy of trained HBNNs, and performed a hyperparameter sweep by varying DistF romAvg in Algorithm 1 when training AlexNet on ImageNet as described in the following sections.

much to our surprise, models trained with the same average bitwidth achieved nearly identical accuracies regardless of distribution.


Especially when there  is combination of severe deformation and edge density is low, GGM pp and GGM lp outperform the selected counterparts.

gGM pm and GGM lm reach significant performance close to state-of-the-art e.g.


We could naively generalize our approach for parallel bandits by observing for T /2 rounds, applying the truncated product factorization to write an expression for each P {Y |a} in terms of observational quantities and explicitly playing the actions for which the observational estimates were poor.

it is no longer optimal to ignore the information we can learn about the reward for intervening on one variable from rounds in which we act on a different variable.


The method proved to be effective achieving also (features) transfer capabilities.

fIRL requires the MDP model to solve the forward problem and the complete optimal policy for the fitting step in order to evaluate the consistency with demonstrations.


Now consider the 2q polynomials The count in Corollary 3 differentiates between positive, negative and zero signs.

we are only concerned with the positivity of YiaXia (in the proof of Theorem 1) or of Xia − Tia (in the proof of Theorem 8), and do not need to differentiate between zero and negative values.


As shown in the last term of 2 error bound of β k , increasing ν will sacrifice its accuracy.

one can remedy this by projecting β k on to a subspace using the support set of γ k , and obtain a good estimator βk with both sign consistency and 2 consistency at the minimax optimal rates.


• META vs. META+ADAPT: META+ADAPT does not improve over META for small values of n, which is in-line with the common observation that SGD-based training is difficult using a small number of samples.

for large values of n, the accuracy of META+ADAPT increases significantly while the META model remains flat.


We provided the contraction rate, under the proposed prior specification and univariate Gaussian kernel, in lemma 3.

theorem 1 is a more general statement and it applies to multivariate mixture density of any kernel.


Both the template parameters and which templates to execute for each state are learned, although the agent's choices are constrained.

the motion templates are parametrized policies designed specifically for the task.


That is, A simple C with V-shaped rows is the classification cost matrix, with entries C y,k = y = k . 1 The classification cost is widely used in multiclass classification.

because the cost is invariant for all kinds of mislabelings, the ordinal information is not taken into account.


When CD was first introduced, algorithms did not differentiate between coordinates; each coordinate i 2 [d] was selected uniformly at random at each time step (see, e.g., [19,20]).

recent works (see, e.g., [10,24,15]) have shown that exploiting the structure of the data and sampling the 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montréal, Canada.


in this work a related but different measure for asymmetric interrelations between time-series is utilized.


If a phase is detected at a station, a corresponding detection is generated.

the parameters of the detection may be imprecise.


We train the LM in an adversarial way by minimizing the loss of LM of real sentences and maximizing the loss of transferred sentences.

since the LM is a structured discriminator, we would hope that a language model trained on the real sentences will automatically assign high perplexity to sentences not in the target domain, hence negative samples from the generator may not be necessary.


In this paper, we prove lower bounds for the δ-related setting, which includes as a special case the commonly-studied setting of randomly partitioned data (in which case δ = O(1/ √ n)).

our bounds do not apply for random partitioning, since they use δ-related constructions which do not correspond to randomly partitioned data.


The optimization problem with fixed γ leads to a new learning problem expressed as Note how the data fitting term elegantly and naturally encodes the transfer of source labels y s i through estimated labels of test samples with a weighting depending on the optimal transport matrix.

this comes at the price of having a quadratic number N s N t of terms, which can be considered as computationally expensive.


Then we made it explicit for RECIST scores monitoring and performed experiments on data from the HEGP.

we have studied that dataset as if all patients behave similarly, which is not true in practice.


[2015] also used the truncated SVD estimator for matrix completion.

his bound depends on the nuclear norm of the underlying matrix which may be √ n times larger than our result.


The absolute location of orientation of the k-th map will be denoted by ξ k = ( x k y k θ k ) T ; here x and y are the Cartesian coordinates and θ is the orientation.

from the scan matcher, we can retrieve relative displacement information of the form δ k,k−1 = ( ∆x k,k−1 ∆y k,k−1 ∆θ k,k−1 ) T which, if scan matching was errorfree, would enable us to recover absolute information via the following recursion (under the boundary condition ξ 0 = (0, 0, 0) T ) scan matching is not without errors.


As new large-scale recording technologies such as the Neuropixels probe are generating simultaneous recordings of spiking activity from hundreds or thousands of neurons [8,21,4], Poisson GLMs will be a useful tool for investigating encoding and statistical dependencies within and across brain regions.

the size of these datasets makes inference computationally expensive.


Our final classifier has the weight vector w = u + v. We refer to the resulting algorithm as CODA (Cotraining for Domain Adaptation), which can be stated concisely with the following optimization problem: min w,u,v log e (u;L) + e (v;L) + γs(L, U, w) subject to: (1) The optimization is non-convex.

as it is not particularly sensitive to initialization, we set u, v randomly and optimize with standard conjugate gradient descent 5 .


Due to the large dataset size, CNN training is widely implemented using distributed methods such as dataparallel stochastic gradient descent (SGD) [9,10,11,12,13,14,15,16], where gradients computed by distributed nodes are summed after every iteration to update the CNN model of every node 2 .

this gradient aggregation can dramatically hinder the expansion of such systems, for it incurs significant communication costs, and will become the system bottleneck when communication is slow.


For example, stimuli that occur at the same time and space are likely to be interrelated by a common cause.

if the visual and auditory cues are incongruent, e.g., when dubbing one syllable onto a movie showing a person mouthing a different syllable, listeners typically report hearing a third syllable that represents a combination of what was seen and heard (McGurk effect, cf.


The inverse Gamma distribution is not conjugate to the exponential family.

the posterior over matrix Γ is tractable.


1: Parameters: a, b, L > 0 2: Initialize: Receive input vector x t ∈ X 6: Adversarial setting: Suffer loss t (f t (x t )) 7: Receive subgradient s t ∈ partial t (f t (x t ))  This bound shows that, if the cumulative loss of the competitor is small, the regret can grow slower than √ T . It is worse than the regret bounds for smooth losses in [9,25] because when the cumulative loss of the competitor is equal to 0, the regret still grows as Õ f instead of being constant.

the PiSTOL algorithm does not require the prior knowledge of the norm of the competitor function h, as all the ones in [9,25] do.


We found that there was close agreement between the posterior distributions of the true generative model and the one learned by the DDC-HM.

the biased recognition of the VAE and IWAE in turn biases the learnt generative parameters so that the resulting posteriors (even when computed without the recognition networks) appear closer to Gaussian.


We have demonstrated that the language model can successfully crack word substitution cipher.

the change of substitution cipher is limited to a one-to-one mapping.


It is worth emphasising the fact that the GPSSM could recover the unobserved velocities in the latent states, which resulted in smooth transitions of the cart and swinging of the pendulum.

it seems that the recovered cart's velocity is overestimated.


In general, it is hard to choose a good schedule for a given path.

consider the set of binned schedules, where the path is divided into segments, some number K j of intermediate distributions are allocated to each segment, and the distributions are spaced linearly within each segment.


Training deep networks is a challenging problem [16,2] and various heuristics and optimization algorithms have been suggested in order to improve the efficiency of the training [5,9,4].

training deep architectures is still considerably slow and the problem has remained open.


The SDE automatically incorporates the geometry and randomness induced by the statistical model, which allows us to establish the exact dynamics of SGD.

classical optimization analysis only yields upper bounds on the optimization error, which are unlikely to be tight in the presence of highly nonconvex geometry, especially around the stationary points that have negative curvatures along certain directions.


• In contrast to common practices (Montavon et al., 2012) and theoretical recommendations (Hardt et al., 2016), generalization keeps improving for a long time at the initial high learning rate, even without any observable changes in training or validation errors.

this improvement seems to be related to the distance of the weights from their initialization.


The "compact grammar" aspect of MDL is close to some traditional linguistic argumentation which at times has argued for minimal grammars on grounds of analytical [10] or cognitive [5] economy.

the primary weakness of MDL-based systems does not have to do with the objective function, but the search procedures they employ.


Since m(t), S(t), λ(t) and Ψ(t) are dependent on these parameters, one needs also to take the corresponding implicit derivatives into account.

these implicit gradients vanish if the consistency constraints for the means (9) and the covariances (10), as well as the ones for the Lagrange multipliers (14-15), are satisfied.


Each residue in an amino acid sequence has two primary degrees of freedom: rotation around the C α -N bond, referred to as the φ torsion angle, and rotation around the C α -C bond, referred to as the ψ torsion angle.

it is difficult to fit a response surface in the space of torsion angles because the energy function is highly irregular in this space; a slight change in a single torsion angle typically causes large global structural changes, which in turn cause large energy changes.


5 It shows that the random projection tends to overestimate the decomposition error.

except for the wine dataset, our method stably estimated the residual error with reasonable approximation errors.


Furthermore, its performance also steadily increases as we increase the number of cascades, and finally KernelCascade recovers the entire network with around 1000 cascades.

the competitor methods seldom fully recover the entire network given the same number of cascades.


ME-TRPO is a purely model-based policy-gradient approach, and uses the ensemble to avoid overfitting to any one model.

sTEVE interpolates between model-free and model-based estimates, uses a value-estimation approach, and uses the ensemble to explicitly estimate uncertainty.


Moreover, in deep linear networks, ensuring that all singular values of the Jacobian are concentrated near 1 can yield a dramatic additional speed-up in learning; this is a property known as dynamical isometry.

it is unclear how to achieve dynamical isometry in nonlinear deep networks.


However, for the squared loss, our problem can be cast as an instance of the trust region problem, which can be solved globally using the DCA algorithm of Tao and An [1998].

note that problem (4) is not jointly convex in q and w. using the dual problem associated to w yields the following equivalent problem, it can be rewritten as follows:  6) is convex since D is a convex set, the first term in ( 6) is convex as a maximum of convex (linear) functions of r. This problem can be solved using standard descent methods, where, at each iteration, we solve a standard QP in ↵, which admits a closed-form solution.


It can be approximated as p(x)q(x) f (x) dx By performing a similar calculation for the denominator of ( 14), it can be shown to be approximately equal to 1 √ P1P1 .

hence, the Laplacian pdf distance can be written as a risk function, given by Note that if P 1 = P 2 = 1 2 , then L = 2P e , where P e is the probability of error when assigning data points to the two clusters, that is Thus, in this case, minimizing L is equivalent to minimizing P e . in the case that P 1 = P 2 , (28) has an even more interesting interpretation.


(Independent of our work, Goldfarb and Ma [12] proposed an algorithm similar to SVP.

their analysis and formulation is different from ours.


A set of methods termed subspace techniques computes a projection of the sensor data onto the signal subspace, which corresponds to brain sources of interest.

these methods rely on thresholding to determine the noise level, and tend to discard information below threshold.


These techniques include GTM [4], self organising maps [5] and others [6,7].

such algorithms often fail when nonlinear structure cannot simply be regarded as a perturbation from a linear approximation; as in the Swiss roll of Figure 3.


Both attention mechanisms have their advantages and disadvantages: parallel co-attention is harder to train because of the dot product between image and text which compresses two vectors into a single value.

alternating co-attention may suffer from errors being accumulated at each round.


As a reference, running ABLR NN and ABLR NN transfer in this setting took 1.2 ± 0.2 and 16.3 ± 1.6 seconds per BO iteration respectively.

bOHAMIANN needed 1335.7 ± 236.5 seconds per BO iteration, which was about 80 times more expensive.


As may be seen, our analytic approximation results for the potential closely match the results obtained via simulations.

the actual query complexity is significantly lower in practice than predicted through our analysis, due to the fact that we assumed a worst case scenario for pairwise queries, and set the number of comparisons to K. For the misclassification ratio, we observe that the general trend is as expected -the larger the number of clusters K, the larger the misclassification ratio.


Compared with the classical 3-layer Fat-Tree topology [5], BCube pays the cost of using more network interfaces on servers.

the number of switches required in BCube is much less than that in Fat-Tree.


It always has a fixed point at c * = 1 as can be checked by direct calculation.

the stability of this fixed point depends on the slope of the map at 1, which is See SM for a derivation of (7).


Low-rank tensor decomposition then arises as a powerful and widely used tool to discover simple low dimensional structures underlying such data.

we currently lack a theoretical understanding of the algorithmic behavior of low-rank tensor decompositions.


Consequently we get for each class a frequency band filter and a pattern (or similar to CSP more than one pattern by choosing the next eigenvectors).

with increasing T the complexity of the frequency filter has to be controlled in order to avoid overfitting.


Convolutional neural networks (CNNs) have recently achieved great success in single-image super-resolution (SISR).

these methods tend to produce over-smoothed outputs and miss some textural details.


We can now show that: The constraint Γ 0 is equivalent to (and can thus be replaced by) the following SPD constraint on a smaller matrix Γ: Since Γ is a principal submatrix of Γ (assuming at least one training label is equal to 1), lemma 2.1 indeed shows that Γ 0 implies Γ 0.

note that by adding a column and corresponding row to Γ, the rank is not increased.


Some examples include [16,1,17] who explored the use of skip connections; [18] who pointed out the distinction of constructing a "deep" RNN from the view of the recurrent paths and the view of the input-to-hidden and hidden-to-output maps.

they did not rigorously formalize the notion of "depth" and its implications in "deep" RNNs.


. . , w m }, to the distributed data-one for each local dataset.

structure between models frequently exists (e.g., people may behave similarly when using their phones), and modeling these relationships via multi-task learning is a natural strategy to improve performance and boost the effective sample size for each node [10,2,5].


[14] uses a matrix of i.i.d Gaussian elements to construct a preconditioner for least square which makes the problem well conditioned.

computing a random projection is still expensive as it requires multiplying a huge data matrix by another random dense matrix.


We assumed no prior knowledge on the dynamics of the hidden trajectories.

in a real application of the model we could simplify the problem by incorporating knowledge about previously identified movement templates.


In order to overcome the nonsparsity issue encountered in TGF-experiment, the SepGlasso estimator was thresholded more than the other methods.

it still performed poorly and identified the wrong gene CSF2RB.


In (1), previous studies often assumed independent firing as the null hypothesis.

for example, when we compare firing patterns in two periods, as control and 'test' periods, there may exist a weak correlation in the control period.


13: By introducing additional artificial variables a, the KKT conditions can be transformed into the linear optimization min i a i subject to the constraints: (22) The only nonlinear constraint is the product α i β i = 0.

this can be effectively implemented in the simplex procedure by modifying the selection of the pivot element to ensure that α i and β i are never both in the set of basic variables.


Hence, this domain pair is more amenable for domain adaptation as is demonstrated by the fact that the other approaches (SOURCEONLY, TARGETONLY, ALL) perform better or atleast as good as TARGETONLY-FULL.

as earlier, EA++ once again outperforms all these approaches including TARGETONLY-FULL.


FNNs with batch normalization exhibit high variance due to perturbations.

sNNs do not suffer from high variance as they are more robust to perturbations and learn faster.


For example, one could replace the spectral norm by the Frobenius norm in Assumptions 2 and 4, which would simplify some of the proofs.

using the Frobenius norm would be a much stronger assumption [18] Theorem 4 holds not just for simple graphs, but in the more general case when Â is a weighted graph (i.e.


Even approximate, sampling-based algorithms tend to take many iterations before they produce reasonable answers.

human recognition of words, objects, and scenes is extremely rapid, often taking only a few hundred milliseconds-only enough time for a single pass from perceptual evidence to deeper interpretation.


System Constraints So far, it is assumed in ( 6) that the classifier score, y t , that is generated after presenting the flash group, f t , is available instantaneously prior to selecting the flash group for the next time step, f s t+1 .

this is typically not the case during real-time BCI implementation as there is a delay between presenting a stimulus and computing the corresponding classifier score due to processing a time window of EEG data, as illustrated in Figure 3.


They also showed that the treereweighted max-product algorithm [9] can be used to solve the LP relaxation given by LOCAL(G), and that this succeeds in finding the MAP assignment for 339 of the 369 proteins in their data set.

the optimal solution to the LP relaxation for the remaining 30 proteins, arguably the most difficult of the proteins, is fractional.


the Noisy OR criteria treats all negative examples as independent negative examples.


Although an attention mechanism can be thought of as modulating the visual features by a language, we stress that such mechanism act on the high-level features.

our work modulates the visual processing from the very start.


In [22], the authors note that the tensor trace norm • tr in equation ( 3) is a convex lower bound to R on the set The key insight behind this observation is summarized in Lemma 4, which we report in Appendix A.

the authors of [22] leave open the question of whether the tensor trace norm is the convex envelope of R on the set G ∞ .


For the other data sets, ASSUMPTION-FREE K-MC 2 is initially disadvantaged by the cost of computing the proposal distribution.

as m increases and more time is spent computing the Markov chains, it either outperforms K-MC 2 (RNA and SUSY) or matches its performance (WEB).


In this paper we only considered real symmetric matrices.

our methods immediately generalize to complex Hermitian matrices, i.e square matrices in C n×n for which S = S T = S * .


Suppose we have intensity images If we sample white noise ∼ N (0 d , I d ) and ask an observer to indicate the class label for a + , most of the time the observer will answer with the correct class label A.

there is a chance that might manipulate a to cause the observer to mistakenly label a + as class B.


MSG is comparable to the incremental algorithm in terms of the the progress made per iteration.

its runtime is slightly worse because it will often keep a slightly larger representation (of dimension k t ).


( 7) is the dual of the entropy-regularized policy search (REPS) objective [5,16] L(q) = E q [R] + KL(q|p θ ).

our learning and sampling procedures are significantly different from REPS, as shown in what follows.


Most popular approaches for addressing this problem are based on purely linear mixing models.

many applications like the modeling of acoustic signals, EMG signals, or movement trajectories, require temporal shift-invariance of the extracted components.


With the emergence of big data, there is an increasing need to parallelize the training process of decision tree.

most existing attempts along this line suffer from high communication costs.


Under mild conditions [14], the ground-truth matrices in M • can be jointly triangularized, that is, there exists an orthogonal matrix U • that simultaneously renders all matrices where low(A) is the strictly lower triangular part of A, i.e., [low(A)] ij = A ij if i > j and 0 otherwise.

when σ > 0 the observed matrices in M σ can only be approximately jointly triangularized, for instance by solving the following optimization problem where • denotes Frobenius norm and optimization is over the manifold O(d) of orthogonal matrices.


As can be seen, the firing rates are a linear function of vestibular input, in agreement with equations 17 and 18 of the Appendix.

a linear firing-rate modulation by head velocity is not universal, for some parameters we have seen asymmetrically head-velocity tuning, with a preference for small head velocities (not shown).


More recently, the empirical kernel (4) was employed in an unsupervised way for multi-task learning to generalize to a previously unseen task [16].

we treat the probability distributions in a supervised way (cf.


This suggests that optimal connectivity might include both static and dynamic elements.

this reading mechanism leads to less stable training.


The student tries to move towards the teacher.

the student network itself can generate examples on which it is trained.


This flexibility is crucial for many DR-submodular maximization problems (see, (1.1)) as in many cases calculating the function F and its derivative is not feasible.

it is possible to provide a good un-biased estimator for these quantities.


Bauschke and Borwein [3] give a general condition for the linear convergence of alternating projections in terms of the value  ⇤ (defined in Section 3.1).

except in very limited cases, it is unclear how to compute or even bound  ⇤ .


In contrast, most experimental studies do not report on binary or steep tuning functions, but show smooth tuning curves only.

the shape of a tuning function always depends on the stimulus set used.


(taken from [16]) For a given sequence of Erdös-Rényi networks G(n, c n ), we have: As previously stated, our results hold for any given graph, and not only asymptotically.

we get an asymptotic behavior consistent with the aforementioned result.


Therefore, This formulation is still problematic because of Dirac-delta function.

we can see that the derivative term is biased towards zero as |u(τ ) − ϑ| increases.


The main reason behind the computational advantage is that once the moment expressions are acquired, the rest of the learning work amounts to factorizing a moment matrix whose size is independent of the number of data items.

it is unclear how to use these algorithms for more complicated models such as Mixture of Hidden Markov Models (MHMM).


The gates and the RBM components make up a high-order RBM.

such a high-order RBM is difficult to train and needs costly iterations during inference.


Note that this definition of static ICA includes methods for blind source separation that make use of temporal structure in the dynamics within the sources themselves [8], as long as the observed mixture of the sources is static.

'convolutive' ICA techniques explicitly assume convolutive or delayed mixtures in the source observations.


Yet another approach to the multivariable case is to use KCDC measures as features in a multiclass classification problem for d-dimensional distributions.

as noted in [13], this approach quickly becomes rather cumbersome as the number of labels grows super-exponentially in the number of variables due to the rapid increase of the number of DAGs that can be constructed from d variables.


By sampling drift only at endpoints, we will fail to sample some of the internal structure that is possible in drifts, e.g., we will fail to distinguish between a linear drift and a spiral pattern with the same endpoints.

the existence of significant, complicated drift patterns within a map segment would violate our assumption of moderate accuracy and local consistency within our low level mapper.


we propose an efficient algorithm that queries f at O(k log d) locations and recovers: (i) the active set S along with (ii) a uniform approximation to each φ l , l ∈ S. the existing error bounds in the statistics community [11,12,15] are in the much weaker L 2 (P) sense.


With multiple observations, the situation is similar.

we now have a set of Gaussian inputs Z (i) for each transition Xi | Xi−1 .


The scaling of weights in these networks is irrelevant because batch normalization re-scales the outputs to have unit variance.

small weights still appear more sensitive to perturbations, and produce sharper looking minimizers.


Our first result shows that this is indeed the case under three of the distance functions listed above, and a similar example works for a fourth.

surprisingly, under the Kendall tau distance the MLE best alternatives are exactly the approval winners, and hence are polynomial-time computable, which stands in sharp contrast to the NP-hardness of computing them given rankings [5].


Such an asymptotic comparison is not very useful in the general case, since the only possible conclusion, if €(hDis,oo) < €(hGen,oo), is that logistic regression is the superior algorithm.

as we saw previously, it is in the non-asymptotic case that the most interesting "two-regime" behavior is observed.


The E-step samples from the posterior and the M-step maximizes the average log joint probability of the samples and the data: where Q(θ) = 1 m m i=1 log p(y, u i |x, θ).

there is a significant drawback to MCEM: If the number of samples m used in the M-step is too low then there is a risk of the hyperparameters overfitting to those samples.


Independently, Wang [24] proved a quantitatively similar bound to Theorem 1.2.

wang's bound only applies to algorithms M that receive only the empirical mean q(X) of each query (as opposed to the whole data set).


These statistics are compared to the summary statistics of the real data, and the simulated parameter is weighted according to the similarity of the statistics to derive an empirical estimate of the posterior distribution.

choosing summary statistics for ABC is challenging because there is a tradeoff between loss of sufficiency and computational tractability.


The much smaller difference in MSE and 1-SSIM suggests that the L-CAE decoder does a better job at generalizing to unseen context-specific classes than the linear decoder.

the images show that there are still important face-specific features (such as nose and eye definition) that are much better decoded by the L-CAE trained only on faces (Figure 5E).


In this paper, we have chosen to focus on saddle-point problems because of their ubiquity in machine learning.

it turns out that our algorithm and, more importantly, our analysis extend to all set-valued monotone operators [8,28].


If the true value function does not lie within the space of these basis functions, the resulting approximation may be arbitrarily inaccurate, regardless of the number of acquired samples.

using predefined basis functions only requires inverting an m × m matrix, which results in a lower computational complexity than NPDP.


In contrast, no tractable algorithm for the NSG learning has been proposed to date.

it has been suggested that the NSG learning it better than the NPG learning due to the three differences [12]: (a) The NSG learning appropriately benefits from the concepts of Amari's NG learning, since the metric F s,a (θ) necessarily and sufficiently accounts for the probability distribution that the average reward depends on.


The drawback of this is a more cumbersome notation, and we have therefore limited the presentation to this simplified version.

the general algorithm is found in the Supplementary material of this paper.


After ICA, a source signal is not linearly predictable from others.

given variance dependencies, a source signal is still 'predictable' in a nonlinear manner.


In this context, the QCP formulation tends to find a doubly stochastic matrix P which minimizes the "average error" between AP and PB.

these spurious mismatching edges can be thought of as outliers, so we would want a metric promoting that AP and PB share the same active set (non zero entries representing edges), with the exception of some sparse entries.


As α approaches 1 from below, this bound leads to a better-behaved estimation problem of the Monte-Carlo gradient.

unless taking the limit of α → 1 (where the objective becomes the KL-bound), V still enters exponentially in the bound.


In most cases, the local maxima of R 2 (X(t), Y ) are indeed relevant for classification.

there are important points for which this quantity does not attain a maximum.


Therefore, the impact of estimation error on the exactly optimal policy is minimal.

we construct approximately optimal policies.


First, note that, since sampling uniformly a set U of size d 1 ( )  1/ allows a good approximation, then we can replace where J can be taken to have cardinality d eff ( ).

this is still costly, and the idea is to repeat and couple approximations at multiple scales.


For example, when the features are constructed using decision stumps, d will be extremely large, but ShareBoost can still be implemented efficiently.

when d is extremely large mix-norm regularization techniques yield challenging optimization problems.


Such bounds have the appealing feature that they provide performance guarantees for classifiers found by any learning algorithm.

it has been observed that these bounds tend to be overly pessimistic.


Statistical decision theory axiomatically assumes that the relative desirability of different options that humans perceive is well described by assigning them optionspecific scalar utility functions.

this assumption is refuted by observed human behavior, including studies wherein preferences have been shown to change systematically simply through variation in the set of choice options presented.


The existing methods can perform this task well, given many training images of zebras and horses.

it seems entirely possible to map a single zebra image to the analogous horse image even without seeing any other zebra image.


The teacher-student framework [14] is also related to our bottom-up approach; a low-cost student model learns to approximate the teacher model so as to meet test-time budget.

the goal there is to learn a better stand-alone student model.


Uniformly randomly pick a set I t from {1, . . . , N} (with replacement) such that |I t | = b 5: Randomly draw η t ∼ N (0, h t ) 6: α i t+1 = θ t for i ∈ I t and α i t+1 = α i t for i / ∈ I t 8: The increased variance in SGLD is due to the fact that we only have information from n N data points at each iteration.

inspired by a minibatch version of the SAGA algorithm [5], we can include information from the remaining data points via an approximate gradient, and partially update the average gradient in each operation.


At development time, one could take a clean dataset and test a defense against a number of poisoning strategies on that dataset.

because of the near-limitless space of possible attacks, it is impossible to conclude from empirical success alone that a defense that works against a known set of attacks will not fail against a new attack.


The layer with a relatively larger filter size had a larger compression ratio because it contains more subtle high-frequency coefficients.

the highest speed-up ratio was often obtained on the layer whose filter number N was much larger than its filter size, e.g., the fc6 layer of AlexNet.


Specifically, learning a shared feature covariance for model parameters [2] is a special case of our proposed framework.

assuming models of all tasks are equally similar is risky.


With the proposed lattice models, one can constrain f (x) to have a ceterus paribus diminishing returns response in the number of guests (for any fixed number of bottles) or in the number of bottles of wine (for any fixed number of guests), but without forcing f (x) to have a diminishing returns response along the diagonal direction defined as one bottle of wine per guest.

with the proposed SCNN, imposing diminishing returns on both wine and guests produces joint concavity, such that f (x) will be concave and decreasing over any line in the two-dimensional feature space.


At each query to the oracle, with probability of 1 − η it samples a random example x ∈ X according to the unknown distribution over X , and returns (x, sign(⟨w * , x⟩)).

with probability η, the oracle returns an arbitrary element of X × {±1}.


While the algorithm is efficient, it cannot be immediately formalized as a cost function with global solution, it relies solely on 2d information, and the affinity functions are handcrafted.

we provide a method that leverages a volumetric 3d scene representation with learned scoring functions for the component parts, and an efficient global linear integer programming solution for person grouping with kinematic constraint generation, and amenable to efficient solvers (e.g.


Supposing that a dot product between two vectors x i , x j can be computed using m operations, the total cost of sampling and then exhaustively solving k−means on the sample is O( msk s log n).

if one asks for a multiplicative (1 + ) approximation, the best running time known depends linearly on n [13].


This choice has shown to be sufficient for traditional low dimensional local features, e.g., SIFT; and typically, good performance can be achieved with only a few hundred Gaussian distributions.

the same number of Gaussians is insufficient to model the feature space spanned by higher dimensional local features, which have become popular recently.


It turns out that this is not true, at least if we restrict to a finite state space.

as the following theorem establishes, the coefficient multiplying min r∈ K J * − Φr ∞ can grow arbitrarily large as α increases, keeping all else fixed.


Testing Unimodality Using a union bound argument, one can use the results on testing monotonicity to give an algorithm with O p n log n/" 2 samples.

this is unsatisfactory, since our lower bound (and as we will demonstrate, the true complexity of this problem) is p n/" 2 .


If the true label is one, the sensitivity (true positive rate) α j for the j th annotator is defined as the probability that the annotator labels it as one.

if the true label is zero, the specificity (1−false positive rate) β j is defined as the probability that annotator labels it as zero.


For instance, SSCNet mistakes the television as another object in the second row of Figure 6, and fails to complete the pictures on the wall in the third row.

sATNet leverages RGB to overcome this difficulty.


Deep neural networks, especially convolutional neural networks (CNNs) [15] and recurrent neural networks (RNNs) [8], have been widely used in a variety of subjects [16].

traditional neural network blocks aim to learn the feature representations in a local sense.


The target error bound using H-divergence has the desirable property that it can be estimated by learning a classifier between the source and target domains with finite VC dimension, motivating the Domain Adversarial Neural Network (DANN) [14].

neural network usually has large VC dimensions, making the bound using H-divergence loose in practice.


2The original articles show that it is not necessary to do all the N steps.

since this is based on mixing approximation t echniques with exact ca lculations, it is not used here as it would hide the real error the approximation makes.


the most error-prone measurement in the experimental reconstruction.

it is important to note that any observed deviation of dendritic path from a straight line, including that due to measurement errors, causes an increase in the most likely values of parameters a and b.


• We show that a factored critic structure is particularly suited for credit assignment, and also allows analytical gradient computation by using results from collective graphical models [22].

• factored critic is not effectively learnable with global rewards.


At each time step, SQL uses two successive estimates of the action-value function that makes its space complexity twice as the standard Q-learning.

this allows SQL to use a more aggressive learning rate for one of the terms in its update rule and eventually achieves a faster convergence rate than the standard Qlearning (see Section 3.1 for a more detailed discussion).


In the applied psychology literature, awareness of sequential dependencies has led some researchers to explore strategies that mitigate relativity of judgment, such as increasing the number of response categories and varying the type and frequency of anchors (Mumma & Wilson, 2006;Wedell, Parducci, & Lane, 1990).

our approach to extracting more information from human judgments is to develop automatic techniques that recover the underlying sensation from a response that has been contaminated by cognitive processes producing the response.


The algorithms presented in [2] determine this point, which is the exponential loss solution x boost £ Xy 3¥ a cb `d ) 4 9 Bv w C i zG (see Figure 1, left).

maximum conditional likelihood estimation for an exponential model with the same features is equivalent to the problem x ml £ y ¦ ¥ a cb ¤d ) 4 #" 9 v w C zG where £¢ u is the exponential family with additional Lagrange multipliers, one for each normalization constraint.


If we do not consider running time, we could construct a low-rank approximation to A as follows: since projecting TAS onto W T is approximately optimal, it follows from Lemma A.1 that with probability 98/100, Let (AS) k = PN T be such that P has orthonormal columns.

then, AS − PP T AS 2 F = AS − (AS) k 2 F and by Lemma A.1 it follows that with probability 98/100, A − PP T A 2 F ≤ A − A k 2 F + A F . even approximately computing a column space P for (AS) k using an input-sparsity time algorithm is no longer sublinear.


In the most common scenarios that X k i is a half-space or ball, this condition is testable in O(n) time.

for arbitrary regions, this condition could be difficult to test.


Tractable learning aims to learn probabilistic models where inference is guaranteed to be efficient.

the particular class of queries that is tractable depends on the model and underlying representation.


We did not try to enforce a weighting with respect to the uniform density.

it would be no problem to compensate the effects of non-uniform sampling by using a weighted from of the Hessian energy.


In this framework, one attempts to construct the smallest ellipsoids around the state estimates that are consistent with the observations and the exogenous disturbance ellipsoids.

the resulting robust filters ignore any distributional information and thus have a tendency to be over-conservative.


Our primary objective in formulating this convex approach to mixture regression is to tackle a difficult problem in video analysis (see below).

to initially evaluate the different approaches we conducted some synthetic experiments.


The DWT and IDWT can be computed in O(n) operations via Mallat's pyramid algorithm [Mallat, 1989].


Neural networks have also been quite successful in working with natural language inputs, and LSTMbased models are state-of-the-art in many tasks [10,27,16].

much like other numerical input, it is not clear whether representations of number words are learned in a systematic way.


The rigid body assumption describes only a fraction of the real world, excluding, for example, soft bodies, cloths, fluids, and gases, and precludes objects breaking and combining.

objects are divisible and made up of a potentially large numbers of smaller sub-parts.


As we showed in Theorem 10, even OAPI, the very simple approximate algorithm for ABP, can perform as well as existing state-of-the art methods on factored MDPs.

a deeper understanding of the formulation and potential solution methods will be necessary in order to determine the full practical impact of the proposed methods.


The standard approach for this problem is to use a multinomial logistic function.

such a function is highly sensitive to the particular choice of negative verbs and using all the verbs as negative ones would be too costly.


Tuning the hyper-parameters for these models is difficult and computationally expensive in this setting as it requires conducting many cross-validation runs over large datasets.

our approach achieves consistently better performance with only 17 parameters, and a single hyper-parameter K which is fixed to 200.


We focus on linear programming (LP) relaxations, which have achieved state-of-the-art performance in many applications.

optimization of the resulting program is in general challenging due to non-smoothness and complex non-separable constraints.


[22] provide methods and code for computing derivatives for Cholesky factorization, QR decomposition, and symmetric eigenvalue decomposition when seen as matrix operators.

they all require that the output function is scalar-valued, and hence, cannot be generalized to higher order derivatives.


These notions have deep consequences for optimization, and the problem of (approximately) optimizing submodular functions has received much attention.

beyond optimization, these notions allow specifying expressive probabilistic models that can be used to quantify predictive uncertainty via marginal inference.


Recent work using string kernels for protein data has achieved state-of-the-art classification performance.

such representations are based only on labeled data -examples with known 3D structures, organized into structural classes -while in practice, unlabeled data is far more plentiful.


More recently, discriminative fine-tuning of generative models with nonlinear neural network observations [6], as well as direct training of CRFs with neural network factors [7], have allowed for training of nonlinear discriminative models.

such models have not been extensively applied to state estimation.


Some probabilistic multi-task feature selection methods have been proposed before [28,2].


Also, the number of parameters to be estimated does not play an important role.

gFM drastically outperforms MEUF in terms of computational complexity.


one can show that only 6 8 b¦ A ' ¢ Ec h8 @ dA c H H . Symmetry, on the other hand, implies that the term must be linear in 8 , too, which proves the claim.


In other words, measuring the effect of ⇣ m,j on @L SGVB D /@b m,j is easy as ⇣ m,j is the only random variable directly influencing this gradient via b m,j .

when sampling random weights, there are a thousand ✏ i,j influencing each gradient term, so their individual effects get lost in the noise.


we introduce a dummy variable gj such that if Y = j, then gj Ix = 1, otherwise gj Ix = 0, where j = 1,•••, J.

we since there may not be any data at Xi = z, the data from the neighborhood of x along dimension i are used to estimate E[gj IXi = z], a strategy suggested in [7].


The resulting high-order terms inside the expansion step also take the form (1) (in fact, Corollary 1 applies here); with no need to build the 'full' high-order graph, this would allow α-expansion to work as a fast alternative to the classic greedy algorithm for facility location, very similar to the fusion-based algorithm in [4].

in Section 3.2 we show that our generalized transformation allows for a novel way to optimize a hierarchical facility location objective.


For example, if P is B(2/3) over h and t, then for n = 3, P ({3}) = P (hhh) + P (ttt) = 1/3, P ({2, 1}) = P (hht) + P (hth) + P (thh) + P (tth) + P (tht) + P (htt) = 2/3, and P ({1, 1, 1} = 0 as this P is binary hence at most two symbols can appear.


It is also known that Gaussian cubature can be viewed as a special (in fact, degenerate) case of a kernel method [46,21].

no overall framework to derive probabilistic analogies of popular cubatures, with corresponding ease-of-use and reliability, has yet been developed.


From 1985 to 2000, this sector is in the cluster 9 which is rather independent from other clusters.

in 2005 the cluster separated, and telecom industry merged with cluster 1, which is a influential cluster.


Similar to these methods, we also make use of the Multi-CNN module to extract local features in our pipeline.

our novel context propagation network shows that propagating semantic representation bottom-up and top-down using a parse three hierarchy is a more effective way to aggregate global context information.


In other words, if H contains a very good policy that has little support under π, we will not be able to detect that by our estimator.

our estimation is safe in the sense that we will never drastically overestimate the value of any policy in H. This "underestimate, but don't overestimate" property is critical to the application of optimization techniques, as it implies we can use an unrestrained learning algorithm to derive a warm start policy.


In most statistical problems where the nonparametric bootstrap is useful, there is no particular reason for implementing it from a smoothed version of the empirical df rather from the raw empirical distribution itself, see [22].

in the present case, smoothing affects the rate of convergence.


Their method calls the SDCA method in a full-dimensional accelerated gradient method in an inner-outer iteration procedure.

our APCG method is a straightforward single loop coordinate gradient method.


Second, the alignment mechanism of both the CTC and RNN Transducer is constrained to be "monotonic" to keep marginalization of the alignment tractable.

the proposed attention mechanism can result in non-monotonic alignment, which makes it suitable for a larger variety of tasks other than speech recognition.


Solving Potts models to optimality is not a big issue anymore due to the recent work [21], which related this problems to the multiway-cut problem [31] and adopted a quite efficient solver based on the cutting plane technique.

we were able to outperform even this specialized solver on hard instances, which we collected in Table 2.


Similarly, as an inductive component of Low Density Separation (LDS) [11], ∆ TSVMs learns the SVM classification model in the primal, which can be used for predicting new data.

the graph part of LDS is transductive, because the kernel and the graph distances are addressed by a prior eigen-decompostion and re-representation (MDS); thus, it is unclear how to make a prediction of a new test point other than by rebuilding the graph with the new test point.


Conventional methods of uplift modeling require every instance to be jointly equipped with two types of labels: the taken action and its outcome.

obtaining two labels for each instance at the same time is difficult or expensive in many real-world problems.


On the one hand, the annotation of position and transcript at line level is costly to obtain.

automatic line segmentation algorithms are prone to errors, compromising the subsequent recognition.


So they instead use an approximation of the second derivative, which can be computed efficiently by the same inference oracle queried for the gradient evaluation.

pure CD is very expensive because it requires to call the inference oracle for the instances related to the current coordinate in each coordinate update.


It was pointed out in [5] that if supp 0 ( ŵ) can exactly identify all non-zero components of the target vector, then in the second stage, the relaxed Lasso can asymptotically remove the bias in the first stage Lasso.

it is not clear what theoretical result can be stated when Lasso cannot exactly identify all relevant features.


Conversely, for the Cambermet, Chimet and Bramblemet sensors, MOSM and CSM provided similar results, though we cannot confirm their difference is statistically significant.

given the high correlation of these signals and the similarity between the MOSM model and the CSM model, the close performance of these two models on this dataset is to be expected.


The presence of a trivial order relation in the shell indices ensures that all shells are placed in a concentric arrangement.

when a k-core is fragmented in two or more components, the diameters of the different components depend also on the relative number of vertices belonging to each of them, i.e.


Our model learns to remove objects primarily by trying to fool object classifiers in a GAN framework.

simply training a generator to re-synthesize the input image to fool object classifiers leads to degenerate solutions where the generator uses adversarial patterns to fool the classifiers.


In this paper, we are also interested in learning L directly.

because we are using constraints between examples and virtual points, we obtain a convex problem with a closed form solution allowing us to learn the metric in an efficient way.


In fact, if the bidder on day t uses two different arbitrary strategies in subsequent subgames depending on his different bids on day t, indeed it is impossible to bound the change in his utility.

we assume that the bidder is rational so his strategy in subsequent rounds satisfies equilibria conditions.


If f is smooth enough, say ↵ > d/2 + 1, and by setting s 2 > 2s 1 = 2↵, we recover the O(n 1 ) rate up to an extra (log n) d 0 factor.

theorem 2 applies to much more general settings.


The method described above is also known as column generation method or cutting-plane algorithm and can be shown to converge to the optimal solution w * [18].

since the computation of F involves many kernel computations and also the number of non-zero α's is often large, solving the problem with more than a few hundred labeled sequences often seems computationally too expensive.


However, it is easily trapped in local optima and one needs to tune the rank for the best performance.

with the observation that the product of two low rank matrices yields a low rank matrix, we will optimize over the new matrices and obtain a convex objective function.


We have sampled a 16×16 patch at every 8 pixel step.

local Self-Similarity computes correlation between a small image patch of interest and its surrounding region which captures the geometric layout of a local region.


samples from an unknown distribution (6) with parameter ✓ ⇤ , one could consider estimating the graphical model structure with an `1-regularized , and where we have collated the edgewise parameters {✓ st;jk } m j,k=1 for an edge (s, t) 2 E into the vector ✓ st .

there is an critical caveat to actually computing this regularized MLE: the computation of the log-partition function A(✓) is intractable (see [18] for details).


the uniform distribution U satisfies 1/ .

there are distributions D that are highly non-smooth relative to U but which still have D 2 / U 2 small.


The topics are still shared globally in the same way as in LDA.

[16] proposed to use different topics for different groups in order to account for the burstiness of the words (points).


(2) The distribution of s is aN-dimensional manifold embedded in aM-dimensional vector space and nominally has a negatively divergent entropy.

as shown in Appendix 1, the probability density of s can be related to the input distribution via the relation: where the susceptibility (or Jacobian) matrix X is defined as: This result can be understood in terms of the singular value decomposition (SVD) of the matrix x.


We improved this result by providing a finite-sample bound with optimal rate of convergence and also analyzed the quality of approximation in L r -norm (1 ≤ r < ∞).

we also proposed an RFF approximation for derivatives of a kernel and provided theoretical guarantees on the quality of approximation in uniform and L r -norms over compact subsets of R d . While all the results in this paper (and also in the literature) dealt with the approximation quality of RFF over only compact subsets of R d , it is of interest to understand its behavior over entire R d . as discussed in Remark 1(ii) and in the paragraph following Theorem 3, RFF cannot approximate the kernel uniformly or in L r -norm over R d . By truncating the Taylor series expansion of the exponential function, [3] proposed a non-random finite dimensional representation to approximate the Gaussian kernel which also enjoys the computational advantages of RFF.


To simplify the notations in the theoretical analysis, we assume that the number of samples for all the tasks are the same.

our theoretical analysis can be easily extended to the case where the tasks have different sample sizes.


An example of a divergence which satisfies the assumption of strong convexity and smoothness of over R d is the Mahalanobis distance which is a divergence induced by the function (x) = x T Ax, for some positive definite matrix A.

we note that many popular divergences such as KL-divergence do not satisfy the smoothness condition over the entire space R d . For such divergences, depends on the distance between the starting point ✓ 0 and the minimizer ✓ and their location in the space.


Over the years, deep learning approaches (see [5,26] for survey) have shown great success in many visual perception problems (e.g., [16,7,32,9]).

modeling videos (building a generative model) is still a very challenging problem because it often involves high-dimensional natural-scene data with complex temporal dynamics.


ROVER calculates the optical flow of consecutive radar maps using the algorithm in [5] and performs semi-Lagrangian advection [4] on the flow field, which is assumed to be still, to accomplish the prediction.

the success of these optical flow based methods is limited because the flow estimation step and the radar echo extrapolation step are separated and it is challenging to determine the model parameters to give good prediction performance.


Row Sampling does not directly apply to our problem because (1) it does not minimize over permutations of the right-hand side, and (2) the approximation factor is too large.

we are able to use it to narrow the search space in our problem.


. . , T T , such that depth(T t ) may equal t. Furthermore, the number of new nodes added to the tree on round t is on the order of t, resulting in T t having O(t 2 ) nodes.

pST implementation tricks in [1] can be used to reduce the space complexity of the algorithm from quadratic to linear in t. The online learning algorithm presented in the previous section has one major drawback, the PSTs it generates can keep growing with each online round.


LS and CP directly regularize the model prediction at each time-step and improve the generalization of the CTC baseline.

their methods indiscriminately regularizes both feasible and unfeasible paths.


One advantage of uniform sampling is its simplicity of construction.

as shown in Section 3.2, it takes nearly input-sparsity time to construct the proposed non-uniform sampling distribution.


gap_per_epoch performs poorly because it is unable to adapt to the variability of the coordinate-wise duality gaps G i that vary a lot from one iteration to the next.

this variation slows down the convergence of B_max_r compared to max_r, but B_max_r is still able to cope with this change by exploring and updating the estimations of the marginal decreases.


Recently, a randomized online PCA algorithm was proposed by [23], whose objective is to minimize the total expected quadratic error minus the total error of the batch algorithm (i.e., the regret).

none of these online PCA algorithms is robust to the outliers.


The strategy shortest-path seems to be intuitively favored over longest-path as it is just  the "nearest-neighbor" prediction with respect to the geodesic distance.

the following proposition shows that it is strictly worse than any fixed-paths strategy in the worst case.


The resulting algorithm is passive whenever the loss is zero, that is, w t+1 = w t whenever (w t ; z t ) = 0.

on rounds for which (w t ; z t ) > 0 we aggressively force w t+1 to satisfy the constraint (w t+1 ; z t ) = 0.


This may be a good approximation when data is drawn from a single population and the goal is to explain the data in terms of a few variables.

in machine learning we often deal with data drawn from several populations and PCA is used to reduce dimensions to control computational complexity of learning.


For simplicity, we are stating our results in terms of only the numerical sparsity.

when the number of zero entries in a row is small, we can always choose that row completely and not do sampling on it.


We allow the annotator to submit the HIT after one correction even if her/his evaluation still points to errors.

we plea to the good will of the annotators to continue in providing feedback.


In summary, if σ 2 y and σ 2 d are given, the optimal D and A B are found by taking the SVD of Y and applying shrinkage/thresholding operations on the singular values of Y.

this method requires setting σ 2 y and σ 2 d manually.


Most existing approaches to distributed sparse regression assume the data is partitioned by samples.

for high-dimensional data (D N ), it is more natural to partition the data by features.


As there is a large amount of overlap in the surface area that the six ODMs observe, this silhouette agreement is enforced to maintain the structure of the object.

this same process occurs for detail carving with C H . we do not require agreement within the constrained depth map predictions.


Given a prior, the Bayes optimal estimators are known to be the posterior mean under MSE, the Maximum A Posterior (MAP) rule under 0/1 loss, and the posterior centroid under Hamming loss [8].

these estimators and their corresponding risks (expected loss) have no closed form for the Ising graphical model and are intractable to analyze.


As a consequence, tests that utilize V b2 usually give lower Type II error then the ones that use V b1 .

v b1 seems to better approximate V -statistic distribution under the null hypothesis.


Given the large label space, it is unrealistic to expect that a label mapping function is readily available.

the two tasks are clearly related and learning them simultaneously is likely to improve performance.


The same query type is also used in [9], which differs from [8] in treating PE as a Partially Observable Markov Decision Process (POMDP).

standard gamble queries are difficult for users to respond to, and naturally lead to noisy responses.


Of course, this cannot exactly accomplish optimal resampling, both because the chance that a particle survives should be normalized with respect to the population, and because the distribution from which a non-surviving particle resamples should also depend on the ensemble distribution.

it has a similar qualitative effect of suppressing conditionally unlikely samples and replacing them ultimately with conditionally more likely ones.


forcing w to be an all 1 vector) would amount to borrowing all examples, which would result in learning a "generic" object detector.

setting λ 1 = λ 2 = 0 would recover the original standard objective of Eq (1), without borrowing any examples.


Real-world experiments illustrate how these shape constraints can be useful and effective in practice.

consistent with the past literature on monotonic neural networks, we found it difficult to control the shape constraints on neural networks as flexibly as with the lattice models.


Since for α small enough d M,α (r, r) > 0 for any r such that h(r) > 0, Sinkhorn distances cannot satisfy the coincidence axiom (d(x, y) = 0 ⇔ x = y holds for all x, y).

multiplying d M,α by 1 r =c suffices to recover the coincidence property if needed.


In both the binary and multiclass scenarios, PCA is inundated by millions of features when using all 5-grams and cannot display good class structure.

compression reduces the feature set to tens of thousands (by two orders of magnitude) and clearly shows class structure.


In general The ACE is simple to estimate in a randomized trial: this follows from estimating the conditional distribution of Y given X under data generated as in Figure 1(b).

in an observational study [21] we obtain data generated by the system in Figure 1(a).


We will show that the idea of k − 1 copies can be understood as forming a compatibility graph between the original database and the released database which is composed of several fully-connected k-cliques.

rather than guaranteeing copies or cliques, the anonymity problem can be relaxed into a k-regular compatibility to achieve nearly identical resilience to attack.


The obtained ML estimators has many desirable properties, such as consistency and asymptotic normality [21].

because of the high dimensional integration/summation, the partition function in q θ oftentimes makes ML learning computationally intractable.


Such techniques rely on estimating the covariances and higher order moments in terms of the model parameters to detect structure and dependencies.

in the mixture of Mallows models problem there is no "natural" notion of a second/third moment.


To learn these hyper-parameters, we can also encode them in particles and filter them together with the hidden states.

since θ is constant across time, naively filtering such particles without regeneration will fail due to particle impoverishment, where a few or even one particle receives all the weight.