Differential privacy is a notion that has emerged in the community of statistical databases, as a response to the problem of protecting the privacy of the database's participants when performing statistical queries. The idea is that a randomized query satisfies differential privacy if the likelihood of obtaining a certain answer for a database x is not too different from the likelihood of obtaining the same answer on adjacent databases, i.e. databases which differ from x for only one individual. Information flow is an area of Security concerned with the problem of controlling the leakage of confidential information in programs and protocols. Nowadays, one of the most established approaches to quantify and to reason about leakage is based on the Renyi min entropy version of information theory. In this paper, we analyze critically the notion of differential privacy in light of the conceptual framework provided by the Renyi min information theory. We show that there is a close relation between differential privacy and leakage, due to the graph symmetries induced by the adjacency relation. Furthermore, we consider the utility of the randomized answer, which measures its expected degree of accuracy. We focus on certain kinds of utility functions called "binary", which have a close correspondence with the Renyi min mutual information. Again, it turns out that there can be a tight correspondence between differential privacy and utility, depending on the symmetries induced by the adjacency relation and by the query. Depending on these symmetries we can also build an optimal-utility randomization mechanism while preserving the required level of differential privacy. Our main contribution is a study of the kind of structures that can be induced by the adjacency relation and the query, and how to use them to derive bounds on the leakage and achieve the optimal utility.
As far as we know, the first work to investigate the relation between differential privacy and information-theoretic leakage was @cite_16 . In this work, a channel is relative to a given database @math , and the channel inputs are all possible databases adjacent to @math . Two bounds on leakage were presented, one for teh R 'enyi min entropy, and one for Shannon entropy. Our bound in Proposition is an improvement with respect to the (R 'enyi min entropy) bound in @cite_16 .
@cite_16
43706173
Differential privacy is a notion of privacy that has become very popular in the database community. Roughly, the idea is that a randomized query mechanism provides sufficient privacy protection if the ratio between the probabilities of two different entries to originate a certain answer is bound by e^e. In the fields of anonymity and information flow there is a similar concern for controlling information leakage, i.e. limiting the possibility of inferring the secret information from the observables. In recent years, researchers have proposed to quantify the leakage in terms of the information-theoretic notion of mutual information. There are two main approaches that fall in this category: One based on Shannon entropy, and one based on Renyi's min entropy. The latter has connection with the so-called Bayes risk, which expresses the probability of guessing the secret. In this paper, we show how to model the query system in terms of an information-theoretic channel, and we compare the notion of differential privacy with that of mutual information. We show that the notion of differential privacy is strictly stronger, in the sense that it implies a bound on the mutual information, but not viceversa. Recent research in quantitative theories for information-hiding topics, such as Anonymity and Secure Information Flow, tend to converge towards the idea of modeling the system as a noisy channel in the information-theoretic sense. The notion of information leakage, or vulnerability of the system, has been related in some approaches to the concept of mutual information of the channel. A recent work of Smith has shown, however, that if the attack consists in one single try, then the mutual information and other concepts based on Shannon entropy are not suitable, and he has proposed to use Renyi's min-entropy instead. In this paper, we consider and compare two different possibilities of defining the leakage, based on the Bayes risk, a concept related to Renyi min-entropy. It was recently shown that estimating the Shannon entropy @math of a discrete @math -symbol distribution @math requires @math samples, a number that grows near-linearly in the support size. In many applications @math can be replaced by the more general R 'enyi entropy of order @math , @math . We determine the number of samples needed to estimate @math for all @math , showing that @math requires a near-linear @math samples, but, perhaps surprisingly, integer @math requires only @math samples. Furthermore, developing on a recently established connection between polynomial approximation and estimation of additive functions of the form @math , we reduce the sample complexity for noninteger values of @math by a factor of @math compared to the empirical estimator. The estimators achieving these bounds are simple and run in time linear in the number of samples. Our lower bounds provide explicit constructions of distributions with different R 'enyi entropies that are hard to distinguish. Consider two parties holding samples from correlated distributions @math and @math , respectively, where these samples are within distance @math of each other in some metric space. The parties wish to agree on a close-to-uniformly distributed secret key @math by sending a single message over an insecure channel controlled by an all-powerful adversary who may read and modify anything sent over the channel. We consider both the keyless case, where the parties share no additional secret information, and the keyed case, where the parties share a long-term secret @math that they can use to generate a sequence of session keys @math using multiple pairs @math . The former has applications to, e.g., biometric authentication, while the latter arises in, e.g., the bounded-storage model with errors. We show solutions that improve upon previous work in several respects. The best prior solution for the keyless case with no errors (i.e., @math ) requires the min-entropy of @math to exceed @math , where @math is the bit length of @math . Our solution applies whenever the min-entropy of @math exceeds the minimal threshold @math , and yields a longer key. We establish formal bounds for the number of min-entropy bits that can be extracted in a timing attack against a cryptosystem that is protected by blinding, the state-of-the art countermeasure against timing attacks. Compared with existing bounds, our bounds are both tighter and of greater operational significance, in that they directly address the key’s one-guess vulnerability. Moreover, we show that any semantically secure public-key cryptosystem remains semantically secure in the presence of timing attacks, if the implementation is protected by blinding and bucketing. This result shows that, by considering (and justifying) more optimistic models of leakage than recent proposals for leakage-resilient cryptosystems, one can achieve provable resistance against side-channel attacks for standard cryptographic primitives. Privacy amplification allows two parties Alice and Bob knowing a partially secret string S to extract, by communication over a public channel, a shorter, highly secret string S'. Bennett, Brassard, Crepeau, and Maurer showed that the length of S' can be almost equal to the conditional Renyi entropy of S given an opponent Eve's knowledge. All previous results on privacy amplification assumed that Eve has access to the public channel but is passive or, equivalently, that messages inserted by Eve can be detected by Alice and Bob. In this paper we consider privacy amplification secure even against active opponents. First it is analyzed under what conditions information-theoretically secure authentication is possible even though the common key is only partially secret. This result is used to prove that privacy amplification can be secure against an active opponent and that the size of S' can be almost equal to Eve's min-entropy about S minus 2n 3 if 5 is an n-bit string. Moreover, it is shown that for sufficiently large n privacy amplification is possible when Eve's min-entropy about S exceeds only n 2 rather than 2n 3. This note describes an information theory problem that arose from some analysis of quantum key distribution protocols. The problem seems very natural and is very easy to state but has not to our knowledge been addressed before in the information theory literature: suppose that we have a random bit string y of length n and we reveal k bits at random positions, preserving the order but without revealing the positions, how much information about y is revealed? We show that while the cardinality of the set of compatible y strings depends only on n and k, the amount of leakage does depend on the exact revealed x string. We observe that the maximal leakage, measured as decrease in the Shannon entropy of the space of possible bit strings, corresponds to the x string being all zeros or all ones and that the minimum leakage corresponds to the alternating x strings. We derive a formula for the maximum leakage minimal entropy in terms of n and k. We discuss the relevance of other measures of information, in particular min-entropy, in a cryptographic context. Finally, we describe a simulation tool to explore these results. We present simple and computationally efficient nonparametric estimators of R 'enyi entropy and mutual information based on an i.i.d. sample drawn from an unknown, absolutely continuous distribution over @math . The estimators are calculated as the sum of @math -th powers of the Euclidean lengths of the edges of the generalized nearest-neighbor' graph of the sample and the empirical copula of the sample respectively. For the first time, we prove the almost sure consistency of these estimators and upper bounds on their rates of convergence, the latter of which under the assumption that the density underlying the sample is Lipschitz continuous. Experiments demonstrate their usefulness in independent subspace analysis. In this paper we obtain new bounds for the minimum output entropies of random quantum channels. These bounds rely on random matrix techniques arising from free probability theory. We then revisit the counterexamples developed by Hayden and Winter to get violations of the additivity equalities for minimum output Renyi entropies. We show that random channels obtained by randomly coupling the input to a qubit violate the additivity of the @math -Renyi entropy. For some sequences of random quantum channels, we compute almost surely the limit of their Schatten @math norms. There is growing interest in quantitative theories of information flow in a variety of contexts, such as secure information flow, anonymity protocols, and side-channel analysis. Such theories offer an attractive way to relax the standard noninterference properties, letting us tolerate "small" leaks that are necessary in practice. The emerging consensus is that quantitative information flow should be founded on the concepts of Shannon entropy and mutual information . But a useful theory of quantitative information flow must provide appropriate security guarantees: if the theory says that an attack leaks x bits of secret information, then x should be useful in calculating bounds on the resulting threat. In this paper, we focus on the threat that an attack will allow the secret to be guessed correctly in one try. With respect to this threat model, we argue that the consensus definitions actually fail to give good security guarantees--the problem is that a random variable can have arbitrarily large Shannon entropy even if it is highly vulnerable to being guessed. We then explore an alternative foundation based on a concept of vulnerability (closely related to Bayes risk ) and which measures uncertainty using Renyi's min-entropy , rather than Shannon entropy. Theories of quantitative information flow have seen growing interest recently, in view of the fundamental importance of controlling the leakage of confidential information, together with the pragmatic necessity of tolerating intuitively "small" leaks. Given such a theory, it is crucial to develop automated techniques for calculating the leakage in a system. In this paper, we address this question in the context of deterministic imperative programs and under the recently-proposed min-entropy measure of information leakage, which measures leakage in terms of the confidential information's vulnerability to being guessed in one try by an adversary. In this context, calculating the maximum leakage of a program reduces to counting the number of feasible outputs that it can produce. We approach this task by determining patterns among pairs of bits in the output, for instance by determining that two bits must be unequal. By counting the number of solutions to the two-bit patterns, we obtain an upper bound on the number of feasible outputs and hence on the leakage. We explore the effectiveness of our approach on a number of case studies, in terms of both efficiency and accuracy. I Entropies for Finite Quantum Systems.- 1 Fundamental Concepts.- 2 Postulates for Entropy and Relative Entropy.- 3 Convex Trace Functions.- II Entropies for General Quantum Systems.- 4 Modular Theory and Auxiliaries.- 5 Relative Entropy of States of Operator Algebras.- 6 From Relative Entropy to Entropy.- 7 Functionals of Entropy Type.- III Channeling Transformation and Coarse Graining.- 8 Channels and Their Transpose.- 9 Sufficient Channels and Measurements.- 10 Dynamical Entropy.- 11 Stationary Processes.- IV Perturbation Theory.- 12 Perturbation of States.- 13 Variational Expression of Perturbational Limits.- V Miscellanea.- 14 Central Limit and Quasi-free Reduction.- 15 Thermodynamics of Quantum Spin Systems.- 16 Entropic Uncertainty Relations.- 17 Temperley-Lieb Algebras and Index.- 18 Optical Communication Processes. Estimating divergences in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential inequality convergence bound derived for any divergence estimators. The main contribution of our work is to provide such a bound for an estimator of R 'enyi- @math divergence for a smooth H "older class of densities on the @math -dimensional unit cube @math . We also illustrate our theoretical results with a numerical experiment. Boosting is a general method for improving the accuracy of learning algorithms. We use boosting to construct improved privacy-preserving synopses of an input database. These are data structures that yield, for a given set @math of queries over an input database, reasonably accurate estimates of the responses to every query in @math , even when the number of queries is much larger than the number of rows in the database. Given a base synopsis generator that takes a distribution on @math and produces a weak'' synopsis that yields good'' answers for a majority of the weight in @math , our Boosting for Queries algorithm obtains a synopsis that is good for all of @math . We ensure privacy for the rows of the database, but the boosting is performed on the queries . We also provide the first synopsis generators for arbitrary sets of arbitrary low-sensitivity queries, i.e. , queries whose answers do not vary much under the addition or deletion of a single row. In the execution of our algorithm certain tasks, each incurring some privacy loss, are performed many times. To analyze the cumulative privacy loss, we obtain an @math bound on the expected privacy loss from a single @math - mechanism. Combining this with evolution of confidence arguments from the literature, we get stronger bounds on the expected cumulative privacy loss due to multiple mechanisms, each of which provides @math -differential privacy or one of its relaxations, and each of which operates on (potentially) different, adaptively chosen, databases. An encryption scheme is said to be entropically secure if an adversary whose min-entropy on the message is upper bounded cannot guess any function of the message. Similarly, an encryption scheme is entropically indistinguishable if the encrypted version of a message whose min-entropy is high enough is statistically indistinguishable from a fixed distribution. We present full generalizations of these two concepts to the encryption of quantum states in which the quantum conditional min-entropy, as introduced by Renner, is used to bound the adversary's prior information on the message. A proof of the equivalence between quantum entropic security and quantum entropic indistinguishability is presented. We also provide proofs of security for two different ciphers in this model and a proof for a lower bound on the key length required by any such cipher. These ciphers generalize existing schemes for approximate quantum encryption to the entropic security model. New results are proved on the convergence of the Shannon (1959) lower bound to the rate distortion function as the distortion decreases to zero. The key convergence result is proved using a fundamental property of informational divergence. As a corollary, it is shown that the Shannon lower bound is asymptotically tight for norm-based distortions, when the source vector has a finite differential entropy and a finite spl alpha th moment for some spl alpha >0, with respect to the given norm. Moreover, we derive a theorem of Linkov (1965) on the asymptotic tightness of the Shannon lower bound for general difference distortion measures with more relaxed conditions on the source density. We also show that the Shannon lower bound relative to a stationary source and single-letter difference distortion is asymptotically tight under very weak assumptions on the source distribution. > Let X1,..., Xn be a sequence of n classical random variables and consider a sample Xs1,..., Xsr of r ≤ n positions selected at random. Then, except with (exponentially in r) small probability, the min-entropy Hmin(Xs1 ...Xsr) of the sample is not smaller than, roughly, a fraction r n of the overall entropy Hmin(X1 ...Xn), which is optimal. Here, we show that this statement, originally proved in [S. Vadhan, LNCS 2729, Springer, 2003] for the purely classical case, is still true if the min-entropy Hmin is measured relative to a quantum system. Because min-entropy quantifies the amount of randomness that can be extracted from a given random variable, our result can be used to prove the soundness of locally computable extractors in a context where side information might be quantum-mechanical. In particular, it implies that key agreement in the bounded-storage model-using a standard sample-and-hash protocol-is fully secure against quantum adversaries, thus solving a long-standing open problem. We study theoretically and numerically the entanglement entropy of the @math -dimensional free fermions whose one body Hamiltonian is the Anderson model. Using basic facts of the exponential Anderson localization, we show first that the disorder averaged entanglement entropy @math of the @math dimension cube @math of side length @math admits the area law scaling @math even in the gapless case, thereby manifesting the area law in the mean for our model. For @math and @math we obtain then asymptotic bounds for the entanglement entropy of typical realizations of disorder and use them to show that the entanglement entropy is not selfaveraging, i.e., has non vanishing random fluctuations even if @math . We give a lower bound of @math for the degree-4 Sum-of-Squares SDP relaxation for the planted clique problem. Specifically, we show that on an Erd "os-R 'enyi graph @math , with high probability there is a feasible point for the degree-4 SOS relaxation of the clique problem with an objective value of @math , so that the program cannot distinguish between a random graph and a random graph with a planted clique of size @math . This bound is tight. We build on the works of Deshpande and Montanari and , who give lower bounds of @math and @math respectively. We improve on their results by making a perturbation to the SDP solution proposed in their work, then showing that this perturbation remains PSD as the objective value approaches @math . In an independent work, Hopkins, Kothari and Potechin [HKP15] have obtained a similar lower bound for the degree- @math SOS relaxation. We give near-optimal sketching and streaming algorithms for estimating Shannon entropy in the most general streaming model, with arbitrary insertions and deletions. This improves on prior results that obtain suboptimal space bounds in the general model, and near-optimal bounds in the insertion-only model without sketching. Our high-level approach is simple: we give algorithms to estimate Tsallis entropy, and use them to extrapolate an estimate of Shannon entropy. The accuracy of our estimates is proven using approximation theory arguments and extremal properties of Chebyshev polynomials. Our work also yields the best-known and near-optimal additive approximations for entropy, and hence also for conditional entropy and mutual information.