Masking and Re-identification Methods for Public-Use Microdata: Overview and Research Problems <s> Re-identification of Microdata <s> A regression methodology based technique can be used to compromise confidentiality in a statistical database. This holds true even when the DBMS prevents application of regression methodology to the database. Existing inference controls, including cell restriction, perturbation, and table restriction approaches, are shown to be generally ineffective against this compromise technique. The effect of incomplete supplemental knowledge on the regression methodology based compromise technique is examined. Finally, some potential complicators of this disclosure scheme are introduced. <s> BIB001 </s> Masking and Re-identification Methods for Public-Use Microdata: Overview and Research Problems <s> Re-identification of Microdata <s> Abstract National statistical offices meet an increasing demand for the dissemination of microdata. This trend can be at variance with the care taken not to disclose data about individuals. Therefore, the risk of disclosure of each microdata set to be released should be assessed. If this risk is too high, measures have to be taken to protect the data set. This article describes the disclosure problem, and explains why it is a real problem. Using the concept of uniqueness, some theory is developed that can help establish the risk of identification. It turns out that useful microdata sets can only be released if some of the disclosure risks are dealt with by legal arrangements, rather than by restrictions on the data to be released. <s> BIB002 </s> Masking and Re-identification Methods for Public-Use Microdata: Overview and Research Problems <s> Re-identification of Microdata <s> This overview gives background on a number of statistical methods that have been proven effective for record linkage. To prepare data for the main computational algorithms, we need parsing/standardization that allows us to structure the free-form names, addresses, and other fields into corresponding components. The main parameter-estimation methods are unsupervised methods that yield 'optimal' record linkage parameters. Extended methods provide estimates of false match rates in both unsupervised and, with greater accuracy, in semi-supervised situations. Finally, the paper describes ongoing research for adjusting standard statistical analyses for linkage error. WIREs Comput Stat 2014, 6:313-325. doi: 10.1002/wics.1317 <s> BIB003 </s> Masking and Re-identification Methods for Public-Use Microdata: Overview and Research Problems <s> Re-identification of Microdata <s> This invention relates to a vehicle sun roof, preferably transparent or translucent such as glass or plastic, which contains the following characteristics: <s> BIB004 </s> Masking and Re-identification Methods for Public-Use Microdata: Overview and Research Problems <s> Re-identification of Microdata <s> A measure of re-identification risk at the record level has a variety of potential uses in statistical disclosure control for microdata. The conceptual basis of such a measure is considered. The risk is conceived of broadly as the evidence in support of a link between the record and the unit in the population from which it is derived. For discrete key variables subject to no measurement error, a measure is derived which reflects the probability that the record is unique in the population. Under certain assumptions, two approaches are described for estimating this measure from the microdata. These approaches are applied to a 10% sample of microdata from the 1991 Census in Great Britain. It is found that the resulting risk measures can indeed be used successfully to establish whether sample unique records are unique in the population. The implications of these findings are discussed. <s> BIB005 </s> Masking and Re-identification Methods for Public-Use Microdata: Overview and Research Problems <s> Re-identification of Microdata <s> Many organizations require detailed individual-level information, much of which has been collected under guarantees of confidentiality. However, simple anonymization procedures, i.e. removing names and addresses, are insufficient for this to be ensured. The records belonging to certain individuals have a high probability of being identified (as their contents, or attributes, are unusual) and therefore have the potential to be recognized spontaneously - such records are referred to as special uniques. Consider, for example, a sixteen-year-old widow in a population survey. Confidentiality of a given dataset cannot be enabled until all special unique records are identified and either disguised or removed. However, to the knowledge of the authors, no exhaustive automated analysis of this nature has been conducted due to the demanding levels of computation and data storage that are required. This paper introduces a new algorithm that locates 'risky' records in discrete data by first identifying all unique attribute sets (up to a user-specified maximum size) and secondly by grading the 'risk' of each record by considering the number and distribution of unique attribute sets within each record. Empirical tests indicate that the algorithm is highly effective at picking out 'risky' records from large samples of data. <s> BIB006 </s> Masking and Re-identification Methods for Public-Use Microdata: Overview and Research Problems <s> Re-identification of Microdata <s> Often a data holder, such as a hospital or bank, needs to share person-specific records in such a way that the identities of the individuals who are the subjects of the data cannot be determined. One way to achieve this is to have the released records adhere to k- anonymity, which means each released record has at least (k-1) other records in the release whose values are indistinct over those fields that appear in external data. So, k- anonymity provides privacy protection by guaranteeing that each released record will relate to at least k individuals even if the records are directly linked to external information. This paper provides a formal presentation of combining generalization and suppression to achieve k-anonymity. Generalization involves replacing (or recoding) a value with a less specific but semantically consistent value. Suppression involves not releasing a value at all. The Preferred Minimal Generalization Algorithm (MinGen), which is a theoretical algorithm presented herein, combines these techniques to provide k-anonymity protection with minimal distortion. The real-world algorithms Datafly and µ-Argus are compared to MinGen. Both Datafly and µ-Argus use heuristics to make approximations, and so, they do not always yield optimal results. It is shown that Datafly can over distort data and µ-Argus can additionally fail to provide adequate protection. <s> BIB007 </s> Masking and Re-identification Methods for Public-Use Microdata: Overview and Research Problems <s> Re-identification of Microdata <s> In this paper we show how a simple model that captures user uncertainty can be used to define suitable measures of disclosure risk and data utility. The model generalizes previous results of Duncan and Lambert. We present several examples to illustrate how the new measures can be used to implement existing optimality criteria for the choice of the best form of data release. <s> BIB008 </s> Masking and Re-identification Methods for Public-Use Microdata: Overview and Research Problems <s> Re-identification of Microdata <s> Coreference analysis, also known as record linkage, object consolidation or identity uncertainty, is a dicult and important problem in natural language processing, databases, citation matching and many other tasks. This paper introduces several discriminative, conditional-probability models for coreference analysis, all examples of undirected graphical models. Unlike many historical approaches to coreference, the models presented here are relational—they do not assume that pairwise coreference decisions should be made independently from each other. Unlike other relational models of coreference that are generative, the conditional model here can incorporate a great variety of features of the input without having to be concerned about their dependencies— paralleling the advantages of conditional random fields over hidden Markov models. We present positive results on noun coreference in two standard text data sets. <s> BIB009 </s> Masking and Re-identification Methods for Public-Use Microdata: Overview and Research Problems <s> Re-identification of Microdata <s> When microdata files for research are released, it is possible that external users may attempt to breach confidentiality. For this reason most National Statistical Institutes apply some form of disclosure risk assessment and data protection. Risk assessment first requires a measure of disclosure risk to be defined. In this paper we build on previous work by [BF98] to define a Bayesian hierarchical model for risk estimation. We follow a superpopulation approach similar to [BKP90] and [Rin03]. For each combination of values of the key variables we derive the posterior distribution of the population frequency given the observed sample frequency. Knowledge of this posterior distribution enables us to obtain suitable summaries that can be used to estimate the risk of disclosure. One such summary is the mean of the reciprocal of the population frequency or Benedetti-Franconi risk, but we also investigate others such as the mode. We apply our approach to an artificial sample of the Italian 1991 Census data, drawn by means of a widely used sampling scheme. We report on results of this application and document the computational difficulties that we encountered. The risk estimates that we obtain are sensible, but suggest possible improvements and modifications to our methodology. We discuss these together with potential alternative strategies. <s> BIB010
The highest standard for estimating the proportion of records that can be reidentified (Lambert  , Domingo-Ferrer and Torra  ) is where record linkage is used to match information from public data with the masked microdata. Alternative effective matching methods are the nearest neighbor methods (Domingo-Ferrer and Mateo-Sanz  ) and the clustering algorithms of Bacher et al.  . Record linkage can also be used in the conservative framework described in section 2 in which a masked file of Y data is matched directly with the original, confidential file of X data that was used in creating it. In some situations, it may be possible to accurately match 0.5-2% of the records in the initial iteration of the potential masked microdata. In this situation, the file is non-confidential and we would apply additional masking procedures to provide disclosure avoidance. The ability to link public information to public-use files has further been compromised by the increased sophistication of record linkage (Yancey et al.  and link analysis methods (McCallum and Wellner BIB009 , Bilenko et al.  ) and the increased availability of high-quality public data. Indeed, due to the quality of public information, Malin et al.  are often able to match public information using purely manual methods. Re-identification of microdata refers to the ability to use publicly available information to attach names, addresses, and other (semi-)unique identifiers to individual records in a public-use file. An identifier is semi-unique if it cannot exactly identify a linkage between two records using the identifier by itself but can allow a reidentification in combination with other variables. For instance, in a file where there are many individuals with the name "John Smith", "John Smith" is a semi-unique identifier. US Postal ZIP code and income may be other semi-unique identifiers for an individual. When the semi-unique identifiers such as name "John Smith", ZIP code, and income are used in combination, they may uniquely determine a linkage between two records. Among the records having name "John Smith" and ZIP code "98100," we may define a metric that allows the income to be matched if the incomes are within 5% of each other. The deviation in the metrics can often be modified according to the types of files being matched, the amount of redundant information that is available for matching, and the stated analytic properties of the public-use microdata. For instance, if we had an additional variable such as profession, it is likely we could re-identify even when the metrics allow significantly larger deviations in income and other characteristics between matching records. In record linkage (Winkler BIB003 ,  ) and in re-identification experiments (Yancey et al.  ), the software is designed to deal with both minor and major errors in a subset of the variables used in matching. In most realistic real-world settings, some of the continuous variables are available in external files and can be used in matching. Further, it is straightforward (Scheuren and Winkler  ) to define new metrics based on relationships between two correlated or related variables to improve matching accuracy significantly. For instance, we might consider the situation of linking two administrative lists of companies in which name and address information are sufficiently poor so most companies in one file might be associated with upwards of fifteen companies in another file. If we also have income on one file, receipts on another file, and a crude function 'f(income) = receipts' that relates income to receipts, then we may be able to significantly increase high quality match rates with the extra, non-name-and-address information. In the situation with a masked file Y, we may have a file Y' that only has a subset of the variables that in the Y file and the subset of Y'-variables is not sufficient for accurate matching. If we can find an extra file Z in which the Z values can be used to predict the Y variables that are missing from the Y' file, then we may have sufficient extra information for significantly improving the matching (and possible re-identification)  . Because many individuals are unable to perform sophisticated or elementary record linkage, a number of other re-identification risk measures have been defined. The first measure is the number or proportion of population uniques in a file. The measure appears to be based on elementary ideas in survey sampling for which sort/merge utilities can be used to determine matches on an exact character-bycharacter basis. A unique is a record with identifying information that distinguishes it from other records. If records are tabulated according to their identifying information, then uniques are those records that have frequency one. There are elementary models that use the frequency distribution of the identifying characteristics in the sample to obtain estimates of the number of the population uniques in the sample. The intuition is that each population unique might be identified using elementary sort/merge methods. A key observation (Elliot et al. BIB006 ,  ) is that a sizable proportion of population uniques in a file with twelve or more variables may be uniquely determined by three, four, or five variables in the records associated with the uniques. These special uniques may be more easily re-identified because the intruder may be more likely to have the smaller number of variables in an external file. There is a class of risk-estimation measures that are based on statistical models in which the number of uniques in the public-use sample is used to provide an estimate of the number of uniques in the original population file. The methods apply quite sophisticated models that relate the distribution of the sample uniques to the distribution of the population uniques. Early papers with these statistical risk measures are by Bethlehem et al. BIB002 , Fienberg and Makov BIB004 , Benedetti and Franconi  and Skinner and Holmes BIB005 . Later papers with enhanced methods are due to Skinner and Elliot  , Rinott  , and Polettini and Stander BIB010 . The apparent intent is to provide a straightforward, rapid method of estimating the proportion of sample uniques that are also population uniques. We will refer to these methods as sample-uniquepopulation-unique (SUPU) methods. There are four obvious criticisms of these SUPU methods. The first criticism is that it is straightforward for the statistical agency to determine the risk of disclosure by directly comparing the sample file with the population file from which it is produced. Indeed, determining the number of population uniques that are also in the sample file can be part of the sampling procedure. The second criticism is that a public-use file will typically contain ten or more variables, both discrete and continuous. Even if the continuous variables are broken into a large number of discrete ranges, it is likely that all or almost all of the records in the sample and many or most of the original population records will be unique. The methods that have typically been applied only use a few discrete variables under the assumption that the intruder will only have access to a few discrete variables. This assumption appears to be naïve given the amount of information that is available from Internet and other sources (Sweeney  , BIB007 ). Because most, if not all, public-use data files contain continuous variables, it is unrealistic not to include them in the statistical models of disclosure risk. The third criticism is that the statistical models only provide an estimate of the proportion of sample uniques that are also population uniques. The methods do not determine which of the sample uniques can be re-identified as would happen in a record linkage experiment (Kim and Winkler  , Sweeney  , Winkler  , Domingo-Ferrer and Torra  ). The fourth criticism is that the models are often severely biased with the bias varying according to the data source. For instance, if we were to produce a public-use file according to the following two procedures, we would obtain severely biased answers from all of the SUPU models. In the first situation, we could sample only from population uniques in producing public-use file D 1 . In the second situation, we would sample only from population records that are not unique (occurring two or more times according to identifying information) in such a manner that every record in the publicuse file D 2 is a sample unique. In each of these situations, every SUPU model would give very biased estimates of the re-identification risk. In general situations where different sampling procedures might be used, we would still likely have subsets of the public-use file where the biases could approach the two extreme situations previously described. A second metric associated with re-identification (disclosure) risk for a onevariable situation is the multiplicative inverse of the Var (X 1 ) (Duncan et al.  ) where the variance is that of an intruder with weak knowledge of the target variable X 1 . For instance, if data are micro-aggregated into a group on n items and each item is given the average value x , the re-identification risk is σ where σ is the variance of the original x-values. Trottini and Fienberg BIB008 have extended this metric to a few two-dimensional situations and have noted that the metric is different from many of the other metrics that are commonly in use. In addition to using methods such as record linkage or nearest-neighbor matching, Lambert  and Palley and Simonoff BIB001 have shown how knowledge of the analytic properties (such as regression parameters) of a masked data file can allow reidentification. De Waal and Willenborg  ,  show how knowledge of sampling weights can allow re-identification. If we have detailed information about the survey frame, the sampling design, and the valid uses of the sampled data, then the sampling weights give us useful information about subdomains in which a record may occur. If the sampling weights are combined with the original sampled data or with masked data, we may have sufficient information to allow some re-identifications. There are several research questions. In many instances, can methods such as record linkage, nearest neighbor, or clustering be used for re-identification? What data will be available on public sources such as the Internet and can be used for reidentification? Can the ideas of constructing functional relationships be substantially extended to allow re-identification in many situations? How can advanced methods such as link analysis be applied for re-identification?