<commit_before>
# Singletons, Cluster-Robust Standard Errors and Fixed Effects: A Bad Mix

(or why `reghdfe` will drop singletons from now on...)

Note: [PDF version with math support available here](http://scorreia.com/reghdfe/nested_within_cluster.pdf)

Singleton groups (groups with only one observations) are increasingly common in regressions with many fixed effects, such as regressions with worker/firm/job title fixed effects that were previously unfeasible due to computational limitations [(see e.g. Carneiro *et al*, 2012)](https://www.aeaweb.org/articles.php?doi=10.1257/mac.4.2.133) . Even though some users may drop them, most are not aware that with more than one fixed effect, singletons need to be dropped iteratively, as e.g. dropping a singleton individual may in turn turn the firm he worked into a singleton, and so on.

Now, what are the effects of keeping singleton groups in regressions where fixed effects are nested within groups/clusters?

1. Coefficients and unadjusted variance estimators don't change.
2. Robust and cluster-robust estimators of the variance decrease due to finite-sample adjustments converging to 1. Note that the asymptotic part of the robust variance estimator (the usual "bread and meat" of the sandwich estimator) remains unaffected, so this is as problem only as long as finite-sample adjustments are relevant (which is the case more often than not). Therefore, **standard errors will be underestimated, and statistical significance will be overstated**.
3. The reported number of clusters will be overstated, potentially misleading users into believing that there are enough clusters to make accurate asymptotic inference (e.g. above 50 clusters).
4. Estimations will run slower, as there is a larger number of ancillary parameters to estimate.

## Finite-Sample Adjustments

Given an estimate of the asymptotic variance of the regression estimates ($V$), $M$ clusters, $N$ observations, $M$ fixed effects (one for each cluster group), and $K$ regressors of interest, then the finite-sample correction that multiples $V$ is:

$$
q = M / (M-1) \times (N-1) / (N-K)
$$

If we add $M_S$ singleton groups, the above becomes
$$
q* = (M+M_S) / (M+M_S-1) \times (N+M_S-1) / (N+M_S-K)
$$

Since $q*$ converges to $1$ as $M_S$ grows, adding enough singleton observations is enough to deem the standard finite-sample corrections moot.


## Toy Example

As an extreme but illustrative example of the first problem, consider the following regressions using the sample Stata dataset:

```stata
* Create toy data based on auto.dta
sysuse auto, clear<commit_msg>Updated nested_within_cluster.md<commit_after>
# Singletons, Cluster-Robust Standard Errors and Fixed Effects: A Bad Mix

(... or why `reghdfe` will drop singletons from now on.)

Note: [PDF version with math support available here](http://scorreia.com/reghdfe/nested_within_cluster.pdf)

Singleton groups (groups with only one observations) are increasingly common in regressions with many fixed effects, such as regressions with worker/firm/job title fixed effects that were previously unfeasible due to computational limitations [(see e.g. Carneiro *et al*, 2012)](https://www.aeaweb.org/articles.php?doi=10.1257/mac.4.2.133) . Even though some users may drop them, most are not aware that with more than one fixed effect, singletons need to be dropped iteratively. For instance, in a matched CEO-firm regression, dropping a singleton CEO may reduce the observation count of the firm he managerd from 2 to 1 observations, turning that firm into a singleton group, and so on.

Now, what are the effects of keeping singleton groups in regressions where fixed effects are nested within groups/clusters?

1. Coefficients and unadjusted variance estimators remain unchanged.
2. Robust and cluster-robust estimators of the variance decrease due to finite-sample adjustments converging to 1. Note that the asymptotic part of the robust variance estimator (the usual "bread and meat" of the sandwich estimator) remains unaffected, so this is as problem only as long as finite-sample adjustments are relevant (which is the case more often than not). Therefore, **standard errors will be underestimated, and statistical significance will be overstated**.
3. The reported number of clusters will be overstated, potentially misleading users into believing that there are enough clusters to make accurate asymptotic inference (e.g. above 50 clusters).
4. Estimations will run slower, as there is a larger number of ancillary parameters to estimate.

## Finite-Sample Adjustments

Given an estimate of the asymptotic variance of the regression estimates ($V$), $M$ clusters, $N$ observations, $M$ fixed effects (one for each cluster group), and $K$ regressors of interest, then the finite-sample correction that multiples $V$ is:

$$
q = M / (M-1) \times (N-1) / (N-K)
$$

If we add $M_S$ singleton groups, the above becomes
$$
q* = (M+M_S) / (M+M_S-1) \times (N+M_S-1) / (N+M_S-K)
$$

Since $q*$ converges to $1$ as $M_S$ grows, adding enough singleton observations is enough to deem the standard finite-sample corrections moot.


## Toy Example

As an extreme but illustrative example of the first problem, consider the following regressions using the sample Stata dataset:

```stata
* Create toy data based on auto.dta
sysuse auto, clear
{'fraction_non_alphanumeric': 0.0718708951022706, 'fraction_numerical': 0.010508538187277162, 'mean_word_length': 4.762162162162162, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7222349', 'n_tokens_mistral': 1502, 'n_tokens_neox': 1362, 'n_words': 740}