Footnote 24: This decrease is in accuracy, not bits; a model may have a large amount of extractable knowledge in bits but not in accuracy. One can also compute knowledge _bits_ in the extractable setting, but we omit such results for brevity.