We provide new estimates of an asymptotic upper bound on the entropy of English using the large language model LLaMA-7B as a predictor for the next token given a window of past tokens.
This estimate is significantly smaller than currently available estimates in,.
A natural byproduct is an algorithm for lossless compression of English text which combines the prediction from the large language model with a lossless compression scheme.
Preliminary results from limited experiments suggest that our scheme outperforms state-of-the-art text compression schemes such as BSC, ZPAQ, and paq8h.