How to count tokens with tiktoken
3. Turn text into tokens with encoding.encode()
The .encode() method converts a text string into a list of token integers.

encoding.encode("tiktoken is great!")

[83, 1609, 5963, 374, 2294, 0]
Count tokens by counting the length of the list returned by .encode().