How to count tokens with tiktoken
Below is an example function for counting tokens for messages passed to gpt-3.5-turbo or gpt-4.

Note that the exact way that tokens are counted from messages may change from model to model. Consider the counts from the function below an estimate, not a timeless guarantee.

In particular, requests that use the optional functions input will consume extra tokens on top of the estimates calculated below: