The current jittering logic in [kubelet token_manager](https://github.com/kubernetes/kubernetes/blob/dd68c5f2409fec7176e6172d6f9d97bd6447c4da/pkg/kubelet/token/token_manager.go#L42) refreshes the token once it has 20% of its lifetime remaining, with 10 seconds of jitter.

Experimentally, this doesn't seem to be enough to really blunt spikes of refresh requests that come from many workloads having a synced refresh.  This can occur when:

* A large deployment is scaled up.  All the new replicas will have synced-up refresh traffic.
* A kubelet restarts.  Because kubelet only holds the tokens in an in-memory cache, it will need to acquire new tokens for all pods on the node after it restarts.  After that, all of the pods will have synced-up refresh traffic.

As an example, here's serviceaccount/token API traffic from a real cluster.  Note how there is a repeating spike every 48 minutes, which is what we would expect from the "refresh after 80%" logic, and the default setting of token lifetime for 1 hour.  Effective jittering logic would be noticeably smoothing these peaks.

We should increase the `maxJitter` value.  Perhaps 5 minutes would be a good value.

![image](https://github.com/kubernetes/kubernetes/assets/1017202/133a6c5c-5a6b-4ad1-a1e8-ffb29fae3059)

/sig auth
/kind bug
Improve the jitter value for kubelet's serviceAccountToken projected volume source
This is a GitHub Issue
repo:kubernetes
owner:kubernetes
Title : Improve the jitter value for kubelet's serviceAccountToken projected volume source
Issue date:

--- start body ---
The current jittering logic in [kubelet token_manager](https://github.com/kubernetes/kubernetes/blob/dd68c5f2409fec7176e6172d6f9d97bd6447c4da/pkg/kubelet/token/token_manager.go#L42) refreshes the token once it has 20% of its lifetime remaining, with 10 seconds of jitter.

Experimentally, this doesn't seem to be enough to really blunt spikes of refresh requests that come from many workloads having a synced refresh.  This can occur when:

* A large deployment is scaled up.  All the new replicas will have synced-up refresh traffic.
* A kubelet restarts.  Because kubelet only holds the tokens in an in-memory cache, it will need to acquire new tokens for all pods on the node after it restarts.  After that, all of the pods will have synced-up refresh traffic.

As an example, here's serviceaccount/token API traffic from a real cluster.  Note how there is a repeating spike every 48 minutes, which is what we would expect from the "refresh after 80%" logic, and the default setting of token lifetime for 1 hour.  Effective jittering logic would be noticeably smoothing these peaks.

We should increase the `maxJitter` value.  Perhaps 5 minutes would be a good value.

![image](https://github.com/kubernetes/kubernetes/assets/1017202/133a6c5c-5a6b-4ad1-a1e8-ffb29fae3059)

/sig auth
/kind bug
--- end body ---

None