Do you have the latest build of Win 11? Lots of complaints and seems that this is a bug in earlier builds of Win 11. Try enabling the tone for cap lock, and then look the Ease of Access screen for an option to display an alert when a tone is played...  
I have a Inspiron 5400 24" all-in-one. It does not have a caps lock indicator on the keyboard and I do not see an indicator in Windows 11. The only thing I have been able to find is an assistance tone.  Any way to see a display indicator in Windows?
Caps Lock Indicator