I managed to solve this by removing the English United States - International keyboard layout and keeping the English United States one.
Is anyone facing an issue with the quote key and the backtick key? They don't register on the first press. On the second press, they register twice. Very frustrating.
XPS 17 9720 keyboard issue