Yes, it is safe. As you can see in Dell's specifications, your laptop is designed to work with either battery. Don't worry about the voltage difference. You may buy genuine Dell batteries direct from Dell. Here is the page showing both batteries for your laptop.  Follow removal and installation instructions in Dell's Service Manual. Whenever touching components or working inside a computer, wear a grounded wrist strap, also called anti-static wrist strap, ESD wrist strap, or ground bracelet. It's a cheap and sensible precaution. Rest the laptop on an anti-static mat or at least a reasonable alternative such as corrugated cardboard.
So here's the situation.  I use my dell as a daily driver laptop that I take with me to university, so I need it have the best possible battery life. It's been 2 years since I bought it and, wouldn't you know it, the battery is dying so I'm looking into a replacement.  My laptop initially came with a 40whr 11.25V battery. However it appears that in the configuration page you could select a 53whr 15V battery instead, whose specific product model I know and on dell's part page is listed as compatible with my laptop. Considering the voltage increase, is it safe to instead upgrade to that battery and not replace the current one with the same part?
Dell vostro 5401 battery