yes it is normal.  my very old Dell dimension pc does the same.
My new XPS 8940 'starts up' for 2-3 seconds when I connect the power cable. Sounds like some fans start spinning fast, probably from PSU. After that the PC is off until I press the power on button. Is that normal? I don't think any of my previous PCs had that behaviour.  
XPS 8940, starts up when connecting power cable, normal?