Do academics look down on well-designed academic websites? Rationally, I should not be concerned about building a well-designed academic website for myself. But academic websites are consistently so visually spare and poorly designed that I would feel like a peacock designing the site in any way. I don't mean anything crazy, maybe a web font or a non-#0000ff link.

I realize this is broad, but I'd be curious if others feel this stigma of trying too hard in this area—same probably applies to clothing—and how they approach it.
As has been stated several times, there is no such thing as a site being "too well designed."  Of course the design should take into account the audience that will be viewing the site—i.e. if you are designing a website gallery for minimalist paintings, it would be very odd indeed to use as many buttons and links as Amazon.com, whereas for an online store that isn't a bad design at all.

However, in my view, the keynote is really not so much design as implementation.

There are two crucial factors: file size (also known as website obesity), and overuse of scripts (which is closely related).

The website you linked in the comments doesn't load at all in my browser, because I have scripts disabled by default.  I enable them for sites that I expect to run scripts, such as gmail or youtube.

I grant that I may be in a minority of the general web browsing population...but I expect a disproportionate number of academics are aware of such things, and are more likely to be irritated by websites which require plug-ins, scripts and high bandwidth simply to display a bunch of text.

Do your design as well as you please, but try to keep it as minimal as possible on bandwidth, plug-ins and scripts.



(Note: My own personal settings aren't the main topic of this answer, but as several people have made noises of shock at my default blockage of scripts, I'll clarify: I don't block first party scripts that are served by the actual website I'm visiting.)