Should I be stressed about asking for an Academic Reference? I need a academic reference for application of a PhD program.
I believe this is the same as what is called a Letter of Recommendation, at other institutions.

I am getting one from my potential supervisor, and I need one other.
I thought through all the lecturers I had, and came up with a priorities list of people to approach. Taking into account (in rough order of priority): How likely they are to remember me, How relevant their units/expertise are to my intended course of study,  How well I did in their Unit/s, and How research/project focused the Unit/s I did under them were.

The person I found most suitable is currently lecturing me, and has done serveral times before.

I figured I would basically send him an email saying:


  "Looking at continuing my studies on X, that I mentioned to you
  before, with a PhD. For this I need a academic reference. I thought
  you might be suitable because of (reasons outlines above). Could we
  meet to discuss this some time in the next few weeks? I am free on
  Monday mornings, Tuesday afternoons ..."


I thought, fairly casual, not really a big deal kinda thing.
But then I got looking around the internet, and it seems like maybe it is a big deal.
There seems like there is lots of advice and template letters out there.

So is it actually worth worrying too much about?
Should I spend say a hour or so checking over my request email,
to make sure I am giving all the right information and that it is formatted nicely and perfectly grammatical?

Or is it more of a casual thing, the email serves its purpose to give the key points of information and to arrange a meeting. It will be a throw away communication.

Is it different because he is currently lecturing me, so it is not so much of a cold call?

If it is cultural depended I am in Australia. I am applying to the same institution I am currently doing my undergraduate/honours at.
If the person who you're asking for a letter of reference knows you well enough to write a good letter of recommendation, you don't really need to be worried about the formality of the request. You should be able to ask: "I would like to talk to you about writing a letter of recommendation for me. Could we set an appointment?" or something similar to that, and it should be fine.

Remember: writing letters of recommendation is one of the responsibilities of faculty members (and other instructors)!