What is the point of a non-confidential reference letter? In this question, it seems that most people think it is acceptable for candidates to see their references. I find this very strange, as in the vast majority of case I have come across it is expected or required that all references are confidential.

What is the point of having a reference that the candidate is able to see? 

The person reading such a reference has no way of telling whether any praise is genuine or simply included to please the candidate. Likewise, the chances of negative points being included are minimal, even if they are very important. Thus the reference becomes little more than a vague sign of support. I understand that in business that is generally sufficient, but in my experience academic references are relied on much more heavily to discriminate between candidates.
From the perspective of UK higher education, the 'point' of a non-confidential reference letter is that there is no such thing as a confidential one; any data held on that student (such as references) can be disclosed by the institution holding it.

How does the old saying go? Dance like no-one's watching, writing an email like it's being read aloud in a deposition.

References are a tricky grey area, and I do preach caution in case a student ends up seeing what you have written - which can be embarrassing, or even libelous if you can't prove statements that purport to be truth.

I do not think that this undermines the referencing process, however; if you can make true statements about a candidate, including your opinions about them, then, in my opinion, you are within your right to do so.  After all, your student has asked you to do this.