Is it commonplace for a recruiter to ask to see an unpublished PhD thesis? I would have thought the document and overall findings are to be a closely guarded secret until defense or publication, so you can imagine my horror that a hiring professor would ask if he can have a pdf of my dissertation. This is in the context of a job application, whilst he decides whether or not to invite me for an interview.  I have already sent them the other standard documentation that was requested in the advert.  Is his request as unorthodox as it seems to me?
In a word, yes.  It is very common for academic employers to want to know about a candidate's research in progress, and they often ask for research plans, unpublished manuscripts, reports on ongoing projects, etc.  From the employer's point of view, they want to know as much as possible about what the candidate is doing, so as to evaluate the promise of their research program and their productivity.  This is especially true for junior candidates who do not already have a large body of published work.  So a request for a draft of a PhD thesis would not be out of line.

When a candidate shares such material as part of their application, the hiring professor or committee has an ethical obligation to hold it in confidence.  They should not circulate it beyond those people within the department who are involved in the hiring decision.  Also, it would be ethically inappropriate for anyone with access to this material to exploit it for the gain of their own research program (e.g. by trying to solve the candidate's thesis problem before they do, or giving it to one of their own students).  As the candidate, you have the right to expect that this will not happen.

Of course, as a matter of practice, if you want the job, you don't have much choice but to give them what they ask.  But I wouldn't see such a request as unusual or unreasonable, and I don't think you need worry about them using it unfairly.  If you are still worried, you could send them the thesis along with a note saying "since this is work in progress, I would ask that you keep it in confidence".

Also, I would say it's an exaggeration to say a thesis should be a "closely guarded secret" or to react with "horror" to a request to share it.  It's generally prudent not to share unpublished work indiscriminately, but it's not as if it were missile launch codes.  If there is something to be gained by sharing it with someone (e.g. useful input from an expert, a potential collaborator, a job) then often that's a good idea.  It seems to be pretty common for people starting out in academia to overestimate the risks of people stealing their work: yes, there are horror stories, but in the long run, you usually have more to lose from excessive secrecy than from reasonable openness.  Paranoia is generally not a helpful trait for an academic.