Q:

Why is it so hard to convince professors to write recommendation letters for me?

I am trying to shift graduate schools to change subjects and over the last year I have been studying the new subject in my current graduate school. I have good grades and in collaboration with a few professors and over the last year, I have been working on an insanely hard research question in this new field. The progress is invariably slow. 
But I am unable to convince any prof to write recommendation letters for me to do this shift! I don't understanding this. (Does it not count that I started a new subject directly at the grad level and could get good grades and start attacking one of the hardest questions in that field?) 

One prof says that if another professor at a different place wants to work with me as their grad student then that prof can contact this prof here to talk about me. Should I take this option in lieu of not getting recommendation letters?  
Officially some profs say that they are not comfortable writing letters for someone who doesn't come with the usual background of having done an undergrad in this new subject. (somehow this gap is not offset by me having done some of thee advanced grad courses in this new subject) 

A:

In general, when you ask someone to write a letter of recommendation for you and they are unwilling to do so then it is likely that the reason they are unwilling to write a letter of recommendation is because they don't feel that they can write a positive and helpful recommendation letter for you and they don't want to write a negative recommendation letter.  
There are many reasons why a letter writer might not be able to write a positive recommendation letter for you, including that they don't think you're well prepared for the program, they don't feel that they're qualified to recommend you for a program outside of their area, they don't feel that your record is sufficient to gain admission to the program in question, or perhaps that they have doubts about your intellectual ability or character.  
In any case, it is almost certainly not in your interest to continue to press for a letter of recommendation from them, since you can be pretty sure that the resulting letter of recommendation will not be positive and won't help you in gaining admission to the program that you're applying to.  
In particular, you should be aware that your professor could be much more candid (and negative) in a telephone conversation than they might be in a written letter of recommendation.    

