Recommender's affiliation I am applying for graduate programs this year. My undergraduate major is physics while my intended specialty is astrophysics. I apply for physics program for the most of schools and astronomy program for the others. One of the main factors which led me to apply for physics program rather than astronomy was the fact that there are much less student openings in astronomy graduate schools, due to its small size of department in general.

I am thinking of 4 recommenders in my mind, and three of them is faculty in astronomy (while two out of the three are theoretical astrophysicists)

I wonder how much the affiliation of recommenders would matter in the consideration of LORs. i.e. Do you think it is `inappropriate' to apply for physics graduate school with three letters from astronomy faculty? 

(This concern recently came up with since I read the requirement of one physics program that at least two of the LORs should be from professor in physics.)
I think it matters much less than you think. A recommender speaks for your past work and for your future potential. The specific field is less important than that they respect you for both aspects. The other elements of your application will show your basic suitability, such as courses taken and such. But the recommenders need to assure the admissions committee that you are serious and hard working as well as adaptable and creative. The specific field matters less. 

Their specific field isn't all that they know. People who specialize in some field also know some things about the requirements of other related fields. So, the examples you cite seem less important than what they will say about you. 

A good letter from a person, in a related field, who is widely known and respected, can be a very positive thing. 

But even a recommendation from a Language professor can be positive (wildly different field) if it supports your suitability for study and making progress. "This person can really think and write" is a good thing to have said about you. (Don't take this as a suggestion that all your letters should be from "wildly different fields", of course)