Is it really the responsibility of a conference organizer to provide the necessary letters to the participants for their visa processing? I got a conference acceptance and my institute registered that. All the expenses to attend the conference were also approved. 

When I checked with the respective embassy for visa processing, they asked for a letter of reason from the host country/conference.

Upon request to the conference representatives they excused. When I told them explicitly, once with a sample from some colleague of what I would need and then with the exact form from the official website of their embassy, then the response was, yes we would try our best to prepare it. However, if we couldn't, you can not blame us because the organizer has not promised any such assistance. 

Such statement was a shock for me. To me, it was as if, I invite someone to my home, once he is at the door, I tell him, I did not promise you to open the door. 

I am not sure if my assumption is wrong or what? maybe that's almost out of their domain but other conferences do prepare these documents. 

So my question is: 

Is it really the responsibility of a conference organizer to provide the necessary letters to the participants for their visa processing?

In any case, how to deal with such an issue/response?
Is it their responsibility?

Yes. I've run into this issue before, but I have never had a conference organizer demur on writing the letters. Not only is it the responsibility of the conference organizer, strictly speaking they are the only person who can provide you with the appropriate letters, since the letter is the justification for your attendance. Someone else cannot write you a letter saying "Tod is attending because of some other conference."

What should you do?

Don't go. On the one hand this is a moot point, because if they don't write the letter, you won't have the visa. On the other hand, the unwillingness to write a letter leads me to believe that this is not a genuine conference. There is an industry of essentially fraudulent journals and conferences. Many of the conferences are held in places like China, I suppose so that people have an opportunity to travel there and because it is relatively cheap, which require letters to travel. Not getting a letter is a huge warning sign.

It is undoubtedly disappointing, but if the conference is not going to do the absolute minimum required, and it sounds like you are paying, you are almost assuredly better off not attending.