Q:

What are the standard responses to this criticism of utilitarianism?

Might not someone else's utility or preferences about what people do infringe upon what's ordinarily taken as right to liberty or privacy? For example, if there were someone who just found it extremely undesirable or distressful that women leave their houses, then isn't it conceivable that, say, a neighbor might be acting immorally if she left her house, just because whatever gain she got would fail to balance someone else's pain at seeing her have that freedom?

A:

Utilitarianism is self evidently difficult to bring to bear on two person hypotheticals such as this: weighing one person's happiness against that of another is, especially without context, rather hard. Standard approaches to dispensing with such problems fall into two broad categories.
Deontological rules of thumb 
Many utilitarian philosophers, such as Mill in Utilitarianism, use utilitarianism as a basis for 'rules of thumb'as a way to avoid such difficulties (see Wikipedia's Rule Utilitarianism page). A variation of the so called golden rule of ethics is (quite) easily derivable in these terms and dispenses with the preference of the prurient neighbour as non-reciprocal or unsustainable by the whole of society.
Specific analysis of the situation
If we looked closely enough at the situation, we can argue that the situation in which the neighbour's increase in happiness at the lady being a shut-in outweighs her increase in happiness at being able to escape is impossible. References to the heirarchy of needs can provide a general basis for rebuttal, but the existence of other solutions (perhaps the neighbour could move, if it is so great an issue!) is also an option.
Nonetheless...
There remains the theoretical possibility that the nosy neighbour is a utility monster of some kind, and in spite of appeals of the latter kind ('Surely noone is actually a utility monster!'), the possibility is very difficult to dismiss a priori and such thought experiments can remain a problem for utilitarianism as a foundation for meta-ethics. 

