Is referencing another PhD dissertation or Master's thesis a reasonable practice in a Master's thesis? Not much literature can be found on what I'm working on and it's all by the people I work with. Am I able to reference their work/text in my thesis or I should just stick to referencing material published in journals and conferences, however remote they might be and just get away with whatever's not published and not back them up?
Yes, you are able to reference their work in your text, as long as you make clear in the bibliography what type of document (master thesis, PhD thesis, Institution) it is.

However, i would not consider it to be ideal if your work critically depends on such references in the sense that a central assumption or starting point in your thesis breaks down if the reference provides false or incomplete results. Particularly in the case of master theses / dissertations it is not generally clear whether and to which extent they are peer-reviewed and what their overall trustworthiness is. With (accepted!) PhD theses things are better, as these normally go through a review process. Furthermore, for PhD theses i would expect that their main points have been published somewhere (conference or journal) and then you clearly should prefer these publications.