Should I index all the specific terms which are included in appendix? I am writing my master thesis (Mathematics field) and I would like to know if I should index all the specific terms which are included in appendix.

Is it necessary?
Institutions that I am aware of generally do not require an index because building a good, useful index is very difficult.  Also, these days people will typically just search for terms in a document rather than looking them up in an index.

The question that I would ask instead is: what is the value that you are trying to get from an index, and is there a better way to get it?  In heavily mathematical documents, I find it very valuable when the author includes somewhere a table of symbols and/or terms, which I can refer to when I'm wading through equations and trying to remember what on Earth v-prime-double-hat and w-double-prime actually mean.  I especially appreciate this when I am the author, and I need to look back at the document a few months later after my brain has flushed its cache.

So if you want your thesis to be accessible to readers, I strongly recommend adding tables of symbols/etc.  As to where: you can put them anywhere you want.  I personally prefer sticking the table right at the beginning of the first place where the symbols start to show up in earnest, both as being most proximate to the need and as a warning sign to the reader indicating heavy going ahead.