What counts as Research Experience? I studied physics at a German university and encounter frequently in application regulations for universities from abroad (US/CA/AU) the wish to have research experience. I don't really know what is meant by that and guess there might be some cultural differences and suspect that many universities might generally not have a well designed curriculum. I personally did quite a lot of research intensive coursework. Particularly I did about 42 lab exercises with 12 being exercises with advanced difficulty concerning topics of modern physics. Both of my thesis projects required me to team up with local research groups and independently research and work on a suggested problem. Further I did seminars in which I was trained in scientific writing and presentation and was required to spend about an half year on writing a research proposal and getting it examined to get allowance to work on my master's thesis. I think my university did quite a good job at preparing me for research. But otherwise, I did not do any research outside of university. Also no publication. Both are not very common in theoretical physics before PhD studies. I would be glad to hear your thoughts on this.
Everything you have named count as research experience and are good enough for applications in graduate programs. At the moment your master thesis is the most important proof/measure of your research experience.

The main goal of a phd program is to train you to be an autonomous researcher. With that in mind, they are not expecting you to have worked as a researcher outside the university before achieving a Ph.D degree. There are students with some luck that are able to publish an article before the phd, but that is not common.