What does non-exclusive publishing rights mean? I have read on open-access journals that some of them allow non-exclusive publishing rights. What does this mean exactly? Is there a general open access policy or consensus about what these publishing rights should be?

Does this mean that I can publish my article in other journals too? Wouldn't that be considered plagiarism, or at least reduce the citations to the original article in the first journal? Does this mean that the author(s) is/are allowed to only self-archive their article on a repository website but cannot publish it elsewhere?

I have also seen this term on the website of Directory of Open Access Journals. So, I suspect that this has to do with open-access policies.
I suspect you're seeing "non-exclusive distribution rights", which means the journal gets the right to distribute the paper, but it's not an exclusive one. Other venues, such as arXiv or ResearchGate, can also distribute it.

If you have an open access paper, presumably it uses the CCBYA license, which means any use, distribution and reproduction in any medium is permissible (i.e. you can publish it elsewhere) as long as the original authors are credited. Journal licenses that only allow the authors to self-archive are more restrictive and usually indicate that the paper isn't open access.

This doesn't mean you can publish your article in other journals. Most journals will have policies that all submitted content must be novel, and if you've already published the paper it's no longer novel.