Can I send my journal article (under review) for a invited Elsevier book (not-to-be-refereed) chapter? I just got an invitation from an editor of a non-refereed book. Is it considered self plagiarism if I send one of my journal article to the book? 

The book is hard to define. It is more like a textbook rather than a review volume.
If you had published your paper on a journal and you had transferred the copyright to that journal, it's a violation of copyright. And typically you transfer the copyright at the time of submission. 

If you want to republish your paper somewhere else, you should first ask permission to the copyright owner.

Beware, moreover, that most of the invitations to "publish your research" in a book come from predatory publishers. In case, you don't want to send them any of your works.