Do editors allow authors to respond to reviewers when a paper has been essentailly rejected? I recently submitted a manuscript to a journal that focuses on short papers about software packages. One reviewer gave a very detailed (and helpful) critique of my software/paper with one major objection: another package already exists that has similar functionality. S/he mandated that I clarify what my package offers that the other package does not.

Unfortunately, I was not aware of the other software package when I started my project (which I'm pretty embarrassed about). On further inspection, I realize that this other package is far superior to my own in terms of functionality and performance. I've come to the conclusion that my project is not salvageable. This is disappointing but not soul crushing as it was a side project that is only tangential to my dissertation work. Frankly, I don't have time to improve my software to point of being a significant contribution, and I'd only do so for the sake of "getting a publication," which does not seem fruitful. Improvements wouldn't be immediately useful to me. and thus I don't have the motivation to submit a revision.

That said, I'd really like to submit a genuine word of thanks to the anonymous reviewer for their detailed critique of my paper/software. Through their comments and working through a revision I learned a lot (how to use Docker, sharing and recording terminal sessions through
asciinema/asciicast, general improvements to my writing, etc.) Do editors allow authors to respond to reviewers even if their manuscript is essentially rejected? It seems as though editors would generally disallow this since authors responding to a rejection may often want to say something nasty. Would it be best to email the editor directly with my request or respond through the submission system?
The most likely scenario is that the authors can respond to the editor, and the editor will decide (at their discretion) whether or not to pass back the message to the referees.

After all, there are valid reasons to message the referees afterwards (for example if the main argument for rejection is based on a misunderstanding). For a simple thank you, it will depend on the editor's preference, but it definitely doesn't hurt to try.