When submitting a journal article, can I cite a paper which cited my arxiv preprint? I had uploaded an arxiv paper about 4 months back which developed an algorithm. After uploading the preprint we got some good feedback from a colleague, following which we have added additional simulation results and a more detailed discussion. 

In this span of 4 months, the paper has been cited a couple of times. The original work was primarily theoretical, whereas the papers which cited our work successfully applied our algorithm (with minor modifications) to power systems. Hence, when submitting to the journal, I would like to cite these papers to show that our methods have practical application.

My question is: would this form a citation loop which is undesirable? They have cited my work, and I cite their work. Is this firstly even allowed?

Field of research: Intersection of applied mathematics, engineering, and computer science. The paper is likely to be submitted to a SIAM journal.
This depends on what the journal's policies are. But in general (i.e. unless the journal overrides this), there's no requirement that the paper you submit be exactly the same as the paper you put on arXiv. It's often the case that the last round of edits you make prior to publication won't be reflected in the arXiv version, and you could consider the addition of citations to be part of that round of edits.

I have seen citation loops in the past, and nobody raised an eyebrow about them. In fields where posting preprints on arXiv is common, it's well understood that this can happen.