Colleague indirectly accusing me of plagiarism? Help? I recently got my PhD degree.

2 years of my PhD were done under supervision of X under their department. The rest was done under supervision of Y in another department. When we separated and I moved to Y, there was an agreement about which artefacts I was allowed to keep to continue my studies *.

Some time ago my boss sent me an email telling me one X's employees has made allegations I have plagiarized their work. There was an email stating 5 separate "cases" where they believed I had done this. The majority included re-creating a figure representing an experimental set-up, and using software developed by X's department, and usage of artefacts I was previously allowed to use (see * above) which I had cited.

I have prepared a response document addressing every point and elaborating how I have not plagiarised any work, nor have taken credit for the work of others (the exception to this MAY be the figure I mentioned earlier, but this was unintentional and I have already contacted the publisher with the intention to amend the acknowledgment section of the paper, if possible). I have developed my own software and have derived my own results.

However, these false accusations were delivered to third parties first. I learned of this case from my boss. Why did X not speak to ME or Y (coauthor)? How do I protect myself in a case where third parties are being told I have plagiarised material when this is not the case?
I worry that you have a misunderstanding about plagiarism and what it means that you had an agreement about things you could use. Normally, plagiarism is pretty obvious on its face. The exception is when there is parallel work by people unknown to each other. 

Even the latter can be complicated in situations where it is fairly clear that the writer should have known about the other work and ignored good research practice. 

But having been given permission to use things doesn't give you permission to use them without citation. You still need to cite the source of anything you use that you didn't develop yourself. 

So, if you did some work with others, and they think they have some claim to originality, so that you obviously know of their work, and they then find a paper of yours that has the work but no citation, it is pretty clearly plagiarism. Doing so accidentally or not understanding the rules is not a defense. You should know the rules and should adhere to them rigorously. 

I don't claim this is what happened here, but it seems that others think it is. Your defense would be to show the citations or to make a claim of parallel, independent, work.  

And they don't owe it to you to tell you before they make a claim. They need to be careful of false claims, of course.