Hobby during PhD studies has taken its toll, how do I recover? In the first two years of my PhD I made a big mistake by spending a large amount and passion on a hobby (making video games). In this hobby, I saw an opportunity to make this into a career, either by being a full developer or by finding a way of fusing game development with my scientific work. 
Looking back at this, I feel guilty for the amount of time and passion spent on this side project, and I understand now that my career ambitions were poorly researched and far from reality.  

I am now in the final year of my PhD in applied maths and I have one paper submitted, another (more ambitious one) in the making. My relationship to my supervisor has been good from the start, however his satisfaction with my work dropped in my second year and he criticised me for somewhat disperse work style and not enough results. He is still supportive but never misses an opportunity to remind to hurry up. But he has remaind supportive throughout. 

I reacted to these warning signs, but slowly. Towards the beginning of my third year I realised that my game development efforts (which have a scientific bend) could not provide a full time career and did not provide better solutions to existing science problems either. So I gradually stopped coding games, focusing on my research project 100%. I finished my first paper, gained a feeling of ownership for my research project, got my scientific curiosity back, adopted a get-it-done mentality in terms of writing up results. But still, every day I keep hating myself for having wasted so much time on an ill-thought, ill-planned childish dream. 

This affects me. I have had a very good work ethic which got me into top schools. It feels like for the first time, I slipped, and I bet more than I should have on the wrong horse. I think about it every day. I want to avoid this feeling of guilt to slow me down, or make me feel that I am not suited or "don't deserve" this or that postdoc position or fellowship.

How do I recover from this?
There's an obvious simple answer to this, that isn't going to even take two sentences:  your game development stint was research.  Research into whether gaming concepts and/or game-style interfaces were useful for certain kinds of activities, and also a "basic training" for you in core computing concepts that will never be irrelevant in your ongoing pursuit.

Professional coders have MOUNTAINS of code that are discarded, serve no useful ongoing purpose, are total feces, and/or seem to be in a content domain that bears no relevance to their work.

Myself personally:  I have about 10k lines of code growing mold in a drawer somewhere for a dartboard app. It isn't worth spit, but the javascript I write NOW for the biggest banks in the world is quite better because of it.

You also discovered that you like your profession more than you like gaming.  That's NOT AT ALL a trivial result.