Should I let my recommendation letter writers know that I didn't submit my application in the end? I feel guilty and not sure of if I should inform my recommenders of this issue. 

A school I intended to apply has received the reference letters from all my recommenders, but as the deadline was approaching I just found that I could not prepare something required by the school in time! So, considering the amount of application fee, I decided to close my application.

Now I feel maybe I should have told my recommenders about this issue, yet, whenever I was about to write a sorry email then I withdrew my draft. 

In such a situation, am I suggested to say sorry to my recommenders formally?
This really depends on the situation. If you only planned to apply to the one school, and ended up not applying, then you've wasted the time of your references, as they did not need to write the letter at all, and could have used the time for other projects.

On the other hand, if this is just one out of some number of schools, then the issue is not as severe. The extra investment to write the "last" letter is not that great. While it is unfortunate that you might not apply, such things happen. 

In either event, though, it may be worthwhile to let them know you didn't submit the application—they may ask the school directly if they have contacts, and it wouldn't be useful to find out from the school that you didn't apply, rather than from you personally.