Will my stressful year be seen as a negative? I am an undergraduate student in mathematics. My academic performance is really good but many people tell me "you are too stressed up", including people who know me for a few weeks.

Due to some serious family issues the last few years have been hell.

As the time passes everything is returning to normal but the professors who don't know about those events might think this is how my personality is.

I am going to ask letters of recommendation from those professors for a graduate school application. Do you think this will negatively impact my application? What I can do about it?

edit: It's just the air around me. Some professors mentioned this: "You are too tense, don't worry you will go alright". Not sure how to explain this better.
"Stressed out" could hurt you insofar as people who would give you otherwise positive recommendations might be tempted to tone them down. 

When asking for letters of recommendation, discuss these issues candidly with the prospective recommenders. Some might not care. Others might "care" but they might also be open to a discussion of extenuating circumstances. A few will have their opinions shaped by your behavior. If and when you identify such a person, you might want to withdraw your request for a recommendation from them.