Will time off spent doing things not related to my research interests count against me when applying to PhD programs? I recently finished my undergraduate degree in physics and I currently have a year-long research internship. My original plan was to apply to graduate school (physics PhD) in the fall. However, the more I think about it, the more I'd like to take additional time off to pursue non-physics related interests (think: traveling, creative writing) and also to spend time at home with family (I grew up in a small-ish city where I doubt I could get a job related to my research interests).

I have been told that I have a pretty decent shot at getting into top-tier graduate schools (4+ years research, honors thesis, > 3.9 GPA, etc.). As has been pointed out previously, there seems to be a consensus that taking time off and getting experience related to your field is viewed positively/not negatively. However, my question is, would taking additional time off for completely non-science related pursuits significantly decrease my chances of being accepted to competitive PhD programs?
Physics isn't my field (Mathematics), but I can't imagine a program that would think less of you for taking time off. 

The danger is that if it goes on too long you lose your "edge" and get "rusty" both at Physics and the academic life in general. 

However, there is little worse than starting an intensive graduate program facing any sort of burn out. You want to start out with a maximum of energy and enthusiasm because the road ahead won't be easy. Your credentials are good, but the competition, if it exists, in grad school will be from people just like you - the best. 

In my own case, I might have finished the doctorate earlier than I did had I taken a year to unwind from a very intense undergraduate program. 

Finally, if you take time away and then apply, make sure you are ready with a good answer if asked why. Life is more important than Physics (for most, anyway).