Do non-academic extra curricular activities matter for grad school applications? I just got into a PhD program and was asked to give a presentation to my juniors regarding grad school applications.

In my country (Asia region), people are fanatical about non-academic extra curricular activities that are totally not related to their topic (sports, debate team, run charities etc). E.g. A student with good grades and is the captain of the school's basket ball team can get picked for a mathematics PhD scholarship over the top mathematics student with zero non-academic activities. The belief is that scholars should be "all rounded".

I think the situation is different in the USA. I would like to encourage my juniors who have excellent academic records but without significant non-academic extra curricular activities to try for the USA because of this.

I understand there are scholarships like the Rhodes which do look at non-academic achievements. But is my general understanding of non-academic activities for USA PhD admissions correct?
In general. non-academic activities are not considered for admissions for schools and most fellowship programs in the US. The primary reason for this is that graduate programs are looking for people who will become excellent scholars and researchers (and sometimes even good teachers!). Success in sports and other highly non-academic activities is less likely to have significant influence in assessments of one's ability to do research, and therefore is not strongly considered. 

However, there are exceptions. People can choose to fund activities however they choose, and can place whatever restrictions on the use of the money they donate. For instance, an alumnus could fund a fellowship for PhD students who play the tuba in a college marching band. While this doesn't happen often, it can be done. But typically admissions committees aren't concerned with such issues, and don't take them into account in making admissions decisions.