Is it a bad sign if I do not get anything from problem discussions with professors or fellow students? This is a question specifically about mathematics.

All math people I've talked with like discussing problems (or math in general) with others, and most people claimed that they learned most from communication rather than from reading textbooks. My situation is, however, different. Most of the time I have math-related conversations, I find that I'm not comprehending what I'm being said, even if it's something not hard (unless I'm familiar with the topic being discussed very well). [In particular, this is true of lectures. However, I am able to understand my notes.] (I'm not sure what the reason is.) In contrast, I learn much more from books and, more importantly, from posting questions on math stackexchange. I was wondering whether this is normal, and whether this will be an obstacle in writing a potential PhD dissertation, since in that case my adviser would probably be the primary source of information, and I wouldn't be able to write a math stack exchange post asking for help.
I suspect I have similar issues as you: when others talk about complicated subjects, I find they're speaking in English, but they're also not speaking in English (because I don't understand them).

I've thought about this a bit and although I am by no means an expert, I think the cause of the issue can be understood with the VAK learning style model. (Note that this model has been heavily criticized; however my perspective is whether it's correct or not is unimportant if it works for your particular case). Basically, people can be very broadly divided into three groups: those that learn best by doing something, those that learn best by listening, and those that learn best by reading. You are good at learning by reading, but poor at learning by listening. If you go to a lecture where the lecturer does not prepare lecture slides, only stands there and talks, you find you struggle. Comparatively if you do get lecture slides, then everything makes sense and you learn smoothly. If you can have only the lecture slide or the talk, you would much prefer the slides.

The bad news is, being bad at learning by listening is of course detrimental. It's just so much more convenient to explain something instead of get out pen and paper. The good news is, if you know what the issue is, you'll able to compensate for it. You'll need to concentrate on these conversations because if you drift, you won't be able to recover easily. Don't be afraid to say "can you repeat that?", and don't be afraid to ask for clarification if something is not immediately obvious to you either. You can also try bringing around a notebook and immediately writing down something you hear, since you are liable to forget quickly otherwise.

If you are so bad at learning by listening that face-to-face meetings with your supervisor are unproductive, try telling him that and suggesting email instead.