Throwaway account

I'm in college, but I still live at home with my parents and my two brothers, the youngest of which has autism. He can't speak, but he was an okay kid until puberty hit him. Now he's gotten more aggressive, yelling and screaming and putting holes in the wall of his bedroom. Therapy and medication isn't helping, and I can't stand the yelling. I tried to be patient with him and ignore him but he pushes me so far that I start screaming back at him. It's gotten so bad that I had to start wearing earplugs whenever I go downstairs, and I spend most of my free time in my room. Sometimes, when my parents aren't home, he targets me specifically for no apparent reason and screams up the staircase to my room until I either lose it and start screaming back or I go to my car and leave. Last time it happened, I just locked myself in the bathroom and cried. Despite my parents being in the same boat and arguably have it worse than me because they're responsible for him, they obviously don't know how to help me as their advice to me either boils down to "ignore him" (which never works) or "calm down" (which upsets me more). My brother shows no signs of improving and I doubt he'll ever talk at this point. It's gotten to where I can't stand living with him and want to leave home. My mom picked up on this and started crying to me about how she doesn't want me to leave because of my brother and that I would be throwing away the rest of my childhood if I do because then I'll have to step into the adult world. I broke down in front of her and told her I wouldn't leave but I felt manipulated afterwards and was angry because she wasn't taking my feelings into account. Now I'm calling those feelings into question. Maybe she wasn't trying to manipulate me and just wants the best for me. But on the otherhand, I feel like my childhood is already gone. Am I selfish for wanting to leave?