How can I do research mathematics without enrolling at a university? My goal is to find the general form for the asymptotic cardinality of a compound arithmetic progression.  I'm not affiliated with a university and I certainly don't have any obligation to continue my research.  However, it is very fulfilling to me.  I spend my time in the library; I study through the books on the shelves and I don't really have anyone to tell me that my efforts are not good enough.

The problem comes in when I realized that if I was stronger in Analytic Number Theory, then I could start to look for the general estimate for the asymptotic cardinality of any compound arithmetic progression, and I'm terribly curious about how it works out.  I am jealous of professional students with academic affiliations, access to reviewers, and journals; terribly.  I admit that and it's a big obstacle.  And I'm envious that they had the opportunity to enjoy more prestigious settings.  I wish I could get a foot hold like that too.  But at the same time I just don't think it's going to be possible for me to be published or be respected on the same level, no matter what I do.  

Is it healthy for me to behave like this?  Or am I being too optimistic in thinking that I have something to contribute that could blossom into graduate level, age appropriate research and/or a funded project?

The point is that I've given up looking for a degree or trying to participate in the research projects of others; it places the burden on them to find a place for me in their work and I'd rather be sharpening my own offering instead of going on a wild goose chase.  Math doesn't require the same kind of budgetary overhead, or anywhere near the same level of oversight as the life sciences.  I don't need a lab.  I don't need dangerous chemicals or expensive equipment, and I don't need a workshop, either.

The other question (of which this was marked as a duplicate) was asked by a much more established individual that already finished a Master's degree and wanted to collaborate on somebody else's research; putting together a CV to apply for a research spot is much easier when there's something to anchor it.
If I understand you correctly, then you long for the access to journals and fellow researchers coming with a university affiliation, but the intensity and committment of being a graduate student has turned out to be bad for you.

It is uncommon, but not entirely unusual, for individuals to be associated with a research group at a university on such level. This might come in some formalized arrangement, eg with a title like "Honorary Research Fellow", or just be handeled by some informal arrangements. Common perks would include access to mathematical journals via the university and being on the mailing list for the local seminars. You may or may not be permitted to claim the university as affiliation when submitting papers, but I believe that this is a very minor aspect anyway.

If you can find a research group, ideally in your geographical area, that is interested in your kind of questions, then it would not be unreasonable to contact them and ask whether this is a possibility. Do brace for the potential of rejection (or being ignored) though.

Whether you personally have a change to produce publishable research in such an environment, or whether this would be benificial or detrimental to your mental health is something we cannot judge here.