Do graduate school admissions committees care about an official bachelors degree, or just the relevant coursework? I am a second year undergraduate in in Math+CS interested in applying to some CS graduate programs (in the US). 

Since I began my degree I purposefully put off taking a series of general education requirements with the intent of instead moving quickly through my main CS and Math coursework so that I could begin doing undergraduate research as soon as possible. If I continue this strategy then by the end of my third year I would have finished all the core coursework for both a Math and CS degree, along with a significant number of graduate courses in both departments. I also am actively working on some research projects which might lead to publishing as a first or second author in a relatively good journal, and some good recommendation letters. 

Overall I feel like I could have a pretty competitive graduate application by next fall, except that I would not have the necessary general education requirements to be officially granted degrees by my university. 

Would a graduate admissions committees be at all concerned that I did not officially complete my degree? Are there any reasons to hold out on applying until my fourth year (other than the possibility of taking more coursework, doing more research, etc.)?
Short answer: You will almost certainly need an official bachelor's degree.

Long answer: Your main obstacle here is not the admissions committee, but the requirements of the university you'd be applying to. Any university in the US surely has it written down officially in multiple places that Masters and PhD students must hold a bachelor's degree (or its foreign equivalent) before enrolling. I believe doing otherwise may even get the university in trouble with their accreditation board. 

Even if there are exceptional circumstances under which this rule can be broken, the university would have to really want you. "Pretty competitive" won't cut it. And indeed, why (from their point of view) should they admit you now if they could admit you in a year, after you have more training under your belt?