Is it possible to change PhD department, but keep supervisor if he has joint appointment in the other department? I am  in my first year at grad school, and I am thinking about changing from EE/CS to BME. My supervisor is the professor in both departments (actually, full professor of the first and joint appointed of the second).

I have taken some modules that are required by both departments, and my project is related to biomedical science. I wonder if there are any possibilities that I can switch? Should I go and talk to the BME administrative before I mention it to my supervisor?
Definitely talk to your advisor first. While I doubt that continuity of funding will be an issue, there could be other ones:


your advisor should be allowed to advise students in BME (sometimes 'joint appointments don't by default come with the ability to be the advisor for a student). You may need to get a co-advisor in BME (or not: it depends on the rules)
you may need to formally reapply. There might even be the possibility that you are rejected (but I'd be very surprised if that happened). It's usually never the case that you can just switch between departments. This also means that there will be deadlines, paperwork and possibly even some fees. 
since you're in the first year in grad school, committee composition is not an issue, but for someone later on in the program that could be a problem as well. 


Your advisor will/should know the answer to some of these questions and the BME administrative staff (graduate advisor or equivalent) will know the rest.