It's been tried in Canada.  The Supreme Court ruled you cannot redirect your taxes to or from specific departments - they all go into one big pot and the government decides to allocate it.  I presume a similar decision would be made in the U.S.