Probably wouldnt change too many people opinions honestly. Its not like someone would be walking along and be like "boy I sure am on the fence about legalization... wait a minute, a pot leaf on a rock? Well thats enough reason for me to be against it I guess. 

Also just say non-smoker. Youre only an "ent" if you identify yourself with /r/trees and decide to label yourself that.