University reputation vs department reputation for industry jobs There are obviously many other factors that are going into choosing a graduate school including supervisors, research interests, etc. but I'd like to leave those out for a moment and focus on just one decision parameter. 

I have to decide between two places A and B for my physics PhD. A is a big state school whose department is ranked very high (top 10 in the US) but the school itself is otherwise unremarkable. Most people outside my field would not have heard of it. B is a very prestigious private university with a decent but not spectacular department (top 30 ish). However, although B isn't great at physics, it consistently ranks among the top 10 schools in the US overall.

If I'm looking at industry jobs after graduation as my primary choice and academia as a secondary choice, which one should I pick?
This is a tricky question, mainly because of the large degree of uncertainty of what you are going to do next. Basically, if you plan to stay within your physics field, then you should be looking for the institution with the best physics department, as ultimately your next job will depend on the quality of your work, and the reputation of the physics faculty you have been working with; while the reputation of your school will matter only if your supervisors won't push you too strongly, or if you want to move across fields.

However, you are mainly concerned with finding a job in industry. In that regard, unless you expect the people in the physics department of choice to directly point you to someone in industry they personally know, then it will be the reputation of your institution - not the department - to be "pushing" you forward. Industry do not and cannot have a fine view of the academic world. The world outside academia is vast, and you will meet someone connected with your home institution only in very rare instances. They will most likely NOT understand what you have done in physics, nor be able to judge the quality of your work; but they will recognize that you were accepted by a prestigious institution, and completed a degree with them.

Personally, I worked in either good or exceptional physics departments, in good institutions. At the end, finding a job in industry was relatively easy because I happened to work on projects that hit worldwide news. I call that luck :) 
If that would not have been the case, my choices would have be far more limited. On the other hand, I had colleagues working on obscure research projects in the world most prestigious institutions; I'd say they have similar or easier times than myself finding jobs.