Which way is length and which way is width? I hear people refer to the dimensions of things as "$2$ by $4$" etc. and I know its length by width, but I can't tell if the length dimension is vertical (up and down) or horizontal (side to side). Does anyone know?

I think "length" and "width" are ambiguous.  I can confirm that native English-speaking college students will sometimes use length=2, width=4, and sometimes use length=4, width=2.  Of course, the area is 8 either way.
When I want to avoid ambiguity, I say Area = (base)x(height) or Area = (width)x(height).  For 3-dimensions, I use Volume = (width)x(height)x(depth) or Volume = (length)x(height)x(depth) to avoid ambiguity.  
(Of course there's nothing wrong with Volume = (length)x(width)x(height), but you can't be sure people will label the dimensions the same way.)
If I'm asked to draw a "2 by 4" rectangle, I'll probably draw it two units wide and 4 units tall (taking "2 by 4" to refer to the x- and y-measurements respectively).  But I think this is also ambiguous, and people may satisfy the request with a rectangle at any orientation.
