Q:

How much of the difference in brain size is attributable to gender, not physical size?

I've been reading a paper regarding the effects of gender on grey matter and white matter, and one passage not directly related to the final result caught my eye:

As expected, intracranial volume in milliliters (ml), consisting of parenchyma, ventricles, and sulci (without subarachnoid space), was (mean ± SD) higher for men (1352.2 ± 104.9) than for women (1154.4 ± 85.1): t = 9.26; df = 78;p < 0.0001. The difference (14.6%) falls between the difference in height (8.2%) and weight (18.7%). Total parenchymal volume was 1229.6 ± 106.2 (range, 1033.9–1469.4) in men and 1072.3 ± 71.5 (range, 895.4–1196.0) in women: t= 7.77; df = 78; p < 0.0001.

The difference I'm concerned with is in volume. I've done some additional reading, and it seems clear that men have a slightly larger overall brain size than women (though individual variation and the size of specific regions in the brain mean that this is not a hard and fast rule).
I'm wondering how much of this difference in brain size is attributable to differences in physical size and how much is attributable to other effects of gender. To be a bit clearer, if a group of men had the same average height and weight as a group of women, would there still be as much of a difference in brain size between genders, or is the majority of this based purely on overall body size?
This is in contrast to the primary result of the study, which seemed to draw neurological conclusions that are not directly related to physical size (regarding percentages of white and grey matter and cerebrospinal fluid).

A:

This is an excellent question. In my attempt to answer it I'll cite sources from the best research on the subject to date, and we'll conduct our own investigation.
Body size vs. Head size vs. Brain volume
In your question, you asked how body size (weight and height) relates to brain volume by gender. Body size is related to brain volume through another measure, head size. Larger body, larger head, larger brain.a Men and women have different body sizes, and different head sizes as well.  

Conducting our own investigation
Sample
In the paper you referenced, Gur et. al. looked at 80 healthy (40 male and 40 female) volunteers aged 18-45 years. They used an automated parcellation approach, meaning they used automated image processing on the brain images to derive their measurements.
In our investigation, we'll use data frome the publicly available Human Connectome Project, or HCP. We'll look at 896 healthy (393 Male and 503 female) volunteers aged 22-36 years. HCP also used an automated parcellation software, called FreeSurfer.b We'll use IntraCranial Volume - the volume inside the cranium - to represent head size.
Total Brain Volume - Males vs Females

We see here that men have larger brains than women. We can also see from the linear model results that the estimated mean difference from our sample is a substantial 153.9 milliliters (as compared to ~200 milliliters in Gur et. al. 2001). For reference, this is about 2/3 one of those half pint boxes of milk you used to get at school lunch.
But how much is total brain volume related to head size?
TBV vs ICV

We can see right away that these two measures are highly correlated. It makes sense. Typically, bigger animals -- with corresponding bigger heads -- have bigger brains:

But what you want to know is whether the difference remains after accounting for head size. In other words, is the difference disproportionate to head size?
Adjusting TBV by ICV
Here we use a method called residualizing to remove the effect of ICV on TBV we observed above. You can find the code we're using here, and a reference describing the procedure in detail here.
Let's check the graph of TBV vs ICV again, to make sure the effect is removed.

Revisiting TBV in Males vs Females
Here we'll reexamine the difference between Male and Female brain sizes using the adjusted values.

The difference is still statistically significant, but smaller. The linear model estimates the adjusted average difference to be 23.6 milliliters (compared to 153.9 before). Going back to the milk reference, this is about 1/10 the box, or one sip.

Conclusions
We found that the difference in brain volumes between men and women is partially, but not completely explained by their difference in head sizes.
The difference is smaller, but still there after adjusting. Now your next question may be, so what? Do these differences in brain volume explain anything about behavior or ability? That's where papers like the one you referenced come in. 
Gur et. al. suggest in their paper that females compensate for smaller total brain volume by having increased gray matter, and therefore more tissue available for computation. Other studies conclude that in order to understand gender differences in cognitive ability, you have to look closer at the parts of the brain that are activated during different tasks.c
The HCP data could help us test these ideas too, but that's outside the scope of your question.

