An item response theory evaluation of the Light and Spectroscopy Concept
  Inventory national data set
  This paper presents the first item response theory (IRT) analysis of the
national data set on introductory, general education, college-level astronomy
teaching using the Light and Spectroscopy Concept Inventory (LSCI). We used the
difference between students' pre- and post-instruction IRT-estimated abilities
as a measure of learning gain. This analysis provides deeper insights than
prior publications into both the LSCI as an instrument and into the
effectiveness of teaching and learning in introductory astronomy courses. Our
IRT analysis supports the classical test theory findings of prior studies using
the LSCI with this population. In particular, we found that students in classes
that used active learning strategies at least 25\% of the time had average
IRT-estimated learning gains that were approximately 1 logit larger than
students in classes that spent less time on active learning strategies. We also
found that instructors who want their classes to achieve an improvement in
abilities of average $\Delta \theta = 1$ logit must spend at least 25\% of
class time on active learning strategies. However, our analysis also powerfully
illustrates the lack of insight into student learning that is revealed by
looking at a single measure of learning gain, such as average $\Delta \theta$.
Educators and researchers should also examine the distributions of students'
abilities pre- and post-instruction in order to understand how many students
actually achieved an improvement in their abilities and whether or not a
majority of students have moved to post-abilities significantly greater than
the national average.
