Teaching First Year Engineering Design Using a Flipped Classroom Model
A team of faculty at Rice University and other institutions has created instructional resources to support a flipped classroom model for first year engineering design. Gone is the traditional 'class' in which faculty lecture on the design process and other professional skills. These 'lectures' are now delivered on video, essentially shifting low cognitive load work to videos that students watch outside of class. In class, students complete active learning exercises focused on the engineering design process. Afterwards, student teams also apply the engineering design process to their specific projects. The authors have created the following educational materials to flip the first year multidisciplinary engineering design classroom: Sixty web based videos featuring student teams and faculty at Rice University as well as three other institutions that focus on steps of the engineering design process and professional skills. Topics include defining the problem, researching the design problem, framing design criteria, brainstorming solutions, selecting solutions with Pugh matrices, project planning using Gantt charts, prototyping, and testing. Twenty one online quizzes (with 10 25 questions each) that cover information discussed in the videos. Quizzes are multiple choice and true/false and test students' knowledge and application of the technical content in the videos. Thirty in class exercises that support active learning in the classroom. The in class exercises typically require students to apply a specific step in the design process to a new problem, critique a completed design step, or synthesize knowledge. During the 2015 2016 academic year, all engineering design and professional skills lectures were flipped. The focus of this poster is to qualitatively and quantitatively evaluate students' use of the videos, quizzes, and in class exercises during the fall 2015 semester. Using analytics from YouTube and our course management software, we evaluated the percent of students who watched the videos, the number of students who started the videos, the average watch time, the percent of students who completed the quiz, and their grades. From this we learned that >80% of the students started the instructor videos, and that the number of student starts and average video watch time declined during a given playlist. Also, students performed well on quizzes, with an average score of 90% These materials are available for others to use. The team is seeking feedback on developing materials that will be helpful for the academic community teaching engineering design. Emergence of Flipped Classroom Model The flipped classroom is a style of teaching that is gaining steam within many schools and universities. The premise behind a flipped class is that students complete the lower level thinking associated with listening to a lecture out of class and complete activities that target higher levels of thinking while they are in class with the instructor present. Typically, the "lecture" material is transferred to rich media that can be watched outside of the classroom. Some STEM educators who have adopted the flipped class model at universities have seen increases in student test scores when compared to students studying in traditional lecture based classes. 1,2 There is increasing evidence that the flipped classroom is an applicable model to technical disciplines such computer science, mechanical engineering, and upper level math. 3,4,5 Student feedback has shown that the ideal video length is less than 15 minutes; students voice concerns about videos that are longer than 20 minutes. 6 Qualitative student feedback from various studies has shown that students find videos to be beneficial to watch but details on watch times are scarce. 5,7,8 One study of an information technology course at Indiana University Purdue University at Indianapolis found that in a flipped class of 27 people, over half of the students reported watching less than 90% of assigned recorded videos. 9 The study also found that a majority of the students reported rarely rewatching videos. 9 As flipped classrooms become more common, it is important to know the extent to which students use faculty produced videos. This paper explores how students utilize videos and analyzes their watching behavior. First Year Design Course Introduction to Engineering Design (ENGI 120) is a one semester multidisciplinary design course for freshman students at Rice University. In ENGI 120, students learn the engineering design process and use it to solve meaningful problems drawn from Rice University campus, local hospitals, local non profits, and international communities. In brief, the first half of the semester is devoted to defining the design problem, developing the design context review, establishing design criteria, brainstorming solutions, using a Pugh matrix to evaluate and select a solution, and then describing the selected solution. During the second half of the semester, student teams focus on physical prototype development and testing. Teams are expected to build prototypes of increasing fidelity, culminating with a functional prototype that meets some of the established design criteria. The course outcomes, structure, and deliverables have been described in detail elsewhere. 10 Methods for Video and Quiz Analytics Before coming to class, students were required to watch videos on the assigned videos and complete the associated quiz. The videos for ENGI 120 were integrated into a university course management system and were also hosted on YouTube as playlists. Quizzes were administered on the university course management system. From our university course software, we pulled the following analytics: Number of people who took a quiz for each module Total possible points of each quiz Score on each quiz for each student The metrics that YouTube reports for each video included the following: Length of video Watch time Views Unique cookies Geographical location Traffic sources Playback location (playlist or embedded page) "Views" is an ill defined metric that is analogous to, but not equivalent to, the number of people who watched the video. While YouTube defines views as "the number of times a video was viewed," YouTube does not report how views are calculated. 11 If one person watches a full video or even a portion of a video twice, that can count as two views. To know how many unique people watched the video, YouTube offers a metric called "unique cookies" that reports unique viewers by planted cookies. This metric only assesses desktops and does not include mobile devices, tablets, or TV views of a video. In surveying the video analytics, it was clear that "unique cookies" was a much better approximation to the number of unique students who started the videos (i.e. unique views) than the number of "views" reported by YouTube. To establish the number of student starts, we used the minimum of the reported number of unique views (i.e. unique cookies) or the number of students in the class (82 in fall 2015) For some modules, not all students watched the videos, as observed by the number of submitted quizzes being less than 82, which is why we used the analytics of unique views. To calculate the average watch time per student, we divided total watch time (defined as the estimated total minutes of viewing time of the video 12 by the number of students who started the video. Overview of Flipped Classroom Materials The team produced flipped classroom materials for 20 modules on the engineering design process and professional skills. For each topic, there are three components of the developed instructional materials: Web based videos featuring student teams and faculty Online quizzes that cover information discussed in the videos In class exercises (ICEs) that support active learning in the classroom Videos To date, the authors from Rice University and their colleagues have produced 60 web based videos, which are grouped into 20 modules (Table 1) Most of these playlists can be found online at http:/goo.gl/5HBkJ5. Table 1. Details of video modules. Playlist links are live. Playlist Title of Instructor Videos of Design Team Videos Playlist Link Engineering design process overview 1 0 https:/www.youtube.com/playlist?list=PLFiVi U_gwuxkjAqcY sriotrEfYbLOlzH Clarifying team assignments 1 1 https:/www.youtube.com/playlist?list=PLFiVi U_gwuxkTUjDxMvmxJAHvDv4Q3Wy7 Understanding the problem and context 5 0 https:/www.youtube.com/playlist?list=PLFiVi U_gwuxkiP aCbjY9jwG _IjOa6RR Design criteria 4 3 http:/www.youtube.com/playlist?list= PLFiViU_gwuxmx q9jgr1fpkOBPm4YYMls User defined scales 1 2 http:/www.youtube.com/playlist?list= PLFiViU_gwuxn0XlompVzPvSgaDTYreHlx Pairwise comparison charts 3 3 http:/www.youtube.com/playlist?list= PLFiViU_gwuxlzKhwcohMGN0o7eFIeUrtp Brainstorming 3 2 http:/www.youtube.com/playlist?list= PLFiViU_gwuxkW5tcLdur2xsec Fb11iz2 Decomposition 1 3 http:/www.youtube.com/playlist?list= PLFiViU_gwuxngirzaJqF6trrtxHL4MH3H Morphological charts 1 3 http:/www.youtube.com/playlist?list= PLFiViU_gwuxmIp P1HEQvIyVdtE3BaN_y Pugh screening matrix 5 3 http:/www.youtube.com/playlist?list= PLFiViU_gwuxnlc2RkX2laBc5QaKnzjuae Pugh scoring matrix 2 3 http:/www.youtube.com/playlist?list= PLFiViU_gwuxkisOyufuyQs Z7GHhXdFgh Prototyping Overview 2 0 https:/www.youtube.com/playlist?list=PLFiVi U_gwuxkWiJSdiQvQM_KaSU53jBFA Prototyping Tools 1 0 https:/www.youtube.com/playlist?list=PLFiVi U_gwuxl56p1MQGYBmMEN_SzMPwci Low fidelity prototypes 2 2 https:/www.youtube.com/playlist?list=PLFiVi U_gwuxkWHQHJFFfqZh2e6zSzE A9 Medium fidelity prototypes 2 1 Testing 4 0 https:/www.youtube.com/playlist?list=PLFiVi U_gwuxnPvcQuvcS_GH1iMn9SbUX2 Teamwork 3 0 https:/www.youtube.com/playlist?list=PLFiVi U_gwuxnulj3Gr8dXlV8WnrV lgId Intellectual property 1 0 Project planning 2 0 https:/www.youtube.com/watch?v= C3aM4suNHGc&list=PLFiViU_gwuxlbq2L6G HRWC9NLm