available topics are pre-defined by the instructor of the course.
Authoring the multiple-choice answers. Similar to the body of the question, Text, images, videos and scientific formulas may be used in the development of each of the multiple-choice answers.
Nominating the correct answer and developing a solution. An ideal solution includes rationale for the correctness of the right multiple-choice answer and the lack of correctness of the other multiple-choice answers.
previewing the question to make sure that it renders correctly and as expected.
submitting the question to be stored as part of the question repository.
Students are able to answer and rate questions that are available on the platform. Once they have answered a question, they are able to view the right answer, the distribution of how the question has been answered by their peers, and the explanation provided for the answer. Students are then able to rate the quality and the difficulty of the question. Figure 2 shows the graphical interface used in RiPPLE for answering questions.
RiPPLE also relies on crowdsourcing to identify inappropriate or incorrect questions. Figure 3 shows how questions may be flagged as inappropriate in RiPPLE.
Users with the “instructor” role have the ability to view, edit and delete questions that are flagged as inappropriate.
4.2 Knowledge Tracing and Recommending Questions
One of the main pages of RiPPLE, as shown in Figure 4, is dedicated to knowledge tracing and recommending questions. The top section of this page provides an interactive visualisation widget that enable learners to select their desired visualisation type for viewing their knowledge state.
“Visualisation Type” allows learners to select from a range of different visualisation techniques so that they can choose a visual display that better suits their comprehension and personal preference. RiPPLE is equipped with a set of different types of visualisations: Bar charts (as displayed in Figure 4
) are helpful in presenting rich information regarding users. They are simple to read and are comprehensible by a wide audience. Colour of the charts categorises competencies into three levels: red demonstrates inadequate competency in a topic, yellow demonstrates adequate competency with room for improvement, and blue demonstrates mastery in a topic. Radar charts are visually striking, and can add interest to what would otherwise be a dry data presentation. One of the strengths of radar charts is that they support visualisation of multiple variables consisting of measures that require different quantitative scales, which a bar chart cannot accommodate. They are very well suited for comparing the knowledge states of users students. Box plots are an effective way of displaying the distribution of data based on minimum, median, maximum, 1st quartile and 3rd quartile. Box plot displays may be preferred when displaying data of a group of learners as they determine if the data is skewed based on where the median sits within the box relative to the inter-quartile ranges. However, they are harder than both bar charts and radar charts to read and comprehend. The widget also supports the use of more recently developed visualisations that are tailored towards education. For example, it supports the use of Topic Dependency Models[Cooper and Khosravi (2018)], which use two-weighted graphs to display the knowledge state of learners not only on based on individual topics but also on combination of two or more topics.
“Compare Data” allows learners to compare their knowledge states against a range of options: “Peers” mode enables them to compare their performance with a selected distribution (e.g. top 20%) of the peers that are currently enrolled in the course. ”Previous Offerings” enables learners to compare their performance with a selected distribution of learners across all offerings of the course. “Topic to Visualise” option enables users to select topics in which their competencies are visualised.
The bottom section of this page, as shown in Figure 4, enables learners to select questi