Beside the duration of the experiment and the lack of breaks, the rapid-design experiment, as proposed in [1]}, seems designed to suppress the very responses that we hope to classify with machine learning methods. Also, the rapid-design experiment proposed in [1]} creates a whole host of additional issues, since object recognition tends to last many hundreds of milliseconds (especially when the items change rapidly from trial to trial). This means that components such as the P300 and the N400 may still be processing the item from one class, when an item from the next class is presented . This signal overlap certainly results in the signal bleeding into the subsequent trial. This is somewhat less of a problem in a block-design study as recognition is known (from decades) to be faster when there is a consistent context . Indeed, the neural processes involved in object categorization are thought to differ greatly between a blocked and interleaved design; when items from a single category are presented in a block, people tend to focus on the similarities of consecutive items, whereas when items from different categories are interleaved (as in [1]}'s rapid design) people tend to focus on the differences between items .
A fair test of the classification rates of randomly presented classes would utilize designs common to the event-related potentials literature. These designs generally include a jittered (randomly timed) blank screen between each stimulus with some minimum duration. In fact, [1]} discusses the need to add jitter between items (page 1), but then fails to do so in the study they present. During this jittered blank interval, a long enough duration is typically included to prevent overlap of the neural responses , i.e., activity generally returns to some near baseline level. Additionally, when event-related potentials are examined, the time right before the stimulus is presented is used to perform a baseline correction, which takes the mean activity for a given channel during a pre-stimulus interval and subtracts it from every sample in that channel . This is expected to eliminate any and all temporal correlation effects not due to subject fatigue or inattention (i.e., a drifting electrical signal over time). Importantly, [1]} failed to include any jitter in their rapid design study. Essentially, all of the features that make a block-design sub-optimal for classification were included to work against classification accuracy in [1]}'s rapid design study.
Additionally, when items are presented in a block, it is possible to make the class very salient (i.e., the participant will notice that they have viewed 50 dogs in a row), whereas the rapid design obscures the point of the study. In this case, if the subjects were even mildly inattentive, they would certainly fail to think about the class of items being presented to them, something that is far harder to miss in the block-design.
Importantly, obscuring the class in the way that [1]} did, without requiring an overt response from the subject, calls into question if the subject was even paying attention to the stimuli, whereas an overt response forces the subject to attend to and more fully process the stimuli to the class level . This flawed design is exacerbated by the effects of the aforementioned prolonged exposition to visual stimuli (much longer than in [8]}, see Table REF ). If the subject is not actively and fully processing the stimulus (i.e., he/she is not paying attention and is, instead, daydreaming), machine learning methods have little hope of being able to classify the stimulus or class from any neuroimaging data.
