In the attempt to invalidate the efforts of [1]}, , , , [2]}, , Li et al. [3]} propose a so-called “rapid-design” experiment. In this case, stimuli across classes are simply shuffled and presented in a randomly organized block of trials. The authors of [3]} then purport to show that such blocks can be classified based on their temporal order rather than class.
We argue that this design maximizes the impact of temporal correlations and suppresses neural signals that would be used for classification.
When replicating this rapid design, we found that the impact of temporal correlation — assessed by classifying randomly-ordered stimuli with block-level labels — is far lower than what was observed by [3]}. Indeed, in our rapid-design study with block-level labels, classification accuracy was inflated by less than 10%, much lower than the approximately 50% found in [3]}.
We claim that the huge impact of temporal correlation observed in [3]} is due to their specific and incorrect experimental design and analysis procedure.
Indeed, Li et al. [3]} altered the acquisition protocol used in [1]}: mainly, they do not report giving subjects any breaks during the experiments. This stands in stark contrast to [1]}, and also to standard recommendations in cognitive neuroscience experiments , as the lack of breaks in [3]} entails the strong risk of resulting in inattentive subjects, since the prolonged exposition to visual stimuli (over 23 minutes, instead of about 4 minutes in [1]}) all but guarantees a vigilance decrement or failure of sustained attention , , .
