Every instant of perception depends on a cascade of brain processes calibrated to the history of sensory and decisional events. In the present work, we show that human visual perception is constantly shaped by two contrasting forces exerted by sensory adaptation and past decisions. In a series of experiments, we used multilevel modeling and cross-validation approaches to investigate the impact of previous stimuli and decisions on behavioral reports during adjustment and forced-choice tasks. Our results revealed that each perceptual report is permeated by opposite biases from a hierarchy of serially dependent processes: Low-level adaptation repels perception away from previous stimuli, whereas decisional traces attract perceptual reports toward the recent past. In this hierarchy of serial dependence, “continuity fields” arise from the inertia of decisional templates and not from low-level sensory processes. This finding is consistent with a Two-process model of serial dependence in which the persistence of readout weights in a decision unit compensates for sensory adaptation, leading to attractive biases in sequential perception. We propose a unified account of serial dependence in which functionally distinct mechanisms, operating at different stages, promote the differentiation and integration of visual information over time.
From moment to moment, human perception is not a high-fidelity replica of sensory input but relies heavily on past experience and short-term dependencies: what happened a moment ago has a strong impact on how we perceive the present. Vision, for example, is inherently contaminated by the immediate past: low-level visual features such as brightness, color, or motion are all perceived depending on their temporal context [1–3], and when the stimulation history changes, identical stimuli may well lead to dissimilar percepts [1,4, 5]. Such short-term dependencies are not peculiar to visual perception but permeate a wide range of cognitive processes, including attention [6–9], decision-making [10–12], memory [13,14], confidence in performance [15], and motor behavior [16]. This implies that, at multiple stages, our cognitive system is anchored and calibrated to the recent history of sensory and decisional processes. From a phenomenological perspective, short-term dependencies may carry opposite effects on perception: negative (repulsive) effects are observed when perception is repelled away from previous stimuli [1,4, 17,18]; positive (attractive) effects, instead, arise when consecutive stimuli appear more similar than they really are, and thus past representations persist and “attract” current perceptions. Attractive and repulsive biases evoked by the history of stimuli, behavioral responses, and memory content have been recognized ever since the dawn of psychophysical and psychological research [11,19–25], but the interest has now regrown considerably after the recent work of Fischer and Whitney [26]. In their collection of experiments, human participants were asked to report the orientation of visual stimuli, trial after trial, using adjustment responses. The authors found that adjustment errors were systematically attracted toward the orientation of stimuli seen one or a few trials before [26]. This led to the hypothesis of a specific neural mechanism, termed the “continuity field, ” that would promote positive serial dependence at the earliest stages of sensory coding [27]. Continuity fields have been reported afterwards in a variety of perceptual tasks [28–33] and for features encoded at multiple stages along the visual hierarchy, including for basic stimulus features, such as orientation [26], spatial position [34], and numerosity [28], but also for more complex stimuli, such as faces [35], emotion [36], and facial attractiveness [37]. Such ubiquity of positive biases has led some authors to hypothesize that serial dependence is an intrinsic mechanism through which our visual system exploits temporal correlations and contextual redundancies by merging similar stimuli, slightly changing over time, into a coherent and “continuous” perceptual field [26,27]. Although the idea of a continuity field fits neatly with recent findings, it stands in sharp contrast with other, widely documented forms of negative dependence. Examples of negative serial dependence are well-known phenomena of visual adaptation and aftereffects [18]. In visual aftereffects, previous exposure to persistent features, such as orientation [4], color [18] or motion [1], causes subsequent stimuli to appear as shifted away from the exposed feature: a stationary object will seem to be moving downward after exposure to upward motion; a vertical grating will look as if it is tilted clockwise after exposure to counterclockwise orientations. These negative biases go plainly in the opposite direction to that predicted by the continuity field, and thus the question arises as to how these two contrasting forces, one attractive and the other repulsive, may coexist in perceptual processing [38]. Although there is a wealth of evidence linking negative aftereffects to transitory decreases in the responsivity of relevant populations of sensory neurons [39–42], less is known about the mechanisms regulating positive serial dependence: is there a shared mechanism responsible for both types of dependencies, or are the two phenomena related to distinct and independent stages of visual processing [30]? According to the original account of serial dependence [26], continuity fields emerge at the very early stages of processing [29,43], likely from the persistence of past activity patterns in sensory areas [26,44]. This view, and the related computational framework [26], suggested a physiological process that closely resembled neuronal adaptation but with an opposite and conceivably distinct outcome: sensory neurons encoding previous stimuli increase their responsivity to future events, thus biasing perceptual processing toward the recent past [26,29]. This entails the coexistence of two additive [29,45] or opposite [30] processes—adaptation and serial dependence—that compete in pulling the appearance of stimuli toward or away from the past [26,29,46]. To explain what factors may determine the prevalence of one or the other, a major role has been assigned to the nature of sensory input: strong and persistent stimuli cause neuronal adaptation and the resulting repulsion [2]; weak, uncertain, and briefly presented stimuli promote positive serial dependence [29]. Since the functional role of these opposite biases appears closely related to the integration (positive serial dependence) and segregation (adaptation) of visual events over time, it has been also suggested that their relative prevalence might depend on the stability or inconstancy of stimuli in the natural environment, with attractive biases for stable attributes (e. g. , the identity or gender of a face) and repulsive biases for changeable features (e. g. , facial expressions) [32], hence conferring a major role to prior knowledge and environmental statistics in determining the direction of biases in a sequence of perceptual events [38,47]. The issue has been recently raised, however, about the stage at which positive serial dependence originates. Several studies, in fact, have presented evidence for opposite effects of the recent past at different stages of processing [30,48]: repulsive biases, akin to negative aftereffects [2,18], at the early stages of perception and attractive biases emerging later and increasing in magnitude as the postperceptual time before a behavioral response increased [30,48]. In light of these findings, an alternative account has been proposed that advocates the coexistence of two types of serial dependence in perceptual judgments: negative dependence evoked by short-term adaptation and positive dependence caused by postperceptual biases in working memory [30,48]. This alternative account denied the low-level and “perceptual” nature of serial dependence, favoring a postperceptual explanation in which previous sensory events do not directly attract the appearance of stimuli [30,48]. Although positive serial dependence at the early stages of perception has been confirmed in more recent studies [29,43,49], the overall scenario remains controversial. First, opposite history biases have been reported under very similar circumstances and using virtually identical stimuli [26,29,30], which makes it unlikely that the only decisive factor is the kind of sensory input. Second, although positive serial dependence has been observed in response to weak and noisy sensory signals, it nevertheless requires attentional resources (but see [50]), whereas repulsive biases come about after nonattended stimuli [26]. Moreover, the increase in neuronal responsivity, which putatively underlies positive serial dependence [26], is a well-known mechanism of gain modulation triggered by feature-based attention [51], but to the best of our knowledge, there is no physiological evidence of the persistence of such increase for several seconds after stimulus offset. Finally, although the two accounts reviewed above make dissociable predictions about the locus of positive serial dependence and its effect on stimulus appearance, other interpretations are equally possible [30,47,52], and a theoretical framework to resolve this puzzling interplay between adaptation and positive dependencies is required. In the present work, we propose and test a new account based on the notion of decisional inertia [53] and inspired by the hierarchy of recurrent neuronal processes that serve the needs of perceptual decision-making [54–58]. We hypothesized that positive serial dependence originates from the persistence of internal templates [59,60] formed during the weighting of past sensory evidence into perceptual decisions, which then become attractor points for future perceptual representations. With the term “template, ” here we refer to the profile with which sensory information is accumulated from a specific region in the stimulus space (e. g. , a visual orientation) and integrated into internal representations and decisional outcomes [53–58,61], a process that may involve the contribution of higher-level processing stages (e. g. , decisional) [30,57,58,61]. For clarity, we will refer to the persistence of such internal templates over time with the term “decisional inertia” [53]. The novelty of our proposal lies in the fact that positive serial dependence by decisional inertia does not deny the impact of previous events on stimulus appearance (i. e. , at the “time of perception” [43,48]); neither does it confine attractive biases to the domain of postperceptual processes. In fact, we reasoned that decisional inertia might alter visual perception independently of the content in working memory but exclusively as a consequence of chains of task-relevant internal representations—decisional events. In line with this hypothesis, we provide compelling evidence for the existence of two separate and functionally distinct stages of visual serial dependence [30,56,62], determined by the independent history of sensory and decisional processes. In a series of experiments, we measured the ability of human participants to reproduce the orientation of consecutive stimuli, and with the aid of multilevel models and cross-validation techniques, we demonstrate that past perceptual decisions, rather than physical stimuli, represent the effective sources of attractive history biases. We found that stimulus-driven serial dependence disappeared when participants were explicitly instructed to withhold their response, whereas repulsive biases emerged under this condition. This result was corroborated by the striking finding that, after streaks of nonreported orientations, positive serial dependence was still dictated by the most recent perceptual decision. This dependency on previous decisions was not the mere consequence of hysteresis in motor responses, as it was strongly modulated by stimulus contrast and had a direct impact on perceptual sensitivity. We further show that the effect of decisional traces generalizes to other tasks and stimuli and is independent of the content in visual working memory. Consistent with our results, we propose a hierarchical model of short-term dependencies in which repulsive biases emerge as a result of low-level adaptation and positive serial dependence arises from the reverberation of internal templates reflecting the reading-out of sensory signals from higher-level decision integrators. Whereas rapid negative aftereffects in response to task-irrelevant stimuli may promote exquisite perceptual sensitivity to relevant environmental changes, decisional traces may reflect the reinstatement of the most informative sensory channels in the recent past, thus promoting perceptual continuity and increasing the efficiency of decisional templates over the long term, as reported in perceptual learning (PL) studies [54]. Our work provides a new perspective on the history of sensory and decisional processes by reconciling opposite forms of short-term dependencies into a unified account of concatenated perception and decision-making.
As a primary goal, we addressed whether serial dependence in visual perception depends more on past physical stimuli or on the response decision in the previous trial. Recent work has shown that past stimuli systematically bias the decision about the present sensory input [26,30,63]. However, the history of decisions may itself represent a source of perceptual bias that combines with or even outweighs the impact of previous stimuli [30,53]. To test this possibility, in the present experiment (Experiment 1) we compared the ability of sensory stimuli and perceptual decisions to predict future errors in an orientation adjustment task [26]. Ten participants were presented with peripheral Gabor stimuli (50% Michelson contrast) oriented randomly between 0° and 165° off vertical (in steps of 15°) and were asked to reproduce the perceived orientation by adjusting the tilt of a response bar (Fig 1A). Data were analyzed after residualizing individual adjustment responses from their orientation-dependent shifts in mean [64], a typical confound that falsely mimics response-related serial dependence [65] (see Methods and Fig 2E and 2F). The biased-corrected behavioral data were then used to compare the influence of past stimuli and responses on current errors. To this aim, we built two competing models. In one model, we predicted errors in the adjustment responses with the difference between the previous stimulus orientation and the current stimulus orientation (delta stimulus [ΔS]). In the alternative model, we predicted errors with the difference between the previous decision and response and the current stimulus orientation (delta response [ΔR]). In both models, errors were fitted to a derivative of Gaussian function (DoG) in a multilevel modeling framework (see Methods), and the amplitude parameter of the DoG (α) was used to quantify the magnitude of serial dependence and to test its significance. Serial dependence for the stimulus model ΔS was significant (α = 1. 32°, p < 0. 05) and had a peak at 9. 95° (Fig 2A and 2B), in keeping with previous reports [30]. When errors were conditioned on previous responses (ΔR), serial dependence was also significant and qualitatively larger than the one observed for ΔS (α = 1. 71°, p < 0. 01, peak at 14. 55°; Fig 2B). To evaluate the best model of serial dependence, we followed two steps: (1) We assessed which model provided a better fit of participants’ errors by computing the difference in the root-mean-square error (rmse) between ΔS and ΔR, and (2) we compared the ability of each model to predict new data with a cross-validation procedure (see Methods). The model comparison in step 1 favored the ΔR model over the ΔS model (rmse[ΔS − ΔR] = 0. 059, p < 0. 001, Monte Carlo permutation test, n = 1,000), and in the cross-validation of step 2, the performance of the ΔR model was considerably higher as compared to the ΔS model (rmse[ΔR] < rmse[ΔS] = 98. 6%; Fig 2C). Thus, both procedures favored the ΔR model, showing that conditioning on previous responses increased the explanatory and predictive ability of the model. The results of Experiment 1 provided strong evidence that previous responses, residualized from orientation-dependent biases, constitute a more efficient predictor of serial dependence than the relative orientation of consecutive stimuli, suggesting that attractive biases may indeed result from the history of perceptual decisions. One possibility is that decisional traces bias perception more than previous stimuli when the sensory input is highly uncertain [53]. Because orientation sensitivity degrades as a function of eccentricity [66,67], the presentation of Gabors in the periphery of the visual field may have favored decisional biases and lowered the contribution of previous uncertain stimuli. Therefore, it could be hypothesized that when stimulus uncertainty decreases, the physical orientation of previous stimuli becomes a more reliable predictor of perceptual errors compared to the previous decision and response. To address this possibility, we performed a second experiment (Experiment 2) in which the same stimuli of Experiment 1 were presented at the fovea. Although there is evidence of stimulus-induced attractive biases at or near the fovea [26], a direct investigation of the impact of previous responses for central stimuli is lacking. In addition, to further reduce stimulus uncertainty (e. g. , to increase orientation discriminability), stimuli were presented with higher spatial frequencies (see Methods) [68]. Following the procedure of Experiment 1, in 10 new participants we compared the contribution of ΔS and ΔR for centrally presented stimuli. In addition, to rule out any confound due to the comparison of one model—the ΔS—built with a discrete predictor against a second model—the ΔR—built with a continuous predictor [69], we varied the relative orientation of consecutive stimuli in the range of ±90° with steps of 1°, making the ΔS predictor fully comparable with the ΔR. Serial dependence at fovea was significant for both ΔS (α = 1. 40°, p < 0. 01, peak at 15. 6°) and ΔR (α = 1. 45°, p < 0. 01, peak at 19°; Fig 2B and 2D). Contrary to the uncertainty hypothesis, however, model comparison again favored the ΔR model (rmse[ΔS − ΔR] = 0. 003, p < 0. 05), which also predicted new data with higher accuracy than the ΔS model (75. 7%; Fig 2C, lower distribution plot), thus fully confirming the results of Experiment 1. The first two experiments converged to one crucial point: previous perceptual decisions represent a better predictor of serial dependence than previous stimuli. Our model comparison revealed that perceptual decisions and behavioral responses are critical, if not fundamental, for serial dependence in orientation discrimination. In their seminal work, Fischer and Whitney [26] reported sequential effects of stimulus presentation in the absence of prior motor responses, ruling out the contribution of postperceptual effects on serial dependence. However, in their experiment and in similar attempts [29,43], participants were not explicitly instructed to withhold their response, and this may have still promoted postperceptual processing of the stimuli (e. g. , in the form of implicit decisions and persisting internal representations). As a consequence, even in the absence of an actual overt motor response, perceptual decisions might have occurred, leading to lingering effects during the interstimulus interval, in turn affecting perception of the subsequent Gabor stimulus. Following this reasoning, we predicted that if serial dependence emerges from previous decisions, then it should disappear when participants are explicitly instructed to ignore the most recent stimulus and to withhold their response and decision. To test this hypothesis, 15 participants performed an experiment (Experiment 3) similar to Experiment 1 but including 40% of “catch” trials in which no response had to be made. Catch trials were explicitly signaled by a black circle appearing at the same location and time as the response bar in the standard trials, indicating that no response—and therefore no perceptual decision—had to be made. Because perceptual decisions were directly manipulated, and to overcome confounds due to orientation biases, we analyzed serial dependence after both catch and response trials only for ΔS. Although serial dependence for ΔS was significant after response trials (α = 1. 76°, p < 0. 01, peak at 12. 30°), not only were attractive biases absent after catch trials, but also ΔS revealed a repulsive effect on current errors (α = −0. 82°, p < 0. 01, peak at −46°) that was significantly different from the effect after response trials (α[catch]–α[response] = 2. 59°, p < 0. 001; Fig 3), ruling out the possibility that serial dependence simply disappeared because of an intervening stimulus (the black circle). Confirming our hypothesis, this experiment showed that prior perceptual decisions are fundamental for positive serial dependence, and it revealed that withholding decisions and responses leads to the emergence of repulsive biases, resembling those observed in negative aftereffects [1,4, 41]. Because the observed positive serial dependence by previous decisions may originate at a later stage than early perceptual processing, one may argue that the reported effect is due to postperceptual biases [30] emerging during adjustment responses and having no direct impact on the quality of a perceptual experience [48]. In order to better characterize the nature of these opposite biases, and before drawing any firm conclusion, in the present experiment (Experiment 4) we investigated whether past decisions and stimuli act directly on perception and on the sensitivity of the perceptual system. To this aim, we abandoned the adjustment paradigm and employed a combination of forced-choice tasks, which have been proposed as a more direct measure of perception [29,30]. Fourteen new participants were asked to perform a dual task with sequences of Gabor stimuli presented at fovea (see Methods and Fig 4). In the first task, they had to discriminate the left or right oblique orientation of a Gabor followed by a noisy mask (45° versus 135°, one-alternative forced-choice [1AFC] symmetric task [70]). In the second task, they had to report whether an ensuing Gabor of variable orientation (±2,5, 15, or 40° off the left or right oblique, positive toward vertical) was more tilted toward the vertical or the horizontal (recoded as a “Yes/No” vertical task, see Methods). In half of the Yes/No trials, the reference oblique (i. e. , the oblique bisecting vertical/horizontal) was the orientation presented or reported in the 1AFC task (Congruent condition). In the other half, the reference oblique was the opposite (Incongruent condition). Gabors in the first 1AFC task were presented with different signal-to-noise ratios (SNRs = 0%, 25%, and 75%), with one condition containing only noise (i. e. , SNR = 0%). The combination of these forced-choice tasks allowed for a direct test of the effect of previous stimuli and decisions on the discriminability and appearance of present sensory events. In adherence with our previous results, we formulated two testable hypotheses: (1) If past stimuli repel perception, then the sensitivity to detect deviations from an oblique orientation should increase if the oblique is presented in a preceding task (e. g. , in Congruent trials). (2) If previous decisions attract current perception, then the net effect of a decision (i. e. , in the absence of sensory signal, SNR = 0%) would be to decrease the sensitivity to small orientation changes by integrating present stimuli close to the previous decisional outcome (e. g. , congruent to the reported orientation). To address these hypotheses, we modeled participants’ sensitivity (d’) in the Yes/No task as a function of the stimulus deviation toward cardinal axes and of the SNR in the preceding 1AFC task. Incongruent trials were treated as a control condition, since stimuli and decisions 90° away from the present orientation should have no effect [29]. As expected, in the Incongruent condition (Fig 5A) a multilevel linear model revealed a significant intercept (β0 = 0. 78, p < 0. 001) and a significant slope for Deviation (the increase of d’ with the increasing deviation from oblique, β1 = 0. 05, p < 0. 001) but no effect of the SNR and no interaction Deviation × SNR (all p > 0. 05). In the Congruent condition (Fig 5B), however, in addition to the intercept (β0 = 0. 60, p < 0. 01) and slope for Deviation (β1 = 0. 06, p < 0. 001), the model revealed also a significant main effect of SNR (β2 = 0. 005, p < 0. 05) with no interaction Deviation × SNR (β3, p = 0. 23). The overall modulation of d’ by the SNR was then compared to a baseline d’ in the Incongruent control condition (see Methods). This baseline-corrected sensitivity showed a decreased d’ for decision-only trials (SNR = 0%, Δd’ = −0. 13) and an increased d’ after high-SNR stimuli (SNR = 75%, Δd’ = 0. 13), with a significant difference between the two (paired t test, p < 0. 05; Fig 5C). To further characterize this difference, we performed a final analysis on the raw proportions of vertical responses over the whole range of orientations tested in the Yes/No task. By subtracting the two Congruent conditions of interest (SNR = 0% and 75%) from the baseline (Incongruent condition), the resulting difference in proportions of vertical responses ΔP (Vertical) showed an evident DoG-like pattern: after decision-only trials (SNR = 0%), participants responded more often “vertical” when the tilt was slightly horizontal and less often “vertical” when the tilt was slightly vertical; conversely, after high-SNR stimuli this pattern was reversed, suggesting a more sensitive discrimination of the vertical/horizontal direction than in the baseline (Fig 5D). The two conditions were well fitted by the multilevel conditional DoG function used in Experiment 3 (Eqs 3 and 4), with a significant positive amplitude for decisions (α = 0. 058, p < 0. 05), a trend for a negative amplitude for stimuli (α = −0. 057, p = 0. 06), and a significant difference between the two (0% SNR − 75% SNR = 0. 11, p < 0. 01). This intriguing pattern confirmed the opposite effects of previous decisions and stimuli in the absence of any potential confound due to postperceptual processes and adjustment procedures. Furthermore, it demonstrated that history biases act directly on perceptual sensitivity: previous decisions (in the absence of sensory signals) attract future perceptual representations, decreasing the discriminability of consecutive stimuli; strong sensory signals repel future stimuli, aiding their differentiation. The opposite effect of past stimuli and perceptual decisions is in sharp contrast with the idea of serial dependence as a low-level visual mechanism [26]. In line with the assumption of low-level continuity fields, indeed, serial dependence would originate from changes in the response properties of early visual neurons and should be immune to neuronal computations at later, decisional stages. However, our results revealed two mechanisms regulating serial dependence: one repulsive, after the mere exposure to sensory stimuli, and the other attractive, observed after perceptual reports. These contrasting forces on visual perception suggest that positive and negative carryover effects may originate at different stages of the processing stream [30,56]. To explore the plausibility of multiple stages of serial dependence, we defined two data-generating models (Fig 6) that implemented serial dependence either as a consequence of low-level processes (Gain model) or as a combination of opposite forces exerted by mechanisms at different stages (Two-process model). In both cases, perception was modeled as an encoding–decoding process performed by a hierarchy of units based on recent models of visual processing [54,71–73]. The model’s architecture (Fig 6A) included a layer of low-level units resembling a population of orientation-selective neurons, a higher-level decisional unit, and a final response stage. The presentation of an oriented stimulus elicited a population response in which the contribution of each low-level unit depended on the distance between the stimulus and the unit’s preferred orientation. The decision unit then decoded and integrated the population profile into a final response (the reported orientation). The decoding stage occurred through a set of weights, mediating the readout of the decision unit from each orientation-selective cell [54]. Thus, orientation discrimination was modeled as a two-stage process: an initial response by a population of perceptual units and a later weighted decoding by a higher-level decisional unit. In the Gain model (Fig 6B), serial dependence was produced by a gain change in the response properties of orientation-selective units [26]. Exposure to a tilted stimulus increased the sensitivity of low-level units coding the same and nearby orientations, and this in turn biased the population response to an incoming stimulus toward the orientation coded in the past. In this model, the decision unit decoded the population profile with uniform and constant weights; thus, serial dependence only emerged from trial-by-trial gain modulations at the level of orientation-selective neurons. In the Two-process model (Fig 6C), serial dependence resulted from the interaction between low-level adaptation mechanisms and changes in decisional weights. Contrarily to the Gain model, exposure to oriented stimuli inhibited the response of low-level units centered onto the same and nearby orientations. This, in turn, biased the population response to an incoming stimulus away from the present orientation [74,75], resembling negative aftereffects observed after both brief and prolonged adaptation periods [76–78]. The critical aspect of the Two-process model was the plasticity at the level of decisional weights. In this model, previous decoding templates persisted, generating a nonuniform distribution of weights with maxima at/near the orientation decoded and reported in the recent past. This way, opposite carryover effects were modeled at different stages of the processing stream, with negative aftereffects arising from low-level adaptation and attractive biases emerging from decisional traces. Because the reweighting process favored the readout from orientation channels that were more informative in the recent past, this effect compensated for trial-by-trial adaptation at the lower level, leading to the positive serial dependence observed after behavioral responses. The reweighting process was triggered by the requirement—and execution—of a behavioral response, and therefore, no weight update followed the mere exposure to irrelevant stimuli. The Two-process model provides a compelling account for the coexistence of both positive and negative dependencies in visual perception [30,56]; however, its plausibility and performance had to be evaluated and tested against the Gain model. To this aim, we first compared the two models in a simulation of behavioral errors for the trial sequences used in Experiments 1 and 2. In this context, the Two-process model closely resembled the observed patterns of serial dependence, and it reproduced the superior predictive ability of ΔR over ΔS that we characterized in our first two experiments (Fig 7A). As a more stringent test, we then compared the predictions of the two models with the true pattern of errors made by participants in a new experiment. In this experiment (Experiment 5; Fig 1B), 15 participants were presented with a sequence of Gabors in each single trial and were asked to report the orientation of the last stimulus after the sequence ended (see Methods). Because previous decisions were always interleaved with a sequence of new stimuli before the next to-be-reported orientation, this paradigm offered an ideal situation in which previous stimuli and decisions were dissociated and both positive and negative serial dependence could emerge. To evaluate the predictions made by the two models in this experimental context, we ran a simulation of Experiment 5 (n = 10,000), and we conditioned the error generated by both models on previous stimuli after perceptual decisions (the reported orientation in the previous trial) or in the absence of decisions and responses (the last nonreported stimulus in the sequence). As expected, the Gain model predicted positive serial dependence for the preceding stimulus in the sequence but no persisting influence of the orientation reported in the previous trial, i. e. , of the previous perceptual decision (Fig 7B). On the contrary, previous stimuli and perceptual decisions had opposite effects on the errors generated by the Two-process model: although adaptation at the low level caused repulsive biases, prior decisional weights attracted the representation of incoming stimuli (Fig 7B). In order to select which data-generating process provided a better approximation of the mechanisms underlying serial dependence, we compared the outcomes of both models with the actual data from Experiment 5. The behavioral results resembled almost exactly the predictions made by the Two-process model (Fig 8A and 8B). At odds with the Gain model, a significant negative serial dependence was observed in response to the previous—nonreported—stimulus in the sequence (α = −1. 10°, p < 0. 01, peak at −17. 9°), whereas positive carryover effects emerged for stimuli reported in the preceding trial (α = 1. 02°, p < 0. 05, peak at 43. 75°), and the magnitude of the two dependency effects was significantly different (α[Reported] − α[Nonreported] = 2. 13°, p < 0. 001; Fig 8B and 8C). Furthermore, the effect of nonreported stimuli was generally repulsive (Fig 8C), whereas opposite biases from previous decisions and past stimuli dominated single perceptual reports with almost equal contribution (dominance for reported stimuli = about 57%; see Methods and Fig 8D). This clear-cut result provided strong evidence in favor of a hierarchical view of serial dependence, and in addition, it showed that decisional traces affect current perceptual decisions even when interleaved with new stimuli. The fact that the lingering effect of perceptual decisions persisted across time, even after streaks of intervening stimuli, is in line with the idea that positive serial dependence emerges from the inertia of a high-level decision unit, which operates in a stimulus-independent manner. An interesting question is therefore whether decisional traces, once formed, become mere abstractions of a (biased) sensory experience or still preserve some low-level properties of the sensory evidence upon which they are based. In the present experiment (Experiment 6), we focused on the role of stimulus contrast on serial dependence by perceptual decisions. The aim was 2-fold: (1) to evaluate the contribution of the actual and previous sensory evidence, as determined by the stimulus contrast, and (2) to further address whether decisional traces only rely on perceptual reports (the orientations confirmed with the response bar) or their impact varies with the strength of the previously decoded stimulus. More specifically, if positive serial dependence requires weak sensory signals and the mere absence of adaptation [29,43], then it should increase for streaks of low-contrast stimuli and should decrease (or even turn negative) for streaks of high-contrast stimuli; contrarily, if attractive biases are due to persistent traces in decisional circuits that counteract adaptation, then the stronger the perturbation of the circuit was (e. g. , when decisions were made on high-contrast stimuli), the larger the actual attractive bias will be. Fifteen participants performed the same orientation adjustment task as in Experiment 1, with stimuli of high (75%) and low (25%) contrast randomly intermixed across trials. In a first multilevel model, we compared the effect of switching between low and high stimulus contrast by extracting the α parameter when a low-contrast inducer was followed by high-contrast stimuli and vice versa when a high-contrast inducer was followed by low-contrast stimuli (Fig 9A). This analysis revealed a strong and significant serial dependence for ΔS only in the high-to-low condition (α = 2. 32°, p < 0. 001, peak at 30. 6°) but no attractive biases in the low-to-high condition (α = 0. 20°, p > 0. 05), with a significant difference between the two (high-to-low minus low-to-high = 2. 11°, p < 0. 01). To further characterize this interaction between the contrast of previous and present stimuli, we then estimated the magnitude of serial dependence at the single-subject level in order to compare four conditions of interest in a full factorial design. Individual α values were submitted to two-way repeated-measures ANOVA with factors Inducer Contrast (Low versus High), Actual Contrast (Same [as the inducer] versus Different), and their interaction. The ANOVA yielded a significant interaction Inducer Contrast × Actual Contrast (F (1,14) = 6. 03, p < 0. 05, η2 = 0. 3) driven by the significant difference between high-to-low- and low-to-high-contrast serial dependence (p < 0. 01, Bonferroni correction; Fig 9B). Thus, decisional traces are stronger after high-contrast stimuli, and their attractive influence increases in relation to weak stimuli. To generalize our findings to domains other than orientation perception, we performed a further experiment (Experiment 7) in which the task required participants to reproduce the average size of an ensemble of circles (see Methods and Fig 10A). Perceptual averaging, also defined as statistical representation or ensemble coding [79,80], refers to the ability of our perceptual system to extract global statistics, such as the mean or variability, from sets of similar stimuli, thus providing a coarse gist of complex visual scenes. It has been recently shown that ensemble coding occurs in space as well as in time and, consequently, the perceived statistics of an ensemble depends on its global properties in the immediate past [31]. These results were discussed as evidence of a spatiotemporal continuity field in visual perception that promotes objects stability and coherence over time. However, based on the evidence reported thus far, it is plausible that serial dependence in summary statistic is due to the same persistence of decisional templates that we have characterized for orientation processing. To test this hypothesis, we predicted errors in the reported average size of 15 new participants with two different multilevel linear models, following the approach of our Experiments 1 and 2. In one model (ΔS), we used the difference between the average size of the previous and present ensemble as the dependent variable. In the other model (ΔR), we used the difference between the reported average size in the last trial and the current average size of the ensemble. Before model comparison, errors were corrected for several confounds that typically affect size adjustment responses (see Methods). The ΔR model was superior to the ΔS model in the overall fit (Akaike information criterion [AIC] for ΔS = 1,484; AIC for ΔR = 1,399; p < 0. 001, theoretical likelihood ratio test [TLRT]) and predicted new data with higher accuracy (99. 9%; see Fig 11B). Moreover, previous responses were significantly associated with errors of reported size (β1 = 0. 054, p < 0. 001), whereas in the ΔS model the effect of previous stimuli was quantitatively smaller (β1 = 0. 02, p < 0. 01) and significantly different from the one of previous responses (β1[ΔR] − β1[ΔS] = 0. 034, p < 0. 001). This result showed that the influence of previous responses/decisions has a stronger impact on perceptual averaging than the average size of previous ensembles (Fig 11A). As an alternative to the Two-process account, one can argue that positive history biases arise during the retention of perceptual or decisional information in a temporary memory storage where persistent traces may interfere (i. e. , the working memory account [30,48]). Although our results appear consistent with decisional inertia in reading-out sensory information, as a final step we found it worthwhile to rule out this alternative interpretation. To this aim, in two versions of this final experiment (Experiment 8a and 8b), we evaluated the persistence of previous decisions when visual working memory was filled with new items (see Methods). Participants were presented with two (n = 6) or four (n = 10) circles at the beginning of each trial and were asked to memorize the diameter of each single circle. A spatial retroactive cue during the retention interval indicated which circle’s diameter had to be reproduced with an adjustment line (Fig 10B). With this method, visual working memory was constantly filled with new and more recent representations (e. g. , the competing stimuli on each display), thus potentially wiping out or at least partly overwriting representations in working memory that might be responsible for positive biases in perception due to recent history. If serial dependence is due to representations in working memory, then more recent traces should have stronger impact on perceptual errors than previous decisions. To test this hypothesis, we predicted errors in the reported diameter size of the retro-cued circle as a function of the competitor (s) relative size (ΔC) and of the ΔS—i. e. , the difference between the previous and the current target circle’s size. In the experiment with two circles (Fig 12A), a multilevel linear model revealed a significant repulsive effect of ΔC (β1 = −0. 14, p < 0. 0001) and an opposite, attractive effect of ΔS (β1 = 0. 10, p < 0. 0001). Similarly, in the experiment with four circles (Fig 12B), the effect of ΔC was repulsive for the two competitors that lay closest to the retro-cued circle (β1 = −0. 085, −0. 082, both p < 0. 0001) but attractive for the distant competitor (β1 = 0. 028, p < 0. 01), while a strong effect of previous target was still present (β1 = 0. 10, p < 0. 0001). Finally, we also performed a model comparison similar to that of Experiments 1,2, and 7, using ΔR instead of ΔS. In both experiments, the model with ΔR explained the data better than the model with ΔS (experiment with two circles: AIC for ΔS = 4,525; AIC for ΔR = 4,477; p < 0. 001; experiment with four circles: AIC for ΔS = 9,481; AIC for ΔR = 9,448; p < 0. 001, TLRT), thus further confirming one of the main findings of this work. These results are in line with previous studies showing strategic and repulsive interactions between items in working memory [81], and they reveal a persistent effect of previous decisions/responses that is independent of the recent content and relational representations in a visual working memory storage.
The content of visual perception is strongly permeated by information lingering from the past. In the present study, we demonstrated that vision is shaped by two contrasting forces arising from the history of physical stimuli and perceptual decisions. Our work, based on multilevel modeling and cross-validation approaches, provides a thorough and comprehensive characterization of visual serial dependence from which we derived the following conclusions: (1) There are two contrasting forces that, at any given moment, interact in shaping our perception, biasing the appearance of visual stimuli toward or away from the past; (2) repulsive forces, resembling low-level adaptation, dominate after stimuli that require no perceptual decision; and (3) attractive forces on perception emerge during serially dependent perceptual decisions. Our primary aim was to compare the ability of previous stimuli and responses to predict participants’ errors in orientation adjustment tasks. This comparison was performed in the first two experiments (and supported by additional analysis in Experiments 7–8) and showed, without ambiguity, that serial dependence in visual perception is explained by the history of perceptual decisions that are required for behavioral responses. Previous responses outperformed past stimuli in predicting participants’ errors for both peripheral and foveal stimuli. This indicated that response-induced history biases are independent of stimulus uncertainty and persist even when the perceptual resolution of stimuli increases. In a following step, we hypothesized that serial dependence occurs exclusively after stimuli passed through a complete perceptual and decisional processing. The results of Experiment 3 revealed the opposite effects of stimulus and response history: after catch trials, in which a perceptual decision was withheld because of task instructions, past stimuli evoked repulsive biases on subsequent orientation judgments; after response trials, however, adjustment errors were systematically attracted toward previous decisions. Other studies have also used a “no response” condition [43] or an “orthogonal” response condition [29] (e. g. , reproducing the orthogonal stimulus orientation), revealing persistent attractive biases and excluding any role for the motor response [43] and response item [29]. It is reasonable, however, that in all these experiments a decision about the relevant stimulus attributes was formed and persisted from trial to trial. In fact, stimulus-related decisions are required for relational (orthogonal) judgments and may reflect the default strategy in tasks in which the demand for a response is uncertain [82], but responding is the most likely case [26,43]. Hence, if positive serial dependence arises from persisting internal representations required to perform the task, from trial to trial, it is no surprise that the presence or type of response plays no role. In line with this idea, here we demonstrated that the same opposite pattern of history biases can be gleaned from discriminative tasks requiring forced-choice responses. The results of our Experiment 4, without adjustment responses, are indeed of critical relevance for two main reasons: (1) They rule out any possibility that our findings may be due to orientation bias confounds in adjustment responses [43,65], and (2) they prove that attractive and repulsive history biases alter directly the sensitivity of our perceptual system and thus the appearance of stimuli. Previous sensory signals may indeed act as an internal reference model against which incoming stimuli are compared [83]. Repulsive biases may contribute to the refinement of trial-by-trial internal references that maximize the system sensitivity during discriminative tasks. Decisional processes, on the other hand, build upon the gathering of sensory evidence over time [84]. When sensory input is weak or scanty (as in our 0% SNR condition), decisional templates may continue to integrate information within the actual decision boundaries, causing a local decrease of perceptual sensitivity. Forced-choice tasks have been already employed in combination with adjustment responses to probe the perceptual/postperceptual nature of serial dependence, leading however to contrasting results. For example, Fritsche and colleagues [30] found repulsive biases in two-alternative forced-choice (2AFC) tasks when the orientation of 2AFC stimuli was ±20° away from that of a preceding adjustment task. Using a similar procedure but including more values for ΔS between adjustment and 2AFC stimuli, Cicchini and colleagues [29] failed to replicate the repulsive bias at ±20° but found positive serial dependence for smaller ΔS (e. g. , ±5°, 10°). Although our findings may not directly resolve this inconsistency, an interpretation in terms of decision inertia can be proposed. First, decisional processes and readout templates may be different between tasks requiring a qualitative scaling (e. g. , orientation adjustment) or comparative/discriminative judgments (e. g. , forced-choice tasks), and their degree of overlap and interaction may be reduced—serial dependence may occur only for highly similar stimuli [29]. Second, when fixed and predictable ΔS are used (e. g. , ±20° [30]), other factors may come into play, including expectations about systematic detectable differences between consecutive stimuli [85–87], which may alter perception, decisional strategies, and sequential effects. It is also worth noting that forced-choice tasks per se are not impermeable to history biases [88] and readout strategies [89] but are more strongly affected by previous decisions/responses than previous stimuli [88]. Hence, the positive serial dependence observed with hybrid tasks [29] may still reflect decisional inertia. The decreased perceptual sensitivity after decisions in the absence of sensory evidence is in line with a recent work showing that decisional inertia increases the tendency of choice repetition after weak stimuli [53]. This supports the notion that the repulsive/attractive consequences of recent events may depend on the quality of the sensory input [29]; however, it also suggests that two contrasting forces coexist at every instant of perception and originate at different stages of the processing hierarchy. The most compelling evidence of such a hierarchical system emerged from the results of Experiment 5, in which past stimuli and responses were dissociated and both positive and negative aftereffects could be revealed in a single perceptual report. Conditioning errors on past stimuli and responses in this key experiment showed the independent profile of the two sources of serial dependence, replicating the repulsive biases for stimuli observed in Experiment 3 and revealing the attractive impact of decisional traces even after streaks of intervening stimuli. One may wonder to what extent repulsive biases reflect visual adaptation in tasks with briefly presented and masked stimuli. Whereas the same consideration applies to low-level positive serial dependence, repulsive biases in orientation perception (e. g. , the tilt aftereffect [TAE] [41,77,77,90]) and other visual domains [91] are evident even after a few milliseconds of exposure [76,77,92], suggesting that this rapid form of adaptation-induced plasticity may reflect an instantaneous and stereotyped behavior of ensembles of sensory neurons. As a consequence, when adaptation is weak (e. g. , after brief stimuli), decisional readout processes prevail. When adaptation is strong (e. g. , after prolonged exposure [26]), instead, decisions will be formed by integrating signals from a drastically adapted system, and adaptation prevails. Additional work could further evaluate the involvement of sensory adaptation by presenting sequences of stimuli at different retinal positions: although in our paradigms retinally overlapping stimuli may have promoted repulsive biases when perceptual decisions were not required, it would be interesting to investigate the direction and strength of history biases in the absence of decisional processes and when adaptation is minimized. Here, we showed that the two competing mechanisms can be disentangled by isolating decisional and sensory traces (Experiment 5). Another possibility is that the observed repulsive bias is not a genuine result of neuronal adaptation but an opposite type of serial dependence that follows response inhibition and stimulus devaluation [93]. Although this is an intriguing possibility that deserves future investigation, the similarity between the repulsive bias in Experiment 3 and the one in Experiment 5 (in which adaptation was facilitated in the absence of backward masking and with short interstimulus intervals) justifies our interpretation of repulsive biases as emerging from low-level adaptation effects. A further intriguing alternative might be that positive and negative history biases result from attentional selection [26,94], which acts as a gating mechanism that enables or blocks the persistence of perceptual traces based on the relevance of sensory events [95]. This seems quite plausible if one assumes that negative biases are the consequence of inhibitory (e. g. , below baseline) modulations of early sensory activity induced by the top-down attentional suppression of irrelevant stimuli. However, such suppressive effects of attention are usually observed in response to salient and distracting stimuli [96,97], whereas in our paradigms saliency was equated across all stimuli, and participants were unaware of the relevance/irrelevance of each stimulus until the response stage. Furthermore, in our Experiment 5, we found no evidence that participants were attending to the last, to-be-reported stimulus while ignoring or “suppressing” the preceding one in the sequence (see Methods). Thus, in light of all evidence considered here, decisional inertia still appears the most satisfactory interpretation of positive serial dependence in our results. Nonetheless, we recognize that the involvement or additional role of attention is clearly an interesting question for future work. To account for the opposite effects of stimuli and decisions, we proposed a Two-process model of serial dependence, combining low-level orientation adaptation and persisting decisional templates in a single hierarchical structure. We modeled low-level adaptation as a repulsive reorganization of a population of orientation-selective neurons [74,98] and the decisional stage as a weighted decoding of low-level information [99–101]. By incorporating a memory trace for readout weights at the decisional stage, we provided a simple model that approximated both low-level adaptation, in the absence of decisional activity, and attractive biases from the decoding stage. The constant reweighting process counteracted and masked the effects of orientation adaptation, leading to the positive serial dependence reported here and in previous work [26,30,63]. The contribution of our work to the perceptual/postperceptual debate about serial dependence depends strictly on the operational definition of perception. If we define perception in terms of stimulus appearance [46] and subjective internal representations, then our results and the Two-process model would certainly classify positive serial dependence as a perceptual phenomenon—it does alter stimulus appearance. What serial dependence is not, according to our perspective, is a process emerging at the earliest stages of sensory encoding. To express this difference in greater details, a parallelism with the PL domain might be useful. PL refers to the improvement in perceptual performance after extensive training, which underlies long-lasting neuronal changes in the brain [102–105]. Although it has long been proposed that such profound changes engage modifications at the level of early sensory neurons (e. g. , V1 [106,107]), growing evidence supports the hypothesis that PL reflects changes at higher-level stages [54,108–110]. According to this idea, PL would arise from improved readout connections between higher-level decisional units and lower-level sensory areas [54], without considerable changes at the lower stages. Thus, in PL, recurrent readout processes tune connection weights and lead to measurable improvements in perception over the long term. In a similar fashion, the inertia in updating these trial-by-trial connection weights, postulated by our model, would transitorily lead to changes in stimulus appearance that can be measured over the short term. The plausibility of our Two-process model is corroborated by extensive evidence of readout processes and integration of sensory information at high-level stages of processing [55,89,99,111] and by the critical role of decisional circuits in recent models of visual perception [54,112]. Furthermore, it is in line with the fact that early visual areas show little to no increase in sustained activity over time [113,114], contrarily to what would be specifically expected from the Gain model [26], although their information content persists longer when engaged in memory and decision-related processes [115–117], whereas task- and decision-related activity reverberates over prolonged delays in higher-level neuronal circuits [57,118,119]. Decisional inertia may therefore reflect the tendency of a specific pattern in decision-making circuits (e. g. , a recurrent readout process, perceptual template) to resist changes in its actual state [119]. This inertia may be proportional to the strength (mass) of a decision-making instance. This possibility fits well with the results of our Experiment 6, demonstrating that decisional traces persist more strongly after the sensory/decision circuit has been perturbed with high-contrast stimuli, which may have caused stronger network activity and slower relaxation dynamics [120]. One may therefore speculate that the stronger the intensity (mass) of a past perceptual/decisional episode, the larger its impact on future stimuli (inertia). Similarly, the fact that serial dependence decreased when high-contrast stimuli followed low-contrast stimuli is in agreement with the idea that salient and strong sensory signals can cause resets in decisional circuits, reducing attractive aftereffects [120]. The results of our Experiment 6 are also relevant for the understanding of the interactions between positive and negative biases in visual perception [46]. Whereas stimulus contrast has been previously manipulated in separate experiments [43], with serial dependence occurring only for low-contrast stimuli, the critical interaction between past and present stimulus strength has not yet been tested. Our findings demonstrate that, in the same participant, response-driven serial dependence by low-to-low- and high-to-high-contrast stimuli does not differ. The difference emerged when comparing low-to-high and high-to-low sequential effects, with attractive biases being higher in the latter case. On the one hand, this suggests that strong signals rapidly dissipate persistent traces from previous perceptual/decision-making events. On the other hand, this indicates that although they may promote higher degrees of adaptation, when readout processes still prevail over adaptation, strong signals exert a larger impact on future weak signals. Hence, our results indicate the existence of two functionally segregated processes of serial dependence that operate at different stages of the processing hierarchy. Attractive biases are not induced by recent stimuli on current responses but are the by-product of chains of decisional processes that are paralleled by repulsive adaptation at a lower stage. The behavior of this hierarchical system generalized also to the domain of ensemble coding [80], demonstrating that the appearance of an ensemble summary is biased toward past decisions more than by the statistical properties of previous stimuli [31]. Although the Two-process model represents a parsimonious and unified account of serial dependence, other alternatives can be formulated. For example, by substituting decisional weights with probability distributions (priors) and low-level population activity with sensory evidence (likelihoods), one can accommodate our hierarchical model into the predictive coding framework [47,121–123]. According to this perspective, higher-level probability distributions built upon past information (e. g. , previous stimuli) would be used to predict and test the incoming sensory input. As a consequence, when two consecutive stimuli are similar in some feature domain (e. g. , orientation), the product of prior knowledge and current sensory evidence would result in mixed representations. It has been proposed that serial dependence emerges from such a predictive mechanism, educated by environmental priors [47] and by the fact that objects in the real world tend to remain stable over time. This implies that in the absence of a reliable temporal structure, the best prediction of future stimuli is that they will match the current percept [38]. Although this represents a suggestive theoretical proposal, it relies on the strong assumption that the brain systematically expects the reoccurrence of the same stimulus in a random sequence. However, if the past in a random series is used as a reference to build predictions, it seems more plausible that perception, shadowing the human reasoning system, would fall into the “local representativeness” heuristic [124], expecting consistent changes within short sequences of perceptual events [8,125] in order to match an internal model of randomness [126,127]. Along a similar line, it has been suggested that, more than from deep-seated priors for object stability, serial dependence originates from optimal perceptual strategies based on adaptive coding and temporal integration [49]. In this view, human perception would resemble an optimal Bayesian estimator (or “ideal observer” in psychophysical terms) that maximizes performance through a weighted integration of highly similar sensory events over time [49]. Although optimality is a questionable term when referring to a mechanism of sequential integration under a stochastic sequence of sensory events (e. g. , independent trials) [128], it is worth noting that the formal aspects of this idea are generally in line with our proposal: irrespectively of whether it produces an optimal or suboptimal behavior, serial dependence arises from an integrative process that carries over information from one trial to the next, biasing stimulus appearance. Here, we show that sequential decisions are fundamental for this integrative process, whereas in their absence, temporal segregation and repulsive biases do prevail. An alternative explanation of positive serial dependence has been proposed based on working memory biases [30,48]. In our final experiment, we compared the impact of visual working memory content and previous decisions in perceptual errors during size estimation tasks. Although, even at the conceptual level, it is difficult to disentangle between working memory content and persistent decisional processes, our Two-process model would predict that only internal templates that have been used for a decision can affect future perception, independently of the amount of new information in working memory. Confirming this prediction, our results showed that traces of previous decisions produce attractive biases, whereas working memory content had a repulsive effect on perception, in line with similar findings reported before [81,129]. The repulsive bias appeared to be stronger for memorized stimuli in the spatial proximity of the to-be-reported stimulus (Fig 12B). As a final remark, few neuroimaging studies have investigated serial dependence so far. For example, there has been only a single attempt to address how positive serial dependence is reflected in brain activity [44]. This work, using functional magnetic resonance imaging (fMRI), investigated orientation-selective activity in V1 during a 2AFC task (left/right oblique). The authors found carryover effects from trial to trial that were reflected by activity patterns in the visual cortex. However, as the same authors discussed [44], their paradigm did not allow determination of whether serial dependence reflected stimulus- or decision-related activity, and when incorrect trials were analyzed, they found perceptual biases toward previous (incorrect) choices, more than previous stimuli [44]. Because of the low temporal resolution of fMRI and the fact that it records mainly input to a given area, more than local computations [130], it is plausible that such findings reflect reentrant feedback from higher-level (decisional) structures to V1. A more recent work has found differences in visually evoked electroencephalographic (EEG) responses that resembled serial dependency patterns in numerosity estimation, even in the absence of an explicit task [50]. In scalp data, such differences emerged at a relatively late and restricted time window (at around 200 ms from stimulus onset), whereas no modulations by serial dependence occurred for the early components typically evoked by numerosity magnitude (approximately 100 ms). A subsequent neural decoding analysis revealed earlier signatures of serial dependence (e. g. , before 100 ms), but the sign (attractive or repulsive) of this bias was not decoded. Therefore, although the report of serial dependence without an explicit task is intriguing, further neuroimaging and EEG work is required to clearly characterize the sites and dynamics of history biases in the brain. What is the functional role of the two sources of serial dependence? We argue that both mechanisms serve the adaptive function of maximizing perceptual efficiency over the short- and long-term range. In free viewing conditions, we tend to make relatively rapid and large saccadic eye movements with subsequent fixations likely to land on stimuli with highly dissimilar structures [76]. The rapid and repulsive reorganization of low-level sensory neurons may thus prepare our perceptual system to decorrelate and resolve different spatial structures, enhancing its ability to discriminate local features in a discontinuous perceptual environment. The lingering of decisional traces, on the other hand, may be explained by the integrative nature of decisional processes. Decisions terminate in a choice/response after sensory evidence is accumulated to some threshold level [84]. Decisional processes may therefore have a long integration and decay time that persist from one trial to the next. Although the persistence of decisional weights can bias perception in the short term, leading to positive serial dependence, it may aid the long-term calibration of optimal decisional weights that fosters PL [54,111,112]. Another intriguing possibility is that the decisional system implicitly counteracts adaptation and compensates for repulsive biases at the lower stage in the attempt to promote visual stability in a top-down fashion. To conclude, in the present work we provide a unified account of the concatenated history of perceptual and decisional processes. We present a hierarchical model of serial dependence in which two separate and contrasting forces, exerted by adaptation and decisional inertia, interact in shaping visual experience and our reactions to environmental stimuli. Building on recent work [30], our study has strong theoretical and experimental implications in dissociating different sources of history biases in perceptual and psychophysics research.
The study was approved by the local Ethical Committees of the University of Verona and of the University of Fribourg and carried out in accordance with the Declaration of Helsinki. A total of 110 subjects (79 females, mean age = 23. 08 ± 3. 93 years) —37 from the University of Verona, Italy, and 73 from the University of Fribourg, Switzerland—participated in this study for monetary payment (7€) or course credits. All participants had normal or corrected-to-normal vision and were naïve as to the purpose of the experiments. Stimuli were presented on a Multiscan G500 CRT 21” monitor (1,024 × 768 pixels, 100 Hz, Experiments 1,3, 5, and 6) and on a Philips 202P7 CRT (1,600 × 1,200 pixels, 85 Hz, Experiments 2,4, and 6) and were generated with a set of custom-made programs written in MATLAB (R2017b) and the Psychophysics Toolbox 3. 8 [131], running on Windows XP–based machines. All experiments were performed in dimly lit rooms, and participants sat at a distance of 45 cm (Experiments 1,3, and 6) or 70 cm from the computer screen, with their head positioned on a chin rest. All stimuli were presented on a gray background (approximately 24 cd/m2). Across experiments, the number of trials varied in order to achieve a minimum of 20 data points for each variable of interest, while keeping the overall duration within roughly 1 hour. Participants were instructed to maintain their gaze at the center of the screen for the entire duration of all experiments. An example of a trial sequence in Experiment 1 is illustrated in Fig 1A. Each trial started with a green central fixation spot (0. 5°) followed by the presentation of a peripheral Gabor (8. 5° to the left or right of fixation). Gabor stimuli had a peak contrast of 50% Michelson, spatial frequency of 0. 5 cycles per degree and a Gaussian contrast envelope of 1. 5°. After 400 ms, a noise mask appeared at the same location of the Gabor for 400 ms. Masks were white-noise patches of the same size and peak contrast of the Gabor stimuli, smoothed with a symmetric Gaussian low-pass filter (filter size = 1°, SD = 2°). After a blank fixation interval of 500 ms, a response bar was presented at the same location of the Gabor. The response bar was a 0. 5°-wide white bar windowed in a Gaussian contrast envelope of 1. 5°, with the same peak luminance of the Gabor stimuli. On each trial, the response bar appeared with a random orientation and participants were asked to rotate the bar until it matched the perceived orientation of the Gabor. The response bar was rotated by moving the computer mouse in the upward (clockwise rotation) or downward (counterclockwise rotation) directions, and the final response was confirmed by clicking the mouse left button. After a random interval (400–600 ms), a new trial started. On each trial, the Gabor was assigned 1 of 18 possible orientations, ranging from 0° (vertical) to 170° in steps of 10°. The series of orientations was pseudorandomly determined in order to keep the relative orientation between consecutive stimuli (ΔS, previous orientation minus present orientation) in the range ±50°, in steps of 10°. At the beginning of the experiment, participants were provided written instructions and performed 20 trials of practice that were not recorded. The experiment consisted of four blocks of 140 trials each, for a total of 560 trials (approximately 30 trials for each orientation tested and approximately 50 trials of each level of ΔS), and lasted approximately 1 hour. Stimuli were presented to the right or left of fixation in separate blocks. The average reaction time in all experiments using the same orientation adjustment task (Experiments 1–3,5, and 6) was 2. 23 ± 1. 26 seconds. The procedure in Experiment 2 was similar to that in Experiment 1, with the following exceptions. Gabor stimuli were presented at the center of the screen and had higher spatial frequency (1. 2 cycles per degree). The orientation of stimuli varied pseudorandomly in the range 0–179° (steps of 1°), and the relative orientation ΔS was a continuous variable ranging from −80° to +80° in steps of 1°. There were four blocks of 110 trials for a total of 440 trials. The procedure in Experiment 3 was similar to that in Experiment 1, with the following exceptions. In order to compare our results with previous work showing serial dependence in the absence of prior motor responses [26], we made our paradigm more similar to the one adopted by Fischer and Whitney ([26]; Experiment 2). Gabor stimuli were presented at ±6. 5° from fixation for 500 ms, and the duration of the mask was increased to 1,000 ms. In about 40% of the total trials (catch trials), the response bar was replaced by a black disk (1. 5° of diameter, 200 ms) explicitly instructing participants not to report the Gabor orientation. The duration of intertrial intervals in catch trials was a running average of individual reaction times during the task. The orientation tested ranged from 0° to 165° in steps of 15°, with ΔS of ±60° in steps of 15°. The increase of the step size for orientations and ΔS with respect to Experiment 1 allowed us to collect a sufficient number of data points for response trials (approximately 30 for each orientation and approximately 40 for each ΔS) and catch trials (approximately 20 for each orientation and approximately 26 for each ΔS) while keeping the overall duration of the experiment at about 1 hour. An example of a trial sequence in Experiment 4 is illustrated in Fig 4. The dual task consisted of a 1AFC symmetric task [70], followed by a Yes/No task. Each trial started with a central reminder of the 1AFC task (two oblique “\” and “/” lines presented in red) for 1 second. After the reminder, an oriented Gabor (1 cycle per degree, contrast = 25%, duration = 500 ms) and a subsequent noise mask (contrast = 50%, duration = 500 ms) were shown at the center of the screen. The SNR of the Gabor was determined by keeping a proportion of randomly selected pixels from the Gabor image and filling the rest with uniformly distributed noise. In the 0% SNR condition, the image contained only noise. When the SNR was higher than 0% (25%–75%), the Gabor could have a left oblique orientation (45°) or a right oblique orientation (135°), randomly intermixed and balanced across trials. Participants had to report the left versus right oblique with their left hand, using two dedicated keys in the computer keyboard. After the 1AFC response (mean reaction times = 559 ± 207 ms) and a blank interval of 500 ms, the reminder for the Yes/No task appeared for 1 second (one vertical and one horizontal line in green). The reminder was followed by a central Gabor (contrast = 50%, SNR = 100%, 500 ms). The orientation of the Gabor deviated from one of the two diagonals (45 or 135°) with eight possible offsets (±2,5, 15,40°, with negative values indicating deviations toward the horizontal axis and positive toward the vertical axis). In half of the trials, the reference diagonal for the Yes/No task was the same as the stimulus orientation in the preceding 1AFC task (e. g. , 45-to-45° and 135-to-135°, Congruent condition). In the other half, it was the opposite (e. g. , 45-to-135° and 135-to-45°, Incongruent condition). In 0% SNR trials, the Congruent/Incongruent conditions were assigned online, depending on the orientation reported by the participant in the 1AFC. Participants had to report whether the Gabor was more tilted toward the vertical or horizontal axis with their right hand, using two dedicated keys in the keyboard. After their response (mean reaction times = 847 ± 184 ms), a random intertrial interval from 500 to 800 ms preceded the next trial. All participants performed a brief training session (20 trials) before the experiment with task-specific trial-by-trial feedback and were not informed about the absence of a detectable orientation in the 0% SNR condition. There were four blocks of 90 trials each, for a total of 360 trials. To test the predictions made by the two data-generating models of serial dependence, we performed an experiment in which stimuli to be reported were interleaved with sequences of stimuli not requiring a behavioral response. In this experiment we pursued an additional aim to measure the role of expectations in serial dependence comparing response biases after regular (i. e. , ABCABC) or random orientation sequences. However, we found no differences between conditions and no explicit learning of the regular sequences, indicating that the employed manipulation of expectation was not effective. Therefore, given the suitability of this paradigm for evaluating the Two-process hypothesis of serial dependence, we used all data from this experiment to test the predictions made by the two data-generating processes. An example of a trial sequence in Experiment 5 is illustrated in Fig 1B. Gabor stimuli were the same as in Experiment 2 and were presented at the same central location. Eighty percent of trials contained a sequence of six Gabors (200 ms each) with varying orientations (from 0° to 160° in steps of 20°) and a ΔS of ± 80°, in steps of 20°. No noise mask was presented after each stimulus, and the phase of the sinusoidal component of each Gabor was changed randomly on each presentation (0–360°, steps of 15°). On the remaining 20% of total trials (catch trials) the sequence ended at a random position before the sixth Gabor (after 1–5 Gabors). These catch trials were used to ensure that participants were paying attention to the entire sequence and, most importantly, to the fifth stimulus before the to-be-reported orientation (fifth versus sixth stimulus error magnitude, t (1,14) = 0. 28, p = 0. 78, two-tailed t test). After 400 ms from the offset of the last Gabor in the sequence, the response bar appeared at the center of the screen, and participants had to report the orientation only of the most recently seen stimulus. Only trials in which the sequence was completed after six Gabors were analyzed. There were four blocks of 71 trials each, for a total of 284 trials. The procedure, stimuli, orientation steps, and ΔS were the same as in the response trials of Experiment 3. The contrast of each Gabor was varied randomly between low (25% Michelson) and high (75% Michelson) on a trial-by-trial basis. The duration of the Gabor was also increased to 1,000 ms, and the following noise mask lasted 500 ms. Reaction times for one subject were not recorded, because of a computer programming error. There were 5 blocks of 80 trials in Experiment 6, for a total of 400 trials. An example of a trial sequence in Experiment 7 is illustrated in Fig 10A. Stimuli were ensembles of 16 light-gray dots of four different sizes. The average diameter of the ensemble was 0. 75°, 0. 945°, 1. 19°, or 1. 5° and was randomly assigned on each trial (see [79] for a similar procedure). The SD of the dot diameter within an ensemble was either 0. 2° or 0. 4°. The dots were arranged in a 4 × 4 matrix (14. 1 × 10°), separated from the monitor’s center by 1. 5° in the horizontal direction and 2° in the vertical direction. Dots were centered on equally spaced cells of 3. 52 × 2. 5°, with a random horizontal and vertical jitter ranging from 1 to 16 pixels. The distance of the ensemble’s center from the fixation spot was 11. 14°. Each trial started with a green fixation spot (0. 5°) at the center of the screen for 400 ms. Then, the ensemble appeared for 200 ms at one of four possible locations (top left, top right, bottom left, or bottom right). After the ensemble offset, a dark-gray response dot appeared at the center of the screen with a random diameter (ranging from 0. 2° to 4°), and participants were asked to adjust the diameter of the response dot to the perceived average size of the ensemble. The adjustment response was performed by moving the computer mouse in the upward (diameter increase) or downward (diameter decrease) directions and then clicking the left button to confirm the response. After a variable interval (500–700 ms), a new trial started. The average reaction time of participants in this experiment was 1. 89 ± 0. 52 ms. The experiment consisted of 4 blocks of 120 trials, for a total of 480 trials. An example of a trial sequence in Experiment 8 is illustrated in Fig 10B. In two separate versions of this experiment, we used two and four light-gray dots of different sizes. The two dots were presented at ±5° from the monitor’s center. The four dots were arranged at the corners of a central imaginary square (diagonal = 10°). The diameter of each dot was randomly chosen from 50 linearly spaced intervals between 0. 5° and 5°. Each trial started with a central green fixation cross for 500 ms, followed by the presentation of the circles for 200 ms. Participants were instructed to memorize the diameter of each dot until a spatial retro-cue (a black X) appeared in one of the dots’ locations after a fixed delay of 1,000 ms. Participants were then asked to adjust the size of a central response line with random orientation (edges = {0. 05°, 8°}) to the diameter of the cued circle, using the mouse. After a variable interval (500–800 ms), a new trial started. The average reaction times were 2. 19 ± 0. 32 seconds with two dots and 1. 91 ± 0. 34 seconds with four dots. The experiment consisted of 4 blocks of 100 trials, for a total of 400 trials.
What occurred a moment ago has a strong impact on how we perceive the present: previous sensory events, decisions, and actions all leave traces that can affect our current perception. In this study, we demonstrate that human perception is constantly shaped by two contrasting forces arising from the history of sensory and decisional events. On the one hand, repulsive forces push perception away from recent stimuli. On the other, attractive forces dominate human perception during sequences of perceptual decisions, biasing the present sensory input to appear more similar to the past than it actually is. We have formalized these findings in a computational, physiologically informed model that takes into account the hierarchical structure of perceptual processing in the human brain, proposing a unifying view of history biases in perception.
classical mechanics
reaction time
social sciences
vertebrates
neuroscience
learning and memory
animals
mammals
dogs
perception
cognitive neuroscience
cognitive psychology
cognition
memory
vision
physics
working memory
psychology
inertia
eukaryota
biology and life sciences
sensory perception
physical sciences
cognitive science
amniotes
organisms
motion
Laws of concatenated perception: Vision goes for novelty, decisions for perseverance