The neuronal basis of insect stereopsis

A puzzle for neuroscience - and robotics - is how insects achieve surprisingly complex behaviours with such tiny brains1,2. One example is depth perception via binocular stereopsis in the praying mantis, a predatory insect. Praying mantids use stereopsis, the computation of distances from disparities between the two retinas, to trigger a raptorial strike of their forelegs3,4 when prey is within reach. The neuronal basis of this ability is entirely unknown. From behavioural evidence, one view is that the mantis brain must measure retinal disparity locally across a range of distances and eccentricities4–7, very like disparity-tuned neurons in vertebrate visual cortex8. Sceptics argue that this “retinal disparity hypothesis” implies far too many specialised neurons for such a tiny brain9. Here we show the first evidence that individual neurons in the praying mantis brain are indeed tuned to specific disparities and eccentricities, and thus locations in 3D-space. This disparity information is transmitted to the central brain by neurons connecting peripheral visual areas in both hemispheres, as well as by a unilateral neuron type. Like disparity-tuned cortical cells in vertebrates, the responses of these mantis neurons are consistent with linear summation of binocular inputs followed by an output nonlinearity10. Additionally, centrifugal neurons project disparity information back from the central brain to early visual areas, possibly for gain modulation or 3D spatial attention. Thus, our study not only proves the retinal disparity hypothesis for insects, it reveals feedback connections hitherto undiscovered in any animal species.

In humans, stereopsis is supported by a complex network spanning multiple cortical 30 areas and involving tens of millions of neurons 8,11 . Praying mantids achieve stereopsis with 31 brains orders of magnitude smaller than vertebrates'. Thus, it is natural to assume that insect 32 stereopsis must be computed in a profoundly different and much simpler manner 7 . Insect 33 stereopsis does differ from humans' in using changes in luminance, rather than luminance 34 directly 12 . However, this does not explain how the mantis brain combines information about 35 the location of luminance changes in the two eyes. In primates, individual retinotopic neurons 36 in the primary visual cortex are tuned to different disparities and thus different locations in 3D- 37 space, but such local computations are often regarded as far too neuronally expensive for an 38 insect brain 7,9 . One theory is that prey is identified in each eye separately and this monocular 39 information is combined in a single late stage in the motor pathway 6,9 . This would require no 40 disparity-sensitive circuitry in the brain. 41 To determine whether neurons tuned to binocular disparities exist in the mantis brain, 42 we recorded intracellularly in the optic lobes, the major visual processing centres in insects 43 (Fig. 1a,b). Animals viewed a computer screen through coloured filters enabling us to control 44 stimuli to each eye separately and thus presenting images in 3D 3 (Fig. 1a). Neurons were 45 stained for subsequent identification. stereoscopically-defined mantis prey. It ramifies in the outer lobes and the anterior lobe of the 48 lobula complex (LOX), a highly structured visual neuropil in the mantis brain 13 (Fig. 1c). We 49 recorded TAOpro's responses to a "spiralling disc" stimulus ( Fig. 1d) which mimics mantis 50 prey. During behavioural experiments mantids readily strike at the disc when its disparity 51 indicates it is in catch range, but not in the control condition with reversed disparity 3 . When 52 the same stimulus was presented to the restrained praying mantis during neuronal recording, 53 the TAOpro-neuron responded vigorously for the disparity indicating catch range, and only 54 weakly for the control condition ( Fig. 1e; Wilcoxon rank sum test, p=5.210 -4 ). 55 To understand the neuronal computation supporting this response, we used our main 56 stimulus comprising vertical bars, 13° wide and flashed briefly at six different, non-overlapping 57 locations independently in each eye ( Fig. 1f and Methods). Vertical bars avoided the need to 58 identify receptive field elevation while enabling us to vary horizontal disparity. For studying 59 potential "prey-detector" neurons, we used dark bars on a brighter background, since mantids 60 strike preferentially at dark prey 14 . Each eye saw either a single bar or a blank screen. In this 61 way we simulated virtual objects at a range of 3D locations in front of the animal (Fig. 1f), as 62 well as "control" locations not corresponding to any single location in space (Extended Data 63 Fig. 1). 64 Tested monocularly, the TAOpro-neuron responded only to bars presented in the left Table 1). A possible neuronal circuit comprises a COcom-neuron transmitting visual 95 information to the contralateral optic lobe, but also receiving information from the contralateral 96 eye via other COcom-neurons, and in this way generating its binocularity (see below). 97 Four of the binocular COcom-neurons showed evident binocular interactions in the 98 central parts of the response fields (at 15-100mm distance and 20 o eccentricity; Fig. 2d,f,g,j). 99 Behavioural experiments 6 have shown that mantis stereopsis operates for prey capture over this 100 region of 3D-space. Three neurons had well-localised excitatory peaks (Fig. 2d,f,g) for a 101 preferred 3D location. These were well modelled by combining binocular excitation at the 102 preferred location with inhibition in peripheral regions (Extended Data Fig. 4 a,b,c). In the 103 fourth neuron the central region was void of excitation, because of inhibition by input from the 104 contralateral eye in the centre of the visual field ( Fig. 2 j,k). In vertebrates such cells are known 105 as "tuned-inhibitory neurons", in contrast to "tuned-excitatory neurons" whose receptive fields 106 have a similar structure in both eyes 22 . The neuron from Fig. 2a,b,c,d also showed disparity 107 tuning to the spiralling disc stimulus (Wilcoxon rank-sum test p=0.0043, Fig. 2e). 108 The morphology of two additional, disparity-sensitive neuron types suggests that they   Fig. 8b,d,f). The example neuron response shown in Fig. 3e has a diagonal excitatory 121 structure, meaning the cell responded to object distances independent of the azimuthal direction 122 leftwards from the midline out to as far as we measured (32 o ). Less extensive diagonal structure 123 was also seen in two COcom neurons (Fig 2d,g). This diagonal structure is a hallmark of 124 vertebrate complex cell responses, which are believed to result from converging simple cell 125 units 10 .
126 Figure 4 summarises the 4 neuronal classes of disparity-tuned insect neurons presented 127 here. A working hypothesis consistent with our data is that disparity sensitivity for dark prey-128 like objects is established in the LOX by brain-spanning COcom neurons which transmit 129 information between the outer lobes in each hemisphere. Disparity information is then passed 130 on to projection neurons like TAOpro and hence to descending neurons which deliver the signal 131 for the predatory strike.

132
TAOpro also receives input in the anterior lobe which is the output region for 133 centrifugal TAcen neurons, tuned to bright background in front of which dark prey moves.  Our work rules out speculations that disparity sensitive neurons are not present in the praying 142 mantis brain at all 9 or occur only in a very late step, fusing pre-processed monocular signals in 143 the central brain 6,9 . In a compelling vindication of the "retinal disparity hypothesis", we have 144 identified neurons which are tuned to different locations in 3D-space, at a range of distances 145 and horizontal eccentricities, confirming disputed deductions from behavioural data 4,6 . A 146 model proposed to explain disparity selectivity in the vertebrate visual cortex 10 also captures 147 the behaviour of most mantis neurons. Binocular disparities are computed early in the visual 148 pathway, in the lobula complex, and fed back even earlier, to the medulla. Disparity feedback 149 has not been reported in vertebrates nor included in machine stereo algorithms, but its discovery  Animals were mounted on custom-made holders with BluTack® and wax; their head and 163 mouthparts were immobilized by wax. A hole was cut into the posterior head capsule to allow 164 access to the brain. Fat and muscle tissue surrounding the brain were removed. The neural 165 sheath was stripped away at the region where the recording electrode was inserted. The gut was 166 removed within the head capsule and prevented from leaking within the thorax by ligating it.

167
A wire platform supported the brain from anterior to further stabilize it. During recording of 168 neural activity the brain was submerged in cockroach saline.  Microsystems) by treatment of the brains with Cy3-conjugated streptavidin (Dianova,191 Hamburg, Germany) as previously described 13 . 192 193 Visual stimulation 194 We used anaglyph technology 3,12 to present 3D stimuli on a computer monitor (DELL U2413  All stimuli were custom written in Matlab (Mathworks) using the Psychophysics Toolbox 28-30 . 201 We presented two main stimuli for the current study. Most importantly we analysed monocular  (Fig. 1f). In this way we covered almost 77° of the fronto-azimuthal visual field.

205
This is slightly wider than the approximately 70° binocular overlap of praying mantids 6 . Bars 206 were presented either to one eye only, for recording monocular response fields, or two bars 207 concurrently, one for the left and one for the right eye, for determining binocular response 208 fields. 209 We used bars instead of structures with smaller vertical extent because of the  After all bar positions had been displayed a pause of 1.7-4.5 s followed, before the procedure 218 started again.

219
The second stimulus was similar to what was found earlier to be a very effective elicitor of the 220 praying mantis prey capture strike 3 . A 22°-diameter dark disc in front of a bright background 221 appeared peripherally and spiralled in towards the centre of the screen (Fig. 1d). On reaching 222 the screen centre, after 5 s, it stayed there for 2 s before vanishing. Small quivering movements 223 were superimposed on the principal spiral trajectory and in the final 2s stationary disc phase.

224
The disc was simulated to float at 25 mm distance in front of the praying mantis in order to 225 simulate an attractive target in catch range of the animal. This was achieved by presenting one 226 disc on the left hand side, which was only visible to the right eye and a disc of identical 227 dimensions slightly shifted to the right, which was only visible to the left eye. We refer to this 228 stimulus condition as the "near" condition. As a control condition, the left and right eye discs 229 were swapped so that the right eye now saw the right hand side disc and the left eye saw the     Table 1). 255 We interpolated all binocular response fields from 6x6 to 100x100 with the Matlab function 256 "imresize" in bicubic mode. An example raw plot and its upsampled version is shown in    input dependent on the eye and location of the stimulation; that is, the model contains receptive 283 fields for both the left and the right eye (Fig. 1h). The inputs from both eyes are filtered by the 284 receptive field and then summed linearly along with a tonic input, necessary to account for a 285 non-zero background rate in some neurons. If the result is negative or zero, the mean response 286 is zero. If the result is positive, the mean response is given by its value raised to some exponent.

287
The value of the exponent, the tonic input, and the 6 azimuthal sub-regions of the left and right

Extended Data Fig. 4 | Response field plots and fitted receptive fields for COcom-neurons.
1 a-f, Left panels show monocular and binocular response field plots for all recorded COcom-2 neurons as presented in Fig. 2. Response field headers state neuron ID and outcome of two-3 way-ANOVA with "L" ("R") being significant left (right) eye input and "I" significant 4 interaction term (see Extended Data Table 1), otherwise "ns" meaning not significant. Middle    Response field headers state neuron ID, stimulus type and outcome of two-way-ANOVA with "L" ("R") being significant left (right) eye input and "I" significant interaction term (see Extended Data Table 1)