To avoid crashing onto the floor, a free falling fly needs to trigger its wingbeats quickly and control the orientation of its thrust accurately and swiftly to stabilize its pitch and hence its speed. Behavioural data have suggested that the vertical optic flow produced by the fall and crossing the visual field plays a key role in this anti-crash response. Free fall behavior analyses have also suggested that flying insect may not rely on graviception to stabilize their flight. Based on these two assumptions, we have developed a model which accounts for hoverflies´ position and pitch orientation recorded in 3D with a fast stereo camera during experimental free falls. Our dynamic model shows that optic flow-based control combined with closed-loop control of the pitch suffice to stabilize the flight properly. In addition, our model sheds a new light on the visual-based feedback control of fly´s pitch, lift and thrust. Since graviceptive cues are possibly not used by flying insects, the use of a vertical reference to control the pitch is discussed, based on the results obtained on a complete dynamic model of a virtual fly falling in a textured corridor. This model would provide a useful tool for understanding more clearly how insects may or not estimate their absolute attitude.
Flying insects are subjected to a broad range of disturbances, for which fast, robust sensorimotor reflexes compensate. The flight stabilization performance of flies are even more impressive in view of the intrinsic aerodynamic instability of their flapping flight [1–4]. Compensating for this passive instability requires an active inner control of the wing kinematic in addition to an outer-loop system which responds to specific sensory cues (looming objects, odours and navigational cues). Discovering insects’ abilities to sense movement via optic flow crossing the compound eye or inertially via the halteres (for dipteran) is still of great interest. However, very few studies have focused so far on how flying insect sense their absolute body orientation in the three-dimensional space (attitude) with respect to a vertical reference. Oppositely, several studies have suggested that flies may lack the ability to perceive the vertical (via graviceptive cues) in order to stabilize their flight [5–7]. To address this point, we used an already designed free-fall procedure [7] with which insects can be briefly exposed to near-weightless conditions in a box lined with horizontal black and white stripes. The present study focused on the following questions: The control of flight speed based on optic flow cues have been confirmed by ethological studies [8–10]. In the same time, in flying insects, as in helicopter, lift vector and body orientation are fixed in time and consequently flight speed and pitch orientation [1,3, 11], and the idea that insects’ attitude may be stabilized on the basis of the optic flow has been tested successfully on a 2 degree-of-freedom flying robot [12]. A pitch rate control process has also been proposed previously to model the drosophila’s forward velocity during flight [13,14]. Based on the existence of coupling between pitch control and optic flow regulation, we challenged the suitability of such closed-loop control compared with hoverflies subjected to an unsteady free fall situation. It has been previously established that the fly’s auto-stabilizer involves several sensory modalities, which interact during flight. First, insect vision is based on two physical structures, compound eyes and ocelli. The fly’s photoreceptors feature a high temporal resolution giving them a great ability to detect fast motion based on contrast changes [6]. Optic flow measurement have shown that motion vision is involved in many visually guided tasks such as flight speed and altitude control, wall following, odometry and optomotor response [9]. Most of the optic flow processing is performed by compound eyes, and local contrast motion measurements are fused by lobula plate tangential cells (LPTC) responsible for detecting large field motion [15,16]. In addition, it has been established that several groups of interneurons, including VSTCs (Vertical Sensitive Tangential Cells) [17] and HSTC (Horizontal Sensitive Tangential Cells) [18], process the various components of visual motion and in particular that they distinguish between the rotational and translational components of the optic flow with respect to the fly’s reference frame [19]. In addition, the ocelli, which are usually composed of three simple unfocused eyes forming a triangle at the top of the head [20], may be involved in the visuo-motor stabilization reflexes that maintain postural equilibrium by detecting the head’s rotational speed [21–25]. Dipteran also possess two minute dumbbell-shaped organs called halteres, which have evolved from hind-wings and beat simultaneously in anti-phase with wings. This active beating along with the campaniform sensilla provide flies with sensitivity to Coriolis forces and consequently to their own body’s angular speed [26–28]. The halteres enable the fly’s autopilot to respond to extremely abrupt changes in attitude with a latency as short as 5ms [29,30]. In addition, insect’s hairs and antennae are sensitive to airflow during flight. Airflow sensing by the Johnston’s organs present in the antennae is known to be involved in flight speed regulation complementary to optic flow regulation [31,32]. All in all, these sensorimotor units are mainly characterized in flies by their high temporal resolution and their low latency response [29]. Flies’ sensors are indeed highly tuned to detecting and quickly counteracting any change in their environment [6]. The combination of various sensory modalities with different bandwidth allows them to cover a wide range of dynamic perturbations. In this study, an insect flight control model was developed, based simply on the closed-loop control of the pitch rate and the regulation of the horizontal component of the optic flow. In a first step, our model was devoid of any kind of absolute reference. The results obtained with this model simulating the fly’s response in unsteady free fall situations are compared with experimental data obtained on plummeting hoverflies in a box lined with horizontal black & white stripes. The model simulated data matched what occurred during the first few milliseconds of the insects flight, but the pitch and speed responses became highly unstable after around 0. 4s. In the second step, the accuracy of the model’s predictions was greatly improved by including two additional feedback loops: one controlling the pitch rate on the basis of the absolute estimation of the pitch orientation and one controlling the lift and thrust forces on the basis of the vertical optic flow. The ability of the fly to measure its pitch orientation with respect to an absolute reference value is discussed in term of the existence of visually mediated responses such as the dorsal light response (DLR).
In a previous study on flight stabilization in plummeting hoverflies [7], we established that the flies’ crash avoidance performance depended more on visuo-motor reflexes than on gravity perception. In order to understand those reflexes more deeply, we modeled a fly’s pitch rate control system based on optic flow cues (see Figs 1 and 2) and compared the results obtained during model simulations with experimental free falling hoverflies. First, we focused on the pitch because we observed that during the period elapsing between the onset of the fall and wingbeat initiation, flies pitched down smoothly, probably because of the pin glued onto their thorax. Therefore, pitch was taken to be the main state to be controlled by the fly’s stabilizer to avoid crash. Secondly, since gravity cues do not seem to be involved in insect flight control [6,7, 33], we assumed that fly’s flight control does not rely on any absolute vertical reference of the environment but that it is based rather on visual and inertial motion perception and compensation. We therefore based our model on previous studies on insects’ flight behaviour providing clear-cut evidence that optic flow-based control are involved during several tasks (for a review see [9]). We considered here that the forward speed was controlled by pitching-down from the nose the body and then orienting the force vector produced by flapping wings [34], as occurred in the case of the helicopter analogy [11]. The pitch rate is set so as to keep the forward optic flow constant, as found to occur in bees traveling in a textured corridor [10]. In order to parametrize the gains in the PD controller in charge of the visual optic flow process in the model (see Fig 2C), we conducted a series of experiments with hoverflies. The parameters of the visual Proportional-Derivative controller (PDV), Kp and Kd, were estimated directly from experimental data. We first selected only the trials in which flies triggered their wingbeats in less than 150ms after the onset of the fall and were able to compensate for the fall by reaching a positive vertical speed (i. e. , a lift force superior to their weight), amounting 44 experimental trials. A simulated falls was then achieved and compared with each of the selected falls as described above with several combinations of Kp, ranging from 0 to 20, and Kd, ranging from 0 to 2. A likelihood estimation (MLE) map was obtained for each fall, giving 44 maps in all, from which we extracted the average map shown in S2A Fig.
As shown in Fig 4A (top view), in the box lined up with stripes only on the lateral walls (X = −20/20), the hoverflies did not seem to express any kind of preference for a specific wall. As expected from our previous study, it can be seen from Fig 4B that no crash occured in presence of visual cues (horizontal periodic stripes) and that most of the trajectories ended with a rising flight, which confirm the ability of hoverflies to control their flight in the free fall tests. The initiation times of the wingbeats, around 100ms in average (see Fig 4C), are also coherent with our previous findings [7]. In this study, we selected only trials featuring a time to wingbeat triggering inferior to 150ms to keep a sufficient margin from the 200ms time limit, after which it is impossible for the fly to stop its fall and avoid crash onto the ground [7]. Fig 5A (dark lines) shows the time course of the mean pitch orientation around the onset of the flies’ wingbeats. Hoverflies pitched down (i. e. head downward) when falling freely, but soon after initiating their wingbeats, they were able to compensate for the misalignment of their body tilt with respect to the horizontal within 150ms. Despite the existence of significant differences in pitch orientation at flight initiation, no difference were observed in terms of the final pitch orientation or correction times between late initiation (125–150ms), medium initiation (100–125ms) or early initiation (75–100ms), which shows the robustness of the reflex response involved. In Fig 5B (dark lines), the mean theoretical optic flow was calculated versus time during free fall and flight recovery phases. The vertical component of ω, ωzRfly, increased to around 0. 04rad. s−1 (for the latest initiation group) during the actual free fall and decreased quickly to zero after initiation of the wingbeats, whereas the horizontal component ωxRFly decreased before the wingbeats was triggered and continue to decreased slightly after the initiation of the wingbeats and reached a mean steady state value of about −0. 04rad. s−1 regardless of the initial conditions. This result supports the idea that hoverflies may control the optic flow in closed-loop so as to keep it constant during flight. The results of the parameters identification, from which the parameters used during the simulations were selected, are presented in supporting information (S2 Fig). Fig 5 shows the results of 150 simulated free falls into the virtual 40cm width corridor and the parameters used. The initial values used in simulation were determined by randomly setting a wingbeat triggering time ranging between 75 and 150ms to fit the data range (Fig 4C). The initial state of the system (i. e. the wingbeat triggering state), θPi, θ ˙ P i, Zi and VZi, was obtained by simulating a free fall without any friction and adding a passive rotation of the body to the pitch dynamics before the onset of wingbeat triggering which was modeled by a third order transfer function (see supplementary materials, S2 Fig). The model accounted successfully for the dynamics of pitch orientation and optic flow observed experimentally during the 0. 2s after the wingbeats initiation (see Fig 5). Fig 6A shows the time course of the mean acceleration produced by the hoverflies, estimated from experimental data after subtracting gravity acceleration. After wingbeat initiation, the acceleration increased immediately to a value around 10m. s−2, which is equal to gravity acceleration absolute, during about 0. 1s. After this initial phase, the acceleration increased within approximately 0. 1s to a value of 25–30m. s−2, representing 2. 5-3 times the absolute value of gravity acceleration, followed by a slight descent phase to around 20m. s−2 at 0. 4s. The force produced by flapping wings in the model was adjusted to these dynamics as shown in Fig 6A (green line). However, the acceleration estimated from 3-D trajectory data is really noisy and could result in some discrepancies between simulated and experimental data. As it can be seen from Fig 5B, the average Z position observed during the simulations shows that the model was able to counteract the fall but the values obtained did not completely match the experimental data on the hoverflies. Nor did the heave and surge speeds match experimental data: they rather showed the occurrence of instability after around 200ms (Fig 5C and 5D).
In this study, control theory was used to model the pitch stabilization process at work in hoverflies placed in free fall situation, using simple rules based on optic flow measurements previously described in navigational tasks context [9]. We proposed a model (Fig 2) based on a virtual fly falling within a textured corridor accounting for the fly’s pitch and speed during about 200ms after the onset of the insect’s wingbeats. As in previous studies [9,10,13,14], we assumed that the pitch control and hence the lift and thrust force control processes rely on the closed-loop control of the pitch rate via the halteres combined with an OF-based feedback loop. In line with [35] and [13,14], we implemented in our model a pitch rate feedback-loop mimicking the halteres via a proportional-integrator (PI) controller. Recent results have suggested that sensory cues delivered by the eyes, halteres and antennae may interact via specific actions and coupling arrangements [37–39]. The present model involves then two nested feedback-loops, one featuring fast dynamics thanks to the halteres and one featuring much slower dynamics due to the presence of a double integrator between the pitch rate and the speed of the fly (see Fig 2). The OF was defined as the ratio between the fly’s speed and its distance to the wall which was constant during the fall and did not vary conspicuously during the 0. 2s analyzed during insects’ flights. The simulated OF measurements are therefore very similar to the airspeeds apart from a different scaling due to dwall. As shown in Fig 1 and described by equation of T →, the forward OF can be controlled directly by adjusting the fly’s speed VRI and thus by controlling the pitch. As shown in Fig 5, a non-null forward OF component was observed during the fall due to a passive pitching of the fly. We simplified the model of hoverflies´ flight dynamics by neglecting any coupling between the pitch control and the other two rotational axes (roll and yaw). There were two main reasons for focusing only on the hoverflies´ pitch attitude control: The main characteristic of the model (see Fig 2) presented here is the total absence of any kind of vertical reference for controlling the pitch in the closed-loop system. This idea was based on previous data showing the absence of graviception in dipteran’s flight control [7]. This model accounts for the fly’s transient response during a period of up to approximately 0. 2s from the onset of the flapping flight. However, as shown in Fig 4B, most of the stabilizing manoeuvres in response to the free fall situation occurred within 0. 2s. The ability of the optic flow model to counteract the fall without requiring any information about the insect’s absolute orientation confirms that optic flow regulation, in addition to navigation processes, may play a stabilizing role [12]. Indeed, a slight tilting of the body and hence of the lift quickly led to a involuntary translation in the environment that results in generating optic flow. Thus, actuating the wingbeats motor system to cancel the generated OF would lead to correcting the attitude. In particular, during an instable flight, the gravity acceleration would induces a permanent increase in the speed toward the ground and a simple strategy such as maintaining a constant forward optic flow will therefore intrinsically induces the pitch to decrease with respect to the horizontal and therefore a restabilization. Indeed, as shown in, the drag force experienced during free fall can be neglected, at least in the range of our experimental paradigm, reinforcing the detection of any heave acceleration by the mean of optic flow variation. In addition, previous studies [8–10] on speed regulation based on optic flow strategy validate the implementation of such feedback loops in flight control system. Future experiments would allow to better describe these sensorimotor regulation by using moving gratings on the walls of the box or a virtual reality setup [40,41] for example. However, some instability in the model´s responses can be clearly seen to have occured after 300-400ms in Fig 6C. This means that a closed-loop pitch control system based only on the optic flow regulation does not suffice to maintain stable flight. Despite the presence of a fast closed-loop control of the pitch rate based on the halteres, a simple PD controller cannot be fast enough to stabilize a system featuring three integrators between the measured forward optic flow and the required pitch (see Fig 2 and eqs 6 and 7). Instead of increasing the complexity of the controller COF (s), we decided to improve the model shown in Fig 2 by adding two biologically plausible feedback loops. First, we added a feedback loop controlling the lift based on the vertical flow ω z R I through a proportional-integrator controller. The integrator cancels the vertical optic flow while keeping a non-null steady lift force, thus simulating the altitude control process observed in dipteran [42,43]. In addition, based on the existence of sensory mechanisms involved in the estimating pitch orientation such as the Dorsal Light Response and that based on an integration of the halteres’ and/or compound eyes’ signal, we added another pitch rate control loop including a Proportional-Derivative controller based on the absolute pitch orientation (Fig 7). Both additional PI and PD controllers has been set manually, gains are given in Fig 7 and all model parameters are summarized in S1 Table (supporting information). These two optic flow and pitch feedback loops combined made it possible to stabilize the simulated pitch and height in steady state (Fig 8). The ability of the improved model to stabilize the hoverflies´ attitude thanks to the addition of a closed-loop control of the pitch orientation suggests therefore a complementary control strategy involving pitch rate, pitch and optic flow measurements. With the model parameters presented here, a single pitch feedback loop would be too slow to stabilize the fly within 200ms as required by the 40cm-high box. Although, the ability of flying insects to estimate their absolute orientation (on the pitch and roll axis) still gives rise to some controversy, the model developed here would certainly provide a basis for studying these sensorimotor reflexes and the coupling that may exists between the sensory modalities involved. It is worth noting that [12] have established that the pitch of an aerial robot can be stabilized without any need for absolute reference value by regulating the dorsal and ventral OF of a 2 degree-of-freedom flying robot. Still our simulation gives controversial results in an unsteady situation such as free-fall recovering. Probably because in their study the rotor forces are adjusted by another control based on ventral optic flow in regard to experimental observation in bees [44]. In addition to visual motion and attitude perception, we can also assume that the fly could probably relies on others sensorimotor reflexes such as those based, for example, on the expansion of the OF [45,46] that we did not challenge in this paper. The hoverflies rely probably on specific sensory channels to estimate its absolute attitude with respect to its environment (i. e. a vertical reference) and to control their attitude as shown by the comparison made here between the two versions of the present model. Although our setup did not include any salient cues such an artificial horizon, the light from above may stimulate the DLR [22,47,48] which could help insects to estimate their attitude. However, a significant improvement in the hoverflies’ ability to stop falling before crashing have been previously observed when the insects were placed in a striped box rather than uniform white environment [7]. The reflex controlling the lift force orientation may not therefore relies solely on a pitch feedback based on the position of the brightest part of the visual field. A combination between optic flow based and DLR-based control may possibly be involved. However, the exact role of the DLR and its contribution to the visual-driven stabilization of insects’ flight is still an open question. It is proposed in the future to investigate more closely how a pitch orientation could be estimated by hoverflies’ sensory system. An argument supporting the idea that pitch estimation is involved in the hoverflies’ response to free fall situations is the relative independence of the responses in regard to the initial conditions. In contrary, our model was found to be over-shooting in short initiation times (75-100ms) and under-shooting in long initiation times (125-150ms). To study the processes that can underlie the insects’ pitch orientation estimation, we can start with some hypotheses: The accuracy of these four hypotheses still remains to be determined, along with the question as to whether any vertical information is really carried by one or more of these processes combined. The comparison with the optic flow strategy made here should help to determine how these various channels combined may serve to maintain a stable attitude during flight.
On the basis of vision-based feedback control of optic flow occurring during insects’ flight, we developed a dynamic model that accounts for the pitch orientation and speed in plummeting flies. We compared the hoverflies’ responses with our model and showed that an optic-flow based control strategy can be used to correct the initial pitch misorientation caused by the free fall situation. To complete the model, we combined the closed-loop control of the vertical optic flow with an additional feedback control loop based on the value of the absolute pitch orientation. The need for this measurement to stabilize the pitch orientation raises the question as whether this is also the case in dipterans. After ruling out the possibility that insects may use gravity acceleration cues to control their flight, for which no experimental evidence has been found so far, we discussed the three main sensory processes possibly involved in in their ability to control their attitude. Our model provides a useful tool for studying the various sensory processes possibly involved in dipterans’ flight stabilization abilities as well as the interactions between these processes.
acceleration
invertebrates
medicine and health sciences
classical mechanics
social sciences
neuroscience
insect flight
biological locomotion
animals
flight (biology)
simulation and modeling
animal anatomy
vision
insect vectors
zoology
research and analysis methods
infectious diseases
wings
disease vectors
insects
arthropoda
physics
psychology
eukaryota
physiology
biology and life sciences
sensory perception
physical sciences
species interactions
organisms
Modeling visual-based pitch, lift and speed control strategies in hoverflies