When deciding between alternative options, a rational agent chooses on the basis of the desirability of each outcome, including associated costs. As different options typically result in different actions, the effort associated with each action is an essential cost parameter. How do humans discount physical effort when deciding between movements? We used an action-selection task to characterize how subjective effort depends on the parameters of arm transport movements and controlled for potential confounding factors such as delay discounting and performance. First, by repeatedly asking subjects to choose between 2 arm movements of different amplitudes or durations, performed against different levels of force, we identified parameter combinations that subjects experienced as identical in effort (isoeffort curves). Movements with a long duration were judged more effortful than short-duration movements against the same force, while movement amplitudes did not influence effort. Biomechanics of the movements also affected effort, as movements towards the body midline were preferred to movements away from it. Second, by introducing movement repetitions, we further determined that the cost function for choosing between effortful movements had a quadratic relationship with force, while choices were made on the basis of the logarithm of these costs. Our results show that effort-based action selection during reaching cannot easily be explained by metabolic costs. Instead, force-loaded reaches, a widely occurring natural behavior, imposed an effort cost for decision making similar to cost functions in motor control. Our results thereby support the idea that motor control and economic choice are governed by partly overlapping optimization principles.
Should I rather bring the groceries from the car trunk to the kitchen in 1 trip or in 2 trips? Even in a seemingly simple decision like this, multiple decision parameters are at odds. When doing a single trip, this bothersome task will certainly be finished more quickly but will require an intense physical effort. This choice might also put one at risk to drop everything on the way. On the other hand, when making 2 trips, each will be less effortful and safer but the task will take longer to complete. When examined through the prism of economics, this example shows 2 alternatives with an equal reward but different amounts and types of costs: risk, time, and effort. Utility theory [1] posits that these decision parameters are combined in a single value, the utility, which characterizes the desirability of each choice as whole. The ways in which costs affect the utility of an option have been well described for risk (prospect theory [2]) and delay (hyperbolic temporal discounting [3]). Defining such a relationship is not straightforward for effort. Physical effort [4], in contrast to mental effort [5], can at least be related to an external, physically measurable variable, in the same way that reward delay is used in the example of temporal discounting. Therefore, we focus on physical effort, defined here as the subjective cost or negative utility associated with a given motor action, independent from the costs resulting from its success rate (risk) or delayed reward (temporal reward discounting). Studies focusing on the brain circuits involved in physical effort-based decision making in humans have used handle-squeezing tasks to produce different effort levels, but they just assumed that subjective effort increases monotonically with isometric squeezing force, without further characterizing the dependency [6,7]. Using a similar task, Hartmann and colleagues [8] showed a quadratic discounting of monetary rewards by squeezing force, suggesting that effort for isometric force production grows proportionally to the square of the force amount. In contrast, by pitting isometric force production with different parameters directly against each other, Körding and colleagues defined effort as a function of both the duration and magnitude of force production, without the need to use an external monetary scale [9]. By using 2 parameters in a force-production task, this latter study highlighted the multifaceted nature of physical effort. The use of isometric force-production tasks to probe physical effort discounting is, however, still limiting compared to the full range of effort-related parameters one could experience when deciding between actual movements. Here we characterize the influence of duration, distance, direction, and force on subjective effort costs in actual reaching movements. From the perspective of motor control, planning and executing a movement, even towards an unambiguous goal, requires commitment to a specific motor act among an infinite amount of potential ways (“choices”) to acquire the goal. Decision making in this context can be seen as part of a continuum that includes motor planning and motor control, and minimizing various cost functions is a core concept of motor control: the stereotypical nature of movement trajectories and velocity profiles has been attributed to minimization of hand jerk [10], endpoint variance [11], and even control effort itself [12,13]. The potential tight link between decision making and motor control is supported by studies showing that action selection can take into account parameters that are related to movement execution, such as biomechanics [14,15] or motor accuracy [16]. Conversely, the vigor with which a movement is executed was shown to be explainable through delay discounting [17]. This raises the question of whether the subjective cost of an action as computed in a decision-making context (i. e. , what we call effort here) is comparable to potential cost functions used for optimization in motor control or, as an alternative, to the metabolic cost of the movement. Here we address this question by investigating how humans assess subjective physical effort in action-selection tasks involving binary choices between different reaching movements. In a first experiment, we varied movement duration, amplitude, and direction as well as resistive force in order to derive isoeffort curves in this duration–amplitude–direction–force space. This allowed us to independently test the sensitivity of subjects to impulse (force × duration) and work (force × amplitude) exerted during movements. In a second experiment, we pitted repeated identical movements against single movements with different resistive forces in order to obtain more precise estimates of the relationship between force and subjective effort in reaching movements.
In both experiments, subjects performed 2-alternative forced choice (2-AFC) tasks in which they compared 2 different actions and were asked to choose the least effortful one (Fig 1). To make informed choices in each trial, subjects first performed both proposed actions (sampling) and then reported the least effortful action by executing it again (choice). The need to repeat the chosen action rendered the choice relevant for the subjects, since genuine selection minimized the overall task effort. Both actions consisted of reach movements performed against different levels of resistive force. In each trial, one of the proposed actions served as a reference action, while the other served as a test action. Note that this distinction was not indicated and not relevant to the subjects but was part of our adaptive experimental design. Within each task condition, the trial-to-trial resistive force level in the test movements was selected with a staircase algorithm [18], while the force level of the reference movements was kept constant. As a consequence, the staircases converged to the force level at which the test action was perceived as being equally effortful as the reference action (equivalent force). In experiment 1, reference and test actions consisted of single movements, differing primarily in amplitude or duration. More precisely, in each trial of the amplitude session, subjects had to choose between 2 movements that differed in amplitude, direction, and force (after sampling both). Conversely, in each trial of the duration session, the choice was between movements that differed in duration, direction, and force. In both sessions, the staircase algorithm adjusted the forces of 1 of the movements, depending on the choice of the subjects, until both movements were subjectively equivalent in effort for the subject. This allowed us to construct isoeffort curves in the force–amplitude–duration movement parameter space (Fig 1A–1D and Methods). In experiment 2, the reference action consisted of 2 identical repeated movements and the test action of a single movement. This allowed us to determine the scaling of subjective effort with force (Fig 1E and 1F). In experiment 1, we asked subjects to conduct naturalistic reach movements against different force levels, either with varied durations independent of amplitude (duration session) or with varied amplitudes independent of duration (amplitude session). As we used constant force profiles, these constraints correspond to dissociations either in impulse (force integrated over time) or work (force integrated over distance), respectively. Fig 2A and 2B depict the average work and impulse produced by the manipulator as a function of force for the different duration and amplitude conditions in both sessions for a representative subject (both integrated from the time of movement onset minus 100 ms to movement offset plus 400 ms). Impulse values were well separated in the duration session but not in the amplitude session, while work values were well separated in the amplitude session but not in the duration session. This confirms that visual instructions about reach-target location and requested movement duration together with the manipulator-controlled resistive force successfully constrained the actual movements of the subjects to the desired parameter ranges in each session (sample trajectories and generated force profiles in S1 Fig and S1 Text). Importantly, the forces imposed by the manipulator had an additive effect on the torques that the subject’s arm actually needed to produce to generate the movements. Since the imposed forces were independent from arm kinematics, a simple biomechanical model of the arm (S3 Text, S3 Fig, S4 Fig) showed that over the duration of a movement, the torques the subjects produced to compensate the imposed forces outweighed the torques produced to compensate for the inertia of the arm. As a consequence, the total work and impulse actually produced by the subjects in the different conditions showed dissociations comparable to those observed in Fig 2A and 2B. Subjects’ choices did not systematically depend on performance differences in the various task conditions of experiment 1, but they depended reliably on force levels (S2 Fig, S2 Text). As a consequence, we could use equivalent forces to titrate the effort subjects associated with the explored movement parameters. In the amplitude session of experiment 1, the equivalent forces did not vary significantly with movement amplitude (linear mixed-effect model [LME], p = 0. 4, effect size 0. 3 N) but varied significantly with reference force level (LME, p < 0. 001, effect size 3. 4 N) (Fig 2C). The result indicates that the 4-N difference between the reference forces was large enough for subjects to judge it as different in effort but that subjects were unaffected in choice by movement amplitude over the tested range when movement duration was kept constant. In other words, a movement of 120 mm against a force of 6 N was rated as effortful by the subjects as a movement of 200 mm against a force of 6 N. Since movement duration was constant in the amplitude session, the observed insensitivity to movement amplitude can also be interpreted as insensitivity to movement speed. In contrast, subjects were sensitive to movement duration when movement amplitude was kept constant in the duration session (Fig 2D). Equivalent force levels were lower for long-duration movements than short-duration movements (LME, p < 0. 001, effect size 3. 8 N). Here, a movement in the 1,300–2,000 ms range against a force of 5 N was judged as effortful as a movement in the 0–800 ms range against a force of 9 N. This indicates that long-lasting reaches were perceived more effortful than brief reaches against the same force level. Additionally, the equivalent forces scaled with the reference force levels (p < 0. 001, effect size 3. 1 N) without interaction between the factors force and duration (p = 0. 4). Physical movement parameters like work or impulse, as defined above, describe the movement properties at the manipulator handle (endpoint movement), irrespective of the required multijoint arm movement. Instead, effort evaluation and the resulting choice were based on subjective experience, to which the biomechanics of the movement could have contributed. In fact, biomechanics of the movement influenced effort judgment in our experiment. In both the amplitude and the duration sessions of experiment 1, equivalent forces depended on the reference movement direction (Fig 2C and 2D). Test movements performed inward (i. e. , towards the left for right-handed subjects and vice versa) required higher force levels to be judged as equally effortful as outward movements (LME, amplitude session: p < 0. 001, effect size 1. 5 N; duration session: p < 0. 001, effect size 0. 9 N). This indicates that at the same force level, outward movements were considered more effortful than inward movements. This difference in equivalent force is likely linked to the use of larger muscles and the higher available strength for inward movements. After showing that duration and biomechanics but not amplitude had an influence on the effort judgment, we asked how effort would depend on the force itself. In Fig 2D, a thin dotted line represents an example isoeffort curve in the force–duration space: it connects the point representing parameters of a reference movement to the points representing parameters of equivalent-force test movements. Similar to this example curve, when averaging over movement directions in the duration session, isoeffort curves are convex for both reference force levels. This indicates that the putative effort cost function supporting the subjects’ choices was a nonlinear combination of force and movement duration. This is because if effort was a linear combination of force F and duration d, such as E (F, d) = αF + βd, the isoeffort curve defining the equivalent force of the test movement would have to be a straight line defined as FT=αFR+βdRα−βαdT. If, on the other hand, effort was a purely multiplicative function of force and duration—i. e. , assuming that the effort cost function is impulse E (F, d) = Fd—then this would lead to convex isoeffort curves shaped as the inverse function FT=FRdRdT. The isoeffort curve in Fig 2D is not straight, and the curvature does not fit the impulse model but is shallower instead. In experiment 2, we determined the precise shape of the force–effort relationship for constant movement durations. Binary choices between 2 options only allow ranking the options in terms of preferred or nonpreferred. To describe effort as a function of force, additional information is needed to turn such a ranking into a scale with a continuous metric. This could be achieved by trying to compensate the effort cost of an action with an independent scalable benefit (e. g. , monetary reward) to achieve equal preference for movements of different force. But the utility function of the benefit must then be known, a task which might be as difficult to achieve as the task of determining the effort cost function itself. As an alternative, in each trial of experiment 2, subjects chose between 2 options after having sampled both: either they opted for performing 2 similar movements in rapid succession against a reference force level FR (the endpoint of the first movement is the starting point of the second movement), or they chose to perform a single movement against a test force level FT (adjusted between trials by a staircase). Here we assume that an action consisting of 2 identical movements is twice as effortful as an action consisting of 1 of these movements, but both give the same benefit (finishing the trial). Under this assumption, the equivalent forces FTeq (staircase convergence point for forces FT in single test movements) as a function of the reference movement force level FR (fixed forces in double movements) should follow the rule E (FTeq) = 2E (FR), that is FTeq = E−1 (2E (FR) ), with E (F) being the function linking force and subjective effort (= cost function) and E−1 being its inverse. As in experiment 1, the observed decision behavior in experiment 2 was best explained by force-based choices rather than performance-based choices (S2 Text). Results from experiment 2 thus allowed to test and fit models for both the cost function and its link to decisions, which we carried out using a Bayesian modeling approach (see Methods). A direct observation of the equivalent force as a function of the reference force level is suited to highlight the properties required of E (F). We computed equivalent forces in 2 ways: first as averages of the test-force levels at the staircase inversions (i. e. , the asymptotic force to which the staircase procedure converged as in experiment 1) and second via points of subjective equality of the psychometric functions that resulted from the Bayesian model. Results of both approaches are illustrated for 2 example subjects in Fig 3A and 3B. Equivalent to the considerations regarding the isoeffort curves in experiment 1, the simplest putative effort cost function is a linear function, E (F) = αF. For this, equivalent forces would have to obey the equation FTeq = 2FR (steepest dashed red line), as we required E (FTeq) = 2E (FR). However, in our data we observed that for large reference forces, the equivalent forces were smaller than predicted from the linear model. For example, for a FR of 9 N, subjects JP and MK showed equivalent forces of 12 and 14 N, respectively, instead of 18 N. This observation indicates that there was a convex nonlinear relationship between force and effort, in line with the results obtained from experiment 1. Additionally, we observed that the equivalent forces for a FR of 0 N were larger than 0 N, confirming the intuition that a movement against no external force still has nonzero effort (i. e. , that E (0) > 0). Therefore, a power function with an offset, (F) = Fα + β, appears as a reasonable minimal model for the force–effort relationship (Eq 2; see Methods), which we will use in the following sections. Our Bayesian modeling approach used the trial-by-trial choices of subjects as dependent data. Thereby, within the same unified framework, we simultaneously modeled (1) how forces affected effort values (see paragraph above, Eq 2) and (2) how the subjects’ choices depended on these effort values (Eq 1). For the dependence of choice on effort, we answered 2 questions. Does utility show a subtractive discounting by effort (i. e. , the difference in utilities on which decisions are based is equivalent to a difference between efforts in our task [Eqs 4 and 5]) or a hyperbolic discounting by effort (i. e. , the difference in utilities corresponds to a difference between effort inverses [Eqs 6 and 7]). Within each of these 2 alternatives, we tested whether effort was represented in a linear (Eqs 5 and 6) or logarithmic scale (Eqs 4 and 7). This 2 x 2 design resulted in 4 alternative models obeying E (FTeq) = 2E (FR), all with the same number of free parameters (see Methods). Notably, all tested models had the same equation for the equivalent force curve (Eq 3); the models only varied in the shapes of the choice curves—in particular, how the reference force FR affected their slope. We assessed model quality on the basis of 3 criteria. First, the percentage of correctly predicted choices (binary predictions based on a comparison between the actual test force and fitted equivalent-force levels) reflected the validity of the equivalent force curve as a decision boundary (in red on Fig 3A and 3B). Second, we evaluated the fit of the model to the full choice probability curves (in blue on Fig 3A and 3B) by examining the corresponding residual distribution (Fig 3F). Last, each fit was also tested by using the Watanabe–Akaike Information Criterion (WAIC) [19], an approximation of cross-validation that allowed us to compute the relative likelihood between models. The model we ultimately selected (and show in Fig 3A and 3B) expressed choice probability as a function of the difference between the logarithms of reference and test efforts (Eqs 1,2 and 4) and outperformed all other models, which we discuss below. The simplest model obeying E (FTeq) = 2E (FR) was based on a difference between test and reference efforts (Eq 5, no logarithms). The difference model predicted subjects’ choices as well (68. 3% correct predictions) as the selected logarithmic difference model (68. 7%), meaning that the equivalent force curves were similar. But the difference model did not fit the choice probability curves nearly as well (WAIC of 4. 50e3) as the selected model (4. 43e3), making the difference model less likely by a factor of 6. 3e–16 compared to the selected logarithmic difference model. Indeed, the distribution of residuals for the simplest model (in red, Fig 3F), is wider than the distribution of residuals for the selected model (in grey, Fig 3F). Both alternative models based on hyperbolic discounting of effort, whether computing choice probability from the difference between the inverse of efforts (Eq 6) or the inverse of effort logarithms (Eq 7), showed lower prediction performances (66. 9% and 63. 8%), higher WAICs (4. 76e3 and 5. 09e3), and wider residual distributions than the selected model and therefore had to be rejected. The results of this model-selection approach thus favor a subtractive discounting of utility by effort (no hyperbolic discounting) and a logarithmic internal representation of effort. By examining the posterior distributions of the selected model’s parameters (Fig 3C–3E, Eqs 1 and 2), we can interpret the equivalent force curves obtained in our subject population. Our main parameter of interest was the population average of the force exponent α in the power function described in Eq 2, as it describes the nonlinear dependency of effort on force. The exponent α showed a narrow posterior distribution centered around 2. This means that on average, the subjective effort rose with the square of the resistive force against a movement. Since the distribution is narrow, the confidence in this estimate is high (95% credible interval for the population average: 1. 56–2. 48). For completeness, the posterior for the effort offset β (95% CI: 4. 9–22), reflecting the effort that subjects associated with performing a movement against no resistive force, and the posterior for γ (95% CI: 1. 1–4. 2), reflecting the sensitivity of subjects to effort differences, are represented in Fig 3D and 3E, respectively. The biomechanics seemed to have played less of a role in experiment 2. For the same reference force levels, we did not obtain different equivalent test forces between the 2 movement directions inward and outward. However, in contrast to experiment 1, subjects never directly compared movements with different directions, as both options were in the same direction. This difference likely made experiment 2 much less sensitive in that respect.
Most studies on physical effort in human decision making operationalize effort by asking subjects to squeeze a handle that measures hand grip force [7,8, 20,21], a device that is easy to use and fMRI-friendly. The effort is then an isometric contraction of varying magnitude, expressed as a percentage of the maximum voluntary contraction (%MVC) that each subject can produce. Subjects are typically required to choose between 2 squeezes with different grip forces, each associated with different rewards or additional decision factors (delay or risk). Hartmann and colleagues associated monetary rewards with grip forces and reported that among linear, hyperbolic, and quadratic effort cost functions, the quadratic cost function explained the subjects’ behavior best [8]. Klein-Flügge and colleagues used a similar task to compare effort discounting and delay discounting and reported that effort cost seemed best described as a sigmoidal function, i. e. , showed a convex dependency for lower forces, as Hartmann and colleagues did, but Klein-Flügge and colleagues found a concave relationship for forces closer to MVC because of saturation of effort cost [20]. Burke and colleagues, instead, compared the integration of physical effort and risk in a similar task and reported a sharp increase of effort cost when approaching MVC [21]. In contrast, Prevost and colleagues used rewarding erotic images instead of money, but found effort cost to fit a hyperbolic function [7]. This means that for isometric force production, the effort cost function is still uncertain or at least depends on the choice task (reward, risk, or delay discounting), while for actual movements hardly any previous data exist. To answer how effort depends on force in our movement task, we first have to address the question of how choice is best linked to effort, since choice is the behavioral readout, while effort is a hidden decision variable. Apart from the study by Prevost and colleagues, the aforementioned studies rejected the idea that choice behavior takes into account physical effort in a similar fashion to delay (i. e. , by hyperbolic discounting). This is not surprising since, intuitively, high physical effort cannot make the subjective utility of a choice decrease asymptotically to zero (as hyperbolic discounting does because of the increasing denominator). Indeed, the subjective utility of a high-effort, low-reward action could well be negative, in which case doing nothing would be preferable. As a consequence, the cost of effort is a value that should be offset from the benefits of an action in the utility space—i. e. , the utility of each action should be computed by subtracting the associated effort. This intuition is confirmed by the results of our experiment 2: models in which the decision variable was a difference between efforts predicted the subject’s movement choices better than models in which the decision variable was a difference between the inverse of efforts. Effort studies with isometric force production modeled the probability of choosing each option by transforming the difference of efforts between the 2 alternatives with the softmax function [7,20]. Here we used an equivalent probit transformation but showed that using the difference of the effort logarithms yielded significantly better results than using the difference of efforts. Indeed, the difference of logarithmic effort as choice variable best captured the decrease in sensitivity we observed for higher efforts (Fig 3A and 3B). How does force affect effort in transport movements? Previous isometric contraction studies do not allow generating a good prediction for this question. The aforementioned studies modeled the subjective cost of effort as convex functions of force expressed as %MVC and titrated efforts against rewards. However, both these features of the experimental design could overemphasize the convexity of the effort cost function. First, producing a force stronger than MVC is by definition impossible. Therefore, the effort cost likely has to undergo a sharp increase when approaching this discontinuity in designs that use the full 0%–100% range of MVC as a force scale. Indeed, Hartmann and colleagues and Burke and colleagues noted that subjects always chose the effortful option when it provided more reward, except when the effort was close to MVC [8,21]. Second, the tendency to almost always make reward-based choices while ignoring moderate efforts also suggests that monetary rewards are too strongly motivating for typical subjects and may not be appropriate to study the cost of the moderate efforts. However, moderate efforts are essential in experimental settings to allow large numbers of trials and to stay away from the MVC discontinuity. In conclusion, a paradigm that uses moderate absolute forces instead of %MVC forces and in which effortful actions are not associated with monetary or social rewards but are compared directly seems more suited to precisely determine effort cost functions. Such a paradigm, which we partly adopted here, was introduced for isometric forces by Körding and colleagues in a study in which subjects had to resist against imposed force profiles of variable magnitudes and durations [9]. This previous study led to a different effort cost function than the one we found. When subjects had to choose between dual and single contractions with the same force profiles, but in which force and duration were varied together, a loss function of the form (FT) α gave the best fit for = 1. 1. Assuming that this fit can be generalized to constant durations, the resulting F1. 1 relationship would indicate a quasilinear influence of isometric force on subjective effort. In contrast, in our experiment 2, in which we extended this approach to actual effortful transport movements and isolated force dependency by keeping duration constant, we found a more convex F2 relationship. Hence, our result is closer to the results obtained in studies using %MVC despite the use of a different task, force scale, and fitting procedure (the force exponent was a free parameter in our model, in contrast with [8]). Nevertheless, our use of moderate forces prevents us from generalizing our findings to movements realized against higher levels of forces (closer to MVC), for which large accuracy and duration changes might bias choice preferences independent of force-dependent effort. In summary, the cost of effort as a function of isometric muscle contraction force has previously been shown to take various forms. Yet, we mainly attribute differences to the quadratic discounting that we observed here to 2 facts that were not fully considered previously. First, compensating effort with rewards is difficult because of uncertainties about the reward utility itself and the need to use large forces. Second, even when avoiding reward–effort competition and comparing effortful actions directly, interactions between force and duration need to be avoided or compensated for, since duration contributes to both effort and reward discounting and thereby may distort measured force–effort relationships, as will be discussed in the following paragraph. To properly understand physical effort in movements, it is important to disentangle the different contributions of force and duration. A previous study obtained V-shaped isoeffort curves in the duration–force space when pitting isometric contractions with force profiles of different durations and magnitudes against each other [9]. For durations below 250 ms, an increase of duration required a sharp decrease of force to maintain effort constant; for durations above 500 ms, the opposite was observed. In other words, contractions of long durations (1–2 s) were considered as effortful as shorter contractions, even with slightly lower forces. This observation contradicts the intuition that longer contractions should be more effortful than shorter contractions. Moreover, performing short contractions allowed finishing the experiment more quickly since total trial duration was not controlled for; therefore, delay discounting should have additionally devalued longer movements. Körding and colleagues interpreted their result as a consequence of increased control difficulty for fast force changes: it was easier for subjects to resist against stronger force profiles when the onset and offset of the forces were slower, which was the case for long-duration force profiles. In this sense, their results marked a compound effect. In contrast, in our task we tied the onset and offset of forces to self-timed movements. This rendered force control less difficult, and, as a consequence, we observed monotonically decreasing isoeffort curves in the duration-force space (Fig 2D) —i. e. , effort increased monotonically with both duration and force, as intuitively expected. Other recent work argued that increasing movement durations requires smaller and smaller decreases of force to maintain effort constant (i. e. , that perceived effort reaches an asymptote instead of growing linearly with duration in isometric force productions). To explain such a counterintuitive effect, the authors assume that effort costs are subject to the same temporal discounting as can typically be observed for reward in economic choice behavior [22]. This hypothetical explanation is, however, not applicable to our results, as we kept the total duration of trials constant. The isoeffort curves observed in our experiment 1 can be explained by the quadratic relationship between force and effort. In conclusion, for effortful transport movements like reaches, effort increases monotonically with movement duration, suggesting that effort is integrated over time. The observations from experiment 1 and 2 provide insight into the internal cost function used by subjects to decide between arm movements. This effort cost function for decisions could potentially be paralleled with motor control cost functions or with the actual metabolic cost of movements. In other words, we can examine whether the choice made by subjects between proposed movements with constrained parameters (duration, speed, force, etc.) reflect natural preferences in the execution of unconstrained movements or minimization of energy expenditure.
Subjects performed the tasks by holding and moving the spherical handle of a parallel-type haptic manipulator (Delta. 3, Force Dimension, Nyon, Switzerland) with their dominant arm (Fig 1A). The manipulator was connected to a computer running our own custom-written software (C++, OpenGL) in charge of visual stimulus presentation, task event control, force computation, and associated data recording. The manipulator and the computer communicated bidirectionally at 2 kHz, with the manipulator sending the 3D position of the handle and the computer requesting forces to be applied at the handle for each iteration of this 0. 5-ms haptic cycle. The movements of the manipulator handle were reproduced in real time for the subject via a spherical yellow cursor displayed in a stereoscopic augmented-reality (3D-AR) environment. Display and haptic device latency were fully compensated by a forward prediction to achieve synchrony between visual cursor and handle movement (Kalman filter with position, speed, and acceleration as state variables). The 3D-AR environment consisted of 2 computer monitors (BenQ XL2720T, screen size 590 x 338 mm, 60-Hz refresh rate, distance 45 cm, Matrox DualHead2Go DisplayPort splitter) that were placed to either side of the subject with the screens facing each other. The subject viewed the screens through a pair of semitransparent mirrors that were angled at 45° relative to the screens. This allowed for the creation of stereoscopic 3D visual stimuli that were perceived as being projected into the haptic device’s workspace. In addition to the visual cursor, which always coincided with the handle' s current physical position, other visual stimuli indicated the starting points and targets of the reaching movements as well as text information. The 3D-AR haptic interface was calibrated for each subject. For this, we made the actual manipulator handle visually coincide with multiple visual targets sequentially presented in the virtual space. Since the control software selected the visual target locations, the manually adjusted handle position could be used to compute the manipulator-to-display transformation matrix for the current geometry of the setup. This calibration was then further adjusted for each subject by setting the location and projection matrix of the virtual openGL cameras according to the subject’s interpupillary distance. To allow the subjects to comfortably operate the haptic manipulator, both monitors and mirrors were tilted to lower the location of the 3D representation (Fig 1A; angle relative to horizontal: 30°). For the same reason, we defined a virtual plane in front of and parallel to the monitor image plane in which all movement targets appeared (distance to mirrors was 430 mm). Subjects were also encouraged to take breaks and relax their arm as frequently as desired. In order to limit the force output of the manipulator to task-relevant forces only, and to allow for natural movement trajectories, subjects could freely move the cursor around the entire spherical workspace of the haptic device. Correct depth perception of the 3D stimuli was thus required for the subjects to be able to acquire the movement targets. Before each experimental session, subjects were trained on simple versions of the tasks in order to familiarize themselves with the setup and the task requirements, notably 3D vision, resistive forces, and time constraints. The haptic manipulator produced forces that resisted the subjects' movements. Our aim was to produce a force with constant magnitude that was only present during the movements and that opposed the instantaneous movement direction, similar to a kinetic friction force. The direct definition of this friction force would thus depend on the velocity of the handle. Yet, the force command sent to the manipulator was not computed directly from online estimates of handle velocity (which is difficult at low speeds) to prevent sudden force onset and direction inaccuracies at low handle speeds. Instead, we implemented the friction force using a virtual point-mass (virtual mass = 100 g) that was connected to the handle via a virtual spring (coefficient = 1 N. m-1). In other words, subjects dragged a virtual mass with the help of a spring, and the kinetic friction force was computed according to the speed of the virtual mass and applied to it. The magnitude of this kinetic friction force (in N) was varied in order to produce the different resistive force levels. For each iteration of the haptic cycle, the dynamic state of this virtual mass was updated according to the forces applied to it (= sum of the spring force and the friction force), and the force resulting from the virtual spring was sent to the haptic manipulator as a command. The position of the virtual point-mass was reset to the handle location, and thereby the spring force set to 0, before the start of each movement. Additionally, the commands sent to the haptic manipulator were modulated by an envelope function (a constant function with linear tapers of varied durations at onset and offsets), which allowed controlling force output outside of the defined movement periods. The course of events of a trial in experiment 1 is presented in Fig 1B (example trial from the amplitude session). Each sampling subtrial started with the subject placing the cursor (6-mm diameter, yellow sphere) within a fixation sphere (20-mm diameter, grey, brightening upon acquisition) and holding this position for a duration randomized between 500 ms and 800 ms. The movement target sphere (diameter 30 mm) was displayed from the start of the subtrial; its color indicated to the subject the modalities of the movement to be executed: across both sessions, a green target indicated the need for a rapid movement (a short-duration movement in the duration session or a large-amplitude movement in the amplitude session), whereas blue and red indicated medium and low speed, respectively. During the acquisition and hold stages, onscreen text announced which sampling movement was currently being performed (“sampling 1” or “sampling 2”). When the fixation sphere disappeared (“go” cue), the subject had to execute the required movement by placing the cursor within the target sphere within the requested time constraints. Movement duration was computed from movement onset (determined online by a combination of speed and distance thresholds) to target acquisition (determined only based on cursor position relative to the target). Resistive forces were turned on when the fixation sphere was acquired (on-taper: linear increase to the desired force value within 200 ms) and were turned off when the target was acquired (off-taper: linear decrease to 0 within 600 or 500 ms in case of successful or failed acquisition). Note that the actual force production by the manipulator was dependent on the subject’s movement and only started at movement onset (see" Force generation" section). If the subject acquired the target faster than the minimum duration set for the movement, or if the subject did not reach the target before the maximum duration set for the movement, the subtrial was interrupted and onscreen text indicated to the subject the type of error committed (“too fast” or “too slow”). Failed subtrials, which also included trials in which the subject broke fixation or left the target too early, were restarted until executed correctly. Once the target was acquired (with movement duration dm), which was signaled by the target sphere becoming brighter, the subject had to hold the cursor within the sphere for a total duration (in ms) dh = 100 + dmmax − dm, with dmmax being the maximum potential movement duration in the session (2,000 ms in the duration session, 1,250 ms in the amplitude session). This ensured that every subtrial had the same duration across conditions within a session, thus preventing temporal discounting, here equivalent to the desire to terminate the experiment early by preferably selecting short movements. After sampling both the test movement and the reference movement by performing each sampling subtrial successfully, the subject had to indicate in the choice subtrial which movement felt less effortful. This subtrial, announced by a “choice” onscreen text, started with the subject acquiring a pre-fixation sphere. Then, the fixation spheres and targets for the 2 alternatives were displayed, and the subject indicated their choice by acquiring the fixation sphere of the chosen movement (Fig 1B, right column). With acquisition of the chosen movement’s fixation sphere, the fixation and target spheres of the nonchosen movement disappeared, and the rest of the subtrial was identical to the sampling subtrial that corresponded to the chosen movement. Subjects performed experiment 1 over 2 sessions on different days, with each session lasting on average 70 minutes. In the duration session, reference and test movements differed in allowed movement duration but had the same amplitude. This allowed us to construct isoeffort curves in the force–duration space. Conversely, in the amplitude session, the reference and test movements differed in amplitude, but not in duration, which allowed us to construct isoeffort curves in the force–amplitude space. With the use of constant magnitude force profiles, the duration and amplitude session allowed us to double-dissociate total impulse (Jx=∫tstarttstopFxdt) and work (Wx=∫tstarttstopFxdxdtdt), respectively. The reference movements had a medium duration in the duration session (800–1,300 ms) and medium amplitude in the amplitude session (160 mm). In both sessions, reference movements were performed against either 6 N or 10 N of resistive force and could be directed either to the right or to the left. The test movements required either low or high values for the session variable of interest (low or high duration or amplitude) and were carried out in the direction opposite to the reference movements (Fig 1D). These combinations lead to 8 conditions per session, which were presented to the subject in a randomly interleaved manner (2 reference movement force levels × 2 reference movement directions × 2 test movement levels of duration or amplitude). For each of these 8 conditions, independent pairs of staircases determined the force level against which the test movements were performed. These staircase pairs followed a one-up one-down rule with a step size of 2 N, with one staircase starting at 0 N and the other at 16 N (the highest force the manipulator could sustainably produce) to compensate hysteresis. In other words, when the subject chose the reference movement in a given trial, the force level of the test movement in the next trial of the same condition and staircase would be decremented by 2 N; and vice-versa, when the subject chose the test movement, the force level of the next test movement of the same condition and staircase was incremented by 2 N. Data collection for each staircase was considered complete after 7 inversions in the subject’s choices. In this subjective-choice task, the choices of the subject could sometimes lead the staircase procedure to propose force values beyond the capabilities of the manipulator (above 16 N) or the interest of the task (below 0 N). As a consequence, the force values were clamped between 0 and 16 N, and each time the force stayed at these boundaries for 2 trials in a row, an inversion was counted in order to allow the staircase to terminate eventually. We used the average force at staircase inversions to determine isoeffort forces in experiment 1. Because of the clamping of the staircase forces, the isoeffort forces were also bounded between 0 and 16 N, which could have caused an underestimation of the observed effects in the rare cases in which the subject stayed at the clamped force limits. Target and fixation locations were selected to avoid confounding biases. Across all 8 conditions, the targets were not placed further than 100 mm from the workspace vertical midline to prevent the effort of reaching towards large eccentricities, which would be considered a confounding factor. To achieve this, the different movement amplitudes in the amplitude session were created by offsetting the locations of the fixation spheres while keeping the targets at constant eccentricities (Fig 1C, vertical dotted lines). For this reason, the 2 alternative movements (towards left and right) also had to be placed at different heights on the workspace (40-mm vertical distance). This made them visually more distinguishable for the subjects, especially in the choice subtrial, in which both targets and both corresponding fixation spheres are displayed. Importantly, the prefixation sphere in the choice subtrial was placed halfway between the 2 alternative fixation spheres to prevent subjects from choosing the movement starting closest to the current cursor location. Both fixation spheres were visually identical and were identified by their vertical position, which was the same as the target of the corresponding movement. While experiment 1 was designed to explore isoeffort curves in the force–duration–amplitude spaces, the similar experiment 2 was designed to provide more details about the shape of the force–effort relationship. Instead of executing a single movement in the reference action, subjects performed an identical movement twice in experiment 2, while the amplitudes and durations of all movements were kept the same across conditions. Assuming that executing a movement twice doubles the associated effort, the experiment allowed us to determine how much force in a test movement was needed to double the effort of a single movement from the reference action. Subjects performed experiment 2 in a single session (average duration 140 minutes). To repeat the reference movement, 2 targets were presented in the corresponding subtrial, and subjects performed 2 reaches in succession. The location of the first target was used as a starting point for the second movement such that no additional movements were required (additional movements would cause more than doubling of effort). Targets were placed such that the 2 movements matched in reach direction and amplitude (Fig 1E). In a reference action, after the movement to the first target, the subject had to maintain the cursor in its location for 500 ms, after which the first target disappeared, indicating to the subject to perform the movement to the second target. The resistive force was tapered in and out for each of these movements (50-ms onset taper on “go” cue and 400-ms offset taper on target acquisition). In all other aspects, the course of events for experiment 2 is identical to experiment 1. In experiment 2, individual movements had a 120-mm center-to-center amplitude and were time constrained between 800 and 1,300 ms (matching the short-amplitude, mid-duration reaches of experiment 1, Fig 1F), for both individual reference movements and the test movements. The total duration of each action, starting from the time the subject left the fixation point to the end of the subtrial, was maintained constant over all subtrials by adding an additional waiting time when holding the target, resulting in a total subtrial duration of 4,000 ms. Contrary to experiment 1, both reference movements and the test movement were in the same direction in each trial. Four levels of reference movement force were probed (0,3, 6, and 9 N), while the forces for the test movements were determined using the same staircase procedure as in experiment 1. Therefore, there was 1 staircase pair for each of the 8 conditions (4 reference movement force levels times 2 movement directions). Data processing and statistical analysis were carried out using Matlab and the gramm [34] toolbox for plotting. In experiment 1, averaged staircase inversion points were analyzed using LMEs (fitlme function in Matlab). For each session, we constructed mixed-effect models fitting the average force at staircase inversion points depending on the varied parameter of the session (duration or amplitude: low, high), reference movement direction (relative to subject handedness: inward, outward), and reference movement force (low, high). All these independent variables were treated as categorical variables. The mixed-effect model included separate random intercepts and random slopes for movement duration and for amplitude across subjects. Main effect sizes were extracted from models without interaction terms. Interactions were tested in separate models and their significance was evaluated by model comparison. Choice data from experiment 2 were modeled using Stan [35], a probabilistic programming language, through its Matlab interface. We used Stan to perform Bayesian inference, using its default implementation of a Markov chain Monte Carlo sampler (NUTS). We fitted a probit hierarchical model, in which the choice of the reference movement in each trial is modeled as a Bernoulli distribution in which the associated probability P (R|FT, FR) is a function of the difference in utility between the test and the reference movement, and the utility for each movement depends on the corresponding movement force. Variations of the model will differ in the way utility is expressed as a function of force-dependent effort. Therefore, for subject i: P (R|FT, FR) i=Φ (U (2Ei (FR) ) –U (Ei (FT) ) γi) (1) where Φ is the cumulative density function (CDF) of the standard normal distribution (probit link), Ei (F) is the effort of a movement executed against the force F for subject i, and U is the utility as a function of force-dependent effort. The factor 2 reflects our assumption that repeating a movement twice should double the effort compared to a single movement and thus imposes the constraint E (FTeq) = 2E (FR). Effort itself in all variations of the model was modeled as power-law function of force: Ei (F) =Fαi+βi (2) The constraint E (FTeq) = 2E (FR), applied on Eq 2, yields the following equation for the equivalent force curve: FTeq=2FRαi+βiαi (3) The force exponent αi, the effort offset βi, and the effort sensitivity γi for each subject were drawn from normal distributions αi ∼ N (μα, σα), βi ∼ N (μβ, σβ), γi ∼ N (μγ, σγ). The resulting parameters of these normal distributions characterize the population-level distributions for α, β, and γ. Bayesian inference requires providing prior distributions for these parameters, which were chosen wide to not constrain the model: μα ∼ N (1,10), μβ ∼ N (0,100). The parameters μγ, σα, σβ, σγ were positive scale parameters and their priors each followed the same half-Cauchy distribution [36] with parameters (location = 0; scale = 20). Posterior distributions were sampled using 4 Markov chains with 1,000 samples each (after a warmup of 1,000 samples). To test our model against alternative hypotheses, we varied the function of the choice probability (Eq 1). We then compared the individual model fits using the WAIC [19], an approximation of cross-validation. In our first model, utility is the negative logarithm of effort and choice probability thus depends on the difference between effort logarithms: U (E) =−log⁡E (4) As second model, we used a simpler model for the choice probability that is based on the difference of effort values and not the log-ratio (difference of their logarithms): U (E) =−E (5) Third, we tested the hypothesis of hyperbolic effort discounting with a model in which effort is on the denominator in each utility term (inverse effort): U (E) =1E (6) Finally, in a fourth model we modified the hyperbolic model to use logarithmic effort: U (E) =1log⁡E (7)
Economic choice in humans and animals can be understood as a weighing of benefits (e. g. , reward) against costs (e. g. , effort, delay, risk), leading to a preference for the behavioral option with highest expected utility. The costs of the action associated with a choice can thereby affect its utility: for equivalent benefits, an action that requires less physical effort will be preferred to a more effortful one. Here, we characterized how human subjects assess physical effort when choosing between arm movements. We show that the effort cost of a movement increases with its duration and with the square of the force it is performed against but not with the distance covered. Therefore, the subjective cost that determines decisions does not reflect the objective energetic cost of the actions—i. e. , the corresponding metabolic expenditure. Instead, the subjective cost has commonalities with the cost that our central nervous system is believed to minimize for controlling the motor execution of actions. Our findings thus argue in favor of action selection and action control sharing common underlying optimization principles.
medicine and health sciences
decision making
experimental design
social sciences
limbs (anatomy)
neuroscience
biomechanics
research design
cognitive psychology
cognition
vision
research and analysis methods
curve fitting
musculoskeletal system
mathematical functions
mathematical and statistical techniques
arms
psychology
anatomy
biology and life sciences
sensory perception
cognitive science
shoulders
What makes a reach movement effortful? Physical effort discounting supports common minimization principles in decision making and motor control