Abstract
Whether the nervous system relies on modularity to simplify acquisition and control of complex motor skills remains controversial. To date, evidence for modularity has been indirect, based on statistical regularities in the motor commands captured by muscle synergies. Here we provide direct evidence by testing the prediction that in a truly modular controller it must be harder to adapt to perturbations that are incompatible with the modules. We investigated a reaching task in which human subjects used myoelectric control to move a mass in a virtual environment. In this environment we could perturb the normal muscle-to-force mapping, as in a complex surgical rearrangement of the tendons, by altering the mapping between recorded muscle activity and simulated force applied on the mass. After identifying muscle synergies, we performed two types of virtual surgeries. After compatible virtual surgeries, a full range of movements could still be achieved recombining the synergies, whereas after incompatible virtual surgeries, new or modified synergies would be required. Adaptation rates after the two types of surgery were compared. If synergies were only a parsimonious description of the regularities in the muscle patterns generated by a nonmodular controller, we would expect adaptation rates to be similar, as both types of surgeries could be compensated with similar changes in the muscle patterns. In contrast, as predicted by modularity, we found strikingly faster adaptation after compatible surgeries than after incompatible ones. These results indicate that muscle synergies are key elements of a modular architecture underlying motor control and adaptation.
Introduction
Human motor skills are remarkably complex because they require coordinating many muscles acting on many joints (Bernstein, 1967). The nervous system might achieve motor coordination by combining basic control modules shared across skills and task conditions (Bizzi et al., 2008). To date, evidence for modularity has come mainly from the observation of low dimensionality in the muscle activity patterns recorded during a variety of motor behaviors in different species. Indeed, muscle patterns can be reconstructed by combinations of a small number of muscle synergies, coordinated recruitment of muscle groups with specific activation patterns (Tresch et al., 1999; d'Avella et al., 2003, 2006; Hart and Giszter, 2004; Ivanenko et al., 2004; Ting and Macpherson, 2005; Torres-Oviedo and Ting, 2007; Overduin et al., 2008, 2012; Dominici et al., 2011). However, low-dimensional output might also be observed in a nonmodular controller due to task characteristics or biomechanical constraints (Todorov, 2004; Kutch and Valero-Cuevas, 2012).
Direct evidence for modularity would come from testing an experimental manipulation that can distinguish a modular controller from a non-modular one (d'Avella et al., 2008; Tresch and Jarc, 2009; d'Avella and Pai, 2010). Here we tested a manipulation of the mapping between muscle activations and hand forces that could make such a distinction. To generate a given force, a controller must find an appropriate muscle pattern. In a nonmodular controller, no further structure is assumed. In contrast, in a modular controller using muscle synergies, the motor command is produced by a weighted combination of muscle synergies, each specifying a specific balance of muscle activations, and the controller must find appropriate combination weights. Consider a surgical rearrangement of the tendons of all the muscles contributing to force generation and altering the force component produced by a given activation of each muscle. A special class of such surgeries could be used to distinguish the two types of controllers. All the surgeries are benign, in that it would still be possible to produce any target force after surgery with an appropriate muscle pattern. However, some surgeries are incompatible with the synergies. Figure 1 illustrates this concept using an idealized arm with two pairs of antagonist muscles. In the intact arm, a hypothetical synergy would recruit two muscles generating forces in the direction of the summed forces of the individual muscles. After the surgery, the individual muscle forces would cancel each other, and the synergy would no longer generate any force. A modular controller would then be unable to generate forces in all directions without new or modified synergies. Thus, modularity predicts that adaptation to incompatible surgeries is harder than adaptation to compatible surgeries, as the latter only requires recombining existing synergies.
Concept of incompatible surgery. Illustration of a tendon transfer surgery that makes a putative muscle synergy unable to generate any force. A, An idealized arm with two pairs of antagonist muscles at two joints (m1 to m4), each generating a force in a specific direction at the end point (f1 to f4), is controlled by a muscle synergy recruiting two muscles simultaneously (m1 and m2). The activation of the synergy generates a force that is the sum of the forces generated by each constituent muscle (fsyn = f1 + f2). B, A tendon transfer surgery affecting the force generated by two muscles. C, After the surgery, one of the muscles participating to the synergy (m1′) generates a force in a direction that cancels the force of the other synergistic muscle (m2). The synergy no longer generates any force (f′syn = f′1 + f′2 = 0).
We constructed such surgeries noninvasively in a virtual environment in which human subjects performed a reaching task using myoelectric signals to control the simulated force applied on a virtual mass. We then tested the hypothesis that the controller involved in the generation of forces at the hand is modular by comparing adaptation rates after compatible and incompatible virtual surgeries.
Materials and Methods
We asked naive participants to reach targets on a virtual desktop by displacing a cursor (i.e., a virtual spherical handle) according to either the force applied on a physical handle (force control) or the force estimated from the EMG activity recorded from many shoulder and arm muscles (myoelectric or EMG control). Initially, the reaching task was performed under force control and, for each individual participant, the force and EMG data collected were used to estimate an EMG-to-force matrix by multiple linear regressions. The same EMG data were also used to identify a synergy matrix by nonnegative matrix factorization. Under EMG control, we could arbitrarily modify the EMG-to-force map, thus performing virtual surgeries on the musculoskeletal system. We could then compare the effects of a compatible virtual surgery with those of an incompatible one.
Participants.
Sixteen right-handed naive subjects (mean age, 24.2 ± 2.4 years, SD; eight females) participated in the main experiment (n = 8) and in a control experiment (n = 8) after giving written informed consent. All procedures were conducted in conformance with the Declaration of Helsinki and were approved by the Ethical Review Board of Santa Lucia Foundation.
Experimental setup.
Each subject sat in front of a desktop on a racing car seat with their torso immobilized by safety belts and their right forearm inserted in a splint, immobilizing hand, wrist, and forearm. The center of the palm was aligned with the body midline at the height of the sternum, and the elbow was flexed by ∼90°. The subject's view of their hand was occluded by a 21 inch LCD monitor inclined with its surface approximately perpendicular to the subject's line of sight when looking at their hand (Fig. 2A). After calibration, the monitor could display a virtual desktop matching the real desktop; a spherical cursor matching, at rest, the position of the center of the palm and moving on a horizontal plane; and spherical targets on the same plane (Fig. 2B). A steel bar at the base of the splint was attached to a 6 axis force transducer (Delta F/T Sensor; ATI Industrial Automation) positioned below the desktop to record isometric forces and torques. Surface EMG activity was recorded from 13 muscles acting on the shoulder and elbow: brachioradialis (BracRad), biceps brachii short head (BicShort), biceps brachii long head (BicLong), triceps brachii lateral head, triceps brachii long head (TriLong), infraspinatus, anterior deltoid, middle deltoid, posterior deltoid (DeltP), pectoralis major, teres major, latissimus dorsi, and middle trapezius. EMG activity was recorded with active bipolar electrodes (DE 2.1; Delsys), bandpass filtered (20–450 Hz), and amplified (gain 1000, Bagnoli-16; Delsys). Force and EMG data were digitized at 1 kHz using an analog-to-digital PCI board (PCI-6229; National Instruments). The virtual scene was rendered by a PC workstation with a refresh rate of 60 Hz using custom software. Cursor position information was processed by a second workstation running a real-time operating system and transmitted to the first workstation. Cursor motion was simulated in real time using an adaptive mass-spring-damper (MSD) filter (Park and Meek, 1995). Either the actual force recorded by the transducer force (force control) or the force estimated in real time from the recorded and rectified EMGs (EMG control) using a linear mapping (see below, EMG-to-force matrix) was applied to a critically damped virtual mass fixed to the cursor's position. To maintain fast response to changes in force while reducing the effect of myoelectric noise, the simulated mass was adapted dynamically as a function of the rate of change of the magnitude of the recorded force (ḟ) according to the following sigmoidal function:
where mmax is the asymptotic mass, a is the slope parameter, and b is the offset (mmax = 1 kg, a = 0.1, b = 0.05 kg, and ḟmax = 50 Ns−1 during EMG control). Thus, when the subject generated a constant force, e.g., when trying to hold the cursor in fixed position, the simulated mass was large, and the MSD filtering was more effective in reducing cursor motion due to myoelectric noise. In contrast, in response to a change of force output, e.g., when the subject tried to quickly reach the target, the simulated mass was small, shortening the delay introduced by the filter and making the cursor more responsive.
Experimental setup and protocol. A, Subjects sat in front of a desktop and applied forces on a transducer attached to a forearm, wrist, and hand splint. A flat monitor occluded the subject's hand and displayed a virtual scene colocated with the real desktop. B, Transparent spheres positioned on a horizontal plane with centers at the same height as the center of the palm indicated force targets that the subjects were instructed to reach with a smaller spherical cursor moving on the same plane according to the force applied (force control) or estimated from EMGs recorded from 13 arm and shoulder muscles (EMG control; see Materials and Methods). C, Subjects were instructed to perform a center-out reaching task in which they had to maintain the cursor in a central start location for 1 s, reach a target as soon as it appeared at one of eight peripheral locations, and maintain the cursor at the target for 0.2 s. D, Each subject performed a single experimental session consisting of 16 trials of maximum voluntary force generation in eight directions, 72 trials of reaching to targets in eight directions at three force magnitudes (10, 20, and 30% MVF) in force control, and the following blocks of 24 trials each in EMG control: one block of familiarization (FAM), two series of 24 blocks for each surgery type (6 baseline blocks, 12 surgery blocks, 6 washout blocks), and 6 additional baseline blocks.
Experimental protocol.
Subjects initially performed two blocks of trials in force control and the rest of the experiment in EMG control. In the first force control block, the mean maximum voluntary force (MVF) along eight directions (separated by 45°) in the horizontal plane was estimated as the mean of the maximum force magnitude recorded across 16 trials in which subjects were instructed to generate maximum force in each direction.
Subjects were then instructed to move the cursor quickly and accurately from the rest position to a target in one of the eight directions by applying forces on the splint. At the beginning of each trial (Fig. 2C), subjects were requested to maintain the cursor within a transparent sphere at the central start position for 1 s (tolerance of 2% MVF). Next, a “go” signal was given by displaying a transparent target sphere while the start sphere disappeared. Subjects were instructed to reach the target as quickly as possible and to remain there for 0.2 s (tolerance of 2% MVF). After successful target acquisition, the cursor and the target disappeared, indicating the end of the trial. Trials had to be completed within 2 s from the go signal.
In the second force control block, subjects performed 72 trials to targets positioned at force magnitudes corresponding to 10, 20, and 30% of MVF (random order within cycles of eight directions). After this block, there was a 5 min pause to process the recorded data to construct the myoelectric controller and the virtual surgeries. All subsequent EMG control blocks (Fig. 2D) consisted of 24 trials with targets at 20% MVF in random order within cycles of eight directions. The first EMG control block served to familiarize subjects with EMG control. Then subjects performed two series of blocks of compatible and incompatible virtual surgeries, respectively (see below, Virtual surgeries and adaptation difficulty section). Each series consisted of 24 blocks: 6 baseline blocks, followed by 12 virtual surgery blocks, followed by 6 washout blocks. In between the two series, subjects rested for 10 min; they were also allowed to rest at any time during the experiment. After the two series there were six additional blocks without surgery. The experimental protocol for the control experiment (see below) consisted of the same block structure and order with compatible and incompatible force transformations instead of virtual surgeries.
EMG-to-force mapping.
If the arm is in a fixed posture, the force generated at the hand is approximately a linear function of the activation of muscles acting on shoulder and elbow:
where f is the generated two-dimensional force vector, m is the 13-dimensional vector of muscle activations, and H is a matrix relating muscle activation to force (dimensions 2 × 13). The EMG-to-force matrix (H) was estimated using multiple linear regressions of each applied force component, low-pass filtered (second-order Butterworth; 1 Hz cutoff), with EMG signals recorded during the initial force control block (dynamic phase, i.e., time from target go until the target has been reached), low-pass filtered (second-order Butterworth; 5 Hz cutoff) and normalized to the maximum EMG activity during the generation of MVF. We verified that the choice of filter parameters for the estimation of the H matrix did not affect the quality of the force reconstruction during EMG control by investigating different force and EMG cutoff frequencies. Figure 3A illustrates the columns of the EMG matrix (i.e., the force associated with each muscle, hi) estimated in Subject 2.
Examples of EMG-to-force matrix, synergies, and virtual surgeries. A, EMG-to-force matrix H estimated for Subject 2 from EMG and force data recorded during the generation of planar isometric forces. Each column of H, representing the planar force generated by one muscle, is illustrated by a colored arrow (1, brachioradialis; 2, biceps brachii short head; 3, biceps brachii long head; 4, triceps brachii lateral head; 5, triceps brachii long head; 6, infraspinatus; 7, anterior deltoid; 8, middle deltoid; 9, posterior deltoid; 10, pectoralis major; 11, teres major; 12, latissimus dorsi; 13, middle trapezius). B, Muscle synergies (matrix W) are identified by nonnegative matrix factorization from the EMG data. Each column of W, a vector specifying a specific pattern of relative level of muscle activation, is illustrated by color-coded horizontal bars. C, Forces associated with each muscle synergy (i.e., columns of the matrix product H W) span the entire force space. D, Forces generated by muscles after a compatible virtual surgery obtained by recombination of the original forces as after a complex rearrangement of the tendons (Tc). E, Synergy forces after the compatible surgery still span the force space. F, Muscle forces after an incompatible surgery generated by a rotation matrix (Ti) that maps a vector in the column space of W into a vector in the null space of H. G, Such rotation aligns the forces associated with all synergies in the same direction; thus synergy forces after the incompatible surgery do not span the entire force space.
Synergy extraction and number of synergies.
Muscle synergies were identified by nonnegative matrix factorization (Lee and Seung, 1999) from EMG patterns recorded during force control from the go signal to target acquisition (dynamic phase):
with W an M × N synergy matrix whose columns are vectors specifying relative muscle activation levels, and c an N-dimensional synergy activation vector where N is the number of synergies and M is the number of muscles. EMG patterns were first low-pass filtered (second-order Butterworth filter; 5 Hz cutoff frequency) and rectified, their baseline noise level was then subtracted, and finally they were normalized to the maximum EMG activity of each muscle recorded during the generation of MVF. Baseline noise was estimated at the beginning of the experiment and updated periodically throughout the experiment while the subject was relaxed. For each possible N from 1 to 13, the extraction algorithm was repeated 10 times, and the repetition with the highest reconstruction R2 was retained. The value N was then selected according to two criteria. The first criterion was a fixed threshold on the fraction of data variation explained by the synergies: N was selected as minimum number of synergies explaining at least 90% of the data variation. The second criterion was the detection of a change in slope in the curve of the R2 value as a function of N. A series of linear regressions were performed on the portions of the curve included between N and its last point (13). The value N was then selected as the minimum value for which the mean squared error of the linear regression was <10−4 (d'Avella et al., 2006). In case of mismatch between the two criteria, we used the criterion that selected the set of synergies with preferred directions (the direction of the maximum of the cosine function best fitting the directional tuning) of the synergy activation coefficients (Eq. 3, c) distributed more uniformly and, in particular, with the smallest number of similar preferred directions. Thus, we arranged the preferred direction vectors on a unit circle, we considered all adjacent pairs, and we selected the set of synergies with the smallest number of pairs with an angular difference below 20°. Figure 3B illustrates the synergies extracted from Subject 2, and Figure 3C the forces associated with those synergies by the EMG-to-force matrix (fi = H wi) estimated in the same subject (Fig. 3A).
Virtual surgeries and adaptation difficulty.
We constructed virtual surgeries that were either compatible or incompatible with the synergies as simulated rearrangements of the tendons. Such virtual surgeries were not fully equivalent to a true tendon transfer, since the pattern of proprioceptive responses after the virtual surgery did not change, but they provided compelling visual feedback on the effect that such a surgery would have had on a manipulated object. Virtual surgeries were performed during EMG control by altering the EMG-to-force mapping (H), as after a simulated rearrangements of the tendons, with a rotation in muscle space (H′ = H T; Fig. 3D,F). For each subject, compatible and incompatible virtual surgeries were constructed according to the identified synergies in force control. Both types of virtual surgeries allowed the generation of any planar force with a new muscle activation vector, m′. However, only for compatible surgeries could all force directions be generated by recombining the existing synergies (Eq. 3, c′; Fig. 3E). Incompatible surgeries, in contrast, were constructed such that muscle activation vectors obtained by synergy combinations could only generate forces in one dimension (Fig. 3G). Thus, if the controller was modular, i.e., synergies generated the muscle patterns, we expected adaptation to an incompatible surgery to be slower because it requires not only adapting the synergy coefficients (c′), as for a compatible surgery, but also learning new synergy vectors (W′). In contrast, if synergies were only a parsimonious description of the regularities in the muscle patterns selected by a nonmodular controller, we expected no differences in learning difficulty if the two surgeries required similar changes of the muscle activations (m′) to compensate for the surgeries.
Construction of compatible and incompatible virtual surgeries.
Virtual surgeries were constructed with specific muscle space rotations. Compatible rotations (Tc) were chosen as rotations in the subspace spanned by the synergies. In contrast, incompatible rotations (Ti) rotated a vector of the same synergy subspace into the null space of H. Specifically, we first identified three subspaces of the M-dimensional muscle space. As the number of synergies (N) is in general larger than the dimension of the force space (D = 2), the null space of H [dimension M − D; Fig. 4A, orange, labeled N(H), illustrated as a set in the Venn diagram; B, illustrated as a vector; C, illustrated as a plane] share a common subspace (dimension N − D, magenta set) with the subspace generated by the synergies [dimension N; Fig. 4, cyan, R(W)]. Thus, using singular value decomposition of the matrix WoT N = U S VT, where Wo is an orthonormal basis of the range of W, N is an orthonormal basis of the null space of H, and S has N − D nonzero singular values, we can identify two orthonormal bases Wc = Wo [u1, …, uN − D] and Nc = N [v1, …, vN − D] of the common subspace between synergies and null space, an orthonormal basis Wnc = Wo [uN − D + 1, …,uN] of the subspace of synergy vectors not in the null space, and an orthonormal basis Nnc = N [vN − D + 1, …,vM − D] of null space vectors not generated by synergy combinations. Then, the compatible rotation Tc was constructed such that a vector w in the span of Wnc is rotated onto a second vector w′ of the same subspace (Fig. 4A,B, green arrow). Any two noncollinear vectors w and w′ in the span of Wnc could be chosen, and we simply randomly selected two distinct vectors of Wnc. In this way, the forces associated with the synergies are altered, but all force directions can be generated by recombining the same synergies. In contrast, the incompatible rotation Ti rotates a vector w in the span of Wnc onto a vector n in the span of Nnc (Fig. 4A,C, red arrows). In this way, the muscle patterns generated by synergy combinations along w do not produce any force after the surgery, and synergy combinations can generate forces only in one dimension (Fig. 3G).
Construction of compatible and incompatible virtual surgeries. A, Venn diagram illustrating the different subspaces of the muscle space used for the construction of the virtual surgeries: the column space of the synergies R(W) (cyan set), the null space of the EMG-to-force matrix N(H) (orange set), and the common subspace of N(H) and R(W) (magenta set). w and w′ represent generic vectors in the column space of the synergies, and n represents a vector in N(H) that is not in R(W). B, A compatible rotation in muscle space maps a vector w in R(W) (cyan plane) that is not in N(H) (orange arrow) into a second vector w′ in R(W). C, An incompatible rotation in muscle space, in contrast, maps a vector w in R(W) that is not in N(H) (orange plane) into a vector n in N(H) that in not in R(W) (cyan plane).
We used rotations in muscle space (i.e., transformations of the muscle activity according to an orthonormal matrix T, TT T = T TT = 1, where 1 is the identity matrix) because they are invertible transformations that do not change the norm of the muscle pattern. However, in general, any pairs of transformations Tc and Ti such that H Tc, H Ti, and H Tc W span the force space and H Ti W does not span the force space (i.e., it does not have full rank) would be appropriate. To construct a rotation in the M-dimensional muscle activation space that maps a vector a into a (noncollinear) vector b (both vectors of unit length), we performed the following procedure: (1) We constructed an orthonormal basis of the M-dimensional muscle space Q = [q1, q2, …, qM] by orthogonal-triangular decomposition (Matlab function qr) of [a, b] = Q R. (2) We computed the angle θ between a and b (cosθ = aT b = R12; sinθ = R22) and the rotated basis vectors QR = [q1r, q2r, …, qMr]: q1r = cosθ q1 + sinθ q2; q2r = −sinθ q1 + cosθ q2; and qir= qi for i = 3…M. (3) We constructed the rotation matrix T = QRQT. In the compatible case, a = w and b = w′, whereas in the incompatible case a = w and b = n.
Finally, to make the changes in the muscle patterns required to compensate the two types of surgeries similar, for each subject, the compatible rotation angle was chosen such that a difficulty index (Idiff), defined as the average change across muscles and force targets in muscle activity required to perform the task after the surgery, was close in the two cases (Table 1). Specifically the difficulty index was defined as follows:
where mik is the activity of the ith muscle for the kth force target, normalized to the maximum across force targets before the surgery, and m′ik the same activity after the surgery. For each force target k, the muscle activation vectors mk and m′k were computed as the minimum norm, nonnegative solution of the equations fk = H mk and fk = H T m′k, respectively.
Summary of individual characteristics and results
Compatible and incompatible force transformations.
For each muscle activation m, a muscle rotation T generates a force transformation of f = H m onto f′ = H m′ = H T m. To assess whether differences in the force transformations associated with compatible (Tc) and incompatible (Ti) muscle rotations could affect adaptation rates, we conducted a control experiment in which subjects were required to adapt to force transformations close to those generated by compatible and incompatible virtual surgeries. We approximated the force transformation generated by a muscle rotation T as a linear force transformation f′ = V f, with V = H T H+ (where H+ is the pseudoinverse of H). Force transformations were constructed from compatible and incompatible muscle rotations, which were generated in exactly the same way as for virtual surgeries. Note that all force transformations are compatible with the synergies because they do not affect the ability of synergies to span the entire force space. If the controller were nonmodular and adaptation after an incompatible muscle rotation were slower because of the nature of the force transformation, we would expect adaptation to an incompatible force transformation (Vi = H Ti H+) to also be slower than the adaptation to a compatible force transformation (Vc = H Tc H+). In contrast, if the controller were modular and differences in force transformations did not affect adaptation difficulty, we would expect similar adaptation rates.
Data analysis.
Task performance was quantified by the angular deviation of the initial movement direction of the cursor with respect to target direction and by the fraction of trials in which the cursor reached and remained in the target within the instructed time intervals. The angular deviation was computed as abs(θtarget −θcursor), where θtarget is the target direction, and θcursor is the direction of the displacement between the position of the cursor at movement onset and at the first subsequent peak of its tangential velocity. Taking the absolute value avoided cancellations when averaging the values of the angular deviations across targets with different signs for the difference between target direction and cursor initial direction. Learning rates of both performance measures were estimated for each subject and type of perturbation by a single exponential fit of block averages. We verified that all results were not affected by calculating the angular error using the position of the cursor at 100 ms after movement onset instead of the position at the first velocity peak. Finally, we quantified the changes of the synergistic organization of muscle patterns throughout the experiment by reconstructing them as combination of the synergies extracted at the beginning of the experiment (force control block) and computing a reconstruction R2 value for each block. The reconstruction of the muscle patterns of each block was performed using the nonnegative matrix factorization algorithm initialized with the synergies extracted at the beginning of the experiments and updating only to the combination coefficients (d'Avella et al., 2006).
Statistical analysis.
Differences in performance measures and learning rates between blocks or perturbation types were assessed by t test statistics (paired, two-tailed) after verifying the normality of the distributions (Lilliefors test). One of the eight subjects participating in the control experiment was unable to properly follow the instructions and to perform the task even in the initial force control block and was excluded from the analysis.
Results
Adaptation to compatible virtual surgeries is faster than adaptation to incompatible virtual surgeries
We compared adaption rates to novel compatible and incompatible surgeries constructed using a virtual environment. Naive participants (n = 8; Table 1) reached targets on a virtual desktop by displacing a cursor according to either the force applied on a physical handle (force control) or the force estimated from the EMG activity recorded from many shoulder and arm muscles (myoelectric or EMG control). A linear model (Eq. 2) estimated during the initial block of trials performed in force control adequately captured the relationship between EMG and force (R2 = 0.80 ± 0.06, mean ± SD; n = 8), and all subjects were able to perform the task immediately after switching to EMG control.
Before undergoing a virtual surgery, using EMG control, subjects displaced the cursor from a central start location to one of eight targets arranged on a circle along approximately straight paths (Fig. 5, first column). Immediately after a surgery, regardless of its type, cursor movements were poorly controlled as the muscle patterns normally used to generate a force toward the target directed the cursor in a different direction (Fig. 5, second column). Task performance, quantified by the angular error of the cursor's initial movement direction with respect to the target direction and by the fraction of trials in which the cursor did not reach and hold the target position in the available time, was significantly worse in the first block after the surgery than in the last block preceding it (p < 10−4, t test, for both surgery types and both performance measurements; Fig. 6A,B, red, green; Table 2). With practice, subjects improved their ability to generate new muscle patterns, and both performance measures improved. However, the improvement was clearly smaller after the incompatible than after the compatible virtual surgery, with a significantly faster learning rate (single exponential fit; see Materials and Methods) during the compatible perturbation (p < 10−4 for initial direction error; p = 0.005 for fraction of unsuccessful trials; t test). Some targets could not be reached even after many trials of practice with the incompatible surgery (Fig. 5, third column). The number of unsuccessful trials did not significantly decrease after the incompatible surgery (p = 0.102, comparison between the first and last block, t test), whereas it did decrease significantly after the compatible surgery (p < 10−4). On average, the performance in the last block after the compatible surgery was significantly better than after the incompatible surgery (p = 0.026, for initial direction error; p = 0.003 for fraction of unsuccessful trials; t test). In the washout phase, clear negative aftereffects were observed after undoing the compatible virtual surgery. For most targets, the initial cursor movement in the first movement in the washout of the compatible virtual surgery (Fig. 5, fourth column) was deviated in the opposite direction with respect to the target than in the first movement after the perturbation was introduced (Fig. 5, second column). Such negative aftereffects were not apparent in the mean angular deviation of the first washout block (Fig. 6, averaged across targets and subjects), because, to prevent cancellations when averaging across targets with different signs for the difference between target direction and initial movement direction, the angular deviation was defined as the absolute value of the angular difference (see Materials and Methods). To quantify negative aftereffects, we computed the ratio between the signed initial angle error in the first eight trials after surgery and in the first eight trials in the washout. The average ratio across targets and subjects for the compatible virtual surgery (0.66 ± 0.45, mean ± SD) was significantly different from the ratio for the incompatible surgery (0.36 ± 0.32; p = 0.04, t test; n = 8). In sum, these results revealed a stronger adaptation to the compatible surgery than to the incompatible one, as predicted by modularity.
Example of cursor trajectories. Trajectories of the cursor on the horizontal plane during individual trials of Subject 2 are shown for different targets (color coded) before undergoing a virtual surgery (baseline; first column), immediately after a virtual surgery (second column), at the end of the exposure to the virtual surgery (third column), and after undoing the virtual surgery (washout; last column), for compatible (first row) and incompatible (second row) virtual surgeries. The motion of the cursor was simulated in real time as that of a mass attached by a damped spring to the center of the real handle under the force applied to the handle estimated from the recorded EMGs (see Materials and Methods). Trajectories are shown from the target “go” signal until the end of the trial.
Comparison of task performance changes during virtual surgery and control experiments. A, Angular error of the initial movement direction with respect to the target direction, averaged across subjects and blocks of 24 trials (solid lines; shaded areas indicate SEM) for compatible virtual surgery (green), incompatible virtual surgery (red), compatible control (cyan), and incompatible control (magenta) experiments. B, Fraction of trials in which the cursor did not reach and hold on the target, averaged across subjects and blocks of 24 trials. Differences between compatible and incompatible conditions in both performance measures were significant at the end of the exposure to the perturbation (surgery block 12) in the virtual surgery experiments, but not in the control experiments.
Summary of performance measures of the main experiment
Compatible and incompatible force transformations show similar adaptation rates
We wondered whether the observed difference in learning rate after compatible and incompatible surgeries might be due only to differences in the forces at the hand required after the two types of surgeries and could have been observed in a nonmodular controller as well. We conducted a control experiment with naive participants (n = 7) in which we perturbed the virtual force generation with hand force transformations (see Materials and Methods) instead of virtual surgeries based on muscle space rotations. These force transformations approximated the change in force generated by the virtual surgeries, but only affected the forces computed with the unperturbed EMG-to-force mapping and did not affect the ability of synergies to span the entire force space. Participants adapted to force transformations with smaller initial directional error and number of unsuccessful trials than after muscle rotations since the first perturbation block (Fig. 6A,B, purple, cyan), indicating that force transformations were generally easier to learn, not surprisingly, as force transformations act on a 2-dimensional force space, whereas muscle rotations on a 13-dimensional muscle space. More importantly, participants did not show significant differences in adaptation rates (single exponential fit, p = 0.54 for initial direction error, p = 0.386 for fraction of unsuccessful trials; t test) or in either error measure between the two types of force transformations in the last perturbation block (p = 0.21 for initial direction error, p = 1 for fraction of unsuccessful trials; t test; Table 3). We thus concluded that the differences in adaptation rates observed with virtual surgeries were not due to differences in the nature of the force transformations associated with muscle rotations, but depended on the difficulty in altering the coordination of the muscle activations required to generate novel patterns not captured by the muscles synergies usually used to perform the task.
Summary of performance measures of the control experiment
Muscle patterns diverge from synergy structure during incompatible surgery
Performance improvements after incompatible surgeries, even if they occurred significantly more slowly than after compatible surgeries, were associated with changes in the muscle patterns that could not be captured by the original muscle synergies. This would be expected if an exploration of novel coordination patterns or an organization of new synergies were occurring. Synergies identified before any perturbations were introduced could reconstruct the muscle patterns observed during the exposure to muscle rotations and force transformations. Figure 7A shows an example of muscle patterns (gray area) for different reaching trials to one target recorded throughout an experimental session and their reconstruction (solid line) by combinations of the synergies extracted from the first baseline experimental block (B, left). These synergies captured the muscle patterns before, during, and after the compatible virtual surgery, as well as right after the incompatible virtual surgery (Fig. 7A, first to sixth columns). However, at the end of the incompatible virtual surgery phase (seventh column), the baseline synergies could not capture all the muscle waveforms activated for the specific target. Specifically, the synergy reconstruction predicted a higher than observed activation of BracRad and a lower than observed activation for BicShort, BicLong, TriLong, and DeltP. These discrepancies between observed and reconstructed muscle activations persisted right after the removal of the incompatible virtual surgery in the first movement of the first washout block (Fig. 7A, eighth column). In contrast, the synergies extracted from the last block after the incompatible virtual surgery (Fig. 7B, right) could reconstruct the muscle patterns of that block and those recorded in the first washout block, but they did not capture all preceding patterns. Thus, the muscle patterns at the end of the incompatible virtual surgeries were captured by synergies that were different from those extracted before any surgery. We then quantified the change of muscle pattern organization throughout the experiment by assessing how well the baseline synergies reconstructed the muscle patterns recorded in all other blocks (Fig. 8). On average, across all subjects, we found a significant reduction of the muscle pattern reconstruction quality at the end of the exposure to incompatible virtual surgery with respect to compatible virtual surgery (p = 0.007, comparison between the last compatible and last incompatible virtual surgery block, p = 0.72 comparison between the first compatible and first incompatible virtual surgery block, t test; first block after compatible virtual surgery, 0.78 ± 0.19, mean ± SD; last compatible block, 0.78 ± 0.07, mean ± SD; first block after incompatible virtual surgery, 0.75 ± 0.12, mean ± SD; last incompatible block, 0.54 ± 0.17, mean ± SD). These results support the notion that a specific adaptation process is involved in overcoming incompatible surgeries and that this process is active when the set of synergies usually used for a task becomes ineffective.
Examples of muscle pattern reconstruction by baseline synergies and synergies extracted from the last block after the incompatible surgery. A, Muscle patterns recorded in Subject 2 for different trials (different columns) to one target (direction 225°; gray areas) throughout an experimental session and their reconstruction by the synergies extracted from the initial baseline block (solid lines) and by the synergies extracted from the last block after incompatible surgery (dotted lines). The vertical lines indicate the time of movement onset (1), the time of the first peak of the cursor tangential velocity (2), and the time of target acquisition (3). The corresponding cursor trajectories and events (1–3) are shown for each trial above the muscle activation waveforms. TriLat, Triceps brachii lateral head; InfraSp, infraspinatus; DeltA, anterior deltoid; DeltM, middle deltoid; PectMaj, pectoralis major; TerMaj, teres major; LatDorsi, latissimus dorsi; TrapMid, middle trapezius. B, Synergies extracted from the initial force control block (left, as in Fig. 3B) and extracted from the last block after the incompatible surgery (right) for the same subject and used for the muscle pattern reconstruction in A. Synergy vectors are normalized to their maximum muscle activation.
Reconstruction quality of muscle patterns by synergy combinations during virtual surgeries. Synergy reconstruction error (R2) averaged across subjects and blocks of 24 trials (solid lines; shaded areas indicate SEM) for compatible surgery (green), incompatible surgery (red), compatible control (cyan), and incompatible control (magenta) experiments. The reconstruction quality was significantly reduced only during the exposure to incompatible surgeries, indicating a reorganization of the muscle patterns.
Discussion
Several studies have investigated the adaptive processes occurring after a change in the mapping between motor commands and visual feedback, such as those induced by looking through prisms when throwing (Martin et al., 1996), by rotating the direction of movement of a hand-controlled cursor on a computer screen during reaching (Krakauer et al., 2000), and by remapping finger joint angles into cursor positions through a linear transformation (Mosier et al., 2005). These studies have suggested that motor adaptation involves adjustments in the parameters of internal models of the body and the environment driven by sensory prediction errors (Shadmehr et al., 2010) as well as changes of the control policies due to reward and repetition (Krakauer and Mazzoni, 2011; Wolpert et al., 2011). In contrast with previous adaptation studies that altered the sensory feedback associated with motion or force, we altered the effect of muscle activation patterns using myoelectric control. Such an approach allowed us, for the first time, to probe the structure of the internal model of the musculoskeletal system by assessing its capability to overcome simulated virtual surgeries that involved complex rearrangements of tendons and to provide direct support to the modular control hypothesis.
A modular organization may allow the CNS to rapidly acquire and efficiently control motor skills, overcoming the complexity inherent in the coordination of the many degrees of freedom of the musculoskeletal system (Bernstein, 1967). Adequate, yet possibly suboptimal, control policies for generating a muscle activation pattern driving an end effector onto a visual target, as in our task, and, in general, for accomplishing a variety of different goals may be constructed by combining a small number of modules (Berniker et al., 2009; McKay and Ting, 2012). Modules such as muscle synergies may capture regularities in the sensorimotor mappings shared across tasks and conditions, reducing the number of parameters to be selected to generate a motor command and to be adjusted to compensate for a perturbation or to acquire a new skill. Thus, the dimensionality of the motor commands generated by a modular controller is expected to be constrained by the number of modules and, if the number of modules is smaller than the maximal dimensionality, to provide an observable signature of modularity. Indeed many previous studies have shown that the muscle patterns recorded in a variety of conditions and behaviors can be reconstructed by the combinations of a number of muscle synergies much smaller than the maximal dimensionality, in the spatial and in the temporal domains, of the motor commands (Ting and McKay, 2007; Bizzi et al., 2008; d'Avella and Pai, 2010; Lacquaniti et al., 2012; d'Avella and Lacquaniti, 2013). However, task constraints and biomechanical demands might also give rise to low-dimensional motor commands independently of modularity (Todorov, 2004; Tresch and Jarc, 2009; Kutch and Valero-Cuevas, 2012) and the observation of low dimensionality does not provide, per se, definitive evidence for modularity. Our approach, in contrast, allows for directly testing a causal prediction coming from a suboptimal characteristic of modularity: as modularity allows efficient learning of control policies by reducing the number of parameters, it also constrains the policies that can be learned with the modules. Thus, our observation that adapting to perturbations that are incompatible with muscle synergies is much harder than adapting to compatible perturbations provides evidence for modularity deriving from a test of a causal manipulation of the controller rather than from a parsimonious description of its output.
Our results suggest that two distinct adaptive processes with different learning rates operate in a modular controller. A fast process may be responsible for reducing the error between the force generated by the synergy combination and the force target by adjusting the synergy activation coefficients, i.e., adapting the sensorimotor transformation, possibly implemented in the cerebellum (Shadmehr and Krakauer, 2008; Taylor et al., 2010; Galea et al., 2011). In our task, such a process is effective in gradually reducing the force error when adapting to compatible virtual surgeries, and it is likely that the same process is involved in adaptation to visuomotor (Martin et al., 1996) and dynamic (Shadmehr and Mussa-Ivaldi, 1994) perturbations. Indeed, in the washout phase after compatible virtual surgeries, clear negative aftereffects were observed, indicating that a modification in the internal model had occurred. In contrast, incompatible surgeries remap synergy forces along a single dimension, and no adjustment of the synergy activation coefficients can reduce the force error in all directions. For this special class of perturbations, a second slower process may be responsible for changing the structure of the synergies to recover their capability of generating forces in all directions. Such a process may require exploring new muscle coordination patterns and acquiring new task-specific synergies. New synergies may be stored in the motor cortex and expressed through the corticospinal connectivity (Kargo and Nitz, 2003; Gentner and Classen, 2006; Rathelot and Strick, 2006; Reis et al., 2009; Gentner et al., 2010; Overduin et al., 2012). Synergy learning might be engaged only when the existing synergies are unable to perform a task, such as after a major change in the musculoskeletal system due to injury or when learning a new motor skill. Indeed, modification of existing synergies or organization of new synergies may be a key neural mechanisms underlying skill acquisition, a process that typically occurs on much longer time scales than motor adaptation (Reis et al., 2009). Thus, differences in skill learning difficulty may be related to synergy compatibility. Processes with different learning rates and different capacities for retention have been proposed to explain savings and anterograde interference (Smith et al., 2006). However, multirate processes have been characterized in adaptation tasks that likely only require recombining existing synergies, and they might be distinct from the process involved in learning new synergies. In contrast, adjusting existing muscle synergies or organizing new synergies may be a form of structural learning in sensorimotor control (Braun et al., 2009). Finally, generalization during adaptation to mechanical perturbations suggests that changes in internal models may be represented by linear combinations of motor primitives that map state variables, such as end-effector position and velocity, into motor commands (Thoroughman and Shadmehr, 2000; Donchin et al., 2003; Sing et al., 2009). Muscle synergies may also function as motor primitives for constructing sensorimotor transformations and for representing changes in internal models through the mechanisms of synergy recombination and synergy reorganization.
Myoelectric control was used in two previous studies to investigate how muscle coordination is affected by simulated changes in limb biomechanics (de Rugy et al., 2012a,b) and to examine muscle covariation while learning to coordinate pairs of muscles (Nazarpour et al., 2012). The observation of habitual rather than optimal patterns of muscle activity after simulated muscle paralysis or increased noise reported by de Rugy et al. (2012b) is fully compatible with the organization and modulation of muscle synergies. However, that study did not directly test modularity and was not designed for that purpose. The study by Nazarpour et al. (2012) tested the flexibility of the patterns of trial-by-trial covariation in the activity of an arbitrary pair of finger and wrist muscles around a single target level of activity, and not the flexibility of the muscle synergies underlying the modulation of a large number of elbow and shoulder muscles across multiple target directions. While it is not clear whether the shaping of trial-by-trial covariation in repetitions of the same task and the modulation of muscle patterns across task conditions share the same mechanisms, the notion of flexible muscle synergies is in line with our idea of an adaptive process underlying synergy exploration and reorganization after an incompatible virtual surgery. However, our results suggest that flexibility is present in a modular architecture both in the synergy combinations and in the synergy structure, yet it is expressed at different time scales.
In the long term, understanding the role of modularity in motor learning may lead to effective neurorehabilitation, usable neural interface systems, and novel biologically inspired adaptive controllers for skilled robots. The ability to distinguish motor impairments caused by neurological lesions due to abnormal synergy recruitment (Cheung et al., 2009) or to altered synergy organization (Cheung et al., 2012) might be used to develop novel diagnosis tools and effective rehabilitation protocols (Safavynia et al., 2011). Specifically, rehabilitation exercises in a virtual environment with myoelectric control could exploit synergy-based feedback to promote recovery of functional synergy recruitment and synergy reorganization, and be a low-cost alternative to robot-assisted therapy. Moreover, one of the key challenges for neural interface systems, such as those using neural signals recorded from the motor cortex to control a computer cursor or a robotic arm (Hatsopoulos and Donoghue, 2009), is to make them more useful aids to people with disabilities by improving the ease with which a user can learn to operate them. Decoding muscle synergy recruitment from neural recordings might result in interfaces that are easier to learn than those based on decoding movement intentions or kinematic plans, as the motor cortex is organized to recruit synergies (Rathelot and Strick, 2006; Overduin et al., 2012), and recombining existing synergies is faster than learning new ones. Finally, today's robots have very limited and inflexible motor skills that are not acquired through practice and must be preprogrammed. The modular organization of biological movement may provide inspiration for the design of adaptive robot control architectures that can quickly learn new and richer motor skills by reusing and recombining modules implementing basic skills.
Notes
Supplemental movies for this article is available at http://www.cs.ubc.ca/research/modularity/. Supplemental Movie 1 illustrates the steps involved in constructing compatible and incompatible virtual surgeries. Examples are based on the data of Subject 2. Supplemental Movie 2 shows examples of cursor kinematics throughout a virtual surgery experiment. The displacement of the cursor during different phases of the experimental session of Subject 2 is illustrated as shown to the subject. This material has not been peer reviewed.
Footnotes
This work was supported by the Human Frontier Science Program Organization (RGP11/2008), the European Community's Seventh Framework Programme (FP7/2007–2013-Challenge 2-Cognitive Systems, Interaction, Robotics, grant agreement No 248311-AMARSi), the Canada Research Chairs Program, the Natural Sciences and Engineering Research Council of Canada, Canada Foundation for Innovation, the Canadian Institutes of Health Research, NIH, and the Peter Wall Institute for Advanced Studies. We thank F. Lacquaniti, E. Bizzi, and R. Ajemian for helpful comments on earlier versions of this manuscript, and D. Borzelli for help with the experimental apparatus.
The authors declare no competing financial interests.
- Correspondence should be addressed to Andrea d'Avella, Fondazione Santa Lucia, Via Ardeatina 306, 00179 Rome, Italy. a.davella{at}hsantalucia.it
