Manual 21014074

Manual 21014074
A Neurorobotic Model of Humanoid Walking
by
Theresa Jean Klein
c Theresa Jean Klein
Copyright A Dissertation Submitted to the Faculty of the
Department of Electrical and Computer Engineering
In Partial Fulfillment of the Requirements
For the Degree of
Doctor of Philosophy
In the Graduate College
The University of Arizona
2011
2
THE UNIVERSITY OF ARIZONA
GRADUATE COLLEGE
As members of the Dissertation Committee, we certify that we have read the dissertation
prepared by Theresa Jean Klein
entitled A Neurorobotic Model of Humanoid Walking
and recommend that it be accepted as fulfilling the dissertation requirement for the
Degree of Doctor of Philosophy.
_______________________________________________________________________
Date: November 15, 2011
M. Anthony Lewis
_______________________________________________________________________
Date: November 15, 2011
Hal S. Tharp
_______________________________________________________________________
Date: November 15, 2011
Ricardo G. Sanfelice
Final approval and acceptance of this dissertation is contingent upon the candidate’s
submission of the final copies of the dissertation to the Graduate College.
I hereby certify that I have read this dissertation prepared under my direction and
recommend that it be accepted as fulfilling the dissertation requirement.
________________________________________________ Date: November 15, 2011
Dissertation Director: M. Anthony Lewis
3
Statement by Author
This dissertation has been submitted in partial fulfillment of requirements for an
advanced degree at The University of Arizona and is deposited in the University
Library to be made available to borrowers under rules of the Library.
Brief quotations from this dissertation are allowable without special permission,
provided that accurate acknowledgment of source is made. Requests for permission
for extended quotation from or reproduction of this manuscript in whole or in part
may be granted by the copyright holder.
Signed:
Theresa Jean Klein
4
Acknowledgments
First, I would like to acknowledge the contribution of my advisor, Dr. Anthony
Lewis, for providing the original inspiration for this research, by introducing me to
the concept of central pattern generators, and by stimulating me to dig deeper into
the early origins of the fields of Robotics and Artificial Intelligence. Of all of the
professors I have worked with, he has had the deepest understanding of the nature of
the problems that we grapple with in these fields.
Second, I would like to thank Dr. Jean-Marc Fellous and Dr. Andy Fuglevand
for their invaluable advice to pursue a comparison of the robot’s walking behavior
to human subjects. In addition, I would like to thank Dr. Darcy Reisman of the
University of Delaware for generously providing sagittal plane joint angle data to
which I could compare the robot.
Thirdly, I’d like to thank my committee members, Dr. Hal Tharp and Dr. Ricardo
Sanfelice, for their careful reading of my dissertation and feedback. In particular, I’d
like to thank Dr. Hal Tharp for his unfailingly fair and professional guidance, as
well as his good humor, and his excellence as a teacher. I’d also like to thank my
former advisor Dr. Michael Marefat for allowing me the freedom to pursue whatever
intellectual interests piqued my curiosity, even though these interests eventually led
in another direction than his.
I’d also like to acknowledge the contributions of Tuan Pham, who worked with me
on the first version of the robot leg, as well as Tim Pifer, whose knack for debugging
electronics enabled me to revamp the robot’s sensor network over the summer of 2010,
without which this research would not have been possible.
Finally, I’d like to thank my close friend Tom Jarvis, for keeping me company,
and keeping me sane, while I was finishing this dissertation, and for enriching my life
in so many other ways.
5
Table of Contents
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
Chapter 1. Artificial Intelligence, Robotics, and Cognition
1.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2. Early Cybernetics . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3. Symbolic AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4. Situated, Embodied, Dynamical Cognition . . . . . . . . . . . . .
1.5. Legged Locomotion . . . . . . . . . . . . . . . . . . . . . . . . . .
1.6. Robotics and Biology . . . . . . . . . . . . . . . . . . . . . . . . .
1.7. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
31
31
31
32
33
36
38
39
Chapter 2. The Biomechanics of Locomotion . . . . . . .
2.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2. Models of Locomotion . . . . . . . . . . . . . . . . . . . .
2.3. Ballistic Walking . . . . . . . . . . . . . . . . . . . . . . .
2.4. Phillipson step cycles . . . . . . . . . . . . . . . . . . . . .
2.5. Mammalian Leg Muscle Architecture . . . . . . . . . . . .
2.5.1. Biarticular Muscles . . . . . . . . . . . . . . . . . .
2.5.2. Work Transfer Equations . . . . . . . . . . . . . . .
2.5.3. Lombard’s Paradox and Function of the Hamstrings
2.6. Biarticular Muscles and the SLIP Model . . . . . . . . . .
2.7. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
41
41
41
43
44
45
47
49
52
54
55
Chapter 3. The Neurobiology of Locomotion .
3.1. Introduction . . . . . . . . . . . . . . . . . . . . .
3.2. Neurobiology Primer . . . . . . . . . . . . . . . .
3.2.1. Action Potentials . . . . . . . . . . . . . .
3.2.2. Neuromodulation . . . . . . . . . . . . . .
3.2.3. Motor Control and Sensory Feedback . . .
3.3. Central Pattern Generators . . . . . . . . . . . .
3.3.1. Sensorimotor interactions in locomotion .
3.3.2. Entrainment of the CPG . . . . . . . . . .
3.3.3. Role of Reflex Circuits in Locomotion . . .
3.3.4. Inter-joint coordination in multi-segmented
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
56
56
56
56
60
63
67
71
74
76
77
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
limbs
.
.
.
.
.
.
.
.
.
.
.
6
Table of Contents—Continued
3.4. Modeling the CPG . . . . . . . . . . . . . . . . .
3.4.1. Spiking Neuron Models . . . . . . . . . . .
3.4.2. Abstract Mathematical Oscillator Models .
3.4.3. Neuromechanical Models of the CPG . . .
3.4.4. Neurorobotic Models . . . . . . . . . . . .
3.5. Summary . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
81
81
84
85
91
92
Chapter 4. Robotic approaches to Bipedal Locomotion . . . . . .
4.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2. Conventional Approaches to Bipedal Walking . . . . . . . . . . . . .
4.2.1. Zero Movement Point . . . . . . . . . . . . . . . . . . . . . . .
4.2.2. Optimal Control and Model Predictive Control . . . . . . . .
4.3. Dynamical Approaches to Bipedal Walking . . . . . . . . . . . . . . .
4.3.1. Passive Dynamic Walking . . . . . . . . . . . . . . . . . . . .
4.3.2. Minimally Actuated Walkers: Technical problems with adding
actuation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4. Robots using Central Pattern Generators . . . . . . . . . . . . . . . .
4.4.1. Limitations of conventional robot architecture . . . . . . . . .
4.4.2. CPG Controlled Passive Walking . . . . . . . . . . . . . . . .
4.4.3. Reflexes in Robots . . . . . . . . . . . . . . . . . . . . . . . .
4.4.4. Coordination of lower limb motion . . . . . . . . . . . . . . .
4.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
94
94
95
95
97
98
99
100
104
105
106
107
108
109
Chapter 5. Design of A Bipedal Robot based on Biomechanical
Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2. Mammalian Leg Muscle Architecture . . . . . . . . . . . . . . . . . .
5.3. Implementation of Limb Model . . . . . . . . . . . . . . . . . . . . .
5.3.1. A motor-strap system for mimicking muscles . . . . . . . . . .
5.3.2. Custom Force Sensors . . . . . . . . . . . . . . . . . . . . . .
5.4. Overall Leg Design . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.1. System Level Block Diagram . . . . . . . . . . . . . . . . . . .
5.4.2. Fully assembled robot . . . . . . . . . . . . . . . . . . . . . .
5.5. Experiment: Work Transfer . . . . . . . . . . . . . . . . . . . . . . .
5.6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
110
110
110
112
112
113
116
116
119
119
123
Chapter 6. Neural Architecture . . . .
6.1. Introduction . . . . . . . . . . . . . . .
6.2. Neuron and Synapse Model . . . . . .
6.3. Experiment: Entraining Spiking Neural
126
126
126
128
. . . . . . .
. . . . . . .
. . . . . . .
Oscillators
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
7
Table of Contents—Continued
.
.
.
.
.
.
.
.
.
.
128
132
133
133
138
139
140
140
140
145
Chapter 7. Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2. Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.3. One-level Controller Results . . . . . . . . . . . . . . . . . . . . . . .
7.3.1. Disturbance Rejection: Comparison of CPG Driven Walking to
Purely Reflexive Walking . . . . . . . . . . . . . . . . . . . . .
7.3.2. Central vs. Peripheral Control . . . . . . . . . . . . . . . . . .
7.3.3. Comparison to Human Subject data . . . . . . . . . . . . . .
7.3.4. Passive knee flexion . . . . . . . . . . . . . . . . . . . . . . . .
7.4. Two-level Controller Results . . . . . . . . . . . . . . . . . . . . . . .
7.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
146
146
146
147
Chapter 8. Discussion . . . . . . . . . . . . . . . . . . . . .
8.1. Biomechanics of the robot . . . . . . . . . . . . . . . . .
8.2. The role of reflexes in locomotion . . . . . . . . . . . . .
8.3. Central vs. Peripheral Control . . . . . . . . . . . . . . .
8.4. Load detection and positive force feedback in locomotion
8.5. Locomotory control as a series of nested dynamical loops
8.6. Issue: The generation of complex central control signals .
8.7. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
171
171
172
173
174
176
178
182
Chapter 9. Sensory Reweighting . . . . .
9.1. Simulation Experiments . . . . . . . . .
9.1.1. Inverted Pendulum Model . . . .
9.1.2. Adaptive Kalman Filter . . . . .
9.2. Simulation Results . . . . . . . . . . . .
9.2.1. Sinusoidal Disturbances . . . . .
9.2.2. Amplitude and Phase Responses .
9.2.3. Dealing with Modeling Errors . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
184
187
187
188
192
192
196
197
6.4.
6.5.
6.6.
6.7.
6.3.1. Simulation: Inhibitory, Excitatory, or Mixed Feedback . .
6.3.2. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.3.3. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . .
One-level CPG: A half-center, plus reflexes . . . . . . . . . . . . .
Correspondence of Reflexes to Phillipson Phases of the Step Cycle
6.5.1. Cutaneous afferents: Ground contact sensors . . . . . . . .
6.5.2. Ib afferents: Load sensors . . . . . . . . . . . . . . . . . .
6.5.3. Hip position afferents: AEP and PEP . . . . . . . . . . . .
Two-level CPG . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
151
154
155
158
162
169
8
Table of Contents—Continued
9.3. Robot Model . . . . . . . .
9.3.1. Robot . . . . . . . .
9.3.2. Optic Flow . . . . .
9.3.3. Experiments . . . . .
9.4. Discussion . . . . . . . . . .
9.5. Conclusion . . . . . . . . . .
.1. Robot Parameters List . . .
.2. Simulation Parameters List
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
198
198
202
204
209
213
214
214
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
9
List of Figures
Figure 2.1. The inverted pendulum and spring loaded inverted pendulum
models. One template for legged locomotion treats the limb as a stiff
pole, over which the body vaults on each step. During running the body
is modeled as a spring-loaded pendulum, absorbing energy during landing
and releasing it during take-off. In reality the human leg exhibits some
spring like behavior during walking. The difference is a matter of degree.
Figure 2.2. The simplest walking model. On a downhill slope, a system of
two inverted pendulums of appropriate mass and length results in a stable
limit cycle in 2D space (From [65]). . . . . . . . . . . . . . . . . . . . . .
Figure 2.3. Phillipson step cycle phases. The classical division of the step
cycle into functional phases. F and E1 occur during swing, while E2 and
E3 occur during stance. F flexes the limb after take off while E1 extends
it in preparation for touchdown. E2 absorbs energy, while E3 releases
energy and propels the body forward. From [199]. . . . . . . . . . . . .
Figure 2.4. A 5-link model of the human body used by Taga [211]. This
model has thigh and shank pieces and can be controlled at the hip and
knee joints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 2.5. Left: Agonist/antagonist muscle configuration. A flexor bends
the joint while the extensor straightens it. A biarticular muscle also extends the joint. In our model extensors are all load bearing muscles that
straighten the leg during stance up to toe-off, while flexors bend the leg
during swing phase. Right: Model of the human leg. TA is tibialis anterior, SO is soleus, GA is gastrocnemius, VL is vatus lateralus, RF is
rectus femorus, IL is illiacus, HA is hamstrings, BFS is biceps femorus
short, GM is gluteus maximus. Redrawn from [178]. . . . . . . . . . . .
Figure 2.6. Assuming the biarticular muscle is held at a fixed length, any
motion about the knee forces motion about the ankle. . . . . . . . . . .
Figure 2.7. Biarticular muscles and the SLIP model. By coupling the joints
together, biarticular muscles allow the entire leg to act together to absorb
or release energy. During loading, the leg bends and extensor muscle
stretch to store energy. That energy can later be released, with added
force on toe-off, during propulsion phase. This process allows the limb to
behave like a single spring. Thus a leg model incorporating biarticulators
can be a biomechanical anchor for the SLIP template. . . . . . . . . . .
43
44
45
46
48
50
53
10
List of Figures—Continued
Figure 3.1. Diagram of a neuron. The “inputs” to the neuron arrive on the
dendrites or cell body. When an action potential is formed, it travels as
a wave down the axon. Myelin can help the wave travel faster without
decay. At the end of the axon, synaptic terminals receive the action
potential, causing the release of neurotransmitters at synapses, which
terminate on other neurons. neurotransmitters released into the synaptic
cleft bind with the post-synaptic neuron, which may inhibit or excite
action potentials on the post-synaptic neuron. . . . . . . . . . . . . . . .
Figure 3.2. Action potentials. As the membrane voltage of the neuron passes
the threshold of activation, sodium channels open, allowing more sodium
ions into the cell. This effect causes a positive feedback loop increasing the
membrane voltage and opening more sodium ion channels. Eventually, at
a higher voltage, potassium ion channels open, allowing positively charged
potassium ions out of the cell resulting in a negative feedback loop. Eventually the sodium ion channels deactivate, and the cell membrane returns
to a resting state. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 3.3. Modulatory interneurons and neuromodulators. Serotonin, a
type of neurotransmitter, acts as a neuromodulator in many sensorimotor
interactions. A serotonin neuron synapses on the pre-synaptic terminal of
the sensory neuron where it excites the motor neuron. When the sensory
and modulatory neuron fire simultaneously, the serotonin supplied by the
modulatory neuron interacts with the synapse in a way that dramatically
increases the amount of neurotransmitter released, resulting in a much
greater probability of the motor neuron firing. Modulatory interneurons
can also have long-lasting effects on the synapse by stimulating the development of more ion channels or even more synapses. . . . . . . . . . .
Figure 3.4. The serotonin neuron is stimulated by a painful stimulus applied
to the tail. By pairing this stimulus with a touch to the siphon skin activating a sensory neuron, the modulatory neuron and the sensory neuron
fire simultaneously, increasing the motor response. Over time, the synapse
between the sensory neuron and the motor neuron is strengthened. Later,
when the siphon skin is touched, the motor neuron will fire, even if no
painful stimulus is present. This process is thought to underlie behavioral
conditioning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
57
59
60
62
11
List of Figures—Continued
Figure 3.5. Gating interneurons vs. transaxonal axons. Here a sensorimotor
circuit is shown with the sensory neuron in blue and the motor neuron in
green. The modulatory interneuron (orange) acts directly on the synapse
(left), or via a gating interneuron (pink, right). Excitatory synapses are
indicated using open circles, while inhibitory synapses are indicated by
filled circles. While modulation can induce long-term changes in network
connectivity by acting directly on the synapse (left), modulation can also
be used to refer to the effect of switching between sensorimotor circuits
by inhibiting (right), or disinhibiting gating interneurons (bottom). If
learning is not involved, it is often simpler to write the circuit as a network
of gating interneurons. . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 3.6. Organization of motor control in the central nervous system.
At the lowest level, sensory signals are mapped to motor neurons via
local circuits in the spinal cord and brain stem. However, descending
modulatory signals from higher brain centers, such as the basal ganglia
and cerebellum, can modify these circuits as well as act directly on motor
neuron pools. Image is from [184]. . . . . . . . . . . . . . . . . . . . . .
Figure 3.7. Sources of proprioceptive sensory feedback in the muscles. Muscle spindles (a) detect the lengthening of the muscle as it stretches. Two
groups of fibers as associated, Ia fibers, which are more associated with
velocity, as well as Group II fibers which are associated with absolute
muscle length. Golgi tendon organs (b) detect the load on the muscle
and activate Ib fibers. Other sources of afferent feedback include joint
receptors and cutaneous afferents (not shown). Images from [164]. . . .
Figure 3.8. Brown Half-center Oscillator. Filled black dots represent inhibitory connections. A tonic drive stimulates the neurons to fire. Due
to mutual inhibition, one neuron will suppress the other, until adaptation
causes firing to slow, eventually to the point it is no longer able to inhibit
the other, at which point they will switch. Thus alternating bursts of
firing are generated. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 3.9. Entrainment of the CPG. The central pattern generator is driven
by sensory feedback from the system it is controlling. Since that system’s dynamics are different than the oscillator’s endogenous frequency,
the frequency of the overall system is modified. This process is called
“entrainment.” The entrained frequency of the system as a whole generally lies somewhere between the natural frequency of the system, and the
endogenous frequency of the CPG. . . . . . . . . . . . . . . . . . . . . .
64
65
66
68
69
12
List of Figures—Continued
Figure 3.10. Load regulation of CPG rhythm. Both load related Ib afferents
from extensor muscles and hip position afferents are involved in the regulation of the step cycle, and the rhythm of the CPG. During stance, load
receptors in the limb excite extension of the leg. As long as these load
receptors are active, excitation of the extensor half-center prevents the
initiation of swing. The dynamics of the body-environment system also
plays a role. As the stance leg propels the body forward, the contralateral
limb swings forward. The swing leg initiated ground contact and becomes
loaded, reducing the load in the stance limb. Once the stance limb is unloaded, hip position signaling initiates flexion, transitioning the leg into
swing. The formerly swing leg enters stance as the result of the loading
of extensor muscles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 3.11. The role of reflexes in locomotion [244]. During swing phase,
ground contact triggers a stumbling corrective response, flexing the limb
to clear an obstacle. However in late swing these reflexes are phase modulated. Ground contact now triggers responses from the foot such as leg
extension to place the limb to support weight. During stance, reflexive
load feedback generate load support in the extensor muscles and propulsive force in the limb. At the end of stance phase, the unloading of the
limb triggers phase resetting and flexion. . . . . . . . . . . . . . . . . .
Figure 3.12. A two layer model of the CPG. The rhythm generator (RG) layer
generates the basic half-center pattern that regulates the overall step cycle, while the pattern formation (PF) network integrates afferent signals
to shape this pattern as necessary to the demands of the environment.
Phase modulated reflex effects from Ib and Ia afferent signals are controlled by the output of the pattern formation network. . . . . . . . . .
Figure 3.13. A. Unit Burst Generator. This model assumes that each joint
is controlled by a separate half-center, which controls groups of synergist
muscles at the joint, and which is coupled to the other joints to produce
the necessary phase offsets. B. The distributed pattern generator, based
on a two-layer model of the CPG, contains a presiding rhythm generator
(RG), and a pattern formation (PF) layer which accepts afferent input
from all parts of the limb, while outputting controls to groups of synergist
muscles to the joints. The PF neurons are not necessarily arranged in
mutually inhibitory pairs about each joint. . . . . . . . . . . . . . . . .
75
77
79
80
13
List of Figures—Continued
Figure 3.14. Neuromechanical model used by Taga [211]. The human body
is modeled by a five-link system. Meanwhile, the CPG is modeled by
sets of Matsuoka oscillators, with one oscillator per joint, connected by
weak coupling. Outputs of the Matsuoka oscillators are mapped to joint
torques while the body is simulated using multi-body dynamics. Feedback
from joint angles is used to entrain the Matsuoka oscillators controlling
the joints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
90
Figure 4.1. Robots based on conventional control theory with ZMP stability criterion. (a) The Honda ASIMO, (b) HRP-2 developed by AIST in
Japan, (c) HRP-4c female humanoid. Note the similarities in the stance
and gait displayed by the robot’s posture. The knees are bent and the
feet kept close together. Also note that the limbs appear relatively massive. The large limb size is due to the necessity for high joint torques to
maintain control against uncooperative natural dynamics. . . . . . . . . 96
Figure 4.2. Purely passive dynamic walking robots. (a) McGeer’s original
walker. The only actuators are to lift the swing leg slightly for clearance.
(b) Based on McGeer’s robot, a team at Cornell built a walker with
knees. [64] (c) A fully 3D walker with counterbalancing arm motion [33].
The arms compensate for body twist. Compared to the conventional
robots above these walkers have light, skinny limbs, and walk much more
naturally. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Figure 4.3. Actuated limit cycle walkers. Based on the original work in
purely passive walkers, passive-based walkers with minimal actuation were
developed. (a) CMU Electric Biped, (b) Runbot [131], (c) Dribbel [37],
(d) Meta [195], (e) Spring Flamingo. These robots tend to be based on
light structures with low gear ratio motors to allow for passive swings at
the hips. As an alternative, Spring Flamingo (e) uses virtual model control
to mimic passive-like behavior, although this is less energy efficient. . . 101
Figure 4.4. Actuated 3D walkers. Most wobble laterally, however Flame uses
hip abduction (a) MIT Toddler (b) Cornell 3D Biped (c) Flame. . . . . 103
Figure 5.1. Model of the human leg. TA is tibialis anterior, SO is soleus, GA
is gastrocnemius, VL is vatus lateralis, RF is rectus femoris, IL is illiacus,
HA is hamstrings, BFS is biceps femoris short, GM is gluteus maximus.
Redrawn from [178]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Figure 5.2. Force sensor fitting. The force sensor is screwed into the fitting
which is buckled into the strap that mimics the muscle. . . . . . . . . . 114
14
List of Figures—Continued
Figure 5.3. Schematic diagram for load sensor boards. The board contains an
instrument amplifier, microcontroller, and RS485 transceiver. The board
samples the output of the FUTEK force sensor, converts it to a 10 bit
integer and outputs a packet on the RS485 bus. . . . . . . . . . . . . .
Figure 5.4. The assembled sensor board. An external oscillator was added
to boost baud rate on the sensor bus. . . . . . . . . . . . . . . . . . . .
Figure 5.5. Left: CAD Drawing of fully assembled leg design. Kevlar straps
representing tendons are shown in gray. The straps are attached (inset
left) to the motors by brackets. Force sensors are attached by custom
fittings (inset right) buckled into the straps. All major muscle groups
involved in planar walking are modeled. Right: The fully assembled leg.
Inset Left: Close-up of motor with strap attached. The motor pulls upward on the strap mimicking muscle contraction. Inset Right: Force
sensor assembly. This sensor is buckled into the kevlar strap to measure
tension in the muscle. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 5.6. System level block diagram of the robot. The central pattern
generator runs on a PC running Ubuntu RT Linux. Two RS-485 buses
are connected via USB-2-RS485 converters. There are 16 motors and 28
sensors. Position commands are sent to the motors to actuate the robot.
Load, joint angles, and ground contact sensor data is transmitted back
on the sensor bus. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 5.7. Fully assembled robot. The robot is placed in a cart, which
confines it to the saggital plane. The robot is also supported by bungee
cords. The cart resembles a “baby walker” training device. . . . . . . .
Figure 5.8. Passive swing motion of the leg when perturbations are manually
applied. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 5.9. Leg Extension Test. The leg pushes off the floor and lifts itself
to standing on tip-toe. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 5.10. Tension in monoarticular muscle tendons when only monoarticular muscles are used. The majority of the force is borne by the SO and
VL, while the GM bears relatively little. . . . . . . . . . . . . . . . . . .
Figure 5.11. Tension in monoarticular muscle tendons when biarticular muscles are used. The majority of the force is borne by the GM and VL,
consistent with force being transfered from proximal to distal limb segments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 5.12. Forces in biarticular muscles. At the start of the test, these
muscles are placed under tension. After the extension is completed, these
muscles are more relaxed, consistent with their role in power transfer
rather than weight support. . . . . . . . . . . . . . . . . . . . . . . . .
115
115
117
118
119
120
121
122
123
124
15
List of Figures—Continued
Figure 5.13. Work Performed in the Leg by various motors. When the biarticular muscles are active, very little work is done by the SO. The VL and
GM bear most of the load. . . . . . . . . . . . . . . . . . . . . . . . . . 125
Figure 5.14. Work transfer performed by RF and GA. As expected WGA,Knee
and WGA,Ankle are approximately equal, and WRF,Knee and WRF,Hip are
approximately equal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Figure 6.1. Example of output from spiking neuron model, synapse, and motor signal. Membrane voltage (left), synapse output (center), and motor
signal (right). Spikes received by the synapse model are integrated, passed
through a logistic function and filtered to produce a smooth motor output scaled between 0 and 1. This output is used to compute a position
command for the motor. . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Figure 6.2. Possible schemes for providing feedback to the neurons of the
CPG. Using only inhibitory feedback (a), leg extension inhibits the ipsilateral extensor and the contralateral flexor. Conversely leg flexion inhibits the ipsilateral flexor and the contralateral extensor. Using only
excitatory feedback (b) has the opposite connectivity. Leg flexion excites
the ipsilateral extensor and contralateral flexor. Another possibility is
to interact only with ipsilateral neurons (c). Leg extension inhibits the
extensor neuron and excites the flexor. . . . . . . . . . . . . . . . . . . 129
Figure 6.3. Endogenous CPG for two possible sets of neuron parameters.
Two neurons shown for left (blue, thick, solid) versus right (red, thin,
dashed) extensors. The low frequency network has a=0.005, b=0.25, c=50, d=0.3, with tonic current I = 5, and synapse parameters dt = 2,
τmem = 10, τs = 10, and τṡ = 50. The high frequency network has
a=0.02, b=0.25, c = -50, d = 1.5 and I =10. and synapse parameters
dt = 2, τmem = 20, τs = 10, and τṡ = 50. . . . . . . . . . . . . . . . . . . 130
16
List of Figures—Continued
Figure 6.4. Examples of various methods for entraining a spiking neural oscillator. All four neurons are shown so that both flexors and extensors are
visible. Colors: Left extensor (blue, thick, solid), left flexor (pink, thin,
dashed), right extensor (red, thick, solid) and right flexor (green, thin,
dashed), feedback signal (black, thin, dotted). The inhibitory feedback
and ipsilateral feedback methods result in poor behavior, while the excitatory feedback allows the system to adapt to a wide range of signals.
Left: For 0.4 Hz endogenous frequency with applied 1.25 Hz feedback
signal. Right: For 1.25 Hz endogenous frequency and a 0.5 Hz feedback
signal. Top: Using only inhibitory feedback. Middle: Using only excitatory feedback. Bottom: Feedback is applied to only the ipsilateral side of
the body. The inhibitory method does not successfully entrain, while the
ipsilateral method produces undesirable phase offsets. Only excitatory
feedback succeeds in entraining the network. . . . . . . . . . . . . . . . 131
Figure 6.5. Neural Architecture of Single Level Simplified CPG model. Four
central neurons control hip flexion and extension. Environmental feedback
from ground contact sensors adapt lower leg motion in the knee and ankle.
Ground contact (GC) and load (Load) sensors feed back to knee and hip
extension. Anterior extreme position (AEP) angle sensors extend the knee
to prepare for foot touchdown, while posterior extreme position (PEP)
sensors extend the ankle during toe off, and trigger flexion at the hip
and knee. In addition, weak descending connections link the four central
neurons to the knees and ankles. As in the McCullough-Pitts neuron [140],
we assume that inhibitory signals are absolute - inhibition of the neuron
shuts down firing regardless of how much excitatory ihnput it receives.
134
Figure 6.6. Reflex action correspondence with phases of the step cycle. The
four phases of the step cycle defined by Phillipson loosely correspond
to the effect of AEP, PEP, ground contact and load reflexes described
by [244]. These reflexes are position and phase modulated so that their
activity is confined to a certain portion of the step cycle that makes
them useful in generating the walking cycle. Our reflexes are designed to
correspond to this system, with ground contact reflexes being modulated
by the swing vs. stance phase of the CPG, while toe-off and leg extension
are initiated by hip position afferents. . . . . . . . . . . . . . . . . . . . 138
17
List of Figures—Continued
Figure 6.7. Simplified Version of Two-level architecture described in [192].
In our robot, we treat the internal PD position control loops of the motors
as models of the effects of Ia and group II afferent feedback from stretch
receptors in the muscle. If we assume that the stretch receptors are part
of a muscle length control feedback loops, we can isolate this from the
system. We therefore eliminate these circuits from the network, since
we assume they are self-contained. By removing the Ia feedback and
associated interneurons, we vastly simplify the complexity of the CPG
network, allowing us to focus on load and position dependent sources of
afferent feedback. Also, for the sake of simplicity we will pretend that a
neuron can provide both excitatory and inhibitory signals, which allows
us to avoid drawing the inhibitory interneurons and instead connect the
CPG neurons directly. These simplification eliminate clutter and make it
easier to understand the function of the circuit. What is left is the Ib and
cutaneous feedback to the flexor and extensor half centers, the RG level,
the PF level, and the motor neurons. . . . . . . . . . . . . . . . . . . . 142
Figure 6.8. Neural Architecture of Two Level CPG model with legs. Four
central neurons make up an RG level, which contains flexor and extensor
neurons for the left and right legs. Six PF layer neurons for each leg are
assigned to the hip, knee and ankle flexors and extensors. Environmental
feedback from ground contact sensors adapt lower leg motion. Ground
contact (GC) and load (Load) sensors feed back to knee and hip extension. Anterior extreme position (AEP) angle sensors extend the knee to
prepare for foot touchdown, while posterior extreme position (PEP) sensors extend the ankle during toe off, and trigger flexion at the hip and
knee, as well as phase resetting of the RG. In addition, weak descending
connections link the four central neurons to the lower limb. . . . . . . . 144
Figure 7.1. One step cycle of the robot. Top: Frame captures from video of
the robot walking. Bottom: GC, L and hip angle signals from the left
(yellow) and right (red) legs. Membrane voltages are shown for hip, knee
and ankle flexors and extensors. A: Toe-off on left leg, triggered by PEP.
Foot touchdown on right. B: Leg extension on right leg triggered by GC
and L. Swing on left. C: Passive forward swing of left leg. Just prior to
initiation of PEP reflexes on right. D: Toe-off on right in response to PEP
reflex. Ground contact on left. E: Swing phase on right. Stance on left,
L and GC excite leg extension. F: Right AEP, right knee extends. Left
leg propels body forward. G: Left ankle extension and toe off, preparing
for stance to swing transition. Touchdown on right. . . . . . . . . . . . 149
18
List of Figures—Continued
Figure 7.2. Motor work performed at the hip. The IL performs very little
work, indicating that most of the forward swing of the leg is passive. . .
Figure 7.3. Responses to afferent signals from the purely reflexive controller.
GC, T and hip angle signals from the left(left) and right (right) legs.
Flexor and extensor membrane voltages for hip, knee, and ankle. Comparing afferent signals to muscle activations, it appears that steps are less
regular. The lack of a tonic signal driving the hip flexor and extensor
muscles (since they are relying purely on afferent signaling), results in
less steady activation, especially of flexors. Step length is also less even.
Toe stubbing events on the contralateral toe (see 47s) tends to result in
the ipsilateral extensor being inhibited. . . . . . . . . . . . . . . . . . .
Figure 7.4. Phase Plot comparison of walking cycle for reflexive versus CPG
driven walking, under perturbations. Means have been corrected to line
up the reflexive and CPG average hip angle. Left: Sensors on the right
leg are disabled so that the robot cannot detect ground contact events. In
this case, stance duration is extended on the right leg, so that its phase
extends further behind the robot before lifting. By comparison the CPG
hip angles remain symmetrical. Right: A weight is attached to the right
leg of the robot. In this case, the right leg lags significantly behind the
left in the reflexive version, but remains relatively even with the left when
the CPG was enabled. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 7.5. Synapse output for the knee extensor versus the hip angle, for
various endogenous CPG frequencies and values of wd . (A) For the 0.15Hz
CPG. (B) for the 0.5 Hz CPG. (C) For the 1.1 Hz CPG. As the strength
of the coupling to the half-center is increased, knee flexion is increasingly
synchronized to the hip. It also becomes more regular, showing more
uniform activation across the step cycle. For the faster CPGs, weak or
no coupling to the CPG tends to result in less continuous knee extension.
Central control therefore adds robustness to the system. . . . . . . . . .
151
152
154
156
19
List of Figures—Continued
Figure 7.6. Comparison between robot saggital plan angles and human subject angles during normal walking. These figures show averages, with
standard deviation, across one step cycle from heel strike to heel strike.
Hip (green, solid), knee (red, dashed) and ankle (blue, dotted). For the
single level controller with descending signals from the CPG to lower limb
of I=5, and for the 0.5Hz CPG. There is a reasonable match between many
of the features of the step cycle, particularly the hip and knee. The ankle’s
toe-off seems less shifted over to the right, however. This characteristic
could be a result of deliberate activation of the ankle extensors earlier
in the step cycle to increase the chances of contralateral foot clearance.
Error bars are included to show the standard deviation of the data over
many step cycles. The robot exhibits more variation, however, we also
had fewer step cycles for comparison. . . . . . . . . . . . . . . . . . . .
Figure 7.7. Change in angular behavior with respect to degree of central
control coupling, for robot hip (right), knee (middle) and ankle (left)
angles. Plots are arranged in order of decreasing values for wd - 10, 5, 3,
and 0, from top to bottom. When greater central control is exerted (top
row), the joints move more synchronously. This coordination increases
the chance of foot clearance, since the knee is more flexed during swing
phase, but makes the behavior less human like. By comparison when the
lower limb is purely reflexive (bottom row), the behavior is more like a
human’s but the chance of toe stubbing is much greater. . . . . . . . .
Figure 7.8. Comparison of robot left leg joint angles to human for various
strengths of coupling to the CPG. Joint angles behave more like a humans
when reflexive control of the lower limb is greater. . . . . . . . . . . . .
Figure 7.9. Comparison of robot right leg joint angles to human for various
strengths of coupling to the CPG. Joint angles behave more like a humans
when reflexive control of the lower limb is greater. . . . . . . . . . . . .
Figure 7.10. Comparison of leg angles to human subject data when all knee
flexion signals are removed, for the left (left) and right (right) legs. Hip
(green, solid), knee (red, dashed) and ankle (blue, dotted).In this case
the knee flexion is purely a result of the biomechanics of the limb during
swing phase. This response matches the most closely with the human
subject data, suggesting that in the human knee, flexion is largely passive
rather than active. Ankle extension was increased to avoid toe stubbing.
Figure 7.11. Snapshots sequences of the robot walking as the central control
component is decreased. . . . . . . . . . . . . . . . . . . . . . . . . . .
157
159
160
161
162
164
20
List of Figures—Continued
Figure 7.12. Ankle, Knee and Hip angles as the degree of central control is
reduced and peripheral control is increased. Left leg (blue solid) and
right leg (red dotted). When hip, knee, and ankles are coupled to the
rhythm generator entirely, all three joints act in phase, in roughly square
waves. As the central control is reduced and peripheral (reflexive) control
is increased, the ankle extension peak shifts later in the step cycle. The
knee also begins to extend shortly after flexion and before heel strike. See
Figure 7.11 for comparison. The hip also appears to gradually shift from
being more square to being more sinusoidal, more similar to the human
data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 7.13. Left Leg comparison with human subject data for various degrees of central vs. peripheral control, for the two-level controller, with
decreasing degrees of central control shown from red to blue in the spectrum. The two sets of human data are shown by thick black dashed and
dotted lines for comparison. As in Figure 7.12, as central control signal
(red) is decreased and peripheral control (blue) increases, the step cycle
behaves more like the human subject data, except for pure peripheral
control. When no central control component is present (purple) the step
cycle is also much less stable. However, a small amount of central control
with mostly reflexive control (blue, cyan) produces the nearest match to
human data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 7.14. Right leg comparison with human subject data for various degrees of central vs. peripheral control, for the two-level controller, with
decvreasing degrees of central control shown from red to blue in the spectrum. The two sets of human data are shown by thick black dashed and
dotted lines for comparison. As in Figure 7.12, as central control signal
(red) is decreased and peripheral control (blue) increases, the step cycle
behaves more like the human subject data, except for pure peripheral
control. When no central control component is present (purple) the step
cycle is also much less stable. However, a small amount of central control
with mostly reflexive control (blue, cyan) produces the nearest match to
human data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 7.15. Comparison of MSE between robot and human subjects for various degrees of central vs. peripheral control. Pure central control on the
left to pure peripheral control on the right. Curves for the robot were
matched against human subject data, the MSE between the robot and
the human data over the step cycle was then taken. Here, it is clear that
a small, but non-zero amount of central control produces the lowest MSE
over the step cycle for this controller. . . . . . . . . . . . . . . . . . . .
165
167
168
169
21
List of Figures—Continued
Figure 8.1. Conceptual models of dynamical walking as a set of nested loops,
each modulated and manipulated by descending signals from more deeply
nested structures. (a) Passive dynamic walking as a pure limit cycle involving the body and environment. (b) Brain-body-environment model of
dynamical cognition. The brain manipulates the body, producing a larger
dynamical loop uniting the environment and the mind. (c) CPG model
of walking. Higher brain centers control the CPG, which acts to stabilize
and inject energy into the passive limit cycle. (d) Reflex/CPG model.
The reflex system interacts more immediately with the passive walking
cycle, while the CPG modulates reflex circuits, and controls walking at
the hips. Feedback from the environment excites reflexes and entrains the
CPG in separate loops. Descending signals from higher centers in turn
can modify CPG and reflex circuits in a task-dependent manner. . . . . 177
Figure 8.2. Hypothetical Learning Network. At birth, the CPG begins as
a half-center rhythm generator producing alternating patterns of synchronized flexion and extension. When stimulated by afferent feedback,
reflex circuits, modulated by the half-center, generate appropriate motor responses. As in babies and SCI patients in treadmill training, the
treadmill induces motion of the body that produces feedback of appropriate timing, stimulating reflex responses that assist the walking cycle
at the right moments. We hypothesize that a learning network develops
after birth which learns to reproduce these motor responses centrally, in
connection with the phase of the rhythm generator. The reflex system,
in combination with the dynamics of the musculo-skeletal system, acts
to provide appropriate feedback, which acts as a “teacher signal” to the
network - providing it with a set of appropriate sensory signals and motor
responses, which can then be mapped to the phase of the rhythm generator. The learning network can then reproduce the signal centrally and
is less reliant on peripheral control. This system can also be regarded as
an inverse model of the body dynamics. . . . . . . . . . . . . . . . . . . 180
Figure 9.1. Model of Postural Control. The body is modeled as an inverted
pendulum. Sensory information is available from visual and vestibular
inputs (optic flow and gyro in out robot), as well as proprioceptive inputs
(foot pressure sensors in the robot). The visual input is perturbed by
false movement of the visual field. Parameters are given in Appendix A.
188
22
List of Figures—Continued
Figure 9.2. Block diagram of model sensory reweighting system. The body
is modeled as an inverted pendulum at angle θ. Visual and vestibular
pathways detect the rate with some added noise, plus the effect of motion
of the visual scene on the visually observed rate. The state estimator
compares the forward prediction of the state to the observed visual rate,
and computes an estimate of noise in the visual channel based on the
residual. The Kalman gain is then recomputed and used to update the
state, based on visual and vestibular rate estimates. . . . . . . . . . . . 189
Figure 9.3. Simulation results in response to a sinusoidal disturbance of 1.2
degrees/sec (or degrees for the FPS) and 1Hz in each sensory channel in
turn. Standard Kalman filter (blue), Adaptive Kalman filter (red) and
non-predictive Bayesian filter (green). Error in estimated angle (top),
angular rate (middle), and applied torques (bottom). Disturbances are
added to each channel at 12.5, 25 and 37.5 seconds of simulation time. In
the standard Kalman filter, significant errors are present in the estimate,
resulting in larger torques applied at the ankle to compensate. In the
adaptive Kalman filter and non-predictive filter, the estimate remains
accurate. The non-predictive Bayesian filter also reweights data, but is
not at good at filtering out disturbances, particularly when applied to the
proprioceptive sensor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Figure 9.4. Simulation results in response to a sinusoidal disturbance of 1.2
degrees/sec (or degrees for the FPS) and 1Hz in each sensory channel in
turn, for the adaptive Kalman filter (top) and the non-predictive Bayesian
filter (bottom). Gains (circles) and variances (squares). Kalman gain
for the visual rate (red), vestibular rate (blue) and proprioceptive angle
(cyan). Estimated variance in visual rate (light green) vestibular rate
(dark green) and proprioceptive angle (black). Disturbances are added to
each channel at 12.5, 25 and 37.5 seconds of simulation time respectively.
In both cases, the estimated noise variance rises while the Kalman gain
for the affected channel falls. We can observe that inter-modality effects
are predicted by this model, as the gain in the non-perturbed channels increases when the disturbance is introduced (except when the disturbance
is applied to the proprioceptive channel). . . . . . . . . . . . . . . . . . 194
23
List of Figures—Continued
Figure 9.5. Gain (a) and Phase (b) for increasing stimulus amplitude, for
adaptive Kalman filter (red, dashed, circles) , standard Kalman filter
(green, dotted, squares), and non-predictive Bayesian filter (black, dotdashed, crosses). Both the adaptive Kalman filter and the adaptive
Bayesian filter exhibit downweighting, though the lack of an internal
model leads to larger gains and a significant phase lag for the non-predictive
filter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 9.6. Effect of model errors on performance, with and without assuming noise independence. Top two plots: When a good model is assumed
and used by the filter, no assumption of independence is used and the
model performs better without it. Bottom two plots: However, when the
model is incorrect, the assumption of independence is crucial to maintain
stability. Without this assumption, modeling errors appear as correlations in the sensory noise, resulting in filter divergence. For our incorrect
model we multiplied the g/l term in the A matrix by 1.3. . . . . . . . .
Figure 9.7. Robot used in Experiment. The robot is equipped with a three
axis gyro, a camera, which uses optic flow to measure rotation, and foot
pressure sensors, which are used to estimate angular tilt of the robot. . .
Figure 9.8. Calibration of foot pressure sensors against the tilt of the robot’s
torso. During quiescent standing the robot was perturbed and allowed to
wobble on its feet several times. The angular measurement of the gyro
and the foot pressure sensors were measured against each other. . . . .
Figure 9.9. Example image of stimulus with optic flow vectors. The vectors
track outward motion of the colored dots, simulating the effect of forward
motion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 9.10. Robot Experiment: Time course of noise estimates (blue) and
gains K (green). During the visual disturbance, θv2 rises, causing the gain
to fall. The gain in the gyro and FPS channels increases to compensate.
196
199
200
202
203
206
24
List of Figures—Continued
Figure 9.11. Results from Moving Room Experiment. Estimates (blue), gyro
measurements (red), visual rate measurements (green), and visual disturbance(black). (a) Estimated angles without and reweighting. The gyro’s
angular position output here is used as a comparison to the filter’s estimates, although the gyro position is not used in our filter. (b) Angular
rates with and without reweighting. At about 15 seconds the visual field
begins to move, creating a false sense of angular rate in the visual channel.
Comparing the reweighting case to the no-reweighting case, the resulting
offset from vertical is greater in the no reweighting case. In addition to
the rate error, ankle torques are applied to the erroneous state estimate
causing the robot to actually tilt away from the vertical, as shown in
(a). With sensory reweighting, the robot filters the disturbance out and
remains unperturbed. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 9.12. Comparison of command applied at the ankle with and without
sensory reweighting. Without sensory reweighting, the robot exerts larger
ankle torques. With sensory reweighting, the robot is perturbed but is
able to recover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 9.13. Results from Moving Platform Experiment. Estimates (blue),
gyro measurements (red), visual rate measurements (green), and visual
disturbance(black). Estimated angles with and without reweighting. When
reweighting is not applied, a significant oscillation results from the introduction of the error in the rate estimate, as the state estimate becomes
inaccurate and the controller attempts to compensate, incorrectly. As the
controller attempts to compensate the robot rotates away from vertical
introducing genuine gravitational torques into the system. This behavior
results in oscillation of the robot. . . . . . . . . . . . . . . . . . . . . .
Figure 9.14. Results from Moving Platform Experiment. Applied ankle torques.
Force is applied to the platform to produce errors in the foot pressure sensors. Without reweighting (blue), the estimates of position diverge from
the true state, resulting in greater ankle torques. With reweighting (red)
the foot pressure sensor data is down-weighted, resulting in smaller ankle
torques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
207
207
208
209
25
List of Figures—Continued
Figure 9.15. Results from Moving Platform Experiment. Gain (green) and
estimated variance (blue) for foot pressure sensors. Force is applied to the
platform to produce errors in the foot pressure sensors. The estimates of
variance in the foot pressure sensor measurements increase, resulting in a
decrease in the computed Kalman gain for that channel. In this case, the
introduction of process noise due to movement of the platform causes the
filter to take longer to identify the FPS sensors as the most noisy, and to
perceive noise from other sensor channels as well. Due to inter-modality
effects, there is a relatively small downweighting of the FPS sensors. . . 210
26
List of Tables
Table 6.1. Spiking Neuron Model. Six parameters define the neuron model,
a, b, c, d, and I the tonic current, as well as integration time constant τ .
Feedback to the neuron is provided via If eed . The synapse model has two
parameters, τs and τṡ . The motor output is further smoothed using an
alpha tracker with learning rate α = 0.05. Outputs of these functions for
a typical burst are shown in Figure 6.1. . . . . . . . . . . . . . . . . . . 127
Table 7.1. CPG Parameters. Three sets of parameters were used to experiment with whether the system would entrain using various endogenous
frequencies for the four-neuron CPG. Here tau is the time constant associated with the membrane voltage, taus is the time constant associated with
the synapse, and tauṡ is the time constant associated with the synapse
rate of change. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Table 7.2. Entrained step cycle frequency for various selections of endogenous
CPG frequency and wd . Once sufficient foot clearance is obtained, the
entrained frequency lies at a point in between the endogenous frequency
of the CPG and the natural frequency of the system, which is somewhere
around 0.5Hz. Very high wd may reduce walking speed because the foot
does not extend near the end of stride. . . . . . . . . . . . . . . . . . . 147
27
Abstract
In this dissertation, we describe the development of a humanoid bipedal robot that
fully physically models the human walking system, including the biomechanics of the
leg, the sensory feedback pathways available in the body, and the neural structure
of the central pattern generator (CPG). Using two different models of the CPG, we
explore several issues in the neurobiology and robotics literature, including the role of
reflexes in locomotion, the role of load reception and positive force feedback in generating the gait, and the degree to which central or peripheral control plays in human
walking. We show that the walking pattern can be generated by a combination of a
half-center CPG and reflex interactions phase modulated by the CPG, and that load
receptors in the muscles can play a substantial role in generating the gait, using positive force feedback. We compare the gait of the robot to human subjects and show
that this architecture produces human-like stepping. Varying the degree of direct
central control of lower limb muscles by the CPG, we show that the most humanlike gait is generated with a relatively weak central control signal, which modulates
reflex responses that generate most of the muscle activation. These results allow us
to conceive of locomotion as a series of nested loops, with a central CPG or rhythm
generator modulating lower level reflex interactions, while higher centers modulate
the CPG. Since locomotion is a primary mechanism by which animals interact with
the world, this research is relevant to artificial intelligence researchers. Recent understanding of cognition holds that minds are embodied, situated relative to a set
of goals, and exist in a feedback loop of interaction with the environment. In our
robot, we model the dynamics of the body, the neural architecture and the sensory
feedback channels in a complete dynamical feedback loop, and show that the robot
entrains to the the natural dynamics of the world. We propose the concept of nested
loops with descending phase modulation as a conceptual paradigm for a more general
28
understanding of nervous system organization.
29
A Neurorobotic Model of Humanoid Walking
Theresa Jean Klein, Ph.D.
The University of Arizona, 2011
Director: M. Anthony Lewis
In this dissertation, we describe the development of a humanoid bipedal robot
that fully physically models the human walking system, including the biomechanics of the leg, the sensory feedback pathways available in the body, and the neural
structure of the central pattern generator (CPG). Using two different models of the
CPG, we explore several issues in the neurobiology and robotics literature, including the role of reflexes in locomotion, the role of load reception and positive force
feedback in generating the gait, and the degree to which central or peripheral control
plays in human walking. We show that the walking pattern can be generated by a
combination of a half-center CPG and reflex interactions phase modulated by the
CPG, and that load receptors in the muscles can play a substantial role in generating
the gait, using positive force feedback. We compare the gait of the robot to human
subjects and show that this architecture produces human-like stepping. Varying the
degree of direct central control of lower limb muscles by the CPG, we show that the
most human-like gait is generated with a relatively weak central control signal, which
modulates reflex responses that generate most of the muscle activation. These results
allow us to conceive of locomotion as a series of nested loops, with a central CPG
or rhythm generator modulating lower level reflex interactions, while higher centers
modulate the CPG. Since locomotion is a primary mechanism by which animals interact with the world, this research is relevant to artificial intelligence researchers.
Recent understanding of cognition holds that minds are embodied, situated relative
to a set of goals, and exist in a feedback loop of interaction with the environment.
In our robot, we model the dynamics of the body, the neural architecture and the
sensory feedback channels in a complete dynamical feedback loop, and show that the
30
robot entrains to the the natural dynamics of the world. We propose the concept of
nested loops with descending phase modulation as a conceptual paradigm for a more
general understanding of nervous system organization.
31
Chapter 1
Artificial Intelligence, Robotics, and
Cognition
1.1
Introduction
Biology has been a source of inspiration not just to philosophers and artists, but engineers and scientists as far back as the Greek myth of Daedalus. As our understanding
of the world has advanced, our sources for inspiration have become more sophisticated. Initial attempts to design artificially intelligent computers, for instance, were
based on relatively early or limited understanding of how our minds and bodies work.
Research on these problems often advanced assuming that the internal mechanisms
employed by the brain were based on symbolic logic or were linear and deterministic
in nature. But in the real world, and especially the world of biology, very few systems
are truly linear. The most interesting problems, in particular our understanding of
how intelligent, reactive, living agents are produced are not. A truly intelligent, or
“alive” system is characterized not by the logical, predictable, or deterministic nature
of its behavior, but by the opposite - its unpredictability, the apparent indeterminism
of its decisions, and the often counter-intuitive behavior it exhibits. These abberant
characteristics render these systems resistant to conventional analysis. Yet, in spite
of, or rather because of those characteristics, these systems have the ability to solve
problems that traditional techniques have been incapable of solving.
1.2
Early Cybernetics
Man’s attempts to reproduce the independent, intelligent capacity for action of humans, or even just animals has a long history. The field of cybernetics emerged in
the mid 20th century out of a collaboration of neuroscientists, roboticists, mathe-
32
maticians, psychologists, and others. While it evolved into a broader field studying
goal-directed behavior, its early focus was on the study of intelligence in biology,
particularly human intelligence. The field quickly split along the issue of whether
the mind was “digital” or “analog” [173]. Those in the digital camp argued that
the informational processes of the mind could be abstracted away from the neural
substrate of the brain, and therefore reproduced in a machine. One key development
of this concept was the McCullough-Pitts (MCP) neuron [140], a precursor of the
perceptron. This model of the neuron was based on the idea that neuron spikes were
digital “all-or-nothing” in nature and so could be thought of as implementing a form
of propositional logic. This seemed to comport well with a symbolistic approach to
theories of the mind; if the brain works as a logic processing system, then symbolistic
reasoning appears to be the basis of intelligence. By contrast, those in the “analog”
camp stressed the importance of the continuous non-digital nature of the physical substrate, noting that digital models of the mind were merely models that lost important
aspects of the real systems in the process of abstracting away the informational process. These researchers showed that complex, lifelike behavior could be produced by
simple “analog” models of neurons. For instance, W. Grey Walter’s “turtles” [230]
were capable of locating a recharging station by phototaxis. Ashby’s homeostat [7],
based on analog control theory, exhibited learning and adaptation against a changing
environment well before symbolistic machine learning techniques were developed.
1.3
Symbolic AI
Despite these early successes, the symbolistic approach won out largely for pragmatic
reasons. Propositional logic and binary mathematics were well developed concepts
at that time, making it possible to build a symbolic processing machine, such as
the ’Logic Theorist’ [158], a precursor to the modern computer. Subsequently, this
approach emerged as the new field of Artificial Intelligence (AI). The symbolistic
33
approach to AI proposes that intelligence is a process of symbol manipulation. Statements are either true or false, and are connected by logical propositions, and these
statements can be used to determine the true or false status of other propositions.
Objects have discrete attributes such as shape and color and size. Later, the binary
perceptron model of the neuron was abandoned due to the misperception that such
networks could not reproduce the XOR function [147] (though this was only true for
single layer networks), resulting in an AI field dominated heavily by purely abstract
symbolistic approaches to intelligence, far removed from even abstract models of the
brain or neurons.
As it turns out, the early “analog” proponents may have been correct in warning
that the “digital” model of the mind might lose something in the process of abstraction, as the symbolistic approach to intelligence ran into a number of philosophical
problems [137]. In addition to the “frame problem”, which deals with the fact that
the truth status of a statement can change with time, many roboticists also quickly
realized that symbol manipulation was the least of the problems with building a true
intelligence. Before symbol manipulation can occur, visual and auditory processing
must take place that extracts symbols from the visual and auditory stream. This
turns out to be an extremely difficult problem, much more difficult than symbolic
logic. Moreover, as our understanding of the mind has advanced, we now realize
that this process is not simply a feedforward process, but one in which the mind is
constantly engaged, manipulating attention and altering our perceptions in a context
dependent manner.
1.4
Situated, Embodied, Dynamical Cognition
In the late 80s and early 90s, competing approaches to symbolic AI began to emerge,
some reviving ideas espoused by the early cyberneticists such as Ashby and Walter.
After the revival of artificial neural networks (ANNs) in the 80s, the connectionist
34
paradigm [203] was developed, holding that cognition emerges from complex interactions between simpler nodes at the level of neurons. Rodney Brooks advanced a
non-representationalist approach [20] to robotics, which involved building robots that
do not attempt to achieve higher cognition, yet produce seemingly complex behavior
from purely reactive rules, much like W. Grey Walter’s turtles. Brook’s robots are
based on a “Subsumption Architecture” [21] in which layers of such reactive rules
are arranged in hierarchical fashion, with higher level layers sending simpler signals
to manipulate the lower level layers. Evolutionary algorithms such as genetic algorithms [72], were developed, seeking to mimic the actions of DNA and natural
selection by applying a fitness function and evolving parameters iteratively. Reinforcement Learning [207], draws upon biological inspiration by modeling learning as
a process by which an agent adjusts its actions in response to feedback from the environment, often keeping track of the rewards offered by various actions within a value
function.
A common theme in these new approaches is, first, biological inspiration, and
second that cognition is embedded in the real world in a body and situated relative to a
set of goals. This perspective links robotics to artificial intelligence by emphasizing the
need for a mind to have a body, along with sensors with which to perceive the world.
It also reshapes our perspective about cognition. A cognitive agent is not a general
purpose problem solver as in traditional AI. Instead, the agent has a real problem
to solve. Animals must be on the hunt for food, or sex, or shelter, and must seek
out those things that are specific to the organisms particular needs. Consequently,
what an organism’s mind has evolved to pay attention to, and thus the nature of the
organism’s cognition, is determined by what features of the environment are relevant
to its survival. Is the organism an insect or a bird? What does it eat? What eats
it? How does it find a mate? All of these issues will shape the nature of its visual,
auditory and higher neural functions.
Moreover, the way in which the agent interacts with the world is integrated with
35
the mechanics of its body. Does it have legs, or fins, or is it a biped with opposable
thumbs? Consequently, a large portion of our minds are devoted not just to perceiving
the environment in relation to our goals, but to manipulating our bodies so as to
achieve them. We act upon the world in a constant feedback loop, sensing it, making
decisions, and taking actions intended to achieve some goal. In animals, learning
based on feedback from the environment, or reinforcement learning is thought to
underly many of the learning processes in the brain. On a more basic level, the
body interacts with the environment through biomechanical feedback. For instance,
walking is now believed to be at least partially determined by the interaction of the
dynamics of the body with feedback in the form of foot strikes, to produce a limit
cycle behavior [88], a dynamical system that generates self-sustaining oscillations (the
step cycle) from the coupled behavior of the limbs and the environment.
More broadly, the dynamical theory of cognition [12,13,66,217] holds that the mind
is dynamically coupled with the body and the environment. From this viewpoint, the
brain-body-environment system should be studied as a whole system, from whose
complete interaction behavior is produced, rather than separate pieces with detailed
interfaces through which precisely calculated control signals are passed. In philosophy
of mind, a parallel concept known as the extended mind hypothesis [29] has emerged
which holds that objects in the external world function as part of the mental processes
which make up higher level cognition. This new concept of cognition illuminates what
may be missing from earlier symbolistic efforts to develop artificial intelligence.
In symbolic AI, the mind is considered an informational process that it independent of, and can be lifted from the body, and replicated in a machine. By comparison,
in real brains and neurons, as in the dynamical view of cognition, knowledge is not
contained (solely) in the spikes (information) but in the physical substrate of the brain
itself (the body). The structure of the interconnections between neurons, the behavior of different types of neurons, and the way various neurotransmitters affect the
system all contain knowledge that has been encoded by evolution and development.
36
This knowledge is not merely replicated from a genetic program, but influenced by
developmental processes over the course of an animal’s life that create and build the
structure of the brain, while the animal is interacting with the environment. Finally,
brains are recurrent, resulting in dynamical feedback loops that include internal processes as well as interactions with the environment, and these loops can themselves
be altered by neuromodulation on both long and short time scales. While these processes can potentially be simulated on a computer, they require the inclusion of a
more detailed model of how the physical brain actually works to capture, as well as
a world with which the agent can interact.
This new understanding of cognition emphasizes the need to embody agents in
robots. If cognition is dynamically coupled to body and environment, then it is
impossible to build realistically intelligent systems without bodies, and without an
environment to learn from. Part of this task can be accomplished by creating simulated environments for the simulated agents to explore, but simulations generally
limit the richness of the real environment which organisms are exposed to, which may
play a limiting role in attempting to model cognitive processes. Further, simulations
may obscure important aspects of cognition by abstracting away aspects of sensory reception that are dynamically relevant. For instance, the visual processing and object
recognition tasks discussed earlier. By abstracting away the complex neural processes
involved in object recognition, classical AI obscures the nature of real vision and important aspects of cognition with it. In addition, the process of attempting to build
robots based on biological principles may also uncover challenges which can further
our understanding of how animal brains and bodies function [232].
1.5
Legged Locomotion
What does this have to do with legged locomotion? If we stipulate that cognition is
dynamically coupled with the body and world, then motor control becomes a primary
37
entry point for investigation. The body is the interface by which the mind affects
the world. Without actively affecting the environment, the dynamic loop between
the mind and the external environment is not closed. Moreover, navigation within
the world is a primary function of all organisms with brains. Researchers such as
Beer [14, 15] have studied insect walking as an entry point for the study of animal
behavior as a dynamical systems. In this dissertation, we describe the development
and control of our own walking system based on new dynamical approaches to locomotion. These include passive dynamic walking, and central pattern generators,
two biologically based concepts of how walking is produced that are in contrast to
conventional approaches.
Conventional approaches to locomotion in robots lie through classical control theory. These methods attempt to explicitly control joint torques and/or joint angles
throughout the step cycle, based on some predetermined criteria for stability. A common criteria is Zero-Movement Point (or “ZMP”) control [227]. In ZMP, the center
of pressure or “zero-movement point” is controlled to lie within the support perimeter, such as with the famous Honda Asimo robot [86, 194]. Not only does this cause
the robot to consume excessive power, it also gives the robot an unnatural gait. For
example, robots based on ZMP use a bent-knee, bent-hip gait. From a control standpoint a bent-knee, bent-hip gait appears more desirable, since it avoids the singularity
that occurs when the joint is straightened, allowing for more rapid reaction times.
This gait exists in primitive primates [34], but is energy inefficient and unlike natural
upright human walking.
By comparison, humans exhibit very free, flexible, dynamically stable gaits that
easily adapt to changing terrain conditions. Natural gaits make extensive use of
passive dynamics to minimize energy consumption [43], applying torque only as necessary to maintain the walking cycle, primarily through energy injection at toe off.
McGeer [141] investigated the passive dynamic walking in the early 1990s, showing
that it was possible to design walking systems which could walk downhill under only
38
the effects of gravity by designing the body mass properties appropriately. The interaction between the body and the environment falls into a limit cycle, creating a
periodic stepping motion as the robot walks downhill. In other words, natural walking
is part of a body-environment dynamical feedback loop. Inspired by McGeer, other
researchers have designed robots which attempt to incorporate passive dynamics into
the walking cycle along with minimal injects of propulsive energy at toe-off. Whereas
classical control techniques might plan a joint trajectory explicitly and apply torques
to make the limb motion conform to it, this dynamical approach requires you to allow
the limb to remain relatively uncontrolled and inject energy only at certain points.
The other recent development is that of central pattern generators (CPGs). The
CPG is an intrinsic neural rhythm generator within the mammalian spinal column
[22, 79], which contains groups of mutually inhibitory neurons that produce oscillations. These oscillators are used to generate complex alternating patterns of muscle
movement. The CPG can be modified by interaction with sensory feedback from the
environment, via direct afferent feedback, and via central modulation. Rather than
precisely controlling joint torques or angles, the CPG approach is to interact with
environmental feedback and passive limb dynamics to produce a stable cyclic gait.
1.6
Robotics and Biology
Developing robots based on CPGs is interesting not only as part of a process of designing better walking robots. While biology is an inspiration to roboticists, robotics
can also help model biological systems [232]. Since CPGs function as part of a complete dynamical feedback loop, involving the body and environment, they can only
be properly studied when implemented within the context of the biomechanics of
a body [218]. While some neuromechanical simulations have been performed [165],
simulations often exist in an idealized world, for instance one without sensory noise
or where the floor is perfectly level. Accurate neuromechanical simulations of walking
39
can also be extremely difficult to develop, because of the difficulty of accurately modeling the effect of ground contacts. Developing a robot based on the neurobiology of
the CPG allows us to model the CPG in a real physical body, without the need to
develop a detailed simulation. Moreover, while many robots using CPGs have been
developed, these robot’s CPGs are only loosely based on the biological literature.
Most use abstract neural oscillators such as Matsuoka oscillators [134], and map the
outputs to joint angles or torques rather than to muscle activation. Thus far, there
are no robots that combine an accurate model of both the biomechanics and the neurobiology of walking. Implementing these elements in a robot allows us to produce a
“neurorobotic” model, which we can use as a testbed to explore and validate certain
hypotheses about locomotion in the biological literature.
Several issues which we explore in this research include the interaction between
reflex circuits and CPG circuits in producing the walking cycle, the use of load detection and positive force feedback in generating the gait, and the degree of central
(feedforward) versus peripheral (feedback) control in stabilizing the walking cycle
against perturbations.
1.7
Summary
In this dissertation, we describe the development and control of our bipedal robot,
Achilles, which is designed to produce walking as a dynamical process, based on a
detailed neuromechanical model of human walking. In Chapter 2 we overview the
biomechanics of the human body, in light of the need to design a walking system in
which walking is produced by a brain-body-environment feedback look. In Chapter
3, we review the neurobiology of walking, beginning with a basic primer on neurobiology, and discussing the most recent models of the mammalian locomotory CPG. In
Chapter 4, we review previous robotics research involving bipedal robots, including
passive dynamics walkers and CPG-based robots in more detail.
40
In Chapter 5, we describe the design of our robot, based on principles of human
leg muscle architecture. In Chapter 6, we describe the design of our neural controller,
based on the neurobiology of CPGs in humans and mammals. In Chapter 7, we
describe the experiments which we performed, validating various concepts of the
neurobiological processes underlying locomotion. In Chapter 8, we discuss the results
with reference to the neurobiology of the CPG. Finally, in Chapter 9, we also discuss
some additional research relating to the reweighting of sensory inputs from various
sources in human postural control.
The contributions of this dissertation include the development of the robot as a
physical model of walking as a complete system, incorporating the biomechanics, central pattern generator, and sensory feedback mechanisms. This model is validated by
comparisons to human subject data, which shows that the robot’s behavior is in fact
similar to a human, thereby indicating that we have correctly modeled the important
major components of this system. In addition to this major contribution, we also are
able to explore several important issues in neurobiology and robotics, making contributions to the robotics literature by engaging concepts from neurobiology, while also
using the robot to test hypotheses in the neuroscience literature. These incldue three
major issues. First, the use of load detection and positive force feedback, which is
tested in the robot, and we believe could be a significant contribution to the robotics
literature. Second, the use of phase modulated reflexes in a CPG driven architecture, which provides a potential solution to the problem of inter-joint coordination
during walking. Third, the relative roles of central versus peripheral control over the
course of the locomotory cycle. We will explore these contributions in greater detail
in Chapters 7 and 8.
41
Chapter 2
The Biomechanics of Locomotion
2.1
Introduction
In this chapter, we briefly overview the biomechanics of legged locomotion. Legged
locomotion is a complex task involving many degrees of freedom, with multiple redundant sets of muscles capable of performing the same action [18]. These muscles are
non-linear actuators with varying elastic properties, and are often connected across
multiple joints, dynamically coupling their activity. This configuration makes controlling the muscle architecture of the body by itself a complex task. To complicate
matters further, the dynamics of the body, and the timing and position of interactions with the ground, determines the propulsion that the animal can produce. Joint
torques are not simply commanded in a top down fashion, but arise from a combination of body dynamics, interactions with the environment and both feedforward
and feedback control signals at multiple levels of the nervous system [38]. However,
a variety of simplifications of the dynamics of walking are often used by researchers,
which give surprisingly accurate descriptions of legged locomotion across insects and
mammals. Full and Koditschek have described this as a system of “templates” and
“anchors” [61,91], where templates are simple abstract models describing locomotion,
and an anchor is a more complex (but still simplified) biomechanical model that can
generate the behavior of the template.
2.2
Models of Locomotion
There are at least three basic templates that are used to describe walking and running: the inverted pendulum, the spring-loaded inverted pendulum, and the lateral
leg spring (Figure 2.1). Usually walking is described using the inverted pendulum
42
template. This template models the limb as a stiff rod with a point mass on top. As
the animal moves forward it vaults over its stance leg (Figure 2.1 (left)), and lands
upon the contralateral limb. Given its simplicity, the inverted pendulum forms the
basis for more complex models such as ballistic walking (Section 2.3). However, it is
now known that the dynamics of the inverted pendulum do not explain the motion
of the center of mass in humans or insects [61].
During running, however, it has been established that the leg is capable of absorbing and storing energy during stance and using this energy for propulsion at the
end of stance. This characteristic allows the leg to be modeled as a spring, as in
the spring-loaded inverted pendulum (SLIP) model (Figure 2.1 (right)). This model
has been shown to effectively model the dynamics of running across a wide range
of species from mammals to insects [61, 91]. Although the SLIP template was developed to describe running gaits, it also conforms to walking gaits for low system
energies [197].
A third template is the lateral spring-loading inverted pendulum. In this model,
the springs not only move forward and backward but also bounce the body from sideto-side. As the body swings laterally away from the leg, it can be transitioned into
the swing phase. This behavior allows the model to expand into three dimensions.
While these templates imply a clear distinction between walking and running, the
limb probably exhibits a degree of spring like behavior during walking. These models
are useful since they reduce the complexity of the problem and simplify both the
description and control of the step cycle. For instance, if the leg can be reduced to
controlling a passive spring, then only the hip joint needs to be controlled so long as
the limbs are properly designed to resemble a spring. For example, the MIT leg lab
developed a hopping biped based on the SLIP model [89] that could run.
43
Figure 2.1. The inverted pendulum and spring loaded inverted pendulum models.
One template for legged locomotion treats the limb as a stiff pole, over which the
body vaults on each step. During running the body is modeled as a spring-loaded
pendulum, absorbing energy during landing and releasing it during take-off. In reality
the human leg exhibits some spring like behavior during walking. The difference is a
matter of degree.
2.3
Ballistic Walking
As early as the late 19th century, toy walking devices were designed which could
stably walk downhill with no energy input at all [55]. However, it was not until
many decades later that Mochon and McMahon [149] demonstrated mathematically
that a biped could walk down hill with no energy inputs, simply on the basis of the
interaction of passively swinging legs with ground contact events. They termed this
interaction “ballistic walking”. Under inelastic impact conditions, the step-to-step
equations of motion of such a system results in a periodic limit cycle [76]. This
limit cycle is dynamically stable, meaning that perturbations from it will tend to
result in the return of the walkers motion to the stable pattern (although the basin of
attraction for pure passive walkers is known to be relatively small [196]). On the basis
of Mochon and McMahon’s results, McGeer [141] developed a passive walking robot
that demonstrated the utility of this concept for robotics, inspiring many subsequent
robots. Later, this concept was refined into a model called the “simplest walking
44
Figure 2.2. The simplest walking model. On a downhill slope, a system of two
inverted pendulums of appropriate mass and length results in a stable limit cycle in
2D space (From [65]).
model” by Garcia [65], shown in Figure 2.2. The essence of this model is that, if the
mass properties of the limbs are chosen appropriately, over one half-step, the walker
will return to its starting orientation, with the limbs reversed.
2.4
Phillipson step cycles
The step cycle of human walking is often divided unto swing and stance phases.
These phases are sometimes confused with flexion and extension parts of the step
cycle, though they are not identical. Functionally, the step cycle can be divided
into more distinct phases according to their role in locomotion. The Phillipson step
cycle [172] is divided into flexion(F) and extension (E1,E2,and E3) phases. As shown
in Figure 2.3, the flexion (F) phase occurs during the first part of swing as the limb
45
Figure 2.3. Phillipson step cycle phases. The classical division of the step cycle into
functional phases. F and E1 occur during swing, while E2 and E3 occur during stance.
F flexes the limb after take off while E1 extends it in preparation for touchdown. E2
absorbs energy, while E3 releases energy and propels the body forward. From [199].
is lifted so that the toe clears the ground during forward swing. E1 occurs during the
latter part of swing as the limb is extended forward to prepare for foot touchdown.
E2 occurs during the first part of stance, during this phase the limb absorbs energy
like a spring, as in the SLIP model, and is loaded with the body weight. E3 occurs
during the latter half of stance, where the limb extends to release energy and add
propulsive force to the step.
2.5
Mammalian Leg Muscle Architecture
The human body has several hundred muscles and over 200 degrees of freedom, making it highly redundant and complex to control. Although real bodies do not generally
have uniform mass distributions it is often simpler to model limbs as systems of rigidly
connected links of uniform mass. It also simplifies modeling to confine motion to the
sagittal plane, which is reasonable in humans where lateral displacements are rela-
46
Figure 2.4. A 5-link model of the human body used by Taga [211]. This model has
thigh and shank pieces and can be controlled at the hip and knee joints.
tively small. Sagittal plane models of 3-9 links have been used, including progressively
more joints in the two legs, along with a torso. A 3 link model includes two legs and
a torso with two hip joints. A five link model expands this to knees, a 7-link model to
ankles, and a 9-link model to toes. Link based models simplify the task of modeling
and dynamically simulating these systems. A 5-link model is shown in Figure 2.4,
including two shank links, two thigh links and a torso link.
Further reductions in complexity can be obtained using the fact that muscles often
work in synergistic groups. The most important difference between human mechanics
and that of typical robots is that human limbs work on the basis of a flexor-extensor
system (Figure 2.5 (left)). In this system, agonist-antagonist pairs of muscle synergists
work across a joint to control flexion or extension. Extensor muscles are those that
extend, or straighten, the limb, while the flexors bend it. Extensor muscles have
been observed to produce more force [157] and have a larger cross-sectional area than
either flexors or biarticular muscles [236], because they play a larger role in support
of body weight. In addition to single-joint or “monoarticular” muscles, some muscles
(or groups of muscle synergists) span more than one joint. Muscles that span two
joints are called “biarticular.” These muscles are believed to play a special role in
47
transferring power within the limb [177].
A reduced model of the human limb in the saggital plan will include a multi-link
model of the skeletal system, along with flexor-extensor pairs of muscles, representing
groups of muscle synergists, as well as the necessary biarticular muscles to actuate
the walking cycle efficiently. One such proposed model is given in [178]. This reduced
model, shown in Figure 2.5 (right), includes 9 muscles in a 3-link model of a leg
(this would be a 7 link system with two legs and a torso). The leg includes, thigh,
calf, and foot links. There are three pairs of antagonist monoarticular muscle groups
around each of the hip, knee and ankle joints, as well as three biarticular muscles.
The hip joint is actuated by the illiacus (IL, flexor) and gluteus maximus (GM,
extensor), the knee joint by the biceps femoris short(BFS, flexor) and vastus lateralus
(VL, extensor), and the ankle by the soleus (SO, extensor) and tibialis anterior (TA,
flexor). The biarticular rectus femoris (RF) spans the hip and knee joints from the
front, acting as a hip flexor and knee extensor. By contrast, the hamstrings (HA) acts
as a hip extensor and knee flexor. The gastrocnemius (GA) acts as an ankle extensor
and knee flexor. Keep in mind that these are not individual muscles but represent
synergist groups of muscles. In other similar models, different muscles are chosen as
the representative muscles.
2.5.1
Biarticular Muscles
Biarticular muscles are believed to have a role in transferring energy between joints
in the limb [177]. This happens due to the effect of the muscle adding a constraint
which couples motion at one joint to motion at the other. To see this, let us consider
the case of the gastrocnemius (GA) muscle, which spans the ankle and knee in the
model above.
The ankle is controlled by three muscles: the tibialis anterior (TA) and the soleus
(SO), a flexor/extensor pair, and the gastrocnemius (GA). If the GA is relaxed, only
48
Figure 2.5. Left: Agonist/antagonist muscle configuration. A flexor bends the joint
while the extensor straightens it. A biarticular muscle also extends the joint. In our
model extensors are all load bearing muscles that straighten the leg during stance up
to toe-off, while flexors bend the leg during swing phase. Right: Model of the human
leg. TA is tibialis anterior, SO is soleus, GA is gastrocnemius, VL is vatus lateralus,
RF is rectus femorus, IL is illiacus, HA is hamstrings, BFS is biceps femorus short,
GM is gluteus maximus. Redrawn from [178].
49
the SO and TA act to exert moments around the ankle. However, if the GA is held
at a fixed length, it acts as a constraint which couples knee motion to ankle motion.
When the knee is straightened by the force of the VL, the fixed length of the GA
will result in a moment about the ankle. As a result, the moment that the vatus
lateralis (VL) exerts on the knee adds to the moment about the ankle. This coupling
illustrates how biarticular muscles can be used to transfer energy within the limb.
Similarly, the rectus femoris (RF) acts on the hip and knee joints, allowing the
Gluteus Maximus (GM) to do work on the knee, which is in turn transferred to the
ankle. This combination leads to chain of energy transfer,
Gluteus Maximus −→Rectus Femoris −→
Gastrocnemius −→ Ankle
in which the VL, SO and GM all contribute to exerting force at the toe, greatly
increasing the total amount of force that the robot is able to project as it pushes off
the ground.
2.5.2
Work Transfer Equations
Here we give a mathematical formulation to the above qualitative description of the
effect of power transfer allowed by biarticular muscles. Biarticular muscles allow
work to be transferred between joints by coupling (and decoupling) the joints to one
another. When a biarticular muscle is activated, any torque around one joint produces
a torque around the other joint. This coupling can be seen from Figure 2.6.
We can show theoretically how this joint coupling allows work performed on proximal joints to be transferred to work about distal joints. In our experiment, we will
only be using the VL, GM, SO, RF and GA, so we will omit contributions from other
muscles.
50
Figure 2.6. Assuming the biarticular muscle is held at a fixed length, any motion
about the knee forces motion about the ankle.
The work performed around a joint by an individual torque is computed by:
Z
W = τ · δθ
(2.1)
θ
For multiple torques operating around the same joint, we approximate this for
small angle changes as:
W =
X
τi · ∆θ
(2.2)
i
The torque around the ankle is:
τankle = τGA + τSO
(2.3)
Multiplying by the change in angle, we have:
WAnkle = WSO + WGA,Ankle
(2.4)
Similarly for the knee and hip we have:
WKnee =WV L + WRF,Knee − WGA,Knee
WHip =WGM − WRF,Hip
(2.5)
(2.6)
51
Where WGA,Ankle and WGA,Knee represent the work performed by the GA on the
Ankle and Knee respectively (similarly, for the RF). When the GA and RF are active,
the straps are of fixed length under tension and the motors do not move. This action
couples knee motion to ankle motion as shown in Figure 2.6. Since the length of the
strap remains unchanged, using ~rGA,Knee and ~rGA,Ankle to represent the vector from
the joint to the attachment point for the muscle, for small angle changes we can write:
∆θKnee · ~rGA,Knee ∼
= ∆θAnkle · ~rGA,Ankle
(2.7)
Taking the cross product with the tension in the strap F~GA , and rearranging, gives
us:
∆θKnee · (rGA,Knee
~
× F~GA ) ∼
~
× F~GA )
= ∆θAnkle · (rGA,Ankle
(2.8)
And thus the work transfer effect:
∆θKnee · τGA,Knee ∼
= ∆θAnkle · τGA,Ankle
(2.9)
WGA,Knee ∼
= WGA,Ankle
(2.10)
From the above equations ( 2.4, 2.5, and 2.10), we can see that when the GA is
active, work is subtracted from the knee and added to the ankle by the force in the
GA. Hence we can write:
WGA,Ankle = WGA,Knee = WV L + WRF,Knee − WKnee
WAnkle = WSO + WV L + WRF,Knee − WKnee
(2.11)
(2.12)
Similarly, we can compute when the RF is active the associated “muscle” is of
fixed length and under tension, so that:
WRF,Knee = WRF,Hip
= WGM − WHip
WAnkle = WSO + WV L + WGM − WKnee − WHip
(2.13)
(2.14)
(2.15)
52
In other words, the RF transfers work from the hip to the knee, and we can see that
when the RF and GA are active that work performed by the VL and GM contribute
positively to work at the ankle.
Similarly, upon landing during a jump, the biarticular muscle allows the muscles
closer to the body (proximal) to absorb energy of the joints further (distal) from the
body. In other words, through the same mechanism, during landing, force on the toe
can be translated into rotation about the hip and knee, via biarticular muscles. Power
transmission and shock absorption via biarticular muscles has been documented in
humans [177].
2.5.3
Lombard’s Paradox and Function of the Hamstrings
While the RF and GA are seemingly the only muscles needed for leg extension, the
hamstrings (HA), can also aid in limb extension and in walking. The hamstrings are
a set of synergistic biarticular muscles that flex the knee and extend the hip. The
main function of the hamstrings is thought to be deceleration of the lower leg during
running, by action as a knee flexor. However, despite generally being regarded as
antagonists, during standing from a squat both the rectus femoris and hamstrings
activate simultaneously, a phenomenon known as Lombard’s paradox [125]. An intuitive explanation for this is that the rectus femoris extends the knee while the the
hamstrings extends the hip. However, each muscle has an antagonistic effect on the
other muscle. An alternative explanation is that the straight limb corresponds to the
lowest energy state, since stretching of the muscle around a joint will result in a longer
total length, and thus more total stretching of the tendons and ligaments of the limb.
Assuming the muscles exert equal force and all tendons have identical elasticities,
the limb will tend to straighten, unless other forces (i.e. downward compression) are
acting on it. Consequently, the hamstrings are also useful in any task that involves
leg extension.
53
Figure 2.7. Biarticular muscles and the SLIP model. By coupling the joints together, biarticular muscles allow the entire leg to act together to absorb or release
energy. During loading, the leg bends and extensor muscle stretch to store energy.
That energy can later be released, with added force on toe-off, during propulsion
phase. This process allows the limb to behave like a single spring. Thus a leg model
incorporating biarticulators can be a biomechanical anchor for the SLIP template.
The concept of power transfer via biarticulars has been criticized by Zajac [243].
This criticism is on the grounds that all muscles exert forces indirectly on the whole
body via joint intersegmental forces. For example, if a force is acting on the knee
an additional moment will be induced at the ankle, which will change the angular
motion of the ankle (or, alternatively the amount of force in the biarticular). We note
that this does not affect the way our formulation works. As long as the biarticular
remains at a fixed length, the joints are constrained to move together. In addition,
the force measured in the biarticular is not generated by the muscle, it is induced in
it by the effect of other joint forces and torques. The biarticular only has to develop
enough force to maintain its fixed length.
54
2.6
Biarticular Muscles and the SLIP Model
Because the joints are constrained to move in synchrony a collapse of dimension
is possible [91]. This collapse allows the entire leg to act in concert as a single
unit. By transferring work between joints, biarticular muscles allow all muscles in
the limb to jointly contribute to the ability of the limb to absorb and release energy.
Elastic elements such as muscles, ligaments, and tendons physically store this energy
throughout the limb. During the E2 loading phase of stance, ground impact forces
induce moments at the ankle, which are transmitted up the limb through the GA
and RF muscles, allowing energy to be stored throughout the limb in these elastic
elements. During the E3 propulsion phase, this energy is released, with additional
muscle force, and is transfered down the limb to the toe. The storage of energy within
muscles and tendons also allows energy to be injected easily by contracting the muscle
in series with the spring element.
The leg as a whole therefore acts as described by the spring-loaded inverted pendulum model given above during the stance phase when the biarticulars are active. The
simplified musculo-skeletal model shown in Figure 2.5 (right) is therefore an “anchor”
for the SLIP model [61], since it is a biomechanical implementation that exhibits the
spring-like behavior of the template. It can also serve as an anchor for the ballistic
walking template, since it permits the use of passive motion in the joints, so that
the full model becomes a spring-loaded ballistic walker. Thus the high-dimensional
behavior of the multi-jointed limb is reduced to a two-dimensional control problem.
The leg can flex or extend, and it can rotate (at the hip) forward or backward), vastly
simplifying the control problem. Biarticular muscles have also been identified by others [68, 97, 198] as a mechanism that stabilizes the control of multi-segmented limbs,
and permits the slip model to be extended to walking.
We note that using biarticular tendons (as opposed to controllable muscles) of
fixed length would permanently couple the joint motions to each other, eliminating
55
the ability to make certain motions independently. For instance, near the end of a
typical leg swing we would want to be able to flex the foot while also straightening
the knee. In order to maintain the functional flexibility of a real limb, the biarticular
muscles must be able to couple and decouple joint motions depending on the task.
That is, it allows voluntary control of the dimensionality of the limb. The joints
may be constrained to work in concert like a spring loaded inverted pendulum, or
decoupled to allow more complex movements. During walking this may be useful for
foot placement, especially in rough terrain, or during swing phase to clear obstacles.
While biarticular muscle structure allows the limb to behave as a single spring,
the elasticity of elements in the limb is not sufficient to fully explain the spring
like behavior of the leg. It has been suggested that positive force feedback through
simple reflexes could add spring-like behavior by amplifying muscle reaction. Through
modeling efforts it was shown that positive force feedback could generate hopping and
could generate a stable walking cycle through simple, reflexive mechanisms [68, 69].
We will return to this concept later in our discussion of the neurobiology of walking
(Chapter 3).
2.7
Summary
In this section, we reviewed the biomechanics of walking and showed how a leg architecture based on the muscle architecture of a mammal could be an “anchor” for
the SLIP model “template” as described in [61]. The flexor-extensor architecture of
the muscle allows the joints to make use of passive dynamic properties of the limb.
Meanwhile, the biarticular muscles allow all three joints to be coupled together so
that the 3-link 9-muscle model of the leg can be made to behave like a spring-loaded
inverted pendulum. This model serves as the basis for our leg design in Chapter 5.
56
Chapter 3
The Neurobiology of Locomotion
3.1
Introduction
In this chapter, we will review important concepts from neurobiology that are relevant
to the process of motor control and walking in animals. We will pay particular attention to the central pattern generator (CPG) and afferent feedback pathways associated
with locomotion. A background understanding of the biology of neurons, including
the concept of spiking, inhibitory and excitatory connections, and neuromodulation
is important. Therefore we begin with a primer on basic concepts of neurobiology in
Section 3.2. In Section 3.3 we will review in detail the literature on central pattern
generators in mammals, focusing on the various sensory feedback path ways and the
mechanisms for generating locomotion. In Section 3.4, we will discuss various models
for the locomotory CPG that have been proposed in the biology literature, as well as
neuromechanical simulations of these models.
3.2
3.2.1
Neurobiology Primer
Action Potentials
In vertebrates, most neurons (Figure 3.1) interact by the transmission of action potentials (Fig. 3.2). An action potential is caused by a biochemical positive feedback
loop. Inside a neuron, an electrical membrane potential is maintained by the flow of
ions through the cell membrane, particularly sodium, potassium, and calcium ions.
The membrane potential increases with the amount of positively charged sodium
ions in the cell. Built into the cell membrane are sodium ion channels sensitive to
57
Figure 3.1. Diagram of a neuron. The “inputs” to the neuron arrive on the dendrites
or cell body. When an action potential is formed, it travels as a wave down the axon.
Myelin can help the wave travel faster without decay. At the end of the axon, synaptic
terminals receive the action potential, causing the release of neurotransmitters at
synapses, which terminate on other neurons. neurotransmitters released into the
synaptic cleft bind with the post-synaptic neuron, which may inhibit or excite action
potentials on the post-synaptic neuron.
58
membrane potential. As the membrane potential increases, more sodium ion channels open, resulting in an influx of sodium ions and a further increase in membrane
potential. This effect causes a positive feedback loop that results in a rapidly increasing membrane potential - an “action potential” or “spike” (Figure 3.2). Eventually,
this positive feedback is canceled by negative feedback as potassium channels open
at even higher voltages, allowing negatively charged potassium ions into the cell.
Sodium channels also automatically deactivate over time. The action potential however, starts a chain reaction that propagates through the cell. As sodium ions flow
down the neuron’s axon, the action potential propagates along the axon by opening
sodium channels ahead of the potential, resulting in a wave traveling down the axon.
When the wave reaches the synapse, this occurence results in the release of neurotransmitters at the synapse. The presence of neurotransmitters (such as Serotonin
and Dopamine, for example) in the synaptic cleft triggers the opening of ion channels
in post-synaptic neurons, and generates either excitatory or inhibitory post-synaptic
potentials. Enough excitation will trigger an action potential in the post synaptic
neuron. The relative size of the spike on the post-synaptic neuron can be affected
by many factors that influence the synaptic strength. These include the availability
of neurotransmitters in the pre-synaptic terminal, the number of receptors on the
post-synaptic neuron, and the number of synapses between the two neurons.
Depending on the properties of the neuron, such as its membrane characteristics
and the density and type of ion channels it has, it may have very different behaviors in
response to stimulation. A typical model neuron spikes once at a time. However, some
neurons will fire in bursts, or will fire rapidly with little input. Over time, however,
the frequency of spiking will slow, as the neuron runs out of neuro-transmitters, a
process called adaptation. Adaptation is key to producing alternating bursts of spikes
in neural oscillators.
59
Figure 3.2. Action potentials. As the membrane voltage of the neuron passes the
threshold of activation, sodium channels open, allowing more sodium ions into the
cell. This effect causes a positive feedback loop increasing the membrane voltage and
opening more sodium ion channels. Eventually, at a higher voltage, potassium ion
channels open, allowing positively charged potassium ions out of the cell resulting in
a negative feedback loop. Eventually the sodium ion channels deactivate, and the cell
membrane returns to a resting state.
60
Figure 3.3. Modulatory interneurons and neuromodulators. Serotonin, a type of
neurotransmitter, acts as a neuromodulator in many sensorimotor interactions. A
serotonin neuron synapses on the pre-synaptic terminal of the sensory neuron where
it excites the motor neuron. When the sensory and modulatory neuron fire simultaneously, the serotonin supplied by the modulatory neuron interacts with the synapse in
a way that dramatically increases the amount of neurotransmitter released, resulting
in a much greater probability of the motor neuron firing. Modulatory interneurons
can also have long-lasting effects on the synapse by stimulating the development of
more ion channels or even more synapses.
3.2.2
Neuromodulation
The strength of a synapse can be influenced by the presence of specialized types of
neurotransmitter called neuromodulators. Neuromodulators primary function is not
to transmit the electrical signal, but to modify the synaptic strength. Neuromodulators are often carried by specialized “modulatory” interneurons, whose synapses
terminate on other synapses or across the axons of other neurons (a “trans-axonal
axon”). A neuromodulator binds to specialized receptors on either the pre- or postsynaptic terminal of the synapse. These receptors initiate slower “second messenger”
processes that stimulate (or inhibit) the production of neurotransmitters, promote
61
(or inhibit) formation of new ion channels or receptors, or stimulate (or inhibit) the
growth of new synapses. By doing so the synaptic strength can be modified. These
effects may occur on very short timescales (stimulating or inhibiting the production
of neurotransmitters), or relatively long ones (growing new synapses).
Synaptic strength can also be modified in the absence of neuromodulation through
Hebbian learning [83]. Hebbian learning is regulated by synaptic time-dependent plasticity (STDP), which causes the synapse to strengthen if the post-synaptic neuron
spikes soon after the pre-synaptic neuron, and weaken otherwise. In other words connections between neurons that fire sequentially are strengthened. This process occurs
on the order of tens of milliseconds. Neuromodulators may interact with Hebbian
learning by signaling when to amplify or inhibit learning in response to sensory signals such as rewards or punishments, though the exact mechanisms for learning on
longer time scales are not well understood. Neuromodulation can also occur through
so-called “gating” interneurons, as shown in Figure 3.5. Because inhibition often has
a more powerful effect on a neuron than excitation, by interposing an interneuron
between the sensory signal and the motor neuron, the gating interneuron can be used
to “open” or “close” the circuit. Although not strictly true the notion of inhbition
having absolute “veto” power over the neuron was also used in the McCollough-Pitts
neuron model [140]. Inhibiting a gating interneuron thus has the effect of opening a
switch. In this way neural circuits could be thought of as resembling a logic circuit
or a transistor. Figure 3.4 shows a gating interneuron in addition to a direct connection. If the conditioning only takes place at the interneuron, that interneuon can be
inhibited, switching off the learned circuit between the siphon skin and the gill reflex
at will.
Serotonin is known to modulate adversive conditioning in primitive organisms
such as the snail [184]. In this simple case, a single sensory neuron connected to the
siphon skin is mapped to a motor output, the gill withdrawal reflex, (see Fig. 3.4.
The strength of the synapse connecting this sensory input to the motor neuron is
62
Figure 3.4. The serotonin neuron is stimulated by a painful stimulus applied to the
tail. By pairing this stimulus with a touch to the siphon skin activating a sensory
neuron, the modulatory neuron and the sensory neuron fire simultaneously, increasing
the motor response. Over time, the synapse between the sensory neuron and the
motor neuron is strengthened. Later, when the siphon skin is touched, the motor
neuron will fire, even if no painful stimulus is present. This process is thought to
underlie behavioral conditioning.
63
modulated by a serotonin neuron with a trans-axonal connection to the synapse (Fig.
3.3). The serotonin neuron has its output synapses located on or near the synapse
connecting the sensory neuron to the motor neuron, or to an interneuron connecting
the two. When this serotonin neuron is activated by a salient stimuli, namely a shock
to the snail’s tail, serotonin released by the neuron binds to the pre-synaptic terminal
of the sensory neuron, setting off a chain of processes which strengthen the sensitivity
of the synapse. By pairing the salient shock with an otherwise harmless touch of
the siphon skin, the snail will be conditioned to respond to the siphon skin touch
by withdrawing the gill, even if the shock does not occur. This conditioning occurs
by the action of the serotonin neuron modulating (increasing) the synaptic strength
connecting the siphon skin neuron to the motor output.
It is known that neuromodulators play an important role in shaping motor actions
in other animals. For instance, serotonin is known to modulate the CPG in both the
lamprey spinal column as well as neo-natal rats [82, 129]. Since many neurological
structures are preserved throughout the animal kingdom, it is likely that serotonin
also modulates the human CPG. The role of neuromodulation acts on both short and
long time scales. On short timescales, modulation can be used to facilitate or inhibit
specific circuits connecting sensory input to motor responses. On long time scales,
modulation can be used to learn conditioned responses, as well as alter the structure
of the network by modifying network properties, and growing new connections.
3.2.3
Motor Control and Sensory Feedback
At the simplest level, the nervous system maps sensory stimuli to motor outputs, as
in the snail conditioning example above, although this process becomes vastly more
complex within the central nervous system in higher animals such as humans. In animals, sensory neurons called “afferents” feed information from the peripheral nervous
system back to the spinal cord and higher centers. These signals are integrated by
64
Figure 3.5. Gating interneurons vs. transaxonal axons. Here a sensorimotor circuit is shown with the sensory neuron in blue and the motor neuron in green. The
modulatory interneuron (orange) acts directly on the synapse (left), or via a gating interneuron (pink, right). Excitatory synapses are indicated using open circles,
while inhibitory synapses are indicated by filled circles. While modulation can induce
long-term changes in network connectivity by acting directly on the synapse (left),
modulation can also be used to refer to the effect of switching between sensorimotor circuits by inhibiting (right), or disinhibiting gating interneurons (bottom). If
learning is not involved, it is often simpler to write the circuit as a network of gating
interneurons.
65
Figure 3.6. Organization of motor control in the central nervous system. At the
lowest level, sensory signals are mapped to motor neurons via local circuits in the
spinal cord and brain stem. However, descending modulatory signals from higher
brain centers, such as the basal ganglia and cerebellum, can modify these circuits as
well as act directly on motor neuron pools. Image is from [184].
66
(a) Muscle Spindle
(b) Golgi Tendon Organ
Figure 3.7. Sources of proprioceptive sensory feedback in the muscles. Muscle
spindles (a) detect the lengthening of the muscle as it stretches. Two groups of fibers
as associated, Ia fibers, which are more associated with velocity, as well as Group
II fibers which are associated with absolute muscle length. Golgi tendon organs (b)
detect the load on the muscle and activate Ib fibers. Other sources of afferent feedback
include joint receptors and cutaneous afferents (not shown). Images from [164].
the central nervous system and used to generate signals which are sent to muscles
called “efferents.” A reflex pathway involves a relatively direct connection between
the afferent and efferent signals, though it can be modulated by descending signals
or other sensory stimuli as in the example above. Afferent signals are divided into
several classes, such as Group I, Group II and cutaneous fibers. These classifications
are not particularly important for us, except that we know which fibers are associated
with which sensory inputs.
The primary sources of afferent feedback we are concerned with are those associated with the proprioceptive control of locomotion. The source of afferent feedback
from muscles includes muscle spindles (Figure 3.7 (a)), which detect changes in length
of the muscle, as wells as Golgi tendon organs (Figure 3.7 (b)), which detect load,
67
and cutaneous (tactile) inputs. Skin sensors, capable of detecting contact on the feet
also play an important role in locomotion.
Originating in the muscle spindles, Group Ia fibers respond to the rate of change
of the muscle length, while Group II fibers respond to the absolute length. Reflex
circuits associated with the muscle spindles synapse in short circuits back to the motor
neuron, and are therefore often believed to act as a kind of PD control loop controlling
muscle length. The Golgi tendon organs are associated with Group Ib fibers. During
quiescent stance, Ib fibers exert inhibitory effects on the muscle, performing a kind
of load regulation. However, during locomotion, these reflex pathways are inhibited
and other excitatory pathways are facilitated [138]. The role of these afferent signals
will be discussed in more detail in Section 3.3.1.
Other important sources of afferent feedback include cutaneous inputs from tactile
sensors on the soles of the feet, and joint angle proprioceptive afferents. Higher
cortical centers also process other important sources of afferent feedback such as
visual and vestibular inputs, but we will omit discussing these since they do not play
a role in this research.
3.3
Central Pattern Generators
The local circuitry of the spinal column contains what is known as the central pattern
generator for locomotion. Central pattern generators (CPGs) are neural networks
capable of producing rhythmic motor patterns that control many innate rhythmic
behaviors in both vertebrates and invertebrates [132]. These include activities such
as breathing, eating and swallowing, and heart rate, in addition to locomotion. The
simplest model of a central pattern generator is the Brown half-center oscillator (Figure 3.8) [22]. In this model, two neurons are connected via reciprocally inhibitory
connections. Both neurons receive an excitatory tonic drive that will induce spiking. However, if there is sufficient reciprocal inhibition, when one neuron is firing,
68
Figure 3.8. Brown Half-center Oscillator. Filled black dots represent inhibitory
connections. A tonic drive stimulates the neurons to fire. Due to mutual inhibition,
one neuron will suppress the other, until adaptation causes firing to slow, eventually
to the point it is no longer able to inhibit the other, at which point they will switch.
Thus alternating bursts of firing are generated.
69
Figure 3.9. Entrainment of the CPG. The central pattern generator is driven by
sensory feedback from the system it is controlling. Since that system’s dynamics
are different than the oscillator’s endogenous frequency, the frequency of the overall
system is modified. This process is called “entrainment.” The entrained frequency of
the system as a whole generally lies somewhere between the natural frequency of the
system, and the endogenous frequency of the CPG.
the other will be inhibited. Hence one neuron will tend to dominate and suppress
the other. At some point the firing neuron becomes adapted and decreases its firing
rate. This disinhibits the second neuron. Eventually the second neuron begins to fire,
inhibiting the first. Half-center oscillators can be based on many different types of
neuron models, but this essential phenomenon remains consistent.
While a neural oscillator has an endogenous (natural) frequency of its own, it can
also be driven by a cyclic sensory feedback signal, to oscillate at the frequency of
the feedback (within certain limits). When the neural oscillator is used as a pattern
generator for some physical system such as locomotion, the dynamics of the oscillator
and the system become coupled. As shown in Figure 3.9, the oscillator drives the
behavior of the system while the system’s dynamics drive feedback that modifies the
frequency of the oscillator. This process is referred to as “entrainment”.
The most well studied locomotory central pattern generator is that of the lamprey [31,78,82]. Lamprey species have been identified in the cretaceous period and are
considered a basal group for all vertebrates. CPGs have also been identified in multiple species including rats and cats [79, 205]. Indirect evidence for a central pattern
generator in humans has been found by studying spinal cord injury patients [24, 40],
70
as well as infants [215, 216, 241]. While there is not yet direct evidence for a CPG in
humans, it would be very surprising if there were not, given the preservation of this
structure across the animal kingdom.
The lamprey CPG is known to be composed of a series of half-center oscillators
with one half-center per segment. These oscillators are weakly coupled together, resulting in phase lags between the vertebrate, which produce a traveling wave down
the body, in other words swimming. Much of the information on central pattern
generators in mammals comes from studying cats. Early experiments showed that
decerebrate cats could be made to walk on the treadmill through electrical stimulation of the midbrain [79, 201]. Cats and humans are both mammals, leading many
biologists to infer that much of the process udnerlying human walking resembles that
of a cat [50]. The human locomotory CPG is much harder to study, since it is unethical to give humans spinal cord injuries to study them. However, some insights can
be gained by studying humans with accidental spinal cord injuries. In particular, the
ability of some patients to recover locomotor function through treadmill training has
implications for the structure of the CPG [219].
Despite the simplicity of the half center model, the mammalian CPG, at least in
adult animals, is known to be much more complex. While lampreys have no limbs or
fins, mammals must control multi-segmented limbs with multiple joints. Rather than
flexing and extending in synchrony, the mammalian CPG exhibits phase offsets between the various joints. While some variation may be produced by the biomechanics
of the body, it is now known that complex patterns of activation, including phasic
offsets to the muscles of different joints, can be produced by the CPG even without
any sensory feedback [80]. The structure of the CPG for legged locomotion must be
more complex than the half-center model, but it is not yet understood how the CPG
produces these complex outputs. The default hypothesis, mirroring what is known
of the lamprey spinal column, is that each joint is driven by a separate half-center
(a “unit burst generator”) which is weakly coupled to the neighboring joints in the
71
limb. Nevertheless, the neural structure of the locomotory CPG remains largely unknown [233], and it has not been shown that the unit burst generator model could
produce the patterns observed.
While complex patterns can be generated without sensory feedback they do not
result in stable locomotion [71, 73]. In order to produce stable locomotion, the CPG
must be incorporated into a complete feedback loop incorporating the biomechanics
of the body and sensory responses due to interactions with the environment [218].
Consequently, many researchers consider sensory feedback to be an essential component of the CPG [166]. In Section 3.3.1 we will review the research on the known
roles that sensory feedback plays in the locomotory cycle. These roles include both
adjusting the behavior of the CPG, as well as acting via reflexes that operate under
the direct or indirect influence of the CPG. In Section 3.3.2 we will discuss these
sensory feedback effects specifically as they pertain to the entrainment of the CPG.
In Section 3.3.3, we will discuss the lower level reflex circuits, their role in locomotion,
and their interaction with the CPG. In Section 3.3.4 we will discuss hypotheses of
how joints are coordinated within multi-segmented limbs.
3.3.1
Sensorimotor interactions in locomotion
Descending supra-spinal signals modulate reflex and CPG neurons in a phase and
task dependent manner, gating circuits so as to select the appropriate circuits for
the task. During locomotion, afferent signals are reorganized such that their effect
assists in producing the step cycle [138, 244]. As mentioned, there are a few sources
of afferent feedback that affect locomotion: Group I (both a and b) fibers associated
with muscle spindles and Golgi tendon organs, Group II fibers associated with the
muscle spindles, cutaneous afferents, and proprioceptive hip angle afferents [183,219].
Comprehensive reviews of the effects of sensory feedback during locomotion may be
found in [77, 138, 167, 189, 235]. These effects occur both on the level of entrainment
72
of the CPG, and through more immediate reflex pathways that may be influenced by
the CPG.
Muscle Spindles: Ia and Group II Afferents Proprioceptive afferents from Ia fibers
are associated with changes in muscle length. Ia fibers encode the rate of stretching, while Group II fibers encode length [164]. Muscle spindle afferents feed back
directly to the motoneurons controlling the muscle, producing automatic muscle contraction whenever the muscle is stretched. For this reason they are often thought of
as PD control loops regulating muscle length. Feedback from Ia afferents generally
extends to synergist motoneurons, and inhibits antagonist motor neurons. The gain of
these reflexes can be modulated during locomotion, and replaced by phase-dependent
responses [4, 74, 75]. Feedback from Ia fibers may account for 30% of muscle activity [206]. Ia fibers around the hip muscle may also contribute to afferent sensing of
hip position which is used to reset the step cycle.
Golgi Tendon Organs: Ib Afferents During quiescent behavior, it has long been known
that Ib fibers provide inhibitory feedback to muscles, recently it has become apparent
that Ib fibers in limb muscles are reorganized during locomotion to provide excitatory
feedback [168]. It is now clear that the feedback from the load receptors in the
extensor muscles plays a central role in the locomotory cycle [51]. Approximately
30% of extensor muscle activity is directly generated due to force feedback via Ib
fibers [45]. Afferent feedback from Ib fibers also plays a crucial role in regulating the
frequency of the CPG. Stretching of the Achilles tendon in cats results in prolongation
of the extension phase during the walking cycle. Ankle extensor afferents excite
extension in extensor muscles throughout the limb [81], suggesting that they feed
back directly to the CPG. However some effects also appear to be phase dependent,
and resetting to extension during swing phase produces a temporary extension of the
limb, but does not necessarily reset the underlying frequency [117]. This behavior
indicates that these effects occur through multiple pathways, both directly through
73
the CPG, through reflex pathways mediated by interneurons influenced by the CPG,
and directly to the motor neurons [167, 189].
Cutaneous afferents Cutaneous or pressure sensing afferents on the feet also play a role
in locomotion. The role of cutaneous afferent is phase dependent [244]. During swing
phase, stimulation of cutaneous afferents triggers a stumbling corrective response, or
flexion, while during stance cutaneous stimulation can prolong and extend the stance
phase [49], which is consistent with excitatory feedback to extensor muscles. These
responses vary depending on what part of the foot is stimulated. During certain parts
of the step cycle they can also contribute to CPG entrainment [192].
Hip position afferents Hip position afferents play a role in regulating the rhythm of
the CPG [59, 115]. It is known that extension of the hip triggers the transition to
swing phase, although the initiation of swing is inhibited by loading of the stance
leg. Flexion of the hip does not have a symmetric effect on the initiation of stance.
Capsular joint afferents are known to exist [183], but their role in locomotion has
not been identified. Most sources attribute hip position sensing to Ia afferents in the
hip flexor muscles. In either case, some source of hip joint angle position sensing is
available and is used to reset the CPG upon extension.
Hip angle may also play a role in modulating the behavior of other reflex pathways
in a phase-dependent manner. As the leg approaches toe-off and touchdown the
behavior of both cutaneous and load sensing afferents changes [244]. Hip position
appears to modulate some foot placement reflexes in preparation for touchdown. It
has been proposed that position dependent sensory information could gate certain
reflex circuits [77]. Loeb [124] also points out that afferent signals are routed through
interneurons that receive descending inputs and project to other interneurons. This
signalling suggests that one source of afferent feedback may interact with another, for
example by having angle dependent afferents gate reflex circuits from the cutaneous
and Ib fibers.
74
3.3.2
Entrainment of the CPG
Central pattern generators are modified by cyclic sensory feedback signals caused by
the interaction of the system and the environment. This interaction results in the
system as a whole finding a new natural frequency determined by the dynamics of the
complete system. As discussed above, the main drivers of entrainment at the CPG
appear to be hip position sensors and load receptors in the stance limb (Figure 3.10).
Hip afferents inputs have been implicated in exciting flexion due to the stretching of
the flexor muscles at the end of stance [59,115]. However, this effect is counteracted by
the excitatory effect of Ib afferents from limb extensors, particularly at the ankle [81].
While the stance limb is loaded, this excitatory effect will prevent the hip position
afferents from initiating flexion, prolonging the stance phase. As a general rule, load
sensing appears to be critical to controlling the locomotory cycle. The limb cannot
be lifted until it is unloaded [51].
Prochazka implemented a somewhat similar mechanism in artificial prostheses
based on finite state machines [179, 180]. For instance, one example of a rule for the
initiation of swing is written as “IF extensor force low AND hip extended THEN
initiate swing”. Although these methods were not directly based on any biological
mechanism, it is easy to see how these effects might be achieved by a combination of
reflex circuits. For example, one reflex circuit might involve activation of hip flexion
from Ia afferent driven stretch reflexes at the hip flexor muscle, which is counteracted
(and possibly gated by) Ib afferent driven extensor reflexes from the ankle extensor.
A more complex version of this rule, as given by Prochazka is “IF speed is low AND
extensor force low AND hip extended AND contralateral limb loaded THEN initiate
swing ELSE prolong stance”. This rule is more complex, but could be constructed
by a set of reflex circuits which are modulated by descending signals as well as each
other. It is noteworthy that many passive walkers rely on similar finite state control
schemes, which are effectively the same as reflexive controllers.
75
Figure 3.10. Load regulation of CPG rhythm. Both load related Ib afferents from
extensor muscles and hip position afferents are involved in the regulation of the step
cycle, and the rhythm of the CPG. During stance, load receptors in the limb excite extension of the leg. As long as these load receptors are active, excitation of the extensor
half-center prevents the initiation of swing. The dynamics of the body-environment
system also plays a role. As the stance leg propels the body forward, the contralateral
limb swings forward. The swing leg initiated ground contact and becomes loaded,
reducing the load in the stance limb. Once the stance limb is unloaded, hip position
signaling initiates flexion, transitioning the leg into swing. The formerly swing leg
enters stance as the result of the loading of extensor muscles.
76
3.3.3
Role of Reflex Circuits in Locomotion
. During the 20th century a longstanding debate over the role of reflexes occurred
between alternative hypotheses proposed by Brown [22], who argued in favor of a
central pattern generator, and Sherrington [200], who proposed that locomotion was
produced by a series of cascading reflex effects. While this debate continued for several
decades, it gradually became clear that a central pattern generator in some form was
present, although it was not known how complex the central pattern generator was.
Lundberg [127] hypothesized that a simple half-center in combination with reflex
effects might be responsible for producing the complex motion of the limb. However,
this hypothesis appeared to be decisively refuted by the discovery that decerebrate
cats could still produce a complex walking pattern, even after all sensory input was
removed by deafferentation [80]. Nevertheless, without sensory feedback locomotion
is uncoordinated and unstable [71, 73], and it is now generally recognized that reflex
circuits play crucial roles during the walking cycle [244] and may be responsible for the
majority of muscle activation [45,206]. Although some researchers define the CPG to
include all circuity involved in motor control including reflex responses [77], we would
like to draw a distinction between signals that are primarily responses to afferent
feedback, and those that can be produced by the stimulation of rhythmic circuits
in the absence of afferent feedback. It is known that afferent signals project to the
CPG, to interneurons that are influenced by the CPG, as well as directly back to the
motoneuron [189]. Here we define a reflex as a circuit which routes afferent feedback
to motonoeurons either directly, or through interneurons which may be modulated
by, but are not part of, the rhythm generating network, thereby distinguishing these
circuits from the CPG circuits involved in rhythm generation. Reflexes are circuits
that operate below the level of the CPG, on more rapid time scales. By contrast, we
define the CPG as the rhythm generating circuitry which does not depend on any
afferent feedback to produce a cyclic output, though it may be entrained by such
77
Figure 3.11. The role of reflexes in locomotion [244]. During swing phase, ground
contact triggers a stumbling corrective response, flexing the limb to clear an obstacle.
However in late swing these reflexes are phase modulated. Ground contact now
triggers responses from the foot such as leg extension to place the limb to support
weight. During stance, reflexive load feedback generate load support in the extensor
muscles and propulsive force in the limb. At the end of stance phase, the unloading
of the limb triggers phase resetting and flexion.
feedback.
Reflex circuits are modulated in a phase, task, and context dependent manner, to
assist in the function at hand [244]. Supraspinal inputs gate both CPG and reflex
circuits in a task-dependent manner, altering the function of the CPG and the reflex
systems to assist in a given task, while CPG circuits modify reflex circuits in a phase
dependent manner. Many reflex circuits also have context dependent behaviors, suggesting the possibility that reflexes could be modulated by other afferents such as hip
position, or even cutaneous afferents [77, 148]. Zehr [244] divided the effects of the
reflexes into four parts of the step cycle (Figure 3.11), which could hypothetically
be controlled by a combination of phase-modulation by the half-center of the CPG,
along with hip position dependent afferents.
3.3.4
Inter-joint coordination in multi-segmented limbs
One issue that continues to elude researchers is how coordination between joints in
multi-segmented limbs is achieved, particularly in mammals. Based on the lamprey
78
research, the assumption is generally that each joint is associated with its own halfcenter, which is weakly coupled to the other joints. Each of these hypothesized modules is referred to as a “unit burst generator” (UBG) [77], shown in Figure 3.13 (A).
Each UBG controls groups of synergist muscles around a joint, and receives afferent
feedback from sensory inputs as well as coupling from other UBGs. Coordination
between joints is assumed to be achieved via these weak couplings.
While there is strong evidence that the CPG in insects has such an organization,
there is less evidence that this is the case in mammals [23]. Mammalian legs tend to
flex or extend syncronously across the whole limb, which suggests that a single CPG
may control the entire limb. However, under certain conditions individual joints
may be decoupled from that CPG. Some researchers believe that this decoupling is
evidence that each joint has a unit burst generator [23,77]. An alternative hypothesis
is that afferent feedback signals from hip position and ground contact sensor may
exert such a strong influence as to override the CPG [23].
The walking cycle is composed of alternating patterns of flexion and extension
among all muscles in the limb. In infants, this pattern is largely composed of synchronous motion at all joints, and synchronous activity in most flexor and extensor
muscles [216]. Over time, this pattern matures and becomes differentiated [56]. In the
adult human, the gait has evolved into what is called the “plantigrade” gait pattern,
which is characterized by features such as an initial heel strike, flexion of the knee
during mid-stance, and relatively late activation of ankle extensors during the stance
phase. The plantigrade gait shows phasic offsets in motion at the joints as well as
non-synchronous muscle activity.
Based on observations of the effects of various types of feedback on the overall
stepping rhythm, as well as individual muscles, it has been postulated that the CPG
has a multi-layered structure [139, 192] (Figure 3.12). According to this hypothesis
the CPG contains at least two layers, a Rhythm Generator (RG) and a Pattern
Formation (PF) network. The RG network is proposed to be essentially a half-center
79
Figure 3.12. A two layer model of the CPG. The rhythm generator (RG) layer
generates the basic half-center pattern that regulates the overall step cycle, while the
pattern formation (PF) network integrates afferent signals to shape this pattern as
necessary to the demands of the environment. Phase modulated reflex effects from Ib
and Ia afferent signals are controlled by the output of the pattern formation network.
80
Figure 3.13. A. Unit Burst Generator. This model assumes that each joint is
controlled by a separate half-center, which controls groups of synergist muscles at
the joint, and which is coupled to the other joints to produce the necessary phase
offsets. B. The distributed pattern generator, based on a two-layer model of the CPG,
contains a presiding rhythm generator (RG), and a pattern formation (PF) layer which
accepts afferent input from all parts of the limb, while outputting controls to groups
of synergist muscles to the joints. The PF neurons are not necessarily arranged in
mutually inhibitory pairs about each joint.
model, which sends descending controls to the Pattern Formation (PF) layer. Afferent
inputs act on both layers, however, some afferent signals act exclusively at the RG
level, while others act at the PF layer. This interaction explains certain observations,
in which certain types of afferent feedback elicits flexion or extension of the limb,
but does not impact the overall phase of the walking cycle [117]. A single rhythm
generator seems to regulate flexion or extension of the entire limb, as indicated by
the effects on overall rhythm extending to all muscles in the limb [81].
Many reflex effects operate throughout the limb, rather than at individual joints.
For instance, ankle extensor afferents and hip position afferents seem to act in concert
to control the overall flexion of leg at the stance to swing transition. Hip position
afferents clearly seem to initiate extension at toe-off, while ankle extensor afferents
trigger extension at the hip. Precise phase relationships between the hip and knee
also suggest that the timing of knee extensor and flexor activity depends on factors
other than loading of knee muscles and position of the knee joint. It is clear that
muscle activity around a given joint may depend much more heavily on the position
81
of other joints in the limb, as well as the loading of other muscles, than on activity
at that joint. Given these observations, the unit burst generator concept, in which
each joint is associated with a separate oscillator that is its primary driver, seems
less useful. Expanding from their two-level model of the CPG, McCrea et. al. [139]
have proposed that the limb CPG has a single central rhythm generator, which is a
half-center oscillator, and a pattern formation network that controls several synergist
groups of muscles in the limb (Figure 3.13 B). The rhythm generator influences overall
motion of the entire limb, while afferent feedback contributes to shaping that pattern
for the entire limb at the pattern formation layer, as well as to entrainment of the
rhythm generator. The PF layer may contain reciprocally inhibitory pairs of neurons
for each joint, but these do not receive an independent descending tonic drive; instead
they are driven by the rhythm generator. Hence, they are not capable of oscillating
independently of the central rhythm generator.
3.4
Modeling the CPG
CPGs have been mathematically modeled through both systems of abstract coupled
mathematical equations, and through biologically realistic models of spiking neurons.
These simulated neurons can be combined into networks to model the function of the
CPG. This section will review the various models of the CPG, including neuromechanical models implemented in simulation.
3.4.1
Spiking Neuron Models
The Hodgkin-Huxley neuron model [90] represents the most well established model
of the neuron. It is based on descriptions of ion flows across the membrane and
their effect on the membrane voltage and the generation of spikes. As such it is
considered the most biologically accurate model that is commonly used. The main
Hodgkin-Huxley equations are written:
82
I = CM ·
dV
+ ḡK · n4 · (V − Vk ) + ḡN a · m3 · h · (V − VN a ) + ḡL · (V − VL ) (3.1)
dt
dm
= αm · (1 − m) − βm m
dt
dn
= αn · (1 − n) − βn n
dt
dh
= αh · (1 − h) − βh h
dt
0.1 − 0.01V
αn =
exp 1 − 0.1V − 1
βn = 0.125 exp −V /80
2.5 − 0.1V
αm =
exp 2.5 − 0.1V − 1
βm = 4 exp −V /18
αh = 0.07 exp −V /20
1
βh =
exp 3 − 0.1V + 1
(3.2)
(3.3)
(3.4)
(3.5)
(3.6)
(3.7)
(3.8)
(3.9)
(3.10)
This equation essentially states that the current flow across the cell membrane
(I) is the sum of the flow due to the capacitance of the membrane (CM ), plus the
flow of ions through the various ion channels, with maximum conductances given by
ḡ terms, and these multiplied by time-varying functions n,m, and h, that allow those
conductances to change according to a non-linear function. Here V is the membrane
voltage at a given instant, and the various Vx terms are the resting potentials for
the ions (L represents an unspecified leakage). The α and β terms are nonlinear
functions of voltage. This equation will output a non-linear but continuous potential that conforms closely to the shape of an action potential. These equations are
computationally intensive to solve. However, other types of spiking neuron models exist that are computationally easier to solve, if it is not necessary to output a
realistically-shaped action potential.
More basic spiking neuron models involve an “integrate and fire” process, where
input currents modeling the input from synapses are integrated over time, until some
83
threshold is reached. At this point a “spike” is generated and the neuron model is
reset to some lower threshold voltage. A so-called “leaky-integrator” model permits
the neuron to resist firing unless some crucial rate of input is reached. When the rate
exceeds the leakiness of the neuron it will fire.
The leaky integrate-and-fire model is written:
I = CM ·
dV
+ ḡL · (V − VL )
dt
(3.11)
where I is again the current flow across the cell membrane, CM is the membrane
capacitance, gL is a membrane conductance, V is the membrane voltage, and VL is a
resting potential. In this case, the conductance of the membrane to the ion leakage
is a constant value CM , so the solution to this equation follows an exponential curve.
Instead of modeling the full shape of the action potential, the voltage is manually reset
to a low value whenever it reaches some upper “spiking” threshold. This introduces
a discontinuity in the output of the neuron. However, in a typical implementation
some discrete binary recording of the spike event is stored when it occurs. The spikes
can then be input into a separate synapse model, or used in binary computations,
although operations upon binary spikes occurring at asynchronous timing is not a
well developed mathematical proposition. Leaky integrate and fire models allow one
to implement a simple model of the neural oscillator, however, these neuron are not
capable of the complexity of behavior of real neurons, such as intrinsic bursting and
chattering behaviors.
Recently a neuron model that is easy to compute and reproduces a wide variety of
spiking behaviors was developed by Izhikevich [102]. This model has a few parameters
that adjust the resting threshold of the neuron and the parameters that control the
adaptation behavior of the neuron. By tuning these parameters, the neuron can be
made to adapt at various rates, recover from adaptation at various rates, and modify
the “saturation” of the neural adaptation.
84
The Izihikevich neuron is described by the following equations:
v̇ = 0.04v 2 + 5v + 140 − u + I
(3.12)
u̇ = a(bv − u)
(3.13)
If
(3.14)
v > 30,
v=c
(3.15)
u+ = d
(3.16)
Here, v is the membrane voltage, u is the adaptation of the neuron, and I is a tonic
(constant) drive current that originates from other neurons. The parameters a,b,c,and
d control how the adaptation evolves, and the voltage at which the neuron is reset
after firing.
3.4.2
Abstract Mathematical Oscillator Models
In order to study the behavior of networks of neurons, it may be simpler to abstract
away the detailed behavior of individual neurons such as spikes, and use more abstract
models of neurons. For example, a set of coupled differential equations representing an
oscillatory system can be used to represent the behavior of interacting neurons. These
continuous time oscillators include the popular Matsuoka oscillator [135], as well as
the Hopf oscillator [92, 186], van der Pol oscillator [48, 84, 223, 245] and continuous
time recurrent neural networks (CTRNNs) [14–16,176,224]. CTRNNs are a recurrent
version of the classical Artificial Neural Networks (ANNs) that are widely used in
computer science and artificial intelligence. ANNs (and CTRNNs) bear only general
resemblance to biological neurons. Instead of modeling individual spikes, the inputs
and output of an ANN is intended to represent at best an “average firing rate”.
This model cannot account for the speed of many responses in humans, since human
reaction times are known to occur on the time scale of individual spikes, however it
is still useful for modeling the network interactions of oscillators.
85
The Matsuoka Oscillator is written:
v̇i = −vi + βui −
X
yj wji + u0 + Feed
(3.17)
u̇i = −ui + yi
(3.18)
yi = max(0, vi )
(3.19)
Where Feed is an external feedback signal, vi is the simulated membrane voltage, ui
is the simulated adaptation, β is a weight associated with the adaptation, and wji
is the weight connecting neuron j to neuron i. Unlike spiking neurons these abstract
oscillators do not produce discrete spikes, but produce continuous time oscillations,
which may be mathematically easier to deal with computationally, since they do not
contain discontinuities.
The objective of these studies has been to understand the effect of network topology and coupling strength on the behavior of groups of oscillators. Matsuoka [134,135]
demonstrated how pairs of oscillators arranged in loops or chains would phase lock
into different configurations. Chains of neurons tend to fall into an alternating configuration, with one side of the half-center oscillating 180 degrees out of phase with
neighboring neurons. Loops, however, would phase lock so that the outputs of the
pairs would be evenly distributed across 360 degrees of phase. These results generalize to spiking neurons demonstrating the independence of the network behavior from
the specific type of neural model.
3.4.3
Neuromechanical Models of the CPG
Since the isolated CPG can produce “fictive” locomotion, certain aspects of the CPG
can be modeled using isolated networks (e.g. [192]). These models incorporate neurons
modeling afferent feedback, however, without the biomechanical model, the timing
and phasic behavior of the afferent feedback cannot be reproduced accurately. According to the dynamical view of locomotion, the CPG can only be understood from
86
within the complete feedback loop involving the dynamics of the body and environment [218]. Otherwise emergent aspects of behavior that depend on the interaction
between the neural system, musculo-skeletal system and environment (i.e. walking)
cannot be replicated.
Because of this, many researchers choose to model the CPG within the context of
a biomechanical simulation of the body and the environment with which it interacts.
Such simulations have been conducted for lampreys and insects, as well as mammals.
Reviews of this research are available in [58, 165]. The body is usually modeled
using multi-body dynamics where each segment of the limb is a rod of uniform mass
distribution, connected by pin joints. The environment model may consist of gravity
and ground reaction forces, or models of the fluid through which the animal swims
in the case of lampreys. Often these models are controlled by torques applied to
the joints [211]. More advanced models include muscle models based on biological
properties of human muscles [163]. The CPG may be modeled by abstract oscillators
87
[211], or by more accurate spiking neurons. The outputs of the CPG, that is the
activation levels of the neurons, are then mapped either to joint torques or to muscle
models to actuate the biomechanical simulation. The combined simulation of the
biomechanics along with the neural model is called a neuromechanical model.
Neuromechanical models are also useful for allowing hypothesis about locomotion
to be tested in situations that are impossible, or unethical, to test in vivo. In a
neuromechanical model, one can systematically isolate or eliminate specific sources of
afferent feedback or elements of the model and observe their effect upon locomotion.
Cat Hind limb Studies Neuromechanical models of locomotion in cat himdlimbs have
been developed in [52, 101, 191, 229]. In [229], a single cat himdlimb is modeled. The
neural pattern generator (NPG) used in this model is strikingly different from the
half-centered oscillator, and acts as a kind of state machine which cycles through
various phases and phase transition states. These neuronal states help modify the
effects of a number of sensory reflexes which drive the limb motion at distal joints.
Although this model is very different from the known half-center model used in the
lamprey, we cannot rule out the possibility that similar processes may be occurring in
the cat and human CPGs, since their structure is unknown. The model’s alteration
of sensory reflexes depending on the state of the NPG, models the phase-dependent
gating of sensory reflexes that is known to exist in the mammalian CPG.
In [52], the neural model is more explicitly grounded in known biological mechanisms, in particular load-sensing receptors, and joint angle afferents. In this case,
the authors notably did not include a rhythm generator for the limb, but instead
relied entirely on afferent feedback to generate reflexive limb movement. This was
done to avoid having the rhythm generator move the limb independently from the
sensory signals. In this study, it was shown again that load sensing of the limb overrides phase reseting at hip extension. Stable walking is only achieved if the limb is
unloaded before flexion.
88
It is noteworthy that in both of these instances where attempts were made to
implement a CPG in a biomechanical model of the limb that using purely reflexive
control elicited successful stepping responses. In other research [35, 67] reflexive control is effective in producing stable walking, although it is less robust to perturbation.
In [101], a detailed model of the CPG is used to control a cat hindlimb. This
model incorporates many of the afferent feedback effects including stretch reflexes
and Ib feedback from load receptors to the CPG, but does not include any phase
modulated cutaneous or Ib reflexes. To produce inter-joint coordination, there are
four interconnected CPGs that include specialized CPGs for transition states such
as stance-to-swing and swing-to-stance. Each of these CPGs outputs to a set of
muscle groups, allowing the joints to be coordinated by the selection of muscles that
compose the group. In this respect, the CPG resembles the state-machine type of
neural pattern generator in [229]. An earlier version of this research [191] uses one
CPG to control each muscle. While these models succeed in generating inter-joint
coordination, they are not well grounded in the neurobiological literature.
Another hindlimb model is studied in [240]. The purpose of this study was to
determine the importance of stretch reflexes during locomotion. In this case, the CPG
is manually implemented according to pre-determined experimentally-derived EMG
profiles. A finite state machine controls transitions from stance to swing and swing
to stance, according to “IF-THEN” rules derived from Prochazka’s work [179, 180].
Thus, this CPG does not generate its rhythm from the intrinsic properties of the
network or entrain to environmental feedback.
All of the above studies have limitations, especially when it comes to their efforts
to achieve appropriate interjoint coordination. The extent to which they are able to
produce such coordination is greatly dependent on contrivances which depart from a
grounding in the biological literature. Since the structure of the CPG in mammals is
largely unknown, it provides no guidance.
Another model of animal locomotion within a biomechanical simulation comes
89
from a salamander simulation [98]. The neural model used by this research is largely
based on the lamprey CPG. This model is appropriate since the salamander largely
moves using body undulation. However, the salamander does have legs. In this
case, however, the salamander’s legs are not multi-segmented, only the single joint
exists at the hip/shoulder, where it connects to the body, so the issue of inter-joint
coordination is not addressed.
Humanoid Studies The most well known simulation studies of the human CPG have
come from Taga’s research [208–211], though Bay and Hermami performed a simpler
study a few years earlier [11]. In Taga’s research each joint is controlled by a Matsuoka oscillator, as shown in Figure 3.14 (b). This is according to the “Unit Burst
Generator” hypothesis that each joint is controlled by its own oscillator. The human
body is modeled as the five-link system shown in Figure 3.14 (a). Weak coupling
along the leg allows the oscillators to entrain with an appropriate phase relative to
each other. Although each joint is actuated by two neurons, a flexor and extensor
neuron, the outputs of each pair of oscillators is combined to form a joint torque. Although this model successfully walks, it produces unrealistically large ankle torques,
and requires that the feet be pinned to the ground at the contact point to prevent
bouncing and slipping.
A complete biped model of the lower human body is studied in [163]. In this case,
both legs are modeled and detailed models of muscle properties and afferent feedback
pathways are included. The model is tested in a simulated treadmill with 60% body
weight support. However, the neuron model used is an ANN where the timing of
oscillation is manually controlled. The oscillations do not arise spontaneously and
are not entrained by sensory feedback. Manually tuned timing delays are used to
impose phase shifts between the joints. Another model [159], however, is able to
obtain fairly phasically correct joint angles and muscle activation levels by relying
entirely on weighted inputs from a central CPG controlling each leg, combined with
90
(a) Musculo-skeletal model
(b) CPG Model
Figure 3.14. Neuromechanical model used by Taga [211]. The human body is
modeled by a five-link system. Meanwhile, the CPG is modeled by sets of Matsuoka
oscillators, with one oscillator per joint, connected by weak coupling. Outputs of the
Matsuoka oscillators are mapped to joint torques while the body is simulated using
multi-body dynamics. Feedback from joint angles is used to entrain the Matsuoka
oscillators controlling the joints.
weighted afferent feedback. The feedback is recieved from simple models of muscle
spindles, Golgi tendon organs, and ground contact sensors. Critically, the model
allows all muscle spindles to potentially effect all other muscle spindles. A simple
half-center CPG is used to send rhythmic input to all the flexor and extensor muscles
in the leg. The weights which determine the connections between the muscle spindles
and the other motors as well as other weights associated with descending CPG signals
and afferent feedback were learned using a genetic algorithm. Some weaknesses of
this model include the fact that by learning the weights using a genetic algorithm,
it becomes more difficult to derive insights into the neural processes that regulate
locomotion in the evolved network. The solution found by the GA may not necessarily
be robust to sensor noise either. Also, the Golgi tendon organs only feed back to
the muscle they are associated with, though we know that in the mammalian CPG
extensor Ib afferent from Golgi tendon organs at the ankle entrains the entire CPG
91
and affect other muscles in the limb. Finally, the CPG is not allowed to entrain to
the sensory inputs. Nevertheless, it is noteworthy that it shows that it is possible to
obtain interjoint coordination with a half-center oscillator for the CPG and afferent
feedback between muscle groups in the leg.
In a very interesting recent model, Geyer [68] is able to control walking in a
simulated biped using only reflex signals combined with accurate biomechanics. In
this model, the reflexes modify muscle, joint, and leg compliance properties to stabilize
a passive, ballistic walking cycle. Hence the joint coordination is generated purely
by reflex action, rather than being centrally generated. Like previous research [67],
this study shows that appropriate phase offsets between the joints can be obtained by
reflexive responses. The notion that reflex responses are used to modify leg compliance
is also interesting in that it provides more an intuitive means of integrating CPG
control with the biomechanics of passive walking. The CPG may be providing phasic
modulation that tunes reflex circuits to produce appropriate muscle properties for
each phase, which could contribute to controlling the “springiness” of the step as in
the SLIP model.
3.4.4
Neurorobotic Models
What is a neurorobotic model? Since walking can only be understood as the outcome
of a complete system incorporating the body, environment, and neural control, the
CPG must have a realistic body to control and feedback signals that depend upon
the body’s dynamics. While a neuromechanical model may attempt to instantiate
the CPG within a simulated body, a neurorobotic model would be one in which the
model of the CPG is implemented in a physical robot. A physical robot frees one
from the need to develop a detailed simulation of the system, which involves problems
such as modeling impact events, bouncing and slipping.
Both biology and engineering have drawn inspiration for significant developments
92
from each other. In addition to being the basis for better robots, robots provide a
platform for testing biological hypotheses [232]. For instance, in [128], Lungarella
uses a robot that learns to bounce in harmony with the natural resonance of a “Jolly
Jumper” device, to validate the hypotheses that development of motor control in
infants is a self-organizing exploratory process.
While there are many robots that have been built using CPGs, the CPG models in these cases are almost exclusively based on simplified abstract mathematical
oscillators, particularly Matsuoka oscillators, rather than accurate models of spiking
neurons. The output of these oscillators is also usually mapped to the joint angle, or
joint torque of a robot, rather than used to generate muscle force. Although CPGs are
capable of entraining to the dynamics of the system, there are relatively few instances
where the CPG is mapped to a biomechanically accurate model of the body. Given
the close coupling between the biomechanics and neural controller in the production
of walking, human-like walking is unlikely to be produced without a human-like body.
The background on approaches to bipedal walking, including walking using CPGs will
be discussed in the next section.
3.5
Summary
In this chapter, we reviewed the neurobiology related to walking, beginning with a
primer of the basic concepts of neurobiology, focusing on motor control and locomotion. We then reviewed the scientific evidence on central pattern generators, including
the role of various sources of sensory feedback in entraining the central pattern generator and initiating various phases of the step cycle. Key points are that entrainment
of the CPG occurs through phase resetting at the end of stance, when the stance
limb is unloaded. Load sensing from Golgi tendon organs is used in positive feedback
pathways to generate reflexive force in the stance leg. Reflexes are phase modulated
throughout the step cycle to serve functional purposes. We also reviewed models of
93
the CPG that have been proposed. In particular, the unit burst generator model
versus the multi-level model proposed by Rybak and McCrea [139, 192]. Finally, we
discussed several of the neuromechanical simulations, and their results in relation to
the literature on the CPG.
94
Chapter 4
Robotic approaches to Bipedal Locomotion
4.1
Introduction
The development of a humanoid walking robot has been the dream of roboticists
for many decades. Yet bipedal locomotion remains an extremely difficult problem.
Bipeds are high-dimensional highly nonlinear systems with complexly coupled dynamics and discontinuities at impacts. They must balance on two legs instead of four
or more, as in mammals or insects. While recent progress has been made such as
with the Honda ASIMO robot [194], this newer generation of robots still does not
exhibit a natural gait, consumes much more energy than a natural gait, and is far
more sensitive to perturbation. Part of the reason bipedal walking remains an unresolved, difficult area of research may lie in the coupled nature of the dynamics of the
robot and the techniques used to control it. Similar to the approach taken by early
AI researchers in separating the informational process of the mind from the physical
substrate of the brain, roboticists have often proceeded by designing a robot that has
physical properties vastly different from a human body. They then attempt to design
a controller that would nevertheless result in humanoid walking, through a top down
process in which commands are sent to track a trajectory, using as much torque or
force as is necessary. The robot must then be designed to be able to generate sufficient
torques to maintain its trajectory and balance. In this chapter, we will explore the
history of these past attempts to develop humanoid walking robots, and some of the
problems they encounter. We will then review some of the more recent alternatives
which are grounded in a dynamical approach to locomotion.
95
4.2
Conventional Approaches to Bipedal Walking
Just as walking in animals is part of a dynamical process involving the mechanics
of the body and CPG, conventional approaches to locomotion control in robots are
intertwined with the architecture of the body. Typically, bipeds are modeled as
multi-link systems with pin joints, to which torques are to be applied. Similarly, in
conventional robots, motors are placed directly at the joint to apply joint torques.
A survey of the general approach to biped locomotion through conventional control
theory is given in [95]. Generally some target for the robot’s behavior is chosen, and
control applied to cause the robot to track the desired behavior. The control target
itself varies. It may be the center of gravity or “zero-movement point” (ZMP) [227],
minimum energy cost, endpoint conditions for the step cycle, or certain aspects of
locomotion such as step length or walking speed. In this section, we review the
major conventional approaches to controlling robotic bipeds, including ZMP, optimal
control, and model predictive control.
4.2.1
Zero Movement Point
An arbitrary set of joint trajectories that mimic human walking is not sufficient to
produce stable walking in a robot. In order to avoid falling the robot needs to adhere
to some stability criterion. The most basic stability criterion is static walking [107].
According to this method, the robot must be statically balanced throughout the
walking cycle, which is to say that the center of gravity of the robot must be within
the perimeter of the stance leg during single support phases, or the convex hull of both
legs, during double support phases. However, this method neglects the momentum
of the body, so motion must be kept very slow to avoid destabilizing the system. An
extension of this criterion, known as “Zero Movement Point” (ZMP) [228], requires
the center of pressure (CoP, identical to the ZMP), rather than the center of gravity,
to remain within the support polygon. The CoP takes into account the effect of
96
(a) ASIMO
(b) HRP-2
(c) HRP-4 Female Robot
Figure 4.1. Robots based on conventional control theory with ZMP stability criterion. (a) The Honda ASIMO, (b) HRP-2 developed by AIST in Japan, (c) HRP-4c
female humanoid. Note the similarities in the stance and gait displayed by the robot’s
posture. The knees are bent and the feet kept close together. Also note that the limbs
appear relatively massive. The large limb size is due to the necessity for high joint
torques to maintain control against uncooperative natural dynamics.
97
momentum, allowing for faster walking. The control target in this case is usually the
center of pressure itself, but the CoP may also be used to generate joint trajectories
for control as in the Honda robot [86], and others, shown in Figure 4.1. ZMP control
requires the feet to remain flat on the floor, so that the position of the center of mass
and pressure can be computed from joint torques and velocities. In order to correct
its position to maintain balance, the robot must be able to produce large torques at
the ankle, requiring the robot to have powerful motors in the limbs. This design tends
to result in the use of a bent-knee gait, since this provides the robot with the best
ability to react rapidly with the greatest torque. The robot also tends to have very
heavy limbs. Because of these weaknesses, these robots are sensitive to perturbations
and very energy intensive.
4.2.2
Optimal Control and Model Predictive Control
In some cases joint trajectories are pre-computed based on human subject observations [85]. More sophisticated approaches rely on designing trajectories that minimize
some cost function which is consistent with following optimal control techniques. The
cost function may be based on energy consumption [190,193] and the trajectory found
by parametric optimization [19], or variational methods [28].
By adding an internal model of the system, predictions can be made about future
deviations from the desired trajectory. This method known as Model Predictive
Control (MPC) involves modeling the system in real time according to current planned
control signals for many time steps ahead of the present [146]. This approach should
be distinguished from optimal state estimation since the predictions are far ahead of
the future and use planned future control torques, rather than current control signals.
This predictive information can then be used to modify future planned control
torques. This strategy is executed by iteratively computing a sequence of optimal
control problems along the predicted trajectory. On each iteration, predicted motion
98
information is used to decide what the optimal control torque will be to minimize
the cost function. Control torques are then modified and new predictions are made
based on the modified plan. Examples of MPC applied to bipedal locomotion include
[104, 234], which uses MPC to predict and control the position of the ZMP. There
are many other examples of biped locomotion control which use this method such
as [8,9,39,41,42]. Usually, the target of control is still the ZMP, however this method
can also be used to adhere to other criteria for the gait cycle, such as step length and
speed, as in [221].
4.3
Dynamical Approaches to Bipedal Walking
The more conventional approaches to bipedal walking tend to treat the dynamics
of the body as a given, prior to the problem of control. This approach does not
mean that roboticists do not attempt to design controllable mechanics, but rather
that less attention is paid to how the natural dynamics of the body may complicate
or assist adherence to a desired performance criterion, or to designing the robot
to have amenable properties before attempting to control it. Because these robots
track a desired joint or ZMP trajectory, this leads to the need to apply large joint
torques to fight the natural dynamics of the body, which results in excessive power
consumption and requires much greater motor mass. However, from a dynamical
systems perspective, attempting to track joint trajectories derived from humans, for
instance, makes little sense if the robot’s natural dynamics are unlike those of the
human body. If the body dynamics are different from a humans, the natural gait is
necessarily going to be different, and attempting to force the body to track a humanlike gait will require the robot to apply additional torque. Moreover, attempting to
explicitly control the joint motion in this way will prevent it from finding a natural
limit cycle, if one exists, which means the robot will be less stable and more sensitive
to perturbations.
99
(a) McGeer’s Original Robot
(b) Cornell Robot with Knees
(c) Cornell 3D Passive
Robot
Figure 4.2. Purely passive dynamic walking robots. (a) McGeer’s original walker.
The only actuators are to lift the swing leg slightly for clearance. (b) Based on
McGeer’s robot, a team at Cornell built a walker with knees. [64] (c) A fully 3D
walker with counterbalancing arm motion [33]. The arms compensate for body twist.
Compared to the conventional robots above these walkers have light, skinny limbs,
and walk much more naturally.
Therefore, a more dynamical approach to walking begins with the design of bodies whose passive properties are already amenable to walking. In Section 2.3 we
previously discussed ballistic walking [149], passive walking [141], and the simplest
walking model [65]. More broadly, these systems can be referred to as ’Limit Cycle Walking’ [88], since the dynamics of the musculo-skeletal system interacting with
the environment forms a stable limit cycle. In this section, we review research on
robotic limit cycle walkers, including purely passive dynamic walkers, and passivebased walkers with minimal actuation. We will also consider how CPGs have been
used in walking robots and how they can be integrated into limit cycle walking.
4.3.1
Passive Dynamic Walking
The first fully passive walking robot was developed by McGeer [141] (Fig. 4.2 (a)).
Provided that the mass properties of the limbs are properly designed, under idealized
100
inelastic foot strike conditions, the step-to-step equations of motion result in a periodic
solution that is a limit cycle [76]. This limit cycle is dynamically stable, meaning that
perturbations from it will tend to result in the return of the walkers motion to the
stable pattern. However, the basin of attraction for pure passive walkers is known
to be relatively small [196]. The joint dimensions and masses must also be designed
within a fairly narrow range to produce a stable limit cycle.
McGeer’s robot has no knees, so the swing leg was retracted by a small motor
to provide foot clearance. Kneed versions were also explored by McGeer [142], and
built by Andy Ruina’s group at Cornell [64] (Fig. 4.2 (b)). Both of these robots
were laterally stabilized by additional legs. However, some three dimensional passive
walkers have also been built [33] (Fig. 4.2 (c)) which are stabilized by contra-lateral
arm-swinging to counteract the twist induced by the swinging legs.
A passive dynamic walking cycle is found by solving the step-to-step equations
of motion for a fixed point. If a solution exists the walker will have returned to the
same configuration at the end of one step cycle. Stability is determined by finding
the eigenvalues of the Jacobian with respect to this point. If all of the eigenvalues
have negative real parts, then the system is stable. Stability depends on the slope
that the robot is walking on, as well as the physical properties of the robot, the joint
masses, and moments of inertia.
4.3.2
Minimally Actuated Walkers: Technical problems with adding actuation
The development of purely passive walkers has inspired efforts to develop minimally
actuated robots based on passive walkers [3, 32]. These robots attempt to inject just
enough energy into the step cycle to keep the robot moving over level ground or
to adapt to small changes in slope. For example, stable walking can be produced on
level ground by phasically injecting energy during late stance via a toe-off action [222].
101
(a) CMU Biped
(b) Runbot
(d) Meta
(c) Dribbel
(e) Spring Flamingo
Figure 4.3. Actuated limit cycle walkers. Based on the original work in purely
passive walkers, passive-based walkers with minimal actuation were developed. (a)
CMU Electric Biped, (b) Runbot [131], (c) Dribbel [37], (d) Meta [195], (e) Spring
Flamingo. These robots tend to be based on light structures with low gear ratio
motors to allow for passive swings at the hips. As an alternative, Spring Flamingo
(e) uses virtual model control to mimic passive-like behavior, although this is less
energy efficient.
102
Usually these robots use a finite state machine to determine what phase of the step
cycle they are in and thus when to inject energy.
However, some complicating factors have been encountered in adding actuation
to passive joints. Conventional motors require a high gear ratio to deliver sufficient
torque to actuate the joint, but this reduces back-drivability, so the joint can no longer
rotate freely during the swing phase. One the other hand, if a low gear ratio is used,
the motor has less power, and so is less easy to control or requires a heavier motor,
which adds unwanted mass. Several examples of 2D robots that use passive motion
are shown in Fig. 4.3. CMU’s direct drive biped uses low gear reduction motors
at the hips and cables to actuate the knees (note the very large motors located
at the hips). ’Runbot’ [131] and ’Dribbel’ [37] use small servo motors on a very
light structure while shutting off power to the motors during swing phase to increase
joint compliance. ’Meta’ [195, 237] uses DC motors at the hip, but has a compliant
element that permits more of the robot to replicate passive motions. The knees are
unactuated. As an alternative, Pratt’s ’Spring Flamingo’ [174], uses ’Virtual Model
Control’ to mimic passive-like motion in an actuated robot.
Minimally actuated passive robots are generally either stabilized by lateral legs
or by a boom to confine them to the saggital plane. Note that these robots all
have relatively skinny legs compared to the robots in Figure 4.1. They exhibit much
more natural looking motion. However, these robots either do not have feet, or have
unactuated ankles, and generally have weak or unactuated knees. This configuration
means that the propulsive force is applied at the hip, rather than via toe-off, as
in [222].
Three dimensional actuated walkers have also been constructed. The MIT toddler
robot [213], inspired by the original “Wilson Walkie” is one example. its large feet are
designed to produce a stable side to side wobble, while ankle actuators inject energy to
keep the oscillation going. The Cornell 3-D biped [32] is based on the unactuated 3-D
passive walker shown above, with the same contra-lateral arm swinging mechanism
103
(a) MIT Toddler
(b) Cornell 3D Biped
(c) Flame
Figure 4.4. Actuated 3D walkers. Most wobble laterally, however Flame uses hip
abduction (a) MIT Toddler (b) Cornell 3D Biped (c) Flame.
as well as large feet. The most recent example of a passive 3D walking robot is the
Flame robot from Delft University [87]. This robot is the first to effectively use hip
abductors to stabilize lateral motion. It also has actuated ankles, which are driven
by series elastic actuators.
Most actuated passive dynamic walkers have been controlled using relatively simple controllers. Commonly, a state machine is used [3, 242], in which target angles
are pre-defined for each phase of the walking cycle. There may be stance and swing
phases, or double support phases included. Sometimes injections of energy are based
on an intuitive scheme [175], in order to stabilize some aspect of the walking cycle
such as step length or height. Reinforcement learning has also been used in several
instances [32, 152, 195, 214]. However, the minimally actuated limit cycle walkers in
this class still have a relatively small basin of attraction [196], and so are less stable
that human walking.
104
4.4
Robots using Central Pattern Generators
As discussed in Section 3.3, oscillatory circuits in the spinal column are known to
generate locomotory signals. Since initial simulations of walking using CPGs [11,211],
researchers began developing robots which were controlled by networks of oscillators
inspired by CPGs. Here we will review some of the efforts to apply CPGs to control
bipedal walking robots.
The neural connectivity underpinning CPGs in animals is unknown, and control
mechanisms for robots differ significantly from animal bodies. In addition, biomechanical simulations of the CPG often rely on conventional multi-link systems where
torques are applied directly to the joint, much like a robot with motors in the joint,
even though this is a simplification of the muscle forces in a real animal. As a result, many roboticists use CPGs that generate smooth continuous time trajectories
to generate joint angles or torques. Recurrent neural networks (RNNs) were used by
many connectionists and early approaches to dynamical cognition. Beer uses RNNs
to control a hexpodal insect [14–16], which has inspired many subsequent CPG-based
hexapods [10,63,122]. Simulations using bipeds include [151,185,224]. The most popular neural oscillator model is the Matsuoka oscillator [134, 135], which was used in
Taga’s original research and many subsequent efforts [53, 112, 136]. In rare instances,
spiking neurons have only been rarely used [121]. Despite the fact that CPGs represent limit cycle based control, implementations of CPGs in robots have often used
conventional robot designs with non-back drivable joints, and thus have not made use
of passive gaits as part of the control method [5, 153, 156, 212]. As a result, the joint
must be actuated throughout the step cycle, reducing the CPG to a joint trajectory
generation mechanism. For instance, CPGs based on sinusoids have been used in
which the sinusoid determines the position of the body within some nominal cyclic
trajectory, with foot strike conditions resetting the phase of the sinusoid on each
step. Sinusoidal oscillators allow the phase relationship among joints to be explicitly
105
specified, a problem for coordinating the motion of the lower limb with the hip. However, this approach reintroduces the problem of shaping the joint trajectory, since the
arbitrary output of the oscillator will not necessarily be stable. In some cases this
result has led to the ZMP criterion to be reintroduced [105]. Usually, the trajectory
must be carefully shaped using reinforcement learning [53], or some other method.
These robots end up exhibiting the same unnatural restricted bent-knee gait as the
ZMP controlled robots above, since they cannot maintain limit-cycle based stability
through the natural dynamics of the body.
4.4.1
Limitations of conventional robot architecture
Conventional robots generally use motors in the joints to control limb motion. However, using motors in the joints poses problems for producing the necessary passive
behavior of the joints [6]. Unless a low gear reduction is used, motors are not back
drivable, requiring them to be position controlled throughout the gait. Such joints
require a smooth trajectory for the motor control, which tends to drive researchers in
the direction of tuning the CPG output to become a trajectory generation method.
Alternatively, if a low gear reduction is used, the joint may be passive, but the motor
must be much larger, resulting in heavier limbs with motor mass located at the knee
or ankle joint. These robot designs have dynamic properties that vary greatly from
those of real bodies.
As an alternative to pure passivity, to produce spring-like behavior, motors can be
made to behave in a spring-like fashion through the “virtual model control” method
[110, 174], by modifying control parameters governing the motor. This technique
allows researchers to avoid the issue of developing pure passive walkers and focuses
on running based on the SLIP model instead. However, this method does not actually
store and release energy, it only simulates spring-like behavior. Power is dissipated in
the motor and then generated to simulate the spring-like release, making this method
106
relatively inefficient [6].
Producing robots that efficiently model the biomechanical properties of real bodies
demands an alternative architecture. Attempts to build actuated robots capable of
passive walking often use muscle-like actuation [3, 32, 222, 238] by pulling on either
side of the joint (although in one instance RC servos have been used by shutting off
power during swing phase [131]). Some key design features of these robots include
the use of passive springs to produce compliant joints [96], hard stops to produce
locking at the knee during stance, and antagonist artificial muscles, such as series
elastic actuators [187], to allow passive joint motion.
4.4.2
CPG Controlled Passive Walking
Since CPGs are capable of entraining to the passive properties of a system, the
integration of CPG based control with passive dynamics remains a topic of ongoing
interest. Combining passive walking with CPG control has been studied in simulation
[155, 211, 224, 226] since simulated link models have naturally passive joints. These
studies show that the CPG results in improved stabilization of the walker against
perturbations [209], while remaining energy efficient [225]. CPGs and passive walking
have been combined in Kimura’s research with the Tekken robot dog [60, 109–111].
Some implementations have also been made in sagittal plane bipedal robots [17, 54,
136], although these robots suffer from the same design problems as the passive bipeds
previously mentioned, having weak knees, and unactuated ankles (or no feet at all),
or have very massive motors placed distally from the body.
These robots are only loosely based on the neurobiology of CPGs. The CPG is
modeled abstractly using Matsuoka oscillators, and entrained using feedback from the
hip joint angle. Outputs are mapped to joint torques rather than muscle activations.
Sensory feedback mechanisms usually include ground contact sensors and angle sensors, but not load sensors. Phase resetting of the CPG often occurs at heel strike
107
rather than toe-off as in mammals. Often the limb does not have ankles so no toe off
is possible. Lower limb coordination is usually acheived through fixed phase offsets
or mapping of the joint to some function of the oscillator output, though in Kimura’s
case, it is lifted using flexion reflexes.
In animals, afferent feedback from the limb is phase modulated and occurs in
discrete bursts. Conversely, in systems of Matsuoka oscillators feedback is applied
continuously, according to some curve. This method introduces new problems of how
to modulate the strength of afferent feedback from angular joint position signals. For
instance, in [225], angular position feedback is passed through a PID controller before
being input to the CPG, in order to get the widest range of possible entrainment frequencies. In [136], a Q-leaning algorithm is used to map afferent feedback strengths
to CPG states in a bipedal robot, allowing the sensory feedback signals to vary depending on the state of the CPG. In [100], a CPG using a model of neuromodulation
was trained to control a simulated biped. The neuromodulator adjusted weights connecting afferent feedback signals to the CPG, similar to the effect of neuromodulators
on synaptic strength in neurons. This approach resulted in the CPG dynamically
adapting to changing environmental conditions such as a unlevel floor conditions.
4.4.3
Reflexes in Robots
Walking using purely reflexive systems to drive all motor responses has been widely
used in hexapods [10, 35], and has also been demonstrated a few times in bipeds
[67, 204]. Van Der Linde [222] showed that reflexive injections of energy during late
stance could stabilize a passive walking cycle on level ground. However, most research
on bipedal robots uses reflexes only to make corrections to the step cycle within a
larger control scheme. For instance, Huang [94] uses reflexes to adjust foot placement
and swing leg orientation for a robot under classic ZMP control. Kimura [110] also
uses flexion reflexes for postural control and step corrections in a CPG controlled
108
quadruped. The flexion reflexes are the main method of controlling the lower limb in
his robots.
By comparison, most passive bipeds use finite state machines for control. These
state machine based approaches to passive dynamic walking could be characterized
as “reflexive”, since the actuators respond depending on joint positions or ground
contact conditions. For example, Prochazka uses finite-state machines to control behavior of artificial limbs, defining the conditions for stance to swing transitions as a
set of “IF-THEN” conditions [179, 180]. Most passive walkers require only minimal
phasic injections of energy at specific points in the step cycle. Hence, reflexive injections of energy, as opposed to continuous control signals, are well suited to the task
of controlling passive walkers. The design of CPG-controlled passive bipeds might
therefore benefit from greater use of reflexive control.
4.4.4
Coordination of lower limb motion
Coordination of lower leg motion remains a significant problem for CPG based walkers. Based on knowledge of the lamprey spinal column, much of the literature assumes
that a separate unit CPG also controls each joint of the lower limb, with weak coupling
between oscillators determining the relative phase of these joints [77, 209]. However,
this approach has not proven to be effective in robots. CPGs in conventional robots
use some kind of trajectory shaping to obtain appropriate phase offsets. For example,
by using the oscillator to generate a foot position rather than hip and knee angles
directly [53], or by using sinusoids as the CPG and applying an arbitrary phase offset [99, 120, 153, 156], or by leaving the joint unactuated as in passive robots, or by
combining multiple signals from the CPG to obtain phase offsets between the hip
and knee [123]. By contrast, the reflexive controllers described above obtain phase
offsets by actuating the joint depending on the position of the limb and ground contact events, in other words, in response to the environment. This thinking opens
109
the possibility of combining CPGs with reflexive systems in a manner that allows
overall leg motion to be driven by a combination of centrally generated CPG signals,
and peripherally generated reflex signals, thereby obtaining the desired phase offsets
by response to the environment. Indeed, according to the neurobiology literature,
biological CPGs are responsive to sensory feedback at multiple levels and have fast
reflex pathways that adapt motion rapidly in response to environmental feedback.
These reflexes are modulated by descending signals from the CPG to produce phase
dependent reflex responses suited to driving locomotion. Recently, these concepts
have been proposed as a model for proper locomotory control in robots [51].
4.5
Summary
In this chapter, we reviewed efforts to develop bipedal walking robots. First we
discussed the mainstream conventional approach to bipedal walking, exemplified by
the Honda Asimo, including ZMP and Model Predictive Control as the major thrusts
of this approach. Second, we reviewed passive dynamic “limit cycle” walkers as well
as minimally actuated limit cycle walkers. Thirdly, we dicussed various efforts to
develop CPG based walkers and some of the limitations and difficulties these robots
have encountered due to the conventional design of robot bodies. We then reviewed
efforts to combine passive dynamic and CPG-controlled walking, in particular the use
of reflexive control in passive walkers and the potential to combine centrally driven
CPG control with peripherally driven reflexive control signals in a way that resolves
some of these issues.
110
Chapter 5
Design of A Bipedal Robot based on
Biomechanical Principles
5.1
Introduction
In this chapter, we describe the design of our bipedal robot, Achilles. The robot
has a unique architecture which is based on the biomechanical principles described in
Chapter 2. First we review principles of mammalian muscle architecture compared to
conventional robots covered in Chapter 2, and describe the biomechanical architecture
we use for our leg design. We then describe our technical implementation of this leg
model in a robot, including sensors and actuators that model the sensory mechanisms
and muscles of the human body. Finally, we describe tests performed demonstrating
that the leg is capable of transferring energy via biarticular muscles and projecting
force at the toe.
5.2
Mammalian Leg Muscle Architecture
Conventional CPG-based robots are biologically inaccurate in several respects. As
discussed in Section 4.4, these architectures use a continuous-time neuron model
(Matsuoka) often applied directly to a joint torque or position. Yet, in vertebrates,
neuron activation is applied to muscle contraction, rather than joint torques. Unlike
most robots, human muscle architecture is designed to take advantage of passive
characteristics. As shown in Figure 5.1, each joint is actuated by at least two muscles,
an extensor and an opposing flexor. If neither muscle is active the joint can move
passively. In animals, much of the body motion is produced by the natural dynamics
of the limbs. Thus the muscle, and the motor neurons which control it, need only
111
Figure 5.1. Model of the human leg. TA is tibialis anterior, SO is soleus, GA
is gastrocnemius, VL is vatus lateralis, RF is rectus femoris, IL is illiacus, HA is
hamstrings, BFS is biceps femoris short, GM is gluteus maximus. Redrawn from [178].
activated when it needs to bear load.
A simplified model of the human leg (Fig. 2.5) incorporates nine muscle groups,
which include three pairs of flexor-extensor muscles associated with the hip, knee and
ankle, as well as three biarticular muscles, or muscles that span two joints. Biarticular
muscles play the important role of transferring energy within the leg and absorbing
shocks [177]. When biarticular muscles are activated they have the effect of coupling
joint motions to one another, allowing work done at one joint to be transferred to
another joint. In addition, muscles and tendons have spring-like properties which
allow energy to be stored within the limb, allowing it to behave like a spring [2].
Muscles in animals also contain sensory feedback mechanisms which tell the animal
when the muscle is contracted (Golgi tendon organs) and when it is stretched by an
opposing force (muscle spindles). These afferent feedback signals are used directly
to activate reflexes and have complex interactions with the CPG depending on the
phase of the gait.
112
5.3
Implementation of Limb Model
We adopt the model of [178], and use it as the basis for the design of our robot. Since
this model allows the limb as a whole to behave like a spring, as described in Section
2.6, this simplifies control of the walking cycle. In order to implement this model we
need a system capable of mimicking muscles, joints that can swing passively, as well
as sensors that can reproduce some of the afferent feedback available to the CPG in
the human body.
5.3.1
A motor-strap system for mimicking muscles
R
Each muscle is modeled by a Robotis
RX-28 motor [188] attached to a bracket, to
which a kevlar strap is buckled. The contraction of muscles is mimicked by the motor
rotating the bracket upward, pulling on the kevlar strap. The strap is attached to a
hard point on the other side of the joint, which incorporates a force sensor. When
the extensor motor pulls on the strap the joint straightens, while if the flexor on the
other side is actuated, the joint bends. Both muscles can also be activated at the
same time, controlling joint stiffness. When both motors are relaxed the joint can
swing passively.
Each muscle is attached to the limb either by buckling the kevlar strap around
a hard point, or by buckling it to a sensor assembly that is attached to the hard
point (but is free to rotate to the line of force). The monoarticular extensor muscles
are also attached by bungee cords that are looped around the hard point, to allow
a degree of elasticity at the joint. Only the monoarticular extensors have elasticity.
The biarticulars do not so as to maintain a fixed coupling between the joints, and
the flexors do not because they do not need to store significant amounts of energy.
The elasticity allows the leg to absorb shocks by bending at impacts and to transfer
energy up the limb and store it at the VL and GM motors, as well as the SO.
113
5.3.2
Custom Force Sensors
To measure the amount of force in each ‘muscle,’ we designed a custom force sensor
assembly based on a Futek FBB300 strain guage [62]. These force sensors can be
considered to loosely model Golgi tendon organs, since they measure tension in the
strap. Meanwhile, internal to each servo motor we have a PID control loop that
maintains the motor position, or muscle length. Since the muscle spindles and Ia
fibers are thought to act as feedback control of muscle length, we can consider the
internal PID control a self-contained model of the muscle-spindle, Ia fiber control
loop, although we note that this does not allow us access to the control parameters
to modulate the strength of muscle spingle feedback during locomotion. In addition,
we have contact sensors on the heels of the robot and foot pressure sensors on the
toes, analogous to cutaneous afferents. We also have angle sensing potentiometers in
the joints, to measure the angle of the joint. These sensors model the roles played by
afferent feedback systems during locomotion in humans. An additional force sensor
is added at the Achilles tendon, which acts as a calibration check and allows force to
be measured directly at the ankle.
Force Sensor Assembly To work as a tension sensor the force sensors needed to be
buckled into the motor strap system. A plastic fitting was designed which could be
buckled to the kevlar strap, or attached to a hard point on the limb, allowing the
strap to pull on the bracket, in line with the tension. The fitting needed to pull on
either end of the sensor without twisting, otherwise the sensor reading would become
non-linear.
Microcontroller Sensor Board Each force sensor was connected to a custom designed
sensor board as shown in Figure 5.3. This board allows the sensors to be networked
on an RS-485 bus, which simplifies communication with the PC. A fully assembled
sensor board is shown in Figure 5.4.
114
Figure 5.2. Force sensor fitting. The force sensor is screwed into the fitting which
is buckled into the strap that mimics the muscle.
The board contains an instrument amplifier, which receives the output of the
force sensor, and converts it into a signal, referenced to 2.5V. A microcontroller on
the board receives this output on its A/D port and converts it to a 10 bit integer.
The microcontroller then constructs a packet which identifies the sensor ID and the
integer value read at the A/D port, and outputs it to an RS-485 transceiver which
puts it on the RS-485 bus. The gain of the instrument amplifier is controlled by a 1K
potentiometer.
An assembled load sensor board is shown in Figure 5.4. The board was modified
with the addition of a 10MHz oscillator to allow the PIC to reach its maximum 40MHz
clock speed, using a phase-locked loop to boost the clock speed by four times. This
clock speed allowed us to increase the baud rate on the sensor bus to 1.25 mbps.
Thus, we were able to achieve a sampling rate of 20 data points per second per sensor
with 28 sensor boards at this rate.
Calibration of Sensors Sensors were calibrated by measuring the voltage reading from
the sensor in line with a Vernier force sensor. Torques about the knee and ankle by
115
Figure 5.3. Schematic diagram for load sensor boards. The board contains an
instrument amplifier, microcontroller, and RS485 transceiver. The board samples the
output of the FUTEK force sensor, converts it to a 10 bit integer and outputs a
packet on the RS485 bus.
Figure 5.4. The assembled sensor board. An external oscillator was added to boost
baud rate on the sensor bus.
116
the GA and RF were calibrated with respect to the sensors, by pulling on the strap
at an angle perpendicular to the radial vector from the joint axis to the connection
points. The reported voltage was observed to be linear with respect to the load for
all sensors. A check was made on the calibration by comparing the observed force at
the Achilles tendon, and the sum of the tension in the GA and SO measurements.
The sum of the GA and SO force sensors approximated the force measured at the
Achilles tendon closely throughout the period when the leg was moving.
5.4
Overall Leg Design
The overall leg design is shown in Figure 5.5 (left). This image shows a CAD drawing
including the shape of the foot, calf, thigh, and hip with the placement of motors inside
each part. These parts were built out of ABS plastic, using a 3D part printer. Joint
dimensions were proportioned to human limb dimensions [236], assuming a height for
a full humanoid of 80cm. An image of the fully constructed leg is shown in Figure
5.5 (right).
5.4.1
System Level Block Diagram
Figure 5.6 shows a system level diagram of the robot. The robot is controlled by a
PC running Ubuntu real-time Linux. The use of real-time Linux allows commands to
be timed with millisecond precision, whereas standard Linux may have latencies up
to 100 ms. The neural simulator that runs on the PC will be described in the next
chapter.
There are two RS-485 buses, one for sensors and one for motors. Since the RS-485
bus contains a power line, it was simpler to separate them. The sensor bus runs
at 5V while the motors run at 15V-18V. Both motors and sensors are daisy chained
together to reduce the amount of wiring involved. The sensor and motor buses split at
a hub in the hip section into separate legs, the sensor bus is looped around each leg to
117
Figure 5.5. Left: CAD Drawing of fully assembled leg design. Kevlar straps representing tendons are shown in gray. The straps are attached (inset left) to the motors
by brackets. Force sensors are attached by custom fittings (inset right) buckled into
the straps. All major muscle groups involved in planar walking are modeled. Right:
The fully assembled leg. Inset Left: Close-up of motor with strap attached. The
motor pulls upward on the strap mimicking muscle contraction. Inset Right: Force
sensor assembly. This sensor is buckled into the kevlar strap to measure tension in
the muscle.
118
Figure 5.6. System level block diagram of the robot. The central pattern generator
runs on a PC running Ubuntu RT Linux. Two RS-485 buses are connected via USB2-RS485 converters. There are 16 motors and 28 sensors. Position commands are sent
to the motors to actuate the robot. Load, joint angles, and ground contact sensor
data is transmitted back on the sensor bus.
minimize communication failures. The motor bus runs at 1mbps, while the sensor bus
runs at 1.25 mbps, which is the maximum that the microcontroller can maintain at
its maximum clock speed. The sensor bus is polled using a single broadcast packet, to
which each board responds with a sensor packet delayed by a number of clock cycles
proportional to its ID. Since there are 28 sensors, the spacing between broadcast
commands must be sufficient to allow all 28 sensor boards to respond. This permits
all 28 sensor boards to respond once in approximately 5ms, which translates to a
sampling rate of 20 Hz.
The completed robot (Fig. 5.7) weighs approximately 4.5kg and stands 0.6m tall
with the legs fully extended. Each leg contains only one motor in the calf, three
motors in the thigh, and four in the hip segment. No motors are in the feet. This
makes the feet and lower limbs very light, and the hips relatively heavy. The hip
joint is also placed slightly behind the center of mass of the hip. A consequence
of this mass distribution is that the robot has a tendency to fall forward, causing
119
5.4.2
Fully assembled robot
Figure 5.7. Fully assembled robot. The robot is placed in a cart, which confines
it to the saggital plane. The robot is also supported by bungee cords. The cart
resembles a “baby walker” training device.
the swing leg to rotate passively forward. While the robot’s mass distribution is
not explicitly designed to produce a stable passive walking gait, the incorporation of
passive properties reduces power consumption and allows the CPG to entrain itself to
the underlying system dynamics, as is shown in Chapter 7. Passive oscillations were
measured empirically by elevating the robot and allowing the legs to swing, giving a
resonance frequency for the swing phase of 1 Hz (See Fig. 5.8).
5.5
Experiment: Work Transfer
In order to verify the work transfer capability, we commanded the leg to extend from
a crouching position to a standing on tip-toe position (Figure 5.9), thus lifting its
120
Figure 5.8. Passive swing motion of the leg when perturbations are manually applied.
own weight vertically. This test was intended to focus on the role of the biarticular
muscles, particularly the effect of transferring work from the GM and VL to the
ankle. During this motion, only the RF, GA, SO, VL, and GM were used. At the
start of the test, both the GA and RF were activated, then a delay was added before
activating the GM, VL, and SO, so that the work transfered between joints could be
isolated from any work contributed by the GA and RF. The motors controlling the
GA and RF do not move during the test. We compared this to a test using only
the monoarticular muscles (SO, VL, and GM) to compare work done by each motor
in each case. Because the motors are operating in the position commanded mode,
the torque exerted by the SO and VL should increase to compensate for the lack of
assistance from force transferred through the GA and RF.
We measured the tension in all sensors while the leg was in motion. In the case
where only monoarticular muscles are used, we can see large spikes ( 40N) in the
tension in the SO and VL (Fig.
5.10), showing that these motors are doing the
majority of the work on the knee and ankle. Meanwhile the GM is bearing relatively
little load ( 5N). We verified that the forces in the biarticular muscles were zero, but
have omitted this figure for the sake of space. By comparison, when the biarticular
121
Figure 5.9. Leg Extension Test. The leg pushes off the floor and lifts itself to
standing on tip-toe.
muscles are activated (Figs. 5.11 and 5.12), the GM experiences a large spike ( 55N)
while the SO undergoes little tension. The spikes in the VL and GM during motion
can be compared with the tension in the RF and GA, showing that force is being
transferred. This result demonstrates that using biarticular muscles relieves load
from distal body parts as expected. We also note that when using the monoarticular
motors only, the SO frequently overloaded, showing that a monoarticular system
would require much larger and heavier ankle motors to perform the same task.
Work performed at the ankle was computed by multiplying the differential changes
in angle at each sample by the force at each time step, and summing cumulatively
(Figure 5.13), then verifying that the work conformed to the equations given above.
As expected, the sum of the work done by the GA and SO adds up to close to that as
measured by the sensor at the Achilles tendon. Work performed by the RF and GA
122
Figure 5.10. Tension in monoarticular muscle tendons when only monoarticular
muscles are used. The majority of the force is borne by the SO and VL, while the
GM bears relatively little.
was computed from both the ankle and knee perspective and is shown in Figure 5.14.
Consistent with Equations 2.10 and 2.13, the work in the RF at the knee and hip,
and the GA at the knee and ankle are equal. This result represents the amount of
work being transferred by the biarticular muscles, work is subtracted from the knee
and added to the ankle by the GA. We can see that the work performed by the VL
is virtually consumed by the knee motion. However, the GM is carrying the load for
the remainder of the knee work as well as the ankle, while the SO does very little,
showing that effectively, the work performed by the GM is being transfered all the
way down the leg to the ankle.
As a check, we can compare the work calculations to what should be expected in
extending the leg. The total work done by the robot can be estimated as the work
done in lifting the center of mass as W = mgh. The robot hip (which contains the
most mass) moves approximately 7cm higher, giving a work estimate of 0.96 N-m.
This work value is approximately the same as the amount of work performed at the
123
Figure 5.11. Tension in monoarticular muscle tendons when biarticular muscles are
used. The majority of the force is borne by the GM and VL, consistent with force
being transfered from proximal to distal limb segments.
ankle as shown in Figure 5.13.
5.6
Summary
We have constructed a functioning robot leg based on human muscle architecture.
This leg employs a flexor-extensor architecture as well as biarticular muscles, modeling
all nine major muscle groups in the human leg involved in planar walking. We are
now able to accurately measure the force in all “muscles” in the leg, allowing us to
observe the transfer of power within the limb for the complete leg.
We tested the ability of the leg to transfer force from proximal to distal motors
by commanding it from a crouching to a standing position. During leg extension
this robot leg effectively transfers power from the hip to the ankle. Work performed
by the GM is projected down the leg by way of the GA and RF biarticulate muscles. Most of the load is carried by the GM, the muscle nearest the body. Force
124
Figure 5.12. Forces in biarticular muscles. At the start of the test, these muscles
are placed under tension. After the extension is completed, these muscles are more
relaxed, consistent with their role in power transfer rather than weight support.
sensor measurements show appropriate changes in tension in both biarticulate muscles, demonstrating force being transfered from proximal to distal limb segments. By
comparing forces in the muscles with and without biarticular muscles, we can see that
much more force is needed at the ankle without the biarticular muscles. This behavior demonstrates that biarticular muscles allow greater mass to be concentrated near
the body, and allows the leg to project force outwards by transferring work down the
limb. This result shows that our robot leg design, incorporating biarticular muscles,
exhibits one key characteristic of biological limbs important for biologically realistic
walking.
125
Figure 5.13. Work Performed in the Leg by various motors. When the biarticular
muscles are active, very little work is done by the SO. The VL and GM bear most of
the load.
Figure 5.14. Work transfer performed by RF and GA. As expected WGA,Knee and
WGA,Ankle are approximately equal, and WRF,Knee and WRF,Hip are approximately
equal.
126
Chapter 6
Neural Architecture of the CPG
6.1
Introduction
In this section, we will describe the neural architecture we use to control the robot,
based on concepts we reviewed in Chapter 3. We create two different neural architectures: a one-level model which links the hip motion directly to the CPG, and a
two-level model based on the work of Rybak and McCrea [139,192], which has a separate rhythm generator and pattern formation layer. Both models use a half-center as
the CPG and use phase modulated reflex circuits to modify the lower limb behavior.
These architectures take advantage of lessons learned from the success of reflexive
architectures in robots [35, 67], to address the problem of the coordination of lower
limb motion in multi-segmented limbs, but combine these reflex circuits with a CPG
that stabilizes the step cycle against perturbations.
6.2
Neuron and Synapse Model
The robot is controlled by a spiking neuron CPG. Most other CPGs in robots rely on
Matsuoka oscillators or continuous time recurrent neural networks. These oscillators
output continuous time signals which are usually mapped more or less directly to
joint angles, precluding passive motion. However, our robot is actuated very differently, using flexor-extensor pairs of motors, which can be simultaneously activated or
relaxed, modifying joint compliance. Spiking neurons firing in alternating bursts have
the property of either being in a firing state or an inhibited state, producing a signal
which maps more intuitively to a muscle being flexed or relaxed. Also, additional
excitation or inhibition is easy to inject into the neuron to alter the response of the
muscle at any point in the step cycle, and thus produce passive behavior or stiffening
127
Neuron Model:
vi+1 = vi + (dt/τ )(0.04vi2 + 5vi + 140 − ui + I + If eed )
ui+1 = ui + (dt/τ )a(bvi − ui )
If vi > 30,
vi = c
ui + = d
f iredi = 1
Synapse Model:
ṡi+1 = ṡi + (dt/τṡ )(−ṡi + wsyn f iredi )
si+1 = si + (dt/τs )(−si + ṡi )
mi+1 = mi + α ∗ (−mi + 1/(1 + exp(−si ∗ 10 + 5))
Motor Output:
Table 6.1. Spiking Neuron Model. Six parameters define the neuron model, a, b,
c, d, and I the tonic current, as well as integration time constant τ . Feedback to the
neuron is provided via If eed . The synapse model has two parameters, τs and τṡ . The
motor output is further smoothed using an alpha tracker with learning rate α = 0.05.
Outputs of these functions for a typical burst are shown in Figure 6.1.
of the joint when desired. In human beings, muscles are activated phasically, only
during the time when they are needed [222], and receive burst of phasic sensory feedback at ground contact and extreme angular positions. Thus, in addition to being
more biologically realistic, spiking neurons firing in alternating bursts seem to us to
be more appropriate for our robot’s architecture.
In our robot, one Izhikevich neuron is used to control one muscle in the leg, which
gives us a total of 18 neurons, since we are basing our robot body on the 9-muscle
model of the leg. We used the Izhikevich spiking neuron model [102], described in
Table 6.1. Four parameters control the behavior of the neuron as well as a tonic
input. As shown in Figure 6.1, we have added a synapse model, which filters spikes
so that connections between neurons transmit relatively continuous stimulation, and
a motor neuron for motor outputs, so that the signal sent to the motor is smooth.
Each motor in the robot is controlled by the output of one spiking neuron, by passing
the synapse output of the neuron through a logistic function which limits it to the
range of 0 to 1, and then through a low pass filter. This output is then scaled between
128
Figure 6.1. Example of output from spiking neuron model, synapse, and motor
signal. Membrane voltage (left), synapse output (center), and motor signal (right).
Spikes received by the synapse model are integrated, passed through a logistic function
and filtered to produce a smooth motor output scaled between 0 and 1. This output
is used to compute a position command for the motor.
high and low motor positions to produce a position command.
6.3
Experiment: Entraining Spiking Neural Oscillators
A key issue in designing the neural architecture was to determine the most effective
way to entrain a spiking neuron CPG to environmental feedback. Matsuoka oscillators are generally driven by a smooth continuous oscillation, but biological neurons
generally receive either excitatory or inhibitory signals, which will cause the neuron
to either fire or cease firing depending on its effect on the membrane voltage. Though
the strength of the input can increase the firing rate once it reaches the firing threshold, the behavior of the neuron is very non-linear. In this study, we used an addition
to the tonic input term (I) to inhibit or excite the neuron.
6.3.1
Simulation: Inhibitory, Excitatory, or Mixed Feedback
In a MATLAB simulation, we explored some methods for entraining spiking oscillators using simple fixed feedback signals. Technically this is not “entrainment” since
the feedback signal does not arise from a physical system controlled by the oscillator,
but some qualitative effects on the oscillator network’s behavior can be observed. The
network we used is made up of four neurons with mutually inhibitory connections, as
129
(a) Inhibitory Feedback
(b) Excitatory Feedback
(c) Ipsilateral Feedback
Figure 6.2. Possible schemes for providing feedback to the neurons of the CPG.
Using only inhibitory feedback (a), leg extension inhibits the ipsilateral extensor and
the contralateral flexor. Conversely leg flexion inhibits the ipsilateral flexor and the
contralateral extensor. Using only excitatory feedback (b) has the opposite connectivity. Leg flexion excites the ipsilateral extensor and contralateral flexor. Another
possibility is to interact only with ipsilateral neurons (c). Leg extension inhibits the
extensor neuron and excites the flexor.
130
Figure 6.3. Endogenous CPG for two possible sets of neuron parameters. Two
neurons shown for left (blue, thick, solid) versus right (red, thin, dashed) extensors.
The low frequency network has a=0.005, b=0.25, c=-50, d=0.3, with tonic current
I = 5, and synapse parameters dt = 2, τmem = 10, τs = 10, and τṡ = 50. The high
frequency network has a=0.02, b=0.25, c = -50, d = 1.5 and I =10. and synapse
parameters dt = 2, τmem = 20, τs = 10, and τṡ = 50.
in Figure 6.2. These neurons represent both left and right hip flexors and extensors,
which should oscillate approximately 180 degrees out of phase. A sine wave at some
desired frequency was used for the feedback signal. If the feedback signal crossed
above 0.5 or below -0.5 a constant input current was applied to the appropriate neurons of the network (via the I term), according to one of three possible paradigms:
purely inhibitory feedback, purely excitatory feedback, and purely ipsilateral feedback. When using inhibitory or excitatory feedback only, the signal was applied to
either the ipsilateral-flexor/contralateral-extensor pair, or vice versa, depending on
the sign of the signal. For the ipsilateral paradigm, an excitatory signal was sent to
the extensor and an inhibitory signal to the opposing flexor. Recall that inhibitory
connections between the four neurons should inhibit (or disinhibit) opposing neurons
on the contralateral side.
We examined two sets of parameters which produced endogenous frequencies of
131
Figure 6.4. Examples of various methods for entraining a spiking neural oscillator.
All four neurons are shown so that both flexors and extensors are visible. Colors: Left
extensor (blue, thick, solid), left flexor (pink, thin, dashed), right extensor (red, thick,
solid) and right flexor (green, thin, dashed), feedback signal (black, thin, dotted).
The inhibitory feedback and ipsilateral feedback methods result in poor behavior,
while the excitatory feedback allows the system to adapt to a wide range of signals.
Left: For 0.4 Hz endogenous frequency with applied 1.25 Hz feedback signal. Right:
For 1.25 Hz endogenous frequency and a 0.5 Hz feedback signal. Top: Using only
inhibitory feedback. Middle: Using only excitatory feedback. Bottom: Feedback is
applied to only the ipsilateral side of the body. The inhibitory method does not
successfully entrain, while the ipsilateral method produces undesirable phase offsets.
Only excitatory feedback succeeds in entraining the network.
0.4 and 1.25 Hz. Endogenous behavior of the CPG is shown in Figure 6.3. Both sets
of parameters were selected to produce long oscillating bursts in mutually inhibitory
pairs in the presence of a tonic stimulus, but no activation without a tonic stimulus.
These criteria allow the same parameters to be used in both the CPG (which must
produce oscillating bursts) and the reflexes (which must only activate when excited).
For the excitatory stimulus we used +5, and for the inhibitory stimulus we used
-1. These values were determined by trial and error to be the minimum necessary
to switch on or off the neuron they were applied to. More current did not have a
meaningful effect on the output.
132
6.3.2
Results
Results of this simulation study are shown in Figure 6.4. In this case, a 1 Hz feedback
signal was applied to a CPG with a 0.4 Hz endogenous frequency. The four neurons,
due to the structure of the mutual inhibition network between them, pair off so that
the left flexor and right extensor (and the right flexor and left extensor) act together.
This coordination produces a nice symmetry between the legs. We found that if only
inhibitory feedback is used (which might seem like a reasonable idea), the range of
frequencies we could entrain to was relatively limited. In Fig. 6.4 (left, top), using
only inhibitory feedback results in two activations for each pair of neurons. This
behavior is traceable to the fact that the adaptation levels of the antagonist neurons
have not recovered to the point that the neurons are able to fire. The disinhibition of
a neuron is not sufficient to induce firing, and the system is unable to entrain itself
to the frequency of the feedback signal.
This is not the case with excitatory feedback. When only excitatory feedback is
used (Fig. 6.4 (left, middle)), the target neuron fires and inhibits opposing neurons.
Symmetry between the two sides is maintained, and the resulting frequency matches
the 1Hz input signal.
We also tried using feedback only to the ipsilateral side of the body, by stimulating
the extensor neurons while inhibiting the flexors (Fig. 6.4 (left, bottom)). This
strategy also succeeded in producing entrainment in terms of frequency, however, it
also produced undesirable phase lags and amplitude variations between the neurons.
Using excitatory feedback also allowed the entrainment to frequencies lower than
the endogenous frequency, (Figure 6.4 (right)). This behavior occurred due to the
ability to extend firing by exciting the neuron beyond the end of the endogenous cycle.
Using a CPG with an endogenous frequency of 1 Hz, we applied a 0.5 Hz sinusoid. In
this case, the inhibitory feedback again does not succeed in entraining the system, due
to the endogenous CPG behavior overtaking the feedback frequency, causing multiple
133
activations before the stimulus changes. Also, the ipsilateral feedback model again
proves unsatisfactory, at the inhibitory stimulus to the flexors is slower than the CPG
endogenous behavior. The excitatory feedback model however, results in entrainment
to the 0.5 Hz signal.
There are limits to the range of possible entrained frequencies: 0.5Hz was about
the lower limit for the 1Hz CPG, while the upper limit was around 4Hz. The limits for
the 0.4Hz CPG were 0.1 Hz and 4Hz. These limits depend on the synapse parameters
as well as neuron parameters, since the frequency partially depends on how fast the
synapse responds, however the range is consistently wider for the purely excitatory
feedback method.
6.3.3
Conclusion
In summary, it appears from our exploration that the best method for entraining the
CPG in a spiking neuron system is to use purely excitatory feedback, symmetrically to
both the ipsilateral extensors and contralateral flexors (or vice versa), as this produces
a symmetric result that can be adapted over a wide range of feedback frequencies and
neural parameters. Inhibitory feedback may be useful for reflex interactions in the
lower limb, but does not seem to be useful for entraining the central CPG pattern.
6.4
One-level CPG: A half-center, plus reflexes
The neural architecture for the first system we implemented in the robot is diagrammed in Figure 6.5. The CPG and Reflex neurons in the diagram are simulated
Izhikevich neurons [102]. The gating interneurons are not simulated, but are conceptual representations of how the phase modulation is implemented - using the
property that inhibition has absolute “veto” power over a neuron (similar to the
McCullough-Pitts neuron). The CPG has four central Izhikevich neurons similar to
the network of [123]. These neurons control the hip flexors and extensors directly,
134
Figure 6.5. Neural Architecture of Single Level Simplified CPG model. Four central
neurons control hip flexion and extension. Environmental feedback from ground contact sensors adapt lower leg motion in the knee and ankle. Ground contact (GC) and
load (Load) sensors feed back to knee and hip extension. Anterior extreme position
(AEP) angle sensors extend the knee to prepare for foot touchdown, while posterior
extreme position (PEP) sensors extend the ankle during toe off, and trigger flexion
at the hip and knee. In addition, weak descending connections link the four central
neurons to the knees and ankles. As in the McCullough-Pitts neuron [140], we assume that inhibitory signals are absolute - inhibition of the neuron shuts down firing
regardless of how much excitatory ihnput it receives.
135
and are arranged in reciprocally inhibitory pairs. When isolated these four neurons
will oscillate in pairs, 180 degrees out of phase, as shown in the experiments above.
Flexor and extensor neurons control the knees and ankles. Signals from the four
central CPG neurons can be added to the knees and ankles to couple them to the
four neuron CPG. As the robot moves, the dynamics of the body results in sensory
events, creating a feedback loop between the musculoskeletal system and the nervous
system. Note that the timing of ground contact events will be strongly influenced by
the motion of hips, which are controlled directly by the CPG, and that the CPG’s
rhythm is entrained in turn by sensory feedback. Hence, there are two levels to the
dynamical feedback loop controlling motion, one at the level of the CPG at the hips,
and one at the level of immediate reflexes at the knees and ankles.
This reflex control circuitry was originally adapted from [67] but has been modified
substantially to conform more accurately with what is known from the neurobiology
literature about the role of afferent feedback and reflex circuits during locomotion
[138, 189, 219, 235, 244]. As previously reviewed in Chapter 3, this literature states
that afferent feedback to the system comes largely from Ia, Ib, and group II fibers, and
cutaneous inputs from the feet. We treat the PD position controllers in the motors
as self-contained models of the Group II and Ia afferent, spindle-fiber feedback loops,
so these pathways are not included in our model.
To be consistent with the naming convention used in [67] and [35], we name our
reflexes ground contact (GC), anterior extreme position (AEP), and posterior extreme
position (PEP). The GC signal is produced by taking the greater of the toe or heel
contact sensors. We also add an additional signal for muscle load (Load). It is known
that load at the ankle extensor excites extensors throughout the limb [81], and that
forces add at the ankle due to the effect of the biarticular muscles, therefore we only
need to use one load sensor at the Achilles tendon per leg. Moreover, we are not using
separate unit burst generators for each joint, but rather a single integrated CPG for
the entire limb. Since the limb is regarded as a single unit that can flex, extend, and
136
rotate at the hip, it is not necessary to include multiple load sensors at each joint.
One load sensor at the Achilles tendon is sufficient.
Sensory feedback is scaled to the range of 0 to 1 and then weighted by a factor for
each sensory mode. Thus, for example the feedback to the left extensor and flexor
neurons would look like:
If eedLHE = wic ∗ sRHE + wii ∗ sLHF + wGC ∗ GCl ∗ mLHE + wL ∗ Ll
(6.1)
If eedLHF = wic ∗ sRHF + wii ∗ sLHE + wp ∗ P EPl
(6.2)
Here the left hip extensor is represented by the subscript LHE, and the flexor
by LHF. Similarly, the right hip flexor and extensor are denoted by RHE and RHF
subscripts. Each neuron receives ipsilateral and contralateral inhibition from the
synapse output (si ) of the contralateral and ipsilateral neurons, weighted by wii for
ipsilateral inhibition, and wic for contralateral inhibition. The hip extensor receives
excitatory input from the load sensor in the left leg (Ll ), as well as extra excitation
from the left ground contact sensor (GCl ), modulated by the phase. We use the
motor output mLHE to give us a 0 to 1 signal that determines if the limb is in flexion
or extension. This strategy prevents accidental ground contacts during swing from
initiating extension. Ground contact events during swing excite flexion at the knee
instead. The hip flexor is inhibited by ipsilateral extension and contralateral flexion,
but excited by the PEP sensor on the left leg. Thus the hip does not flex until the hip
extensor is no longer being exited by a load. It is disinhibited when the load in the
limb is reduced. Similar equations can be written symmetrically for the right side.
The equations for the knee and ankle are:
137
If eedLKE = wii ∗ sLKF + wGC ∗ GCl ∗ mLHE + wL ∗ Ll + wa ∗ AEPl + wd ∗ sLHE
(6.3)
If eedLKF = wii ∗ sLKE − wa ∗ AEPl + wp ∗ P EPl ∗ Lr + wd ∗ sLHF
(6.4)
If eedLAE = wii ∗ sLAF + wp ∗ P EPl ∗ GCl + wd ∗ sLHE
(6.5)
If eedLAF = wii ∗ sLAE + wd ∗ sLHF
(6.6)
Here we follow the same convention for subscripts. We treat the biarticulars as
extensors and so they receive the same signals as the extensors on either joint they are
associated with. That is, the GA receives the feedback from both SO (ankle extensor) and VL (knee extensor). By activating the biarticular muscle during extension,
and relaxing them during flexion, we allow the leg to act as a single spring during
stance phase, as in the SLIP template for walking. During flexion, relaxation of the
biarticular muscles allows individual joints to be extended separately, for instance to
extend the knee when the hip is flexed.
In the above equations, we can see that the knee extensor is excited by phasemodulated GC, L, and AEP, as well as descending coupling from the hip. The flexor
receives inhibition from AEP, and is excited during PEP if the contralateral leg is
loaded (knee flexion in late stance). It also receives coupling from the hip flexor.
The ankle is extended during PEP, but only while ground contact is present. Ankle
flexor and extensors are also coupled to the hip joint. All muscles receive inhibitory
feedback from the ipsilateral antagonist.
Parameter values for these equations are wic = wii = 5, wGC = 5, wL = 10,
wAEP = 20, wP EP = 20, and wd is varied experimentally through 1, 3, 5, and 10.
Note that the tonic current to the CPG is I=10, for comparison.
138
Figure 6.6. Reflex action correspondence with phases of the step cycle. The four
phases of the step cycle defined by Phillipson loosely correspond to the effect of AEP,
PEP, ground contact and load reflexes described by [244]. These reflexes are position
and phase modulated so that their activity is confined to a certain portion of the
step cycle that makes them useful in generating the walking cycle. Our reflexes are
designed to correspond to this system, with ground contact reflexes being modulated
by the swing vs. stance phase of the CPG, while toe-off and leg extension are initiated
by hip position afferents.
6.5
Correspondence of Reflexes to Phillipson Phases of the
Step Cycle
We have designed our reflex architecture to correspond to the Phillipson phases of
the step cycle as shown in Figure 6.6. According to Zehr [244], phase and position
dependent reflexes play a role in generating and adapting the response of the limb
in various phases of the step cycle. Zehr outlines four phases: swing, swing-tostance transition, stance, and stance to swing transition, with different reflex effects
which act during these portions of the step cycle. These phases loosely correspond
139
to the four Phillipson phases of the step cycle: S, E1, E2, and E3, respectively, as
discussed previously in Section 2.4. We have designed our reflexes to be modulated
by a combination of phase (swing vs. stance) and hip angular position, dividing the
step cycle into four quarters as shown in Figure 6.6. During the swing to stance
transition, the AEP reflex serves to extend the leg, corresponding to the E1 extension
phase prior to touchdown, and the swing to stance transition described by Zehr.
We activate the three biarticular muscles at this point to prepare the leg to absorb
shock upon touchdown. During the early stance phase, the hip and knee, but not the
ankle, are excited, allowing the leg to support the body weight and absorb energy,
corresponding to the E2 phase. During late stance phase, the PEP reflex extends the
toe, corresponding to the E3 propulsion phase. Once the limb is unloaded, the PEP
reflex resets the CPG and initiates flexion.
6.5.1
Cutaneous afferents: Ground contact sensors
There are a wide variety of cutaneous receptors on the feet which play a multitude
of roles in walking, many of which are phase dependent. During swing phase, except
when approaching foot touchdown, certain cutaneous inputs produce a stumbling
corrective response. This response is highly complex and is not something we attempt
to model here, given that we have only a simple set of ground contact sensors on the
soles of the feet. However, cutaneous afferents on the soles of the feet are known to
have phase modulated effects as well. Stimulation of these nerves will excite extension
during extension phase and flexion during flexion phase [49, 189]. In previous work
with robots [35, 67], ground contact sensors are usually used only to excite extension.
We attempt to mimic the roles played by cutaneous afferents in humans by connecting
these sensors in a phase dependent manner, as shown in Figure 6.5. The GC sensors
are gated in such a way that if they are felt during flexion, the flexors will be excited,
and if felt during extension, the extensors will be excited.
140
6.5.2
Ib afferents: Load sensors
The Ib fibers have an excitatory effect on extensor muscles during locomotion. During
late stance, a reduction in excitation in the Ib extensor fibers triggers the onset of
flexion during the stance to swing transition. During the flexion phase, loading of
the limb will also trigger the early onset of extension. In general, load sensors always
have an excitatory effect on the limb. We implement excitatory feedback pathways
to both the hip and the knee, as well as to the contralateral hip flexor to maintain
the phase offset of the half-center.
6.5.3
Hip position afferents: AEP and PEP
Research also indicates the existence of position sensing afferents in the hip joint,
whether from receptors in the muscles around the hip, or receptors in the joint itself.
The AEP and PEP signals in this model play the role of regulating position dependence of reflexes. Because knee extension is known to precede foot touchdown, we
implement an AEP reflex, which inhibits the knee flexor and excites the extensor.
AEP also reroutes the GC-related reflex pathways to make them excitatory in late
swing. PEP triggers toe off and leg flexion during the stance to swing transition,
also entraining the CPG at the hip. Ankle extension is modulated by GC, so that
flexion will occur once the foot is lifted. In addition, Biarticular muscles are excited
synergistically when either joint they are connected to is extended. (For example, the
RF is excited when either the hip or the knee extends).
6.6
Two-level CPG
The previous section describes a simplified one-level model for CPG, in which the CPG
is directly connected to hip motion. This model differs from more recent models of the
CPG, which have been developed from studies of fictive locomotion [139, 192]. The
141
more recent models involve two levels of pattern generation, a central half-centered
rhythm generator, and a pattern formation network that maps to groups of muscle
synergists, such as flexor-extensor pairs across a joint. This model contains many
interneurons which gate the afferent feedback as well as the mutual inhibition between
CPG neurons. The role of these connections allows for phase and task dependent
changes in behavior. However, many of these phase and task dependent effects are
not necessary to generate walking.
Since our system is designed only for walking and does not need to switch between
tasks such as running or jumping, we eliminate these neurons and simplify the reflex
circuits to a minimal form used for walking. In addition, since we are treating the PD
motor control loops as self-contained models of the muscle-spindle and Ia pathways,
we do not need those either. This structure is shown in Figure 6.7. We have eliminated
all neurons associated with Ia feedback. Also, for the sake of simplicity and clarity,
we assume that a neuron can generate both excitatory and inhibitory feedback. This
is not biologically correct, but allows us to avoid drawing out separate neurons for
inhibitory pathways, thus making the diagram easier to read. The Ib positive feedback
pathway is preserved, but is essentially always open, since this network is distilled
down to only locomotory function.
The above network shows one “unit burst” half-center. However, we wish to control the entire limb, and a single half-center is not sufficient to generate the complex
locomotory pattern. Rybak and McCrea also proposed a multilevel unified architecture for the entire limb as shown earlier in Figure 3.13 (B), though they did not specify
the connectivity of the PF layer or how it would be integrated with reflexes. As above
for the simplified one-level CPG model, we use afferent feedback to adapt the halfcenter pattern. The two levels allows us to control the hip independently from the
rhythm generator and to separate entrainment of the rhythm generator from reflexive
control of the limb. This extended model is shown in Figure 6.8. Here, the central
RG is the four neuron network we used previously. Each side of the RG controls three
142
Figure 6.7. Simplified Version of Two-level architecture described in [192]. In our
robot, we treat the internal PD position control loops of the motors as models of the
effects of Ia and group II afferent feedback from stretch receptors in the muscle. If
we assume that the stretch receptors are part of a muscle length control feedback
loops, we can isolate this from the system. We therefore eliminate these circuits from
the network, since we assume they are self-contained. By removing the Ia feedback
and associated interneurons, we vastly simplify the complexity of the CPG network,
allowing us to focus on load and position dependent sources of afferent feedback. Also,
for the sake of simplicity we will pretend that a neuron can provide both excitatory
and inhibitory signals, which allows us to avoid drawing the inhibitory interneurons
and instead connect the CPG neurons directly. These simplification eliminate clutter
and make it easier to understand the function of the circuit. What is left is the Ib
and cutaneous feedback to the flexor and extensor half centers, the RG level, the PF
level, and the motor neurons.
143
pairs of reciprocally inhibited PF neurons, associated with the hip, knee and ankle
joints. Note that these three reciprocally inhibited half-centers are not really “unit
burst generators”, since they do not receive their own tonic drive. Afferent feedback
is routed to the three sets of neurons according to a reflexive scheme which allows the
joint to be decoupled from the rhythm generator and controlled reflexively instead of
centrally. The two level controller allows reflexive feedback to also modify hip motion.
In this model, most of the equations for feedback to the knee and hip are similar,
with the wd term coupling leg flexors and extensors not to the hip flexor and extensor,
but rather to the rhythm generator. In addition, since the hips are no longer directly
controlled by the RG, they now receive both coupling from the RG and reflexive
feedback from various sources. For instance, ground contact during swing phase also
flexes the hip. The RG layer also receives separate feedback signals for entrainment,
so the weight of the load and gc signals at the hip do not need to be identical. Thus,
If eedLHE = wii ∗ sLHF + wGC ∗ GClef t ∗ mRGLE + wL ∗ Llef t ∗ mRGLE + wd ∗ sRGLE
(6.7)
If eedLHF = wii ∗ sLHE + wGC ∗ GClef t ∗ mRGLF + wd ∗ sRGLF
(6.8)
Here RGLE and RGLF stand for the rhythm generator left extensor and flexor,
respectively. The hip receives signals from the rhythm generator, at the same strength
as the rest of the limb, and receives phase modulated reflex feedback from the GC
and L sensors. The hip neurons maintain their ipsilateral inhibition, but no longer
have contralateral inhibitory connections to the other leg.
In this model, the parameters used in the network were wic = wii = 5, wGC = 10,
wL = 20, wAEP = 20, wP EP = 20. In addition, we introduce separate parameters
for feedback from PEP and L to the RG layer: wLcpg = 15 and wpcpg = 10. We also
set wd = 5 and introduce a new parameter wcontrol which varies between 0 and 1 and
scales the relative strength of the reflexive signals vs. central control signals.
144
Figure 6.8. Neural Architecture of Two Level CPG model with legs. Four central
neurons make up an RG level, which contains flexor and extensor neurons for the left
and right legs. Six PF layer neurons for each leg are assigned to the hip, knee and
ankle flexors and extensors. Environmental feedback from ground contact sensors
adapt lower leg motion. Ground contact (GC) and load (Load) sensors feed back to
knee and hip extension. Anterior extreme position (AEP) angle sensors extend the
knee to prepare for foot touchdown, while posterior extreme position (PEP) sensors
extend the ankle during toe off, and trigger flexion at the hip and knee, as well as
phase resetting of the RG. In addition, weak descending connections link the four
central neurons to the lower limb.
145
6.7
Summary
In this chapter, we described the neural control architecture for our robot. We described the spiking neuron model we use as the basic element of the control network,
as well as a synapse and motor neuron model that can be used to generate motor commands from the output of networks of these neurons. We performed some
simple experiments to discover effective methods to entrain the CPG to afferent feedback signals arising from the interaction of the body and the environment. We then
described two neural architectures, based on a combination of central control and
phase-modulated reflex interactions with the lower limb. One uses a simpler singlelevel architecture while the second separates the CPG into a rhythm generator and
pattern formation layer as in [139,192]. Both of these networks rely on reflexive feedback to modify a simpler half-center CPG structure to adapt the limb’s motion to
the needs of the step cycle. In the Chapter 7, we will describe experiments with the
robot using these two controllers.
146
Chapter 7
Experiments
7.1
Introduction
In this chapter, we will describe the experiments performed with the robot using our
two controllers. The robot is placed in a cart that confines its motion to the saggital
plane and is supported by bungee cords, so it will not completely fall over. Hence,
the cart resembled a baby walker, much like the “Jolly Jumper” in the experiments of
Lungarella [128], but which allows the robot to walk forward and learn the dynamics
of its own body through entrainment to the natural frequency of walking.
7.2
Experimental Setup
The robot is controlled by a Real-Time Linux PC, which simulates the spinal neurons,
receives sensory data from the sensor bus, and commands the motors via a second RS485 bus. The endogenous frequency of the CPG was measured by setting all feedback
weights to 0 and recording the output of the simulated neurons. We experimented
with CPGs with endogenous frequencies of 0.15 Hz, 0.5 Hz, and 1.1 Hz for the onelevel controller. Parameters are shown in Table 7.1. The robot was able to walk using
all three endogenous CPG frequencies.
Below, we show entrainment behavior to the 0.5 Hz CPG. Second, we compare
the behavior of a purely reflexive controller versus the CPG based system, demonstrating that the addition of a CPG stabilizes walking against sensory disturbances
and environmental perturbations. Third, we examine the usefulness of CPG control
of the lower limb versus reflexes. We compare entrainment for various strengths of
wd for all three sets of neural parameters.
147
Parameters
Low Frequency (0.15Hz)
Middle Frequency (0.5 Hz)
High Frequency (1.1 Hz)
a
0.002
0.005
0.02
b
0.25
0.25
0.25
c
-65
-60
-50
d
0.1
0.5
1.4
I
10
10
10
τ
10
10
10
τs
50
50
50
τṡ
50
50
50
Table 7.1. CPG Parameters. Three sets of parameters were used to experiment with
whether the system would entrain using various endogenous frequencies for the fourneuron CPG. Here tau is the time constant associated with the membrane voltage,
taus is the time constant associated with the synapse, and tauṡ is the time constant
associated with the synapse rate of change.
Endogenous/Descending
0.15 Hz
0.5 Hz
1.1 Hz
0
0.2666
0.4373
0.5833
3
0.3333
0.4686
0.6666
5
0.3333
0.5291
0.6866
10
0.4000
0.4706
0.6667
Table 7.2. Entrained step cycle frequency for various selections of endogenous CPG
frequency and wd . Once sufficient foot clearance is obtained, the entrained frequency
lies at a point in between the endogenous frequency of the CPG and the natural
frequency of the system, which is somewhere around 0.5Hz. Very high wd may reduce
walking speed because the foot does not extend near the end of stride.
7.3
One-level Controller Results
All three sets of CPG parameters were capable of producing a stable walking cycle,
with the entrained frequency of the step cycle significantly altered from the endogenous behavior of the CPG. See Table 7.2 for details. Although the knee flexes passively
when the hip begins to swing forward, the amount is not always sufficient for foot
clearance, thus there is a tendency for toe stubbing to occur, reducing the stepping
frequency (the toe stub does not initiate a new step due to the phase modulation
of the GC reflex). However, with a small amount of coupling, there is additional
knee flexion when the hip is flexing, which increases the chances of foot clearance.
Once foot clearance is achieved, the speed roughly levels off. The entrained frequency
tends to differ depending on the endogenous frequency of the CPG. The entrained
frequency is somewhere between the most natural walking frequency of the system
148
and the endogenous CPG behavior. The 0.5 Hz CPG probably comes closest to the
natural walking speed, since the results hover close to 0.5 Hz, varying from 0.46 Hz
to 0.53 Hz. For very high wd the knee tends to be overly coupled to hip flexion, which
may reduce walking speed in some cases. The high frequency CPG results in faster
walking, however it has some undesirable features, such as a tendency to relax the
knee before stance or swing is completed. In nature, the endogenous frequency tends
to be slightly lower than the entrained walking frequency. The ability to entrain to a
lower frequency is also consistent with out earlier simulation experiments, where we
found that the network could be entrained to a lower frequency using only excitatory
feedback, although the range of possible frequencies was smaller on that end.
Snapshots of one step cycle of the robot are shown in Figure 7.1 (top). At (A), the
left leg (yellow) is at toe-off, while the right leg (red) has just initiated stance. PEP
feedback to the CPG triggers hip and knee flexion in the left (yellow) leg, initiating
swing phase. Meanwhile, the load (L) reflex initiates weight support on the right
(red) leg. At (B), the left (yellow) leg is in swing phase while the right (red) is in
stance, fully extended in response to L and GC reflex circuits. At (C), AEP has
inhibited knee flexion, allowing the leg to passively extend, while the right (red) leg
continues to extend. At (D), the robot has fallen forward onto the swing (left, yellow)
leg, while the stance (right, red) leg is at PEP. Toe-off occurs at the right (red) ankle
extensor. As the swing (left, yellow) leg is loaded, which triggers extension responses,
the stance (right, red) leg is unloaded, allowing PEP to begin knee flexion, causing
the legs to switch their swing vs. stance roles. In (E), the right (red) leg is now in
swing phase while the left (yellow) is in stance. In (F), the right (red) knee flexor is
inhibited and the leg extends, while the left (yellow) leg continues to extend. At (G),
the stance (left, yellow) leg is now at toe off. PEP triggers ankle extension.
Using the 0.5 Hz CPG parameters with wd = 5, we can examine the interactions
between the afferent feedback and the behavior of the system more closely. Figure 7.1
shows the responses of the membrane voltages of select neurons in the network within
149
(a) Frame captures of robot walking
(b) Left leg afferent feedback and membrane (c) Right leg afferent feedback and membrane
voltages
voltages
Figure 7.1. One step cycle of the robot. Top: Frame captures from video of the
robot walking. Bottom: GC, L and hip angle signals from the left (yellow) and
right (red) legs. Membrane voltages are shown for hip, knee and ankle flexors and
extensors. A: Toe-off on left leg, triggered by PEP. Foot touchdown on right. B:
Leg extension on right leg triggered by GC and L. Swing on left. C: Passive forward
swing of left leg. Just prior to initiation of PEP reflexes on right. D: Toe-off on right
in response to PEP reflex. Ground contact on left. E: Swing phase on right. Stance
on left, L and GC excite leg extension. F: Right AEP, right knee extends. Left leg
propels body forward. G: Left ankle extension and toe off, preparing for stance to
swing transition. Touchdown on right.
150
the left leg to afferent feedback, with markers indicating corresponding snapshots of
the robot. As above, we can see the effect of various afferent signals on the affected
neurons. The effect of GC and L can be seen by comparison to the hip and knee
extensors (H E and K E), when GC and L rise, in general the hip and knee extensors
are firing, except when the contralateral or ipsilateral neurons are inhibiting them.
Note that at line C the toe appears to stub during the forward swing, but this does
not initiate a step transition. This response may be attributed to both the phase
modulation of GC as well as inhibition of the extensor by other neurons. The knee
flexor is inhibited between B and C on the left leg and between E and F on the right,
allowing for a passive forward knee swing. Active flexion is relatively brief on both
legs. On the left leg at C and on the right leg at F, the AEP activates the knee
extensor. For both legs, ankle extension occurs rapidly after the hip reaches PEP.
The most interesting point is during the stance to swing transition, where our
PEP reflex plays a role in resetting the CPG and initiating toe-off. On both left and
right legs, you can see that the ankle extensor (A E) is excited as the leg reaches its
extreme position. In both cases, the hip flexor (H F) activates as L is reduced, well
after the leg has reached PEP, showing that the loading of the limb prevented flexion
from occurring until the leg was unloaded by transfer of load to the contralateral
limb. This effect is consistent with the literature from cats and humans. The effect
of toe-off triggers a shift of body weight forward over the contralateral limb, which
reduces the load in the extensors in the stance limb, and initiates flexion.
Tension in the muscle straps, as measured by tension sensors (Figure 7.2) also
indicates that passive motion dominates the swing phase of the leg. This dominance
can be seen by computing the work performed by the motor over the angle motion.
In the hip, the GM performs significant work during the stance phase. By contrast,
the IL performs very little, and only during small portions of the swing phase (not the
entire swing phase). Power consumed by the robot was also measured over several
walking cycles, at an average of 36W, while the speed was 0.75 m/s, and the weight
151
Figure 7.2. Motor work performed at the hip. The IL performs very little work,
indicating that most of the forward swing of the leg is passive.
is 4.5 kg. This gives an energy cost of transport of about 1.63.
7.3.1
Disturbance Rejection: Comparison of CPG Driven Walking to
Purely Reflexive Walking
In order to determine how useful the CPG at the hips was in this model, we compared
it with a purely reflexive controller. In the reflexive controller, the mutual inhibition
between the four central neurons was set to zero, and the tonic drive was turned
off. Since the neural parameters were selected to allow spontaneous bursting only
with a tonic drive, none of the neurons can then activate without being stimulated
by one of the reflex circuits. This setup results in a reflexive network in which the
ground contact and Achilles tension reflexes excite the extensors at the hips, and the
ipsilateral knee only. In [67], a purely reflexive controller is used, in which ground
contact sensors are connected fully to both flexors and extensors on both the knee
and hip, on both sides of the body. Therefore, we added additional reflex connections
to the contralateral side of the body for each circuit.
Walking was found to be possible using only reflexes (See Fig. 7.3), but behavior
was inconsistent, was sensitive to initial conditions, and often failed if the robot
152
(a) Left leg afferent feedback and membrane volt- (b) Right leg afferent feedback and membrane
ages
voltages
Figure 7.3. Responses to afferent signals from the purely reflexive controller. GC,
T and hip angle signals from the left(left) and right (right) legs. Flexor and extensor
membrane voltages for hip, knee, and ankle. Comparing afferent signals to muscle
activations, it appears that steps are less regular. The lack of a tonic signal driving the
hip flexor and extensor muscles (since they are relying purely on afferent signaling),
results in less steady activation, especially of flexors. Step length is also less even.
Toe stubbing events on the contralateral toe (see 47s) tends to result in the ipsilateral
extensor being inhibited.
153
stubbed its toe. The legs tended to inhibit each other and the robot would come
to a standstill. The inhibitory network at the hips in the CPG seems to facilitate a
regular stepping rhythm, causing sensor noise and toe stubbing to be ignored during
swing phase, and allowing one leg to win out over the other during the initiation
of walking. The CPG driven model was always able to fall into a regular rhythm
no matter what starting position we used. Figure 7.3 shows the afferent signals
and effect on membrane voltages, as above for the CPG, for the fully interconnected
reflexive network. Here, we can see that the neurons are not behaving as rhythmically.
Sometimes firing appears to be suppressed or initiated unexpectedly, the result of
signals from the contralateral foot.
To test the intuition that the CPG helps stabilize the walking cycle, we devised
two experiments to compare walking ability in the presence of physical or sensory
disturbances. In the first experiment, we disconnected the ground contact sensors
from the right leg. In this case, we observed that the reflexive system tended to walk
with a limp, extending stance duration on the right foot (Figure 7.4 (left)). This
is a result of the fact that ground contact could not be detected on the right, so
the initiation of support transfer to the right was delayed. However, the CPG based
model walked normally, since the oscillatory network at the hip drove the transition
to stance in the absence of ground contact.
In a second test, we attached a weight to the right leg of the robot at the ankle,
to simulate a physical disturbance. In this case, the purely reflexive system resulted
in the robot dragging its right foot (Figure 7.4 (right)), sometimes to the point of
stumbling and pitching forward on both legs. This occurred with both the ispilateralreflexive model and the fully connected reflexive model. By comparison, there was
only a slight deviation of the right leg in the CPG-based model. The robot was able to
walk relatively normally. These results show a strong stabilizing effect on the walking
rhythm as a result of the CPG.
154
Figure 7.4. Phase Plot comparison of walking cycle for reflexive versus CPG driven
walking, under perturbations. Means have been corrected to line up the reflexive and
CPG average hip angle. Left: Sensors on the right leg are disabled so that the robot
cannot detect ground contact events. In this case, stance duration is extended on
the right leg, so that its phase extends further behind the robot before lifting. By
comparison the CPG hip angles remain symmetrical. Right: A weight is attached to
the right leg of the robot. In this case, the right leg lags significantly behind the left
in the reflexive version, but remains relatively even with the left when the CPG was
enabled.
7.3.2
Central vs. Peripheral Control
Another issue of investigation was the degree to which direct CPG control of the
lower limb muscles is necessary. In the one-level model, the lower limb may be
entirely driven by reflexes. However, many CPG based models assume at least some
descending control of the lower limb by the CPG. Therefore, we analyzed the effect
of direct control of the lower limb by allowing descending signals, coupling lower limb
flexors and extensors to hip flexors and extensors.
The robot was activated for various values of wd (0, 5, and 10), which couples the
knees and ankles to the hips. Results showed varying effects of these signals (Figure.
7.5). As the strength of the descending inputs are increased, the robot’s behavior
becomes more dominated by the CPG, reducing the phase difference between the
knee and hip joint. The central control signal has the effect of stabilizing the robot
against perturbations to its stepping rhythm. However, we observed that it also
155
resulted in less effective motion for very high descending signal strength, as the knee
remains flexed at touchdown. It also makes it less similar to human walking, which
will be discussed next.
7.3.3
Comparison to Human Subject data
A comparison of the robot’s joint angles was made to human subject data obtained
from Dr. Darcy Reisman at the University of Delaware, Biomechanics and Movement
Science (Research funded by NIH R01NR010786. Principal Investigator Dr. Stuart
Binder-Macleod). Figure 7.6 shows a typical comparison between two human subjects
and the robot’s walking behavior, for the saggital plane hip, knee and ankle joints.
The data is normalized across a step cycle from heel strike to heel strike, since the cycle
length and frequency will differ between humans and the robot. Hence, the time axis
represents a percent of the step cycle completed. The data also represents averages
across several recorded step cycles for both the robot and the human subjects. Two
different human subjects’ data is presented to show the variation between different
humans’ gaits.
The robot’s gait shows several features in common with the human gait. The timing of flexion in the hip, knee, and ankle occurs in approximately the same sequence
with toe-off occurring just after knee and hip flexion begins. The ankle reaches a peak
of extension slightly delayed from the point where the hip and knee start to flex. At
the beginning of the step cycle, just after heel strike, the robot’s knee bends slightly
during loading. This effect is seen (slightly) in human subject 1, though not subject
2. By comparison the hip angular behavior of the robot is more similar to subject 2.
One significant difference is that the robot’s ankle starts to extend partway through
stance. This robot behavior was a deliberate addition to facilitate contralateral foot
clearance during swing. By lifting the robot slightly on one foot, the other foot was
more likely to clear the ground. Angular excursion magnitudes are also similar from
156
(a) 0.15Hz CPG
(b) 0.5Hz CPG
(c) 1.1 Hz CPG
Figure 7.5. Synapse output for the knee extensor versus the hip angle, for various
endogenous CPG frequencies and values of wd . (A) For the 0.15Hz CPG. (B) for
the 0.5 Hz CPG. (C) For the 1.1 Hz CPG. As the strength of the coupling to the
half-center is increased, knee flexion is increasingly synchronized to the hip. It also
becomes more regular, showing more uniform activation across the step cycle. For
the faster CPGs, weak or no coupling to the CPG tends to result in less continuous
knee extension. Central control therefore adds robustness to the system.
157
(a) Human Subject 1
(b) Human Subject 2
(c) Robot - 0.5Hz CPG
Figure 7.6. Comparison between robot saggital plan angles and human subject
angles during normal walking. These figures show averages, with standard deviation,
across one step cycle from heel strike to heel strike. Hip (green, solid), knee (red,
dashed) and ankle (blue, dotted). For the single level controller with descending
signals from the CPG to lower limb of I=5, and for the 0.5Hz CPG. There is a
reasonable match between many of the features of the step cycle, particularly the
hip and knee. The ankle’s toe-off seems less shifted over to the right, however. This
characteristic could be a result of deliberate activation of the ankle extensors earlier
in the step cycle to increase the chances of contralateral foot clearance. Error bars
are included to show the standard deviation of the data over many step cycles. The
robot exhibits more variation, however, we also had fewer step cycles for comparison.
158
the robot and the human subjects.
A more detailed comparison can be made of the behavior of the robot as the
strength of central control is varied. We vary the parameter wd through 4 settings,
0, 3, 5, and 10, and observe how the angular motion changes, as shown in Figure 7.7.
For a strong degree of central control, the joints moved synchronously. As this signal
is decreased, reflexes dominate. Consequently the curves change. For example, the
knee is extended prior to heel strike, and the peak ankle extension occurs later in the
step cycle. Although the hip is controlled by the CPG directly, some change in hip
behavior is also observable, as the timing of heel strike and toe-off, as well as loads
in the limb, affects the regulation of the CPG.
A side by side comparison of the robot joint angles to the human subjects is shown
in Figures 7.8 and 7.9. Here it is easier to see how weak to zero central control of the
lower limb results in behavior more similar to human walking. In these results, the
left toe is stubbing, since a lack of central control reduces the amount of knee flexion
during swing. However, the right leg is not stubbing, the ankle behavior on the right
leg matches the human subject behavior much more closely. We can also see that
weak coupling (wd = 3) on the left leg makes the knee behavior more closely match
a human like behavior.
7.3.4
Passive knee flexion
In an experiment to determine what amount of knee flexion might be achieved passively by the robot, we shut off all reflexive and descending signals to the knee flexors
and allowed the robot to walk using increased ankle extension to reduce the chances
of toe stubbing. The results are shown in Figure 7.10. Surprisingly, there is a fairly
good match between the robot’s knee behavior and the human subject data. This
correlation suggests that knee flexion during swing is largely passive, and also that
our robot’s dynamics do a fairly good job modeling the dynamics of the human body.
159
Figure 7.7. Change in angular behavior with respect to degree of central control
coupling, for robot hip (right), knee (middle) and ankle (left) angles. Plots are arranged in order of decreasing values for wd - 10, 5, 3, and 0, from top to bottom. When
greater central control is exerted (top row), the joints move more synchronously. This
coordination increases the chance of foot clearance, since the knee is more flexed during swing phase, but makes the behavior less human like. By comparison when the
lower limb is purely reflexive (bottom row), the behavior is more like a human’s but
the chance of toe stubbing is much greater.
160
Figure 7.8. Comparison of robot left leg joint angles to human for various strengths
of coupling to the CPG. Joint angles behave more like a humans when reflexive control
of the lower limb is greater.
161
Figure 7.9. Comparison of robot right leg joint angles to human for various strengths
of coupling to the CPG. Joint angles behave more like a humans when reflexive control
of the lower limb is greater.
162
Figure 7.10. Comparison of leg angles to human subject data when all knee flexion
signals are removed, for the left (left) and right (right) legs. Hip (green, solid), knee
(red, dashed) and ankle (blue, dotted).In this case the knee flexion is purely a result
of the biomechanics of the limb during swing phase. This response matches the most
closely with the human subject data, suggesting that in the human knee, flexion
is largely passive rather than active. Ankle extension was increased to avoid toe
stubbing.
We use extra contralateral ankle extension to help the robot’s toe clear during swing,
however, which doesn’t match human data. If humans are not lifting the contralateral toe, and are not actively flexing the knee, how is foot clearance achieved? One
alternative possibility is that humans may be lifting the entire leg during swing, by
tilting the pelvis, rather than extending the toe or flexing the knee.
7.4
Two-level Controller Results
Since the two-level controller separates the rhythm generator from direct control of
the lower limb, it invites a more explicit comparison between central and peripheral
control. To do this, we devised a single parameter wcentral between 0 and 1, to
represent the strength of the central control component. All weights for the signals
extending from the RG to the PF layer were then scaled by this amount. Conversely,
using parameters from the purely reflexive controller, we set wperipheral = 1 − wcentral ,
163
so that the parameter acts as a slider between purely central control with all joints
chained to a single half center, and a purely reflexive controller, with no half-center
component at all. We then varied this value from 1 to 0 starting with pure central
control to compare the behavior of the robot. Note that afferent feedback which
entrains the RG layer is not affected by this, the strength of afferent feedback to the
rhythm generator remains constant so that entrainment of the walking frequency is
not affected.
Figure 7.11 shows the evolution of the behavior of the robot as it shifts from
a purely centrally controlled paradigm to a purely reflexive (peripheral) controller.
When all joints are coupled to the half-center, the joints alternate from flexion to
extension across the entire leg simultaneously. This coupling results in a walking
pattern in which one leg is lifted with the knee bent, while the other is straightened.
The robot then falls forward onto the flexed leg, still in a bent position. At this point,
the stance to swing transition occurs, and the legs reverse positions.
As the central control component is reduced and reflexes are allowed to take over,
the behavior of the robot changes. The robot extends its leg prior to heel strike, as a
result of the AEP reflex. This behavior allows the robot to remain in a more upright
posture, since it does not have as far to fall to catch itself on the forward leg. Figure
7.12 shows the changes in the angular behavior of the joints as the reflexes take over.
As peripheral control is introduced, the joint angular behavior becomes more complex.
Initially, hip, knee and ankle all have a similar “square” shape to their response. As
the reflexes are introduced, the knee extends before heel strike and the peak ankle
extension shifts to a phasic offset after hip and knee flexion. The hip angular behavior
also becomes less squarish. Since the hip neuron is not driven directly by the CPG,
it receives other sources of feedback, such as load and ground contact, which alters
the hip’s behavior somewhat from the pure CPG controlled behavior seen in the onelevel controller. Figures 7.13 and 7.14 show a direct comparison between the human
subject data and the robot’s behaviorfor the left and right legs. Here we can again
164
(a) Pure Central Control
(b) Central Control Component 0.8
(c) Central Control Component 0.6
(d) Central Control Component 0.4
(e) Central Control Component 0.2
(f) Pure Peripheral Control
Figure 7.11. Snapshots sequences of the robot walking as the central control component is decreased.
165
Figure 7.12. Ankle, Knee and Hip angles as the degree of central control is reduced
and peripheral control is increased. Left leg (blue solid) and right leg (red dotted).
When hip, knee, and ankles are coupled to the rhythm generator entirely, all three
joints act in phase, in roughly square waves. As the central control is reduced and
peripheral (reflexive) control is increased, the ankle extension peak shifts later in the
step cycle. The knee also begins to extend shortly after flexion and before heel strike.
See Figure 7.11 for comparison. The hip also appears to gradually shift from being
more square to being more sinusoidal, more similar to the human data.
166
see that as the central control component is decreased the pattern shifts and becomes
more humanlike.
Comparison to human subject data To make it easier to see the increasing similarity
of the robot’s behavior as reflexive control becomes more dominant, we took the MSE
of the error between the robot’s behavior and the human subjects for both subjects
and both legs. We attempted to fit the robot’s behavior as near as possible using the
timing of the hip extension and flexion peaks, by shifting all curves using the same
time and angular offsets (since the zero position is not identical for the robot and
human or for both legs). Once the best possible match was achieved, we computed
the MSE for each set of angles for the robot against both human subjects. Figure
7.15 shows the MSE versus wperipheral for both legs and both subjects. It is now more
clear that as the reflexes have a greater contribution to controlling the limb behavior,
that the match between the limb behavior and the human subjects improves. The
best match occurs when the central control component is only 0.2, since wd = 5 this
translates to adding I=1 to each Izhikevich neuron. However, pure peripheral control
matched the human subjects less well. We note that it also resulted in a less effective
and less stable step cycle.
One difference between the one and two-level controllers is that in the one level
version the hips are always coupled to the CPG, by definition, while the lower limb
may or may not be. By comparison, the two-level controller uses the same coupling
strength for the hip, knee, and ankle joints. Results from the one-level controller show
more stable walking than the two-level version for low values of wcentral however. A
likely possibility is that different degrees of central control are necessary for the hip
versus the knee and ankle joints. We got a very good fit will low coupling of the lower
limb, but strong coupling to the hip earlier. It is probably that the degree of central
control of the lower limb is different from that of the hip, and that this is a more
effective walking strategy.
167
Figure 7.13. Left Leg comparison with human subject data for various degrees of
central vs. peripheral control, for the two-level controller, with decreasing degrees of
central control shown from red to blue in the spectrum. The two sets of human data
are shown by thick black dashed and dotted lines for comparison. As in Figure 7.12,
as central control signal (red) is decreased and peripheral control (blue) increases,
the step cycle behaves more like the human subject data, except for pure peripheral
control. When no central control component is present (purple) the step cycle is also
much less stable. However, a small amount of central control with mostly reflexive
control (blue, cyan) produces the nearest match to human data.
168
Figure 7.14. Right leg comparison with human subject data for various degrees of
central vs. peripheral control, for the two-level controller, with decvreasing degrees of
central control shown from red to blue in the spectrum. The two sets of human data
are shown by thick black dashed and dotted lines for comparison. As in Figure 7.12,
as central control signal (red) is decreased and peripheral control (blue) increases,
the step cycle behaves more like the human subject data, except for pure peripheral
control. When no central control component is present (purple) the step cycle is also
much less stable. However, a small amount of central control with mostly reflexive
control (blue, cyan) produces the nearest match to human data.
169
Figure 7.15. Comparison of MSE between robot and human subjects for various
degrees of central vs. peripheral control. Pure central control on the left to pure
peripheral control on the right. Curves for the robot were matched against human
subject data, the MSE between the robot and the human data over the step cycle
was then taken. Here, it is clear that a small, but non-zero amount of central control
produces the lowest MSE over the step cycle for this controller.
7.5
Summary
In this chapter, we sdescribed the experiments and results performed with the robot
using the two different version of the neural controller. First, we showed how the robot
is able to entrain effectively using the single level CPG. We showed that the robot’s
walking frequency is significantly modified from the endogenous frequency, indicating
that the system has entrained to a new limit cycle behavior. We examined the various
sources of afferent feedback, and how they contribute to the control of the walking
pattern. In particular, we showed that the entrainment of the CPG occurs at hip
extension, as the PEP reflex results in hip flexion, but only once the leg is unloaded,
similar to what is known from the biology literature. Secondly, we showed that the
CPG at the hip level has advantages over purely reflexive control, in that it is more
able to withstand sensory and environmental perturbations, by turning off the foot
sensor or weighting the leg. In both instances, the walking pattern is more resistant
170
to perturbations when the hips are driven by the CPG. Thirdly, we compared the
behavior of the robot to human subject data, showing that the robot’s gait is similar
to the human gait in several respects. One significant difference is that we use ankle
extension to enable contralateral foot clearance, which humans do not use. We also
showed that the robot can flex its knee passively, producing a motion that matches
human behavior relatively closely. Finally, we used the two-level controller to show
a more direct comparison between the effect of central vs. peripheral control. This
comparison shows that the best match to human subject data is obtained using only
a small amount of central control, although some degree of central influence by the
half-center is needed to maintain the stability of the walking pattern.
171
Chapter 8
Discussion
This research represents a complete, if simplified, neurorobotic model of the lower
human body. The robot physically models the biomechanics of the human body, using
a muscle like architecture that models mammalian leg muscles. We have modeled
all of the key afferent feedback signals, and constructed a neural controller that uses
realistic spiking neurons to model the behavior of neurons in the spinal column. These
elements are all key to the process of walking from a dynamical systems perspective.
8.1
Biomechanics of the robot
The robot includes models of the biarticular muscles. These muscles couple different
joint motions together, allowing the robot’s leg to function as a single unit, extending
or flexing all joints in synchrony. We showed that the use of biarticulars allowed
forces to combine at the Achilles tendon, increasing the thrust at the toe through the
combined action of the GM, VL, and SO. With the addition of springs for energy
storage at the joints, this unified action allows the leg to function like a single spring.
We can extend or flex the leg, or, optionally, decouple individual joints by relaxing
the biarticulars. In our robot, we activate the biarticulars during extension phases,
and deactivate them during flexion. This strategy allows the knee to be extended
forward during late swing when the hip is flexed. Activation of the biarticulars
in late swing also prepares the limb for shock absorption upon landing since the
biarticulars are used to transfer energy up the limb. This architecture allows the robot
to behave according to the spring-loaded inverted pendulum template for locomotion,
and simplifies control.
.
172
8.2
The role of reflexes in locomotion
The role of reflexes versus central pattern generator has been debated since the early
20th century, beginning with Brown, who proposed central control of locomotion [22],
and Sherrington [200], who proposed that locomotion was produced by cascading
reflex effects. In the late 70s, Lundberg proposed that locomotion might be produced
by a simple-half center upon which reflexes worked to modify the pattern [127]. While
it is now clear that the spinal column is capable of producing complex locomotory
outputs without afferent feedback [80], the structure of the locomotory CPG remains
a mystery [233]. At the same time, reflexive control has been widely used to control
robots [35, 67], and is able to achieve coordination of lower limb motion including
by augmentation of a CPG [110], though the coordination of multisegmented limbs
remains a problem for roboticists.
We developed two models for how a neural architecture might be constructed that
combine feedforward central control with reflexive feedback from peripheral sensors,
based on the biological literature. In contradiction to the “unit burst generator”
model, a single complex CPG controls the entire limb, allowing for unified flexion or
extension through central control, or modification of this pattern through reflexive
feedback. Both networks are composed of a half-centered oscillator which is entrained
by afferent feedback, and a set of phase and position modulated reflexes that modify
the behavior of individual joints, according to a combination of rules derived from the
robotics and biological literature. The first model uses a single level of control, with
a CPG that is directly connected to the hip muscles, and lower limb neurons that are
largely reflexive but can be coupled to the CPG. The second model is a multi-level
CPG based on the work of Rybak and McCrea [192], which has a separate rhythm
generator (RG) and pattern formation (PF) layer. The separation of the RG and PF
layers allows for explicit modulation of the degree of central vs reflexive control for
all muscles, which we achieve using a single parameter.
173
We are able to produce human-like walking motion with this relatively simple
network, modeling only the known afferent feedback circuits and a half-center based
at the hip. Although there may be no central control of the lower limb, the motion
of the hips dynamically drives lower limb motion by triggering position-dependent
reflex circuits and influencing the timing of ground contact events. We also observed
that, as in observations of cats, flexion of the stance limb does not actually occur
until the stance limb is unloaded, as indicated by a reduction in load in the Achilles
tendon. This effect matches that observed in cats, where it is well established that
the swing phase is initiated by extension of the hip and unloading of the limb. We
achieve this result not by explicitly encoding an afferent that senses limb unloading,
but rather by relying on the interaction of a few circuits. During late stance, the body
falls forward onto the stance leg, assisted by the propulsive effect of ankle extension,
causing a shift of load to the contralateral limb. Consequently, load in the stance
limb is reduced. At this point, the flexion reflex overcomes the excitatory effect of the
Ib afferent and initiates the flexion phase at the CPG. The swing (now stance) limb
meanwhile detects ground contact and load and consequently excites the extension
of the knee and the hip. These effects are similar in both versions of the controller.
8.3
Central vs. Peripheral Control
One issue we explored in our experiments was the extent of central control, versus
reflexive. Reflexive control can also be termed peripheral, since these signals arise
from the peripheral nervous system, while CPG control signals are developed centrally. We examined both the extent to which central control was needed to stabilize
walking, and the degree to which walking patterns matched human-like walking when
more closely coupled to a half-center based RG (or CPG). We found that walking was
significantly stabilized against both sensory and physical perturbations by incorporating CPG control at the hip. However, we also found that using mostly reflexive
174
control caused the robot to behave in the most human like fashion. In the two-level
controller, where we were able to explicitly vary central vs. peripheral control of the
entire limb simultaneously, the most human-like motion was obtained using a very
small central coupling signal (I=1). In the one-level version, where the hip motion
was inherently tied to the CPG, we found that purely reflexive control was best, so
long as foot clearance was made possible. We also found that using very little active
knee flexion, thereby relying on the dynamics of the limb to flex the knee slightly
during swing, produced results similar to human walking. While most researchers
assume a significant degree of descending control of the lower limb by the CPG, these
results show that only very weak central control is necessary. Much of the appropriate
limb motion can be peripherally generated using phase modulated reflex circuits.
8.4
Load detection and positive force feedback in locomotion
Our robot has an unusual architecture in that it relies on muscle-like actuators. This
type of architecture isn’t unheard of, but is relatively rare. One element of that
architecture that is unique is that we have incorporated load sensors into the muscles
as a source of sensory feedback to be used in control. We are not aware of any other
robots that incorporate this feature. Based on the neurobiology literature, it now
seems that load detection in the muscles, via Golgi tendon organs, plays a central
role in the locomotory cycle [51], a reality that seems to be largely overlooked in the
robotics literature. In biological systems, the unloading of the limb appears to be the
dominant trigger for the initiation of swing phase, and the consequent entrainment
of the rhythm generator [52]. This concept has also been incorporated in the use
of neuroprostheses for amputees [179]. A simple rule for locomotion, with is true
across many species from insects to cats, is that a limb cannot be lifted until it is
unloaded. When stated in this fashion it is obvious why this must be the case. Lifting
of a loaded limb would immediately result in instability. The rule is accomplished in
175
mammalian limbs by the opening of positive force feedback circuits from Ib afferents to
the extensor muscles during locomotion. These circuits are powerful, reflexive below
the level of the CPG, and contribute directly to generating a significant portion of
the muscle activation in the limb [46].
While many robots do incorporate ground contact sensors, they are often binary
in nature, as in the reflexive robot of [67]. Foot pressure sensors can be difficult
to implement with a high degree of accuracy and reliability, but have been used
to maintain balance using ZMP control. Moreover, positive force feedback is not
often considered by engineers because positive feedback tends to result in instability.
However, Prochazka showed that this does not happen due to some of the intrinsic
properties of the muscles [181]. Positive force feedback has also been shown to be of
potential use in running gaits [69], though does not seem to be used in robots so far. It
may also be useful in generating muscle compliance [68] that stabilizes passive walking
cycles. In essence, positive force feedback may help lend a “spring-like” behavior to
the limb, allowing locomotion to behave according to the SLIP model discussed in
Chapter 2.
Given these new understandings of the way load detection and force feedback
work during locomotion in animals, it is important to communicate this to roboticists
developing walking robots. Very little research in walking robots focuses on the role
of load detection. In our robot, the addition of the load sensors greatly improved the
stability of the walking cycle. Even with nothing but load sensors, we were able to
produce a walking pattern using positive force feedback, although it was not very well
coordinated. We believe that this is an important area of future research in legged
locomotion that may turn out to be key to developing stable, human-like dynamical
walking in a robot. Other, less biologically-based, robots may also benefit from this
insight. For instance, robots based on conventional architectures may be able to
detect load by measuring power consumption in a joint. This principle applies across
many animal species and may be generally applicable to a variety of walking robots.
176
8.5
Locomotory control as a series of nested dynamical loops
It is well established that reflex networks do not operate solely through rhythm generating networks, but more directly through connections that synapse directly back
to the motoneurons, and through interneurons modulated by descending signals form
the CPG [189], and possibly by position and ground contact signals [77], indicating
that reflexes are phase and position modulated. This phase modulation of reflexes
allows them to play important roles in generating the step cycle [244]. A significant
amount of the muscle activation during locomotion is known to be generated through
reflex effects [45, 46, 240].
We therefore imagine a paradigm of motor control as a series of nested loops
(Figure 8.1), with the highest brain centers represented by the most deeply nested
loops. At the most basic level, keeping in mind that body properties have evolved
to enable such behavior, passive dynamic walking represents a limit cycle produced
by the interaction of the mechanics of the body and the environment (Figure 8.1
(a)). We can build upon this by comparing it to the brain-body-environment model
advanced by Beer (Figure 8.1 (b)). In this case, while the body and environment
interact, the brain regulates this via the body, and receives signals back from the
environment via senses in the body, resulting in a higher level loop. The CPG model
of motor control advanced by Taga and other researchers (Figure 8.1 (c)) introduces
the separation between the spinal CPG and higher centers in the control of movement,
with the CPG more actively regulating a largely passive walking cycle, with simpler
descending signals altering the properties of the CPG and providing a tonic drive.
Hence, the CPG represents a layer between the brain and the body-environment
interaction of passive dynamic walking. With these models organized in this way, we
can expand on this concept by adding a new level of interaction - a new loop involving
reflex interactions. In our model (Figure 8.1 (d)), at the lowest level internal to the
body, reflex interactions directly mediate between environmental stimuli and musculo-
177
(a) Passive dynamic walking
(b) Model of dynamical cognition
(c) CPG based walking
(d) Nested loops model
Figure 8.1. Conceptual models of dynamical walking as a set of nested loops,
each modulated and manipulated by descending signals from more deeply nested
structures. (a) Passive dynamic walking as a pure limit cycle involving the body and
environment. (b) Brain-body-environment model of dynamical cognition. The brain
manipulates the body, producing a larger dynamical loop uniting the environment and
the mind. (c) CPG model of walking. Higher brain centers control the CPG, which
acts to stabilize and inject energy into the passive limit cycle. (d) Reflex/CPG model.
The reflex system interacts more immediately with the passive walking cycle, while
the CPG modulates reflex circuits, and controls walking at the hips. Feedback from
the environment excites reflexes and entrains the CPG in separate loops. Descending
signals from higher centers in turn can modify CPG and reflex circuits in a taskdependent manner.
178
skeletal responses. One step higher, the CPG receives afferent feedback from the
body, and modulate reflex interactions. Rather than directly driving all of the muscle
motion in the lower limb (only the hip oscillation is directly driven by the CPG),
the CPG largely modulates the reflex signals so that the reflexes act to generate
most of the muscle activation, though under CPG guidance. Similarly, signals from
higher brain centers modulate and reorganize CPG circuits, as well as reflex circuits,
in a task-dependent manner. Accordingly, higher centers can also hijack individual
muscle motions by activating inhibitory interneurons that shutdown both CPG and
reflex circuits, and can bring more far flung sensory systems such as the vestibular
system into the dynamical loop. And so forth, until we get higher brain centers able
to modulate individual circuits to make refined specific motions, and incorporate
complex senses such as vision.
Although we do not suggest that our particular set of reflex circuits represents
the final word on what specific interactions exist in mammals, or are most useful
in a walking robot, we believe this method for conceptualizing the role of reflex
behavior may be useful to both roboticists interested in building biologically-inspired
robots, and neuroscientists interested in studying the operation of the neural circuitry
involved in human walking. By conceiving of reflex circuits as operating within the
influence of the spinal CPG, the complex circuitry involved in locomotion can be
reduced to simpler components. In our model, we have a set of relatively simple
CPG, and a relatively simple set of reflex circuits which produce a complex behavior
(dynamical walking), due to the various levels of interaction of each of these circuits
with the environment, and with each other.
8.6
Issue: The generation of complex central control signals
As discussed in Chapter 3, the connectivity between the afferent feedback signals,
CPG, and reflexes in humans is unknown. It is now understood that patterns more
179
complex than a simple half-center can be generated centrally in the absence of any
afferent feedback [80], yet how these patterns are generated is a mystery. On the other
hand, as this and other research [67] shows, the complex patterns needed to generate
the step cycle can be generated peripherally, using reflex circuits in combination with
a simple half-center. This relationship was also shown in [68], where mostly stretch
reflexes were used to generate locomotion in simulation. The use of reflexive circuits
alone is widespread in insect-like robots [35].
This observation leaves us to explain how, if the walking cycle can potentially
be generated using a combination of afferent feedback and a simple half center, the
deafferented cat nevertheless exhibits a complex walking cycle similar to that of the
intact animal. We hypothesize that the more complex walking pattern may be learned
or developed during childhood. While studies of deafferented cats have shown a
complex walking pattern, these studies were performed on adult cats or kittens that
had begun to develop walking ability. Studies of human infants show that babies
tend to move their legs in synchronous alternating motions [56, 215] of flexion and
extension, such as we experience when all muscles are coupled to the half-center
in our robot. Human infants are also capable of surprisingly adult-like stepping if
they are stimulated by appropriate afferent feedback when placed on a treadmill
[56, 215, 241]. The treadmill has the effect of driving the mechanics of the body into
a walking pattern, thus generating feedback at the appropriate times during the step
cycle. Therefore, it seems that human infants are also capable of generating an adultlike walking pattern when stimulated with appropriate afferent feedback, but exhibit
simple half-center-like behavior otherwise.
We hypothesize that a learning network could exist between the central rhythm
generator and the motor neuron layer, and that this learning network could be “bootstrapped” by interactions with the mechanics of the body, sensory feedback and reflex
circuits. The reflex signals could act not only to generate the stepping pattern during
learning, but also as a “teacher signal” to the learning network. That is, while the
180
Figure 8.2. Hypothetical Learning Network. At birth, the CPG begins as a halfcenter rhythm generator producing alternating patterns of synchronized flexion and
extension. When stimulated by afferent feedback, reflex circuits, modulated by the
half-center, generate appropriate motor responses. As in babies and SCI patients in
treadmill training, the treadmill induces motion of the body that produces feedback
of appropriate timing, stimulating reflex responses that assist the walking cycle at
the right moments. We hypothesize that a learning network develops after birth
which learns to reproduce these motor responses centrally, in connection with the
phase of the rhythm generator. The reflex system, in combination with the dynamics
of the musculo-skeletal system, acts to provide appropriate feedback, which acts as
a “teacher signal” to the network - providing it with a set of appropriate sensory
signals and motor responses, which can then be mapped to the phase of the rhythm
generator. The learning network can then reproduce the signal centrally and is less
reliant on peripheral control. This system can also be regarded as an inverse model
of the body dynamics.
181
correct walking pattern is being generated by peripheral stimulation, the learning network learns to map those sensory signals to the appropriate motor responses, through
Hebbian learning [83]. The learned pattern can then be used when those peripheral
signals are later absent or unreliable. Thus, when afferent feedback is removed, such
as in [80], the learning network would still exhibit the learned pattern, which would
resemble the motor output needed for walking.
Such a central learning network would be advantageous to an animal, by being
able to generate the correct pattern centrally as well as peripherally, thereby making the animal more adaptive. When sensory information becomes unreliable, it
may rely more upon complex centrally generated signals, rather than the half-center,
which interferes with phasic relations between the joints as we saw in experiments.
The animal can then modulate the strength of central versus peripheral control (or
feedback vs. feedforward control), depending on conditions. This concept has been
proposed by other researchers such as Kuo [116], who points out that under perfect
sensory conditions, there is no advantage to having any feedforward control, but that
when feedback become unreliable, it is advantageous to be able to make a forward
prediction, as in a state estimator. Kuo argues that the CPG acts as a state estimator
which contains an internal model of the body dynamics. By mimicking the effect of
reflex circuits, the learning network could also act as an inverse model of the body dynamics. One such brain structure is already thought to exist: the cerebellum [1, 133]
is often thought to be an inverse model of the body dynamics [239].
The ability to learn the walking pattern based on afferent feedback is also supported by studies in adult humans with spinal cord injuries. Adults with spinal cord
injuries sometimes recover locomotory function after being stimulated by being placed
on a treadmill (similar to the effect observed in infant studies) [219]. Recent research
also shows that CPG neurons exhibit synaptic plasticity and that properties of the
CPG can be learned by training [150]. Therefore, we conclude that there is considerable support in the literature for our hypothesis that the complex CPG pattern
182
observed in the deafferentation studies is learned, and that the CPG at birth more
closely resembles the simple half-center model of Brown [22]. This hypothesis could
be tested by performing similar deafferentations on kittens (although we note that
the dynamics of the limbs in utero might also begin the training process before birth).
8.7
Conclusion
We began this paper by framing the issue of locomotion within the context of a larger
dynamical approach to cognition. According to this approach, the mind is situated,
embodied, and dynamical in nature. It exists within the context of a continuous
feedback loop between the brain, the body, and the environment. This approach
is a departure from conventional approaches to AI and robotics, which neglect the
physical embodiment of the mind, both in terms of the neural substrate upon which
information is encoded, and by neglecting the need for a physical world for the brain
to interact with.
Over the course of our research we discovered that locomotion could be organized
as a series of nested dynamical loops, with the biomechanics of the body, the reflex
circuits, and the central pattern generator existing as separate layers of interaction.
The concept of a layered architecture has also been used, in a much different form,
in the work of Rodney Brooks [21]. In our architecture, the lowest level is composed
of simple reflex circuits, modulated by the CPG or by higher level circuits. The
CPG behavior can also in turn be modified by changes in descending tonic inputs
[189] and higher level modulatory signals. This layered architecture has both an
intuitive evolutionary explanation and provides an intuitive means of understanding
the organization of the nervous system. Older evolutionary structures, such as simple
reflex circuits in the snail, exist as the simplest level of organization. As the brain
becomes more complex, newer layers are added which modulate older ones. A few
modulatory signals alter the effect of many reflex circuits, so that individual control
183
signals do not need to be sent to individual motoneurons. This layered structure thus
acts to compress information and simplify control.
This perspective also provides a useful way of thinking about the mind, through
the perspective of the dynamic theory of cognition. As each level of the series of
nested dynamical loops interacts with the environment, the mind’s business is to
modulate these layers to change the dynamics of their behavior, and manipulate the
continuous feedback loop they are engaged in. Rather than engaging directly with
the world, we are engaged in a constant process of manipulating layers of interaction
from biomechanics, to reflex circuitry, to the CPG, along with numerous higher level
and more complex structures. However, this organizing principle of nested dynamical
loops, with each layer modulating the one below it, could provide a useful paradigm
for developing a newer and more viable artificial intelligence based on a situated,
embodied, and dynamical view of cognition.
184
Chapter 9
Sensory Reweighting
Most current humanoid robots rely primarily on proprioceptive sources such as foot
pressure sensors to balance, by maintaining the center of pressure (CoP) within the
support perimeter of the robot’s feet [227]. A typical example of this is the Honda
Asimo [86, 194]. However, this technique requires the floor to remain level, stiff and
not in motion, conditions which are not always available in the real world. As a
result, these robots do not adapt well to complex environments. By comparison, human balance involves sensory feedback from multiple channels, which are combined
to adapt dynamically depending on environmental conditions. In particular, humans
use vestibular and visual inputs in addition to proprioceptive signals. When proprioceptive inputs are less reliable, humans use these alternative sources of information
to balance.
How are these sensory systems integrated? Sensory integration in humans is
studied by examining the effect of sensory perturbations on postural control during
standing. Perturbations to the senses lead to postural sway [30, 36, 93, 170], granting
insight into the sensory feedback control system that maintains postural stability [169,
171]. By observing how much the body sways in response to sinusoidal disturbances,
an estimate of the gain and phase in response to each input may be obtained [93,108,
161, 170]. In order to isolate each modality, subjects stand on a motionless platform
within a visual “moving room,” or on a moving platform within a fixed room. Patients
with vestibular loss can be studied under the same test condition, or subjects can
be instructed to close their eyes to remove visual cues. These observations show
that after an initial disturbance, humans dynamically “re-weight” sensory channels,
causing the gain associated with the perturbed channel to fall as the amplitude of the
185
perturbation increases [103, 144, 161, 169], while the phase remains roughly constant.
The unperturbed channels are simultaneously upweighted, indicating inter-modality
effects. Additionally, when a disturbance begins, the affected channel is rapidly downweighted, but then only gradually reintroduced when the visual input returns to its
initial state [160]. These features may provide clues to how sensory information is
integrated in the nervous system.
Several models have been proposed for how these sensory signals are integrated.
Bayesian models can account for human behavior in many motor tasks, showing
that humans may take into account prior observed statistics about the probability
of various observations [113, 114]. In the engineering literature, optimal estimation
theory uses the same technique to produce optimal estimates from noisy signals. One
popular technique, the Kalman filter [106], uses an internal model of the system
to make predictions of future sensor data. A standard Kalman filter assumes that
noise in each sensory channel has a zero-mean Gaussian distribution, and uses prior
assumptions about these distributions to compute optimal gains for each sensory
channel. The state is then updated using what is effectively a maximum likelihood of
the next estimate, based on the prior prediction and the new data. In this sense, it is
a restricted case of a Bayesian filter - one in which the prior distribution is assumed
to be a zero-mean Gaussian [114]. However, standard Kalman filters do not account
for time-varying measurement noise (i.e. changing prior distributions), and hence
do not reweight sensory data as noise characteristics change. Fortunately, adaptive
Kalman filters capable of estimating the noise priors are also well established in the
engineering literature [25,143,154,162]. This type of an adaptive Kalman filter, which
estimates the noise covariance in real time from observation residuals, could account
for sensory re-weighting in human subjects [220]. By learning the noise covariance
matrix from observation, the computed Kalman gain is automatically modified. As
a result, the degree to which each sensory channel is used automatically adapts to
downweight less reliable sensors.
186
While the Kalman filter uses an internal model, however, it is not known how much
the human nervous system models the dynamics of the body or the environment. Van
der Kooij [220] proposes that it not only models the body but also the dynamics of the
external environment. Carver [26, 27] argues for a model that does not estimate the
external environment, but only the internal dynamics of the body. Carver’s model
is motivated by certain results showing that while the gain of the sway response
declines with increasing amplitude, the phase remains roughly constant [161, 170].
Others propose models that do not use any internal state at all, but rely entirely on
Bayesian decision theory, using non-Gaussian priors to form a maximum likelihood
estimate [44].
While understanding sensory fusion and sensory reweighting in humans helps build
better walking robots [130], robots can also be used to model biological systems
and test hypothesis about how they work [232]. Simulations may overlook non-ideal
aspects of real-world systems, leading researchers to overlook significant problems
and the methods biological system use to cope with them. So called ’neurorobotic’
models allow hypothesis to be tested under real world conditions that reveal these
issues. An example of a neurorobotic model includes recent work by Mergner [145],
in which a robot is used to explore the contribution of the vestibular system to stance
control. These explorations can be a benefit to both the robotics and neuroscience
communities.
In this chapter, we implement an adaptive Kalman filter based on [154] in a
bipedal robot with visual, vestibular and proprioceptive inputs, consisting of a gyro,
foot pressure sensor and a camera system that uses optic flow to compute forward
motion in the visual field. In contrast to [220], we do not model the dynamics of the
external environment, only the dynamics of the body, such as gravitational torque.
Consequently, the adaptive Kalman filter treats motion in the visual field not as a
state variable, but as an external disturbance. The internal model is used to help
distinguish ego-motion from motion of the environment. Since the visual motion is
187
treated as noise, we estimate the measurement noise covariance matrix (R) rather than
the process noise covariance matrix (Q). We include visual rate, vestibular rate, and
angular position information in the model, since this matches the sensory modalities
available in the robot. In simulation, we compare the response of this model to
sinusoidal visual inputs at frequencies and amplitudes used by [170, 220]. We also
compare it to a standard Kalman filter and a Bayesian filter at the same points.
We test our robot by subjecting it to “moving room” and “moving-platform” test
conditions and show that it successfully down-weights perturbed sensory channels,
leading to improved stability.
We are able to observe several significant features of human subject data. In addition to the reweighting phenomena, we observe intermodality effects, as the Kalman
gains for the unperturbed channel are upweighted in response to a disturbance in
another channel. We also observe a temporal asymmetry observed by [160], in both
the simulation and the robot, including when an internal model is not used to make
forward predictions of the state. The temporal asymmetry is produced by the effect
of learning noise characteristics by comparing sensory data to a fused state that is
produced by a weighted contribution of the same data. When one channel is downweighted it contributes less to the state, which increases it’s apparent variance relative
to the other channels, leading to slower upweighting. This effect suggests that a fused
internal state (though not necessarily an internal model), is being used by the body,
and that an error signal between the fused state and the observed sensory data is
used to estimate the quality of each sensory channel.
9.1
Simulation Experiments
9.1.1
Inverted Pendulum Model
We model the robot/human body by an inverted pendulum, with angle φ, length L
and mass m. A diagram of the inverted pendulum model is shown in Fig. 9.1. Angular
188
Figure 9.1. Model of Postural Control. The body is modeled as an inverted pendulum. Sensory information is available from visual and vestibular inputs (optic
flow and gyro in out robot), as well as proprioceptive inputs (foot pressure sensors
in the robot). The visual input is perturbed by false movement of the visual field.
Parameters are given in Appendix A.
position is available through foot pressure sensors (See Section 9.3.1). Angular rate
data is available from visual and vestibular inputs (optic flow and gyro in our robot).
Parameters are given in Appendix A.
The following equations describe a linearized discrete time model of the system.
xk = F xk−1 + Guk + wk
(9.1)
zk = Hxk + vk
where,





 

0 1 0
1
x
1 δt 0
x = ẋ F =  gl δt 1 0 G =  ml1 2 δt H = 0 1 0
0 0 1
0
1 0 1
b
(9.2)
Here, we have included an augmented state to remove a persistent bias we observed
in the foot pressure sensors, as in [47, 57].
9.1.2
Adaptive Kalman Filter
A block diagram of our model is shown in Fig. 9.2. There are three sources of
sensory feedback in our model, visual, vestibular and proprioceptive. In humans,
189
Figure 9.2. Block diagram of model sensory reweighting system. The body is
modeled as an inverted pendulum at angle θ. Visual and vestibular pathways detect
the rate with some added noise, plus the effect of motion of the visual scene on the
visually observed rate. The state estimator compares the forward prediction of the
state to the observed visual rate, and computes an estimate of noise in the visual
channel based on the residual. The Kalman gain is then recomputed and used to
update the state, based on visual and vestibular rate estimates.
190
visual estimates of rate may be derived from the optic flow field [118], the vestibular
system (semicircular canals) also provides rate information. Both sources of sensory
feedback are subject to perturbation or noise vvis and vgyr . Position information is
provided by proprioceptive sources such as tactile sensors on the feet, and is affected
by noise vf ps . The state estimator (Kalman filter) receives this information and
updates the state based on the current Kalman gains. A controller using PD control
and gains Kp and Kd outputs torques to the robot. In addition to control torques,
process noise ω also impacts the system. An internal model of the state makes forward
predictions on each time step. These estimates are compared with incoming sensory
data and the residuals are used to update estimates of the noise variance, and update
the Kalman gain.
The process noise wk and measurement noise vk have the following covariance
matrices:


σv12 0
0
0 0
Q=
R =  0 σv22
0 
0 σw2
2
0
0 σf ps
(9.3)
In many situations the noise covariances may be unknown. One method of determining an estimate of the noise variance is to use the autocorrelation of the observation residual [25, 143, 154], the difference between the predicted sensor measurement
and the observed measurement. Under steady state conditions for an optimal filter,
the residual is a zero mean white noise process, and the measurement noise covariance
can be estimated from the sample covariance of the residuals.
Based on formula of Myers and Tapley [154], we estimate the measurement noise
variances using the observation residuals ỹk , so that R becomes Rk , a function of
time. We converted the Myers formula from a sum over the last N elements weighted
equally to an incremental update weighted by N-1/N, as follows:
From [154] Equation 9, using our notation, we can write (in our own notation),
191
N
1 X
N −1
{(ỹk − r̂)(ỹk − r̂)T −
H P̄k H T }
R̂ =
N − 1 k=1
N
(9.4)
N −1
1 X
N −1
R̂ =
H P̄k H T }
{(ỹk − r̂)(ỹk − r̂)T −
N − 1 k=1
N
+
1
1
(ỹN − r̂)(ỹN − r̂)T − H P̄N H T
N −1
N
(9.5)
by taking the formula one time step forward, and isolating the most recent observation, we can turn this into an update equation. We then assign a subscript k to
the most recent (Nth) observation. Thus,
R̂k+1 ≈
N −1
1
1
R̂k + (ỹk+1 − r̂)(ỹk−1 − r̂)T −
H P̄k H T
N
N
N +1
(9.6)
This formula is more computationally efficient, since it does not require storage of
the past N time steps, and requires fewer computations to update. It also weights the
estimate more heavily towards recent data, which is more important from a postural
control perspective. Similarly, we also need to estimate the mean of the noise.
r̂ =
N
1 X
ỹk
N k=1
r̂ =
N −1
1 X
1
ỹk + ỹN
N k=1
N
r̂k ≈
(9.7)
N −1
1
r̂k−1 + ỹk
N
N
In the formula of [154], N represents the number of measurements observed. In
this adaptation, measurements prior to N time steps in the past aren’t completely
discounted, but are weighted by (N − 1/N )n , where n is the number of time steps in
the past. We can consider the 1/N term to be effectively a learning rate for adaptation
in the final equations.
192
9.2
Simulation Results
The adaptive Kalman filter was simulated in MATLAB, and compared with a standard Kalman filter and an adaptive non-predicting Bayesian filter. The non-predicting
filter was created by setting the transition matrix in the model to identity and removing the predictive step from the Kalman filter. This formulation effectively removes
the internal model from the system, and reduces the filter to a maximum likelihood
estimator under the assumption of Gaussian priors. To compensate for the lack of an
internal model, we increased the assumption of process noise in the filter (but not to
generate the noise), essentially treating self-motion as process noise. The equations
for learning the noise variances were retained so the Bayesian filter remains adaptive,
producing the reweighting effect. Gaussian noise generated according to assumed
priors, was added to the true state to produce three simulated sensor data streams,
for gyro, vision, and foot pressure sensors. For better comparison to results of van
der Kooij [220] and others, the parameters were selected to be close to the moment
of inertia and length of the pendulum used in his model. Since we are using a point
mass pendulum, the equivalent mass is 20 kg, with a length of 1.1m, giving a moment
of inertia of 24.2 kgm2 . All parameters were identical for all three models except for
the larger process noise assumption used by the Bayesian estimator.
9.2.1
Sinusoidal Disturbances
To show the effect of reweighting in various modalities, at three separate intervals, disturbances are introduced into the vestibular, visual, and proprioceptive sensor data,
in that order. Simulation results show that the adaptation of the noise variance allows the state estimator to rapidly down-weight the affected pathway and up-weight
the other pathways (See Fig. 9.4). Here we are displaying the largest component of
the Kalman gain matrix associated with each sensory input. So, for the visual and
vestibular channels this is the second row element, which updates the rate element
193
Figure 9.3. Simulation results in response to a sinusoidal disturbance of 1.2 degrees/sec (or degrees for the FPS) and 1Hz in each sensory channel in turn. Standard
Kalman filter (blue), Adaptive Kalman filter (red) and non-predictive Bayesian filter
(green). Error in estimated angle (top), angular rate (middle), and applied torques
(bottom). Disturbances are added to each channel at 12.5, 25 and 37.5 seconds
of simulation time. In the standard Kalman filter, significant errors are present in
the estimate, resulting in larger torques applied at the ankle to compensate. In the
adaptive Kalman filter and non-predictive filter, the estimate remains accurate. The
non-predictive Bayesian filter also reweights data, but is not at good at filtering out
disturbances, particularly when applied to the proprioceptive sensor.
194
Figure 9.4. Simulation results in response to a sinusoidal disturbance of 1.2 degrees/sec (or degrees for the FPS) and 1Hz in each sensory channel in turn, for the
adaptive Kalman filter (top) and the non-predictive Bayesian filter (bottom). Gains
(circles) and variances (squares). Kalman gain for the visual rate (red), vestibular
rate (blue) and proprioceptive angle (cyan). Estimated variance in visual rate (light
green) vestibular rate (dark green) and proprioceptive angle (black). Disturbances
are added to each channel at 12.5, 25 and 37.5 seconds of simulation time respectively. In both cases, the estimated noise variance rises while the Kalman gain for
the affected channel falls. We can observe that inter-modality effects are predicted by
this model, as the gain in the non-perturbed channels increases when the disturbance
is introduced (except when the disturbance is applied to the proprioceptive channel).
195
of the state, and for the proprioceptive channel it is the first row element, which
updates the position element. In this case, we have symmetrical responses to disturbances affecting vestibular and visual rates. This response is because the Gaussian
(background) noise we are assuming in each channel has the same variance and the
disturbance has the same magnitude and frequency. Ideally, these parameters could
be tuned to match the underlying noise variances of human subjects.
The reweighting effect is achieved whether or not an internal model is used, in both
the non-predictive Bayesian estimator (Figure 9.4 (top)) and the adaptive Kalman
filter (Figure 9.4 (bottom)). Since both methods are learning the noise variances
through observation residuals, they exhibit similar profiles. Note that the Bayesian
estimator, without an internal model, has much larger gains, which is a result of
assuming higher process noise and treating body motion as a noise process. The
Bayesian estimator consequently tends to rely more on sensory data than an internal
model and has a harder time filtering out the disturbance.
The reweighting effect can also be seen directly in the position and rate error (Fig.
9.3 top, middle). In the standard Kalman filter, the errors are significant, leading
to increased control torques (Fig. 9.3 bottom) and body sway. Without an internal
model, the Bayesian filter also has greater difficulty filtering out the disturbance
when the disturbance is applied to the proprioceptive channel. This difficulty is
possibly because there is only one proprioceptive channel modeled, so it is harder to
compensate for disturbances in this case.
We also can observe the appearance of intermodality effects when the disturbance
is applied to the vestibular and visual senses. As the Kalman gain declines in the
affected channel, it rises in the others. This effect has been observed in humans [161].
Curiously, this does not seem to occur when applied to the proprioceptive channel.
However, we note that we implemented a full version of the Myers and Tapley [154]
filter in which both the Q and R matrices are adapted, and intermodality effects are
seen for all channels there. There may be some dependence on the process noise
196
(a) Gain vs. Amplitude
(b) Phase vs. Amplitude
Figure 9.5. Gain (a) and Phase (b) for increasing stimulus amplitude, for adaptive
Kalman filter (red, dashed, circles) , standard Kalman filter (green, dotted, squares),
and non-predictive Bayesian filter (black, dot-dashed, crosses). Both the adaptive
Kalman filter and the adaptive Bayesian filter exhibit downweighting, though the
lack of an internal model leads to larger gains and a significant phase lag for the
non-predictive filter.
assumptions.
9.2.2
Amplitude and Phase Responses
For comparison to previous models we simulated the system at the frequencies and
amplitudes used by [220] and [170]. Simulations were run at 7 different random
seeds for each model, and the mean and standard deviation were computed. Figure
9.5 shows the mean gain and phase with error bars at three times the standard
deviation. These results show that, as expected, the standard Kalman filter does not
exhibit reweighting and has a constant gain and phase across all input amplitudes
for both 0.1 Hz and 0.2 Hz inputs. Both the adaptive Kalman filter, and the nonpredicting Bayesian filter do exhibit reweighting, as shown in Figure 9.5 (a), including
a decrease in the threshold at which downweighting occurs at higher gains, also seen
in [170, 220]. The non-predicting Bayesian filter has somewhat larger gains than
the adaptive Kalman filter, and a slightly higher threshold for downweighting. The
197
greater gains are also a consequence of the lack of an internal model, since the filter
necessarily must admit more sensor data to make up for the larger assumption of
process noise. Without the ability to predict body motion, an optimal filter has to
rely on sensory information to a greater extent, which is necessarily going to be true
for any optimal filter that does not use an internal model. (One could force it to use
sensory information less but it would degrade performance.)
The non-predicting filter also exhibits a significantly larger phase lag, compared to
both the standard and adaptive Kalman filters (Figure 9.5 (b)). This large lag is also
a consequence of the lack of an internal model, since the filter estimate necessarily
lags the true state more when not predicting it forward. Both the adaptive Kalman
filter and non-predicting filter exhibit declines in the phase at higher amplitudes,
and a smaller phase offset at the higher frequency stimulus. At higher amplitudes,
the computed phase offsets exhibit a large variance. This larger variance is because
in the adaptive Kalman filter the signal is almost completely filtered out, so the
response is extremely noisy and hard to detect a phase offset. In the Bayesian filter,
the problem is that the system tends to exhibit some oscillations at the natural
mode of the pendulum, due to greater difficulty in keeping up with the pendulum
dynamics, which overwhelms the effect of the visual disturbance. However, at the
0.1Hz frequency we can see a change in phase of about 40 degrees for the adaptive
filter. At the 0.2Hz frequency, there is a change in phase of about 15 degrees for
both the adaptive Kalman filter and the Bayesian filter. We also note that at higher
frequencies the phase continues to flatten with respect to amplitude.
9.2.3
Dealing with Modeling Errors
After implementing the algorithm in a robot, we discovered that because the body
model is not accurately known for the robot, the Kalman filter would tend to diverge.
This problem was not discovered in simulation, because the simulation was using the
198
same transition matrix to model the dynamics as the filter was using to make it’s
predictions. In other words, in the simulation, the agent had perfect knowledge of
it’s dynamics, which is not necessarily the case in real life. It is possible to simulate
this problem, although it would probably not have been discovered if we had not
attempted to implement it in a robot.
Since the predictions of body motion were incorrect, the real motion of the robot
would show up in observation residuals, and appear as correlations between the noise
channels. That behavior resulted in an incorrect calculation of the R matrix, and
ultimately divergence of the filter. The non-predictive Bayesian model also has this
problem if the process noise variance is not assumed to be larger.
However, since we know that noise channels are in fact independent (sources
of noise in vision don’t usually impact the vestibular system and vice versa), the
problem might be mitigated by diagonalizing the covariance matrix and setting all
off-diagonal elements to zero, effectively forcing an assumption of independence in
the filter. It turns out that this does resolve the problem, as shown in Figure 9.6.
The independence assumption also tended to result in better performance against
non-Gaussian noise, such as sinusoids. This result is consistent with the hypothesis
that a separate error signal for each sensory mode is compared with the fused state,
and independently processed.
9.3
9.3.1
Robot Model
Robot
We implemented the model in a bipedal robot shown in Figure 9.7. The robot has
motors in the ankles and hips to control posture in the saggital plane, which are
position controlled RC servos. The robot is equipped with a three axis gyro which
outputs both angular position and angular rate data in three axes. For the purpose of
these experiments, we did not use the gyro’s angle measurement in the filter, instead
199
Figure 9.6. Effect of model errors on performance, with and without assuming
noise independence. Top two plots: When a good model is assumed and used by
the filter, no assumption of independence is used and the model performs better
without it. Bottom two plots: However, when the model is incorrect, the assumption
of independence is crucial to maintain stability. Without this assumption, modeling
errors appear as correlations in the sensory noise, resulting in filter divergence. For
our incorrect model we multiplied the g/l term in the A matrix by 1.3.
200
Figure 9.7. Robot used in Experiment. The robot is equipped with a three axis
gyro, a camera, which uses optic flow to measure rotation, and foot pressure sensors,
which are used to estimate angular tilt of the robot.
201
we use it to measure the accuracy of the filter’s output. The 3-axis gyro models
aspects of the vestibular system (semicircular canals) while a digital camera models
visual input using optic flow, which it used to create an estimate of angular rate. The
robot is also equipped with foot pressure sensors, which are variable resistors with
near linear conductance with respect to force.
Since the RC servos in the ankles are position controlled, torque is not applied
directly at the ankles. Instead a PD controller was devised which rotates the hips
slowly to manipulate the location of the center of pressure. The controller was hand
tuned so that the robot could recover from a forward shove in a few seconds. Parameters were scaled so that the gains used by the filter approximate what would
be required to produce an equivalent torque about the ankles. Since torques are not
being applied directly at the ankles, and the hip position is being adjusted slowly,
the torques about the ankles are primarily induced by the position of the center of
mass under gravity. Under these conditions, the foot pressure sensors can be used to
estimate the angular tilt of the center of mass, as shown in Figure 9.8.
The estimate is obtained by performing a least squares fit between the gyro angle
measurement and the foot pressure sensor measurement over a range of angles produced by giving the robot small shoves and waiting for it to recover. This experiment
was performed with the robot using the output of the gyro exclusively to balance.
Figure 9.8 shows this fit versus the measurement given by the gyro over several perturbations, both for the calibration data (top) and test data (bottom). It is easy to
see that in this case the FPS measurements give a reasonable measure of the angular
position of the robot.
The foot pressure sensors nevertheless exhibited a persistent bias, which can be
seen in Figure 9.8, due to the fact that the robot, and thus the gyro, is not perfectly vertically aligned when it is perfectly balanced, but leans slightly forward. To
accommodate this problem we added an augmented state to the system to produce
a colored noise Kalman filter, as in [47, 57]. This augmentation allows the filter to
202
Figure 9.8. Calibration of foot pressure sensors against the tilt of the robot’s torso.
During quiescent standing the robot was perturbed and allowed to wobble on its feet
several times. The angular measurement of the gyro and the foot pressure sensors
were measured against each other.
detect and filter out a constant undesired offset in the measurement of a particular
channel.
9.3.2
Optic Flow
Estimates of motion may be obtained from observations of the motion of the visual
field. Motion sensitive neurons have been identified in various areas of the primate
visual cortex [184]. One prominent theory of how motion is tracked is the optic flow
field [70, 118], in which visual information is encoded as an array of motion vectors
assigned to features in the field. It has been shown that optic flow plays an important
role in directing human motion for path trajectory following [182,231]. Postural sway
also appears sensitive to velocity information in the visual stream [202], suggesting
203
Figure 9.9. Example image of stimulus with optic flow vectors. The vectors track
outward motion of the colored dots, simulating the effect of forward motion.
that rate information derived from optic flow may be used for postural control as
well. The use of optic flow to control balance also holds the prospect of being easily
integrated with robot locomotion schemes [119].
Several algorithmic implementations of optic flow have been introduced, including
the popular Lucas-Kanade [126] algorithm. We use an implementation of this algorithm available in the computer vision package OpenCV. This algorithm produces
an array of N vectors v~i , shown in green in Figure 9.9, which we can then use to
compute an estimate of rate. The net motion in the frame is then computed by averaging over the movement vectors as shown in Equation 9.10. For forward motion,
the dot product of the vector with the vector to the center of the frame c~i gives a
measure of the “outwardness” of the vector, which corresponds to forward motion.
The vector c~i is normalized, and then scaled again by its magnitude, since vectors
starting further from the center of motion will be larger. The forward and vertical
motion estimates were combined to give an estimate of the robot’s rate of rotation.
For each element, a scaling factor was determined empirically by rotating the robot
and comparing the measurement given by the optic flow (forward and vertical flow)
to the objective measurement of the rate gyro.
204
ynet =
fnet =
1 X
vyi
N i
1 X ~vi · ~ci
N
i
|~ci |2
θ̇vis = Ay ynet + Af fnet
(9.8)
(9.9)
(9.10)
Where Ay and Af are scaling factors determined by comparing the ynet and fnet
computations with gyro rate measurements.
9.3.3
Experiments
To test the robot implementation, we used two experimental platforms designed to
mimic the experimental platforms in the human subject studies. In the first experiment, the “moving room” condition, the visual environment is perturbed to create
the illusion of motion. In actuality, the robot is standing still, causing the robot to
react improperly. Unfortunately, because of the small size of most movement vectors
(a few pixels), it was difficult to obtain an accurate measurement of rate via optic
flow for small and slow motions such as the sinusoidal stimuli in previous studies of
human postural control. Therefore, instead of using sinusoidal stimuli, we decided to
use a constant offset disturbance in the rate. This disturbance would be equivalent to
standing on a fixed platform while a train moves by, or standing on a moving train,
observing the environment.
In the second experiment, the moving platform condition, the platform the robot
is standing on is placed on top of a soft foam surface, which is depressed, tilting the
platform, and causing pressure on the feet to change, thereby injecting error into the
proprioceptive measurement of position. Our effort is to inject error into the foot
pressure sensors. However, this process also injects process noise, since the effect of
tilting the platform applies torques to the ankles. The effect on reweighting is more
complex in this case, but it is still possible to discern.
205
Experiment 1: “Moving Room” The robot is placed approximately 30 centimeters in
front of a visual field produced by 3D graphics of spheres resting in a black background
(Fig: 9.1). Initially the field is stationary, then it begins to move so that the spheres
appear to be moving towards the camera, producing the illusion of falling forward.
The spheres move outwards at a rate of 5 cm/s, which translates to an error in the
apparent visual angular rate of 0.2 rad/s. After 10 seconds, the visual field returns to
stationary. We ran examples of this test with and without reweighting. As the visual
disturbance begins, in the case of reweighting (Fig. 9.10), the variance for the visual
channel increases rapidly, causing K to decline, resulting in the visual disturbance
being filtered out of the state estimate (Fig. 9.11). This decrease in K results in
relatively little disturbance in the control of the robot’s ankle torques. We can also
observe that the gain for the gyro and foot pressure sensors increase during this time
period.
We compared the state estimates for both cases to show that the difference in control torques at the ankle is caused by differences in the accuracy of the state estimate
(Figs 9.11 (a) and (b)). When reweighting is turned off there is a much larger difference between the gyro position and the combined state estimate. When reweighting
is present, the difference is very small. Without reweighting, this translates into large
erroneous torques at the ankles, which contribute to larger errors in the visual flow
versus the state estimate, as the robot rotates unexpectedly under gravity. By contrast, when reweighting is present, the visual errors are almost immediately filtered
out, allowing the robot to rapidly stabilize.
Finally, after the visual field stabilizes the visual information is gradually reincorporated into the combined state estimate (Fig. 9.10).
Experiment 2: “Moving Platform” To test the effect of error in the foot pressure
sensors we placed the robot on top of a platform which rested upon a layer of spongy
material. By applying pressure to the platform we could create a low level motion in
206
Figure 9.10. Robot Experiment: Time course of noise estimates (blue) and gains
K (green). During the visual disturbance, θv2 rises, causing the gain to fall. The gain
in the gyro and FPS channels increases to compensate.
207
(a) Position Estimate
(b) Angular Rate Estimate
Figure 9.11. Results from Moving Room Experiment. Estimates (blue), gyro measurements (red), visual rate measurements (green), and visual disturbance(black).
(a) Estimated angles without and reweighting. The gyro’s angular position output
here is used as a comparison to the filter’s estimates, although the gyro position is
not used in our filter. (b) Angular rates with and without reweighting. At about
15 seconds the visual field begins to move, creating a false sense of angular rate in
the visual channel. Comparing the reweighting case to the no-reweighting case, the
resulting offset from vertical is greater in the no reweighting case. In addition to the
rate error, ankle torques are applied to the erroneous state estimate causing the robot
to actually tilt away from the vertical, as shown in (a). With sensory reweighting,
the robot filters the disturbance out and remains unperturbed.
Figure 9.12. Comparison of command applied at the ankle with and without sensory
reweighting. Without sensory reweighting, the robot exerts larger ankle torques.
With sensory reweighting, the robot is perturbed but is able to recover.
208
Figure 9.13. Results from Moving Platform Experiment. Estimates (blue), gyro
measurements (red), visual rate measurements (green), and visual disturbance(black).
Estimated angles with and without reweighting. When reweighting is not applied, a
significant oscillation results from the introduction of the error in the rate estimate,
as the state estimate becomes inaccurate and the controller attempts to compensate,
incorrectly. As the controller attempts to compensate the robot rotates away from
vertical introducing genuine gravitational torques into the system. This behavior
results in oscillation of the robot.
the platform in a controlled fashion to simulate the effect of standing on an unstable
surface, injecting sensory error into the foot pressure sensors.
As shown in Figure 9.13, the estimates of tilt angle from the foot pressure sensors
are affected by the motion of the platform. In response to this motion, the estimated
variance increases and the Kalman gain drops (Fig. 9.15, highlighted middle). As
a result, smaller ankle torques are applied when reweighting is used (Fig. 9.14).
Because we are introducing both process and measurement noise in this case, the
filter has a harder time distinguishing that the disturbance is occurring in the FPS
channel. Process noise is not being estimated, so the disturbance appears as noise
209
Figure 9.14. Results from Moving Platform Experiment. Applied ankle torques.
Force is applied to the platform to produce errors in the foot pressure sensors. Without reweighting (blue), the estimates of position diverge from the true state, resulting in greater ankle torques. With reweighting (red) the foot pressure sensor data is
down-weighted, resulting in smaller ankle torques.
in the visual and gyro senses as well as in the FPS. Over time, however, the gyro
and visual measurements fit the predictions of the internal model better, allowing the
filter to eventually increase the estimate of noise variance more in the FPS channel,
causing it to downweight that channel relative to the others.
9.4
Discussion
In [145], a robot implements models of vestibular and proprioceptive signals. That
research is primarily focused on understanding the role of the vestibular system, and
therefore does not attempt to incorporate a detailed model of sensory reweighting. it
uses a relatively simple thresholding process rather than learning online from observations. Van der Kooij’s work [220] shows that such thresholds can be achieved by the
adaptive Kalman filter, which we are also able to observe. Similarly, while [130] proposes switching between two pre-defined covariance matrices in a Kalman filter, the
210
Figure 9.15. Results from Moving Platform Experiment. Gain (green) and estimated variance (blue) for foot pressure sensors. Force is applied to the platform to
produce errors in the foot pressure sensors. The estimates of variance in the foot pressure sensor measurements increase, resulting in a decrease in the computed Kalman
gain for that channel. In this case, the introduction of process noise due to movement
of the platform causes the filter to take longer to identify the FPS sensors as the most
noisy, and to perceive noise from other sensor channels as well. Due to inter-modality
effects, there is a relatively small downweighting of the FPS sensors.
211
robot implementation manually forces the change in the gains, without relying on any
automatic detection of the disturbances by the robot. By contrast, our method automatically learns the noise covariances based on filter residuals, resulting in automatic
sensory reweighting under an optimal filtering regime.
It is possible to produce sensory reweighting effects without an internal model.
We did this by setting the transition matrix in the filter to identity, and removing
the predictive step. This configuration reduces the Kalman filter to a Bayesiam maximum likelihood estimate based on Gaussian priors. However, in order to make the
filter stable, we needed to increase the process noise assumption. In other words, egomotion was assumed to be a Gaussian noise process, rather than a dynamic process
that the system has internal knowledge of. The lack of an internal model to make
predictions results in an increased phase lag between the robot and somewhat greater
difficulty in controlling the stance. However, we are still able to obtain downweighting of perturbed senses, by retaining the online learning of the measurement noise
variances (R).
During the process of implementing the model in a robot, we ran into several
problems which forced us to confront problems that we would not have considered
in a simulation. Since the robot is not really an inverted pendulum, and it’s mass is
not evenly distributed, it’s internal body model is only an approximation. Because
of this approximation, we discovered that the adaptive Kalman filter was unstable
under certain conditions, including when there are significant inaccuracies in the internal model. Since there is motion of the robot that is not predicted by the model,
it results in an erroneous interpretation of body motion as correlations between the
noise channels in the R matrix, resulting in poor computations of the Kalman gain.
However, using prior knowledge of the fact that noise in each channel is independent, we diagonalize the covariance matrix, setting all off-diagonal elements to zero.
This configuration prevented body motion from being interpreted as correlations in
the observation residuals, while preserving the ability to downweight the perturbed
212
channel. The algorithm also exhibited improved performance against non-Gaussian
noise. Under certain circumstances, non-Gaussian noise would result in instability
even if the internal model was correct. However, assuming independence in the noise
channels allowed the filter to remain stable. We also adapted the model of [154] from
an average over the past N samples, to an online running average with adaptation
rate 1/N, and added compensation for persistent sensor bias as in [47, 57].
Our model produces the inter-modality effects between the various channels when
disturbances are introduced, similar to those described by [161]. The variances are
found by comparing the predicted state against measurements, and the computation
of the gain depends on the relative variance between the channels. As a result, the less
noisy channels are automatically upweighted when another channel gets noisier, even
if the variance in the non-perturbed modality remains unchanged. In the “moving
platform” case, we are introducing unmodeled process noise along with sensor noise in
the foot pressure sensor. The unmodelled process noise shows up in all of the sensors,
resulting in increased variances in all channels. Over time, however, the internal
model’s predictions track the visual and gyro sensors more accurately, causing the
variance in the FPS to rise relative to the others.
Our model also reproduces a temporal asymmetry observed in human postural
control. The fact that the gain K increases more slowly after a disturbance than it
drops at the start of the disturbance is consistent with the temporal asymmetry measured in the human response [160]. The gradually up-weighting apparently happens
because the noise estimate is derived from the difference between the internal state
estimate and the raw data. Once data is down-weighted, this difference is larger since
the state estimate necessarily tracks the other data sources, leading to continuing
larger estimates of the noise variance for the down-weighted channel. This behavior
can easily be seen by observing the estimates of variance in Figure 9.10. By necessity,
down-weighted sensory information affects the state less, and hence appears to be
noisier in comparison to it. This effect happens whether or not a forward predic-
213
tion of the state is made before updating based on some internal model of the body.
Rather it suggests that some sort of fused estimate of the state is maintained internally and used to estimate noise levels. This fusion could plausibly be implemented
in the nervous system by the generation of an “error signal” between the expected
sensory data, produced by the internal state, and the observation.
An internal model may be helpful to detect the difference between ego-motion and
external motion of the environment. An internal model allows the system to make
predictions of sensory data assuming only ego motion, based on past position and
applied torques. Sensory data which does not match the prediction of the model is
more likely to be environmental. Without an internal model, the system relies more
heavily on external sensor data, i.e. it is consequently less able to filter out environmental motion, and is more likely to exhibit body sway in response to disturbances.
It also exhibits a significant phase lag in its responses to disturbances. These results
might be useful point for comparison to determine if humans are using an internal
model.
9.5
Conclusion
The ability to incorporate multiple sources of sensory data has applications in robotics,
especially robots that must move over irregular terrain, soft or unlevel material. Most
robots to date tend to rely on proprioceptive inputs such as foot pressure sensors and
joint positions. However, incorporating multiple sensory sources may introduce conflicts between the senses, as each channel is subject to different sources of noise.
Sensory reweighting in humans therefore may be a useful model for building better
robots. We show here the successful implementation of an automatic reweighting
scheme in a robot with multiple channels including visual and vestibular functionality. In addition, neurorobotic models can help neuroscientists gain insights into
physical systems in ways that may be overlooked in simulation. We discovered that
214
an inaccurate body model poses problems for models of postural control in which
an internal model is needed to predict the state. In biological systems, the internal
model must be learned in some way, and may never be completely accurate. As a
final question, how does the model compensate for this, and how can the organism
continue to maintain control as it learns a more accurate model?
.1
Robot Parameters List
Name
Description
m
Pendulum mass
L
Pendulum length
δt
Time step
Kp
Proportional Gain
Kv
Differential Gain
σw
Process noise variance
σv1
Gyro noise variance
σv2
Vision noise variance
σvf ps
Foot pressure sensor noise variance
Ay
Vertical motion scaling factor
Af
Outward motion scaling factor
N
History length constant
.2
Value
3 kg
0.5 m
0.1 s
30
10
0.00001 rad/s
0.0001 rad/s
0.0001 rad/s
0.0001 rad
0.009
2.8
50
Simulation Parameters List
Name
Description
m
Pendulum mass
L
Pendulum length
δt
Time step
Kp
Proportional Gain
Kv
Differential Gain
σw
Process noise variance
σv1
Gyro noise variance
σv2
Vision noise variance
σvf ps
Foot pressure sensor noise variance
N
History length constant
Value
20 kg
1.1 m
0.005 s
1518
560
0.00003 rad/s
0.001 rad/s
0.001 rad/s
0.0001 rad
100
215
References
[1] J. S. Albus. A theory of cerebellar function.
10(1/2):25–51, 1971.
Mathematical Biosciences,
[2] R. M. Alexander. Three uses for springs in legged locomotion. Int. J. Rob.
Res., 9:53–61, March 1990.
[3] S. Anderson, M. Wisse, C. Atkeson, J. Hodgins, G. Zeglin, and B. Moyer.
Powered bipeds based on passive dynamic principles. In Humanoid Robots,
2005 5th IEEE-RAS International Conference on, pages 110 –116, dec. 2005.
[4] M. J. Angel, E. Jankowska, and D. A. McCrea. Candidate interneurones mediating group i disynaptic epsps in extensor motoneurones during fictive locomotion
in the cat. The Journal of Physiology, 563(2):597–610, 2005.
[5] S. Aoi and K. Tsuchiya. Locomotion control of a biped robot using nonlinear
oscillators. Autonomous Robots, 19(3):219–232, 2005.
[6] M. A. Armada, P. Gonzlez Santos, J. Hurst, and A. Rizzi. Physically variable compliance in running. In Climbing and Walking Robots, pages 123–133.
Springer Berlin Heidelberg, 2005.
[7] W. R. Ashby. Design for a brain. Wiley, Oxford, England, 1952.
[8] C. Azevedo, P. Poignet, and B. Espiau. Moving horizon control for biped robots
without reference trajectory. In Robotics and Automation, 2002. Proceedings.
ICRA ’02. IEEE International Conference on, volume 3, pages 2762 –2767,
2002.
[9] C. Azevedo, P. Poignet, and B. Espiau. Artificial locomotion control: from
human to robots. Robotics and Autonomous Systems, 47(4):203 – 223, 2004.
[10] D. Barnes. Hexapodal robot locomotion over uneven terrain. In Control Applications, 1998. Proceedings of the 1998 IEEE International Conference on,
volume 1, pages 441 –445 vol.1, sep 1998.
[11] J. S. Bay and H. Hemami. Modeling of a neural pattern generator with coupled
nonlinear oscillators. Biomedical Engineering, IEEE Transactions on, BME34(4):297 –306, april 1987.
[12] R. D. Beer. A dynamical systems perspective on agent-environment interaction.
Artificial Intelligence, 72((1-2)):173–215, 1995.
216
[13] R. D. Beer. Dynamical approaches to cognitive science. Trends in Cognitive
Science, 4(3):91–99, 2000.
[14] R. D. Beer, H. J. Chiel, and J. Gallagher. Evolution and analysis of model
cpgs for walking ii. general principles and individual variability. Journal of
Computational Neuroscience, 7(2):119–147, 1999.
[15] R. D. Beer, H. J. Chiel, and J. C. Gallagher. Evolution and analysis of model
cpgs for walking i. dynamic modules. Journal of Computational Neuroscience,
7(2):99–118, 1999.
[16] R. D. Beer and J. C. Gallagher. Evolving dynamical neural networks for adaptive behavior. Adaptive Behavior, 1(1):91–122, 1992.
[17] H. Benbrahim and J. A. Franklin. Biped dynamic walking using reinforcement
learning. Robotics and Autonomous Systems, 22(3-4):283 – 302, 1997. Robot
Learning: The New Wave.
[18] N. A. Bernstein. The Co-ordination and regulation of movements. Pergamon
Press Ltd., first english edition edition, 1967.
[19] G. Bessonnet, P. Seguin, and P. Sardain. A parametric optimization approach
to walking pattern synthesis. The International Journal of Robotics Research,
24(7):523–536, 2005.
[20] R. A. Brooks. Intelligence without representation. Artificial Intelligence, 47(13):139–159, 1987.
[21] R. A. Brooks. A robot that walks: Emergent behaviors from a carefully evolved
network. Neural Computation, 1:253–262, 1989.
[22] T. G. Brown. The intrinsic factors in the act of progression in the mammal.
Proceedings of the Royal Society of London, Series B, 84(572):309–319, 1911.
[23] A. Büschges. Sensory control and organization of neural networks mediating
coordination of multisegmental organs for locomotion. Journal of Neurophysiology, 93(3):1127–1135, 2005.
[24] B. Calancie, B. Needham-Shropshire, P. Jacobs, K. Willer, G. Zych, and
B. A. Green. Involuntary stepping after chronic spinal cord injury. Brain,
117(5):1143–1159, 1994.
[25] B. Carew and P. Blanger. Identification of optimum filter steady-state gain
for systems with unknown noise covariances. IEEE Transactions on Automatic
Control, 18(6):582–587, 1973.
217
[26] S. Carver, T. Kiemel, H. van der Kooij, and J. J. Jeka. Comparing internal models of the dynamics of the visual environment. Biological Cybernetics,
92(3):147–163, 2005.
[27] T. Carver, S. Kiemel and J. Jeka. Modeling the dynamics of sensory reweighting.
Biological Cybernetics, 95(2):123–134, 2006.
[28] P. Channon, S. Hopkins, and D. Pham. Variational approach to the optimization of gait for a bipedal robot. Proceedings of the Institution of Mechanical
Engineers, Part C: Journal of Mechanical Engineering Science, 210(2):177 –
186, 1996. Bipedal robots;Cost function;Euler equations;Walking motions;.
[29] A. Clark and D. J. Chalmers. The extended mind. Analysis, 58(1):7–19, 1998.
[30] S. Clark and M. A. Riley. Multisensory information for postural control: swayreferencing gain shapes center of pressure variability and temporal dynamics.
Experimental Brain Research, 176(2):299–310, 2007.
[31] A. H. Cohen and P. Walln. The neuronal correlate of locomotion in fish. Experimental Brain Research, 41:11–18, 1980. 10.1007/BF00236674.
[32] S. Collins, A. Ruina, R. Tedrake, and M. Wisse. Efficient bipedal robots based
on passive-dynamic walkers. Science, 307:1082–1085, 2005.
[33] S. Collins, M. Wisse, and A. Ruina. A 3-d passive-dynamic walking robot with
two legs and knees. International Journal of Robotics Research, 20(7):607–615,
2001.
[34] R. H. Crompton, L. Yu, W. Weijie, M. Gnther, and R. Savage. The mechanical
effectiveness of erect and bent-hip, bent-knee bipedal walking in australopithecus afarensis. Journal of Human Evolution, 35:55–74, 1998.
[35] H. Cruse, T. Kindermann, M. Schumm, J. Dean, and J. Schmitz. Walknet–a
biologically inspired network to control six-legged walking. Neural Networks,
11(7-8):1435 – 1447, 1998.
[36] B. L. Day, A. S. Cauquil, L. Bartolomei, M. A. Pastor, and I. N. Lyon. Human body-segment tilts induced by galvanic stimulation: a vestibularly driven
balance protection mechanism. Journal of Physiology, 500(3):661–672, 1997.
[37] E. Dertien. Dynamic walking with dribbel. Robotics Automation Magazine,
IEEE, 13(3):118 –122, sept. 2006.
[38] M. H. Dickinson, C. T. Farley, R. J. Full, M. A. R. Koehl, R. Kram, and
S. Lehman. How animals move: An integrative view. Science, 288(5463):100–
106, 2000.
218
[39] H. Diedam, D. Dimitrov, P.-B. Wieber, K. Mombaur, and M. Diehl. Online
walking gait generation with adaptive foot positioning through linear model predictive control. In Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ
International Conference on, pages 1121 –1126, sept. 2008.
[40] M. R. Dimitrijevic, Y. Gerasimenko, and M. M. Pinter. Evidence for a spinal
central pattern generator in humansa. Annals of the New York Academy of
Sciences, 860(1):360–376, 1998.
[41] D. Dimitrov, P.-B. Wieber, H. Ferreau, and M. Diehl. On the implementation
of model predictive control for on-line walking pattern generation. In Robotics
and Automation, 2008. ICRA 2008. IEEE International Conference on, pages
2685 –2690, may 2008.
[42] D. Dimitrov, P.-B. Wieber, O. Stasse, H. Ferreau, and H. Diedam. An optimized
linear model predictive control solver for online walking motion generation. In
Robotics and Automation, 2009. ICRA ’09. IEEE International Conference on,
pages 1171 –1176, may 2009.
[43] J. Doke, J. M. Donelan, and A. D. Kuo. Mechanics and energetics of swinging
the human leg. Journal of Experimental Biology, 208(3):439–445, 2005.
[44] K. Dokka, R. V. Kenyon, E. A. Keshner, and K. P. Kording. Self versus environment motion in postural control. PLoS Comput Biol, 6(2):e1000680, 02
2010.
[45] J. M. Donelan, D. A. McVea, and K. G. Pearson. Force regulation of ankle
extensor muscle activity in freely walking cats. Journal of Neurophysiology,
101(1):360–371, January 2009.
[46] J. M. Donelan and K. G. Pearson. Contribution of force feedback to ankle
extensor activity in decerebrate walking cats. Journal of Neurophysiology,
92(4):2093–2104, 2004.
[47] J.-P. Drécourt, H. Madsen, and D. Rosbjerg. Bias aware kalman filters: Comparison and improvements. Advances in Water Resources, 29(5):707 – 718,
2006.
[48] M. S. Dutra, A. C. de Pina Filho, and V. F. Romano. Modeling of a bipedal
locomotor using coupled nonlinear oscillators of van der pol. Biological Cybernetics, 88:286–292, 2003. 10.1007/s00422-002-0380-8.
[49] J. Duysens and K. G. Pearson. The role of cutaneous afferents from the distal
hindlimb in the regulation of the step cycle of thalamic cats. Experimental
Brain Research, 24:245–255, 1976. 10.1007/BF00235013.
219
[50] J. Duysens and H. van de Crommert. Neural control of locomotion; the central
pattern generator from cats to humans. Gate and Posture, 7(2):251–263, 1998.
[51] J. Duysens, H. Van de Crommert, B. Smits-Engelsman, and F. Van der Helm.
A walking robot called human: lessons to be learned from neural control of
locomotion. Journal of Biomechanics, 35:447–453(7), April 2002.
[52] r. Ekeberg and K. Pearson. Computer simulation of stepping in the hind legs
of the cat: An examination of mechanisms regulating the stance-to-swing transition. Journal of Neurophysiology, 94(6):4256–4268, 2005.
[53] G. Endo, J. Morimoto, T. Matsubara, J. Nakanishi, and G. Cheng. Learning
cpg-based biped locomotion with a policy gradient method: Application to a
humanoid robot. The International Journal of Robotics Research, 27(2):213–
228, 2008.
[54] G. Endo, J. Morimoto, J. Nakanishi, and G. Cheng. An empirical exploration
of a neural oscillator for biped locomotion control. In Robotics and Automation, 2004. Proceedings. ICRA ’04. 2004 IEEE International Conference on,
volume 3, pages 3036 – 3042 Vol.3, april-1 may 2004.
[55] G. T. Fallis. Walking toy, 1888.
[56] H. Forssberg. Ontogeny of human locomotor control i. infant stepping, supported locomotion and transition to independent locomotion. Experimental
Brain Research, 57:480–493, 1985. 10.1007/BF00237835.
[57] B. Friedland. Treatment of bias in recursive filtering. IEEE Transaction on
Automatic Control, 14(4):359–367, 1969.
[58] A. Frigon and S. Rossignol.
Experiments and models of sensorimotor
interactions during locomotion. Biological Cybernetics, 95:607–627, 2006.
10.1007/s00422-006-0129-x.
[59] A. Frigon, J. Sirois, and J.-P. Gossard. Effects of ankle and hip muscle afferent
inputs on rhythm generation during fictive locomotion. Journal of Neurophysiology, 103(3):1591–1605, 2010.
[60] Y. Fukuoka, H. Kimura, and A. H. Cohen. Adaptive dynamic walking of a
quadruped robot on irregular terrain based on biological concepts. The International Journal of Robotics Research, 22(3-4):187–202, 2003.
[61] R. Full and D. Koditschek. Templates and anchors: neuromechanical hypotheses of legged locomotion on land. Journal of Experimental Biology,
202(23):3325–3332, 1999.
220
[62] L. Futek Advanced Sensor Technology. FUTEK Model FBB300 Planar Beam
Force Sensor.
[63] J. C. Gallagher, R. D. Beer, K. S. Espenschied, and R. D. Quinn. Application of
evolved locomotion controllers to a hexapod robot. Robotics and Autonomous
Systems, 19(1):95 – 103, 1996.
[64] M. Garcia, A. Chatterjee, and A. Ruina. Speed, efficiency, and stability of
small-slope 2d passive dynamic bipedal walking. In Robotics and Automation,
1998. Proceedings. 1998 IEEE International Conference on, volume 3, pages
2351 –2356 vol.3, may 1998.
[65] M. Garcia, A. Chatterjee, A. Ruina, and M. Coleman. The simplest walking
model: Stability, complexity, and scaling. Journal of Biomechanical Engineering, 120(2):281–288, 1998.
[66] T. V. Gelder. What might cognition be, if not computation?
Philosophy, 92(7):345–381, 1995.
Journal of
[67] T. Geng, B. Porr, and F. Wrgtter. A reflexive neural network for dynamic biped
walking control. Neural Computation, 18(5):1156–1196, 2006.
[68] H. Geyer and H. Herr. A muscle-reflex model that encodes principles of legged
mechanics produces human walking dynamics and muscle activities. Neural
Systems and Rehabilitation Engineering, IEEE Transactions on, 18(3):263 –
273, june 2010.
[69] H. Geyer, A. Seyfarth, and R. Blickhan. Positive force feedback in bouncing
gaits? Proceedings of the Royal Society of London. Series B: Biological Sciences,
270(1529):2173–2183, 2003.
[70] J. J. Gibson. The Ecological Approach To Visual Perception. Psychology Press,
new edition edition, September 1986.
[71] C. Giuliani and J. Smith. Stepping behaviors in chronic spinal cats with one
hindlimb deafferented. The Journal of Neuroscience, 7(8):2537–2546, 1987.
[72] D. Goldberg. Genetic algorithms in search, optimization and machine learning.
Addison-Wesley, 1989.
[73] M. E. Goldberger. Locomotor recovery after unilateral hindlimb deafferentation
in cats. Brain Research, 123(1):59 – 74, 1977.
[74] S. Gosgnach, J. Quevedo, B. Fedirchuk, and D. A. McCrea. Depression of group
ia monosynaptic epsps in cat hindlimb motoneurones during fictive locomotion.
The Journal of Physiology, 526(3):639–652, 2000.
221
[75] J. P. Gossard. Control of transmission in muscle group ia afferents during fictive
locomotion in the cat. Journal of Neurophysiology, 76(6):4104–4112, 1996.
[76] A. Goswami, B. Espiau, and A. Keramane. Limit cycles and their stability in
a passive bipedal gait. In Robotics and Automation, 1996. Proceedings., 1996
IEEE International Conference on, volume 1, pages 246 –251 vol.1, Apr 1996.
[77] S. Grillner. Biological pattern generation: The cellular and computational logic
of networks in motion. Neuron, 52:751– 766, 2006.
[78] S. Grillner, T. Deliagina, A. E. Manira, R. H. Hill, G. N. Orlovsky, P. Walln,
. Ekeberg, and A. Lansner. Neural networks that co-ordinate locomotion and
body orientation in lamprey. Trends in Neurosciences, 18(6):270 – 279, 1995.
[79] S. Grillner and P. Zanger. On the central generation of locomotion in the low
spinal cat. Experimental Brain Research, 34(2):1432–1106, 1979.
[80] S. Grillner and P. Zangger. The effect of dorsal root transection on the efferent motor pattern in the cat’s hindlimb during locomotion. Acta Physiologica
Scandinavica, 120(3):393–405, 1984.
[81] P. Guertin, M. J. Angel, M. C. Perreault, and D. A. McCrea. Ankle extensor group i afferents excite extensors throughout the hindlimb during fictive
locomotion in the cat. The Journal of Physiology, 487(Pt 1):197–209, 1995.
[82] R. M. Harris-Warrick and A. H. Cohen. Serotonin modulates the central paterrn generator for locomotion in the isolated lamprey spinal cord. Journal of
Experimental Biology, 116:27–46, 1985.
[83] D. O. Hebb. The Organization of Behavior: A Neuropsychological Theory.
Wiley, New York, new edition edition, June 1949.
[84] R. Heliot and B. Espiau. Multisensor input for cpg-based sensory—motor coordination. Robotics, IEEE Transactions on, 24(1):191 –195, feb. 2008.
[85] H. Hemami and R. Farnsworth. Postural and gait stability of a planar five link
biped by simulation. Automatic Control, IEEE Transactions on, 22(3):452 –
458, jun 1977.
[86] K. Hirai, M. Hirose, Y. Haikawa, and T. Takenaka. The development of the
honda humanoid robot. In Internatioanl Conference on Robotics and Automation, pages 1321–1326, 1998.
222
[87] D. Hobbelen, T. de Boer, and M. Wisse. System overview of bipedal robots
flame and tulip: Tailor-made for limit cycle walking. In Intelligent Robots and
Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, pages 2486
–2491, sept. 2008.
[88] D. G. E. Hobbelen and M. Wisse. Limit cycle walking. In M. Hackel, editor,
Humanoid Robots, Human-like Machines, pages 277–294. InTech, 2007.
[89] J. K. Hodgins and M. H. Raibert. Biped gymnastics. The International Journal
of Robotics Research, 9(2):115–128, 1990.
[90] A. L. Hodgkin and A. F. Huxley. A quantitative description of membrane
current and its application to conduction and excitation in nerve. The Journal
of Physiology, 117(4):500–544, 1952.
[91] P. Holmes, R. J. Full, D. Koditschek, and J. Guckenheimer. The dynamics of
legged locomotion: Models, analyses, and challenges. SIAM Review, 42(2):207–
304, 2006.
[92] E. Hopf. The partial differential equation ut + uux = µxx . Communications on
Pure and Applied Mathematics, 3(3):201–230, 1950.
[93] F. B. Horak and J. M. Macpherson. Postural Orientation and Equilibrium.
Oxford University Press, 1996.
[94] Q. Huang and Y. Nakamura. Sensory reflex control for humanoid walking.
Robotics, IEEE Transactions on, 21(5):977 – 984, oct. 2005.
[95] Y. Hurmuzlu, F. Gnot, and B. Brogliato. Modeling, stability and control of
biped robots–a general framework. Automatica, 40(10):1647 – 1664, 2004.
[96] J. Hurst, J. Chestnutt, and A. Rizzi. Design and philosophy of the bimasc, a
highly dynamic biped. In Robotics and Automation, 2007 IEEE International
Conference on, pages 1863 –1868, april 2007.
[97] F. Iida, J. Rummel, and A. Seyfarth. Bipedal walking and running with springlike biarticular muscles. Journal of Biomechanics, 41(3):656–667, 2008.
[98] A. J. Ijspeert. A connectionist central pattern generator for the aquatic and
terrestrial gaits of a simulated salamander. Biological Cybernetics, 84:331–348,
2001. 10.1007/s004220000211.
[99] A. J. Ijspeert, A. Crespi, D. Ryczko, and J.-M. Cabelguen. From swimming
to walking with a salamander robot driven by a spinal cord model. Science,
315(5817):1416–1420, 2007.
223
[100] A. Ishiguro, A. Fujii, and P. E. Hotz. Neuromodulated control of bipedal locomotion using a polymorphic cpg circuit. Adaptive Behavior, 11(1):7–17, 2003.
[101] D. Ivashko, B. Prilutsky, S. Markin, J. Chapin, and I. Rybak. Modeling the
spinal cord neural circuitry controlling cat hindlimb movement during locomotion. Neurocomputing, 52-54:621 – 629, 2003. Computational Neuroscience:
Trends in Research 2003.
[102] E. M. Izhikevich. Simple model of spiking neurons. IEEE Transactions on
Neural Networks, 14:1569–1572, 2003.
[103] J. J. Jeka, L. Allison, and T. Kiemel. The dynamics of visual reweighting in
healthy and fall-prone older adults. Journal of Motor Behavior, 42(4):197–208,
2010.
[104] S. Kajita, F. Kanehiro, K. Kaneko, K. Fujiwara, K. Harada, K. Yokoi, and
H. Hirukawa. Biped walking pattern generation by using preview control of zeromoment point. In IEEE International Conference on Robotics and Automation,
pages 1620–1626, 2003.
[105] S. Kajita, M. Morisawa, K. Harada, K. Kaneko, F. Kanehiro, K. Fujiwara, and
H. Hirukawa. Biped walking pattern generator allowing auxiliary zmp control.
In Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference
on, pages 2993 –2999, oct. 2006.
[106] R. Kalman. A new approach to linear filtering and prediction problems. Transactions of the ASME - Journal of Basic Engineering, 82:35–45, 1960.
[107] I. Kato. Information-power machine with senses and limbs (wabot 1). First
CISM-IFToMM Symp. on Theory and Practice of Robots and Manipulators,
1:11–24, 1974.
[108] T. Kiemel, K. S. Oie, and J. Jeka. Multisensor fusion and the stochastic structure of postural sway. Biological Cybernetics, 87(4):262–277, 2002.
[109] H. Kimura, S. Akiyama, and K. Sakumara. Realization of dynamic walking
and running of the quadruped using neural oscillators. Autonomous Robots,
7(3):247–258, 1999.
[110] H. Kimura, Y. Fukuoka, and A. H. Cohen. Adaptive dynamic walking of a
quadruped robot on natural ground based on biological concepts. The International Journal of Robotics Research, 26(5):475–490, 2007.
[111] S. Kimura, M. Yano, and H. Shimizu. A self-organizing model of walking patterns of insects. Biological Cybernetics, 69(3):183–193, 1993.
224
[112] T. Komatsu and M. Usui. Dynamic walking and running of a bipedal robot using
hybrid central pattern generator method. In Mechatronics and Automation,
2005 IEEE International Conference, volume 2, pages 987 – 992 Vol. 2, july-1
aug. 2005.
[113] K. P. Körding and D. M. Wolpert. Bayesian integration in sensorimotor learning. Nature, 427(6971):244–247, Jan. 2004.
[114] K. P. Körding and D. M. Wolpert. Bayesian decision theory in sensorimotor
control. Trends in Cognitive Sciences, 10(7):319 – 326, 2006. Special issue:
Probabilistic models of cognition.
[115] D. J. Kriellaars, R. M. Brownstone, B. R. Noga, and L. M. Jordan. Mechanical
entrainment of fictive locomotion in the decerebrate cat. Journal of Neurophysiology, 71(6):2074–2086, 1994.
[116] A. D. Kuo. The relative roles of feedforward and feedback in the control of
rhythmic movement. Motor Control, 6(2):129–145, 2002.
[117] M. Lafreniere-Roula and D. A. McCrea. Deletions of rhythmic motoneuron
activity during fictive locomotion and scratch provide clues to the organization of the mammalian central pattern generator. Journal of Neurophysiology,
94(2):1120–1132, 2005.
[118] D. Lee. The optic flow field: the foundation of vision. Philosophical Transaction
of the Royal Society of London, 290(1038):169–179, 1980.
[119] M. A. Lewis. Detecting surface features during locomotion using optic flow. In
International Conference on Robotics and Automation, 2002.
[120] M. A. Lewis and G. A. Bekey. Gait adaptation in a quadruped robot. Autonomous Robots, 12:301–312, 2002. 10.1023/A:1015221832567.
[121] M. A. Lewis, R. Etienne-Cummings, M. Hartmann, Z. R. Xu, and A. H. Cohen.
An in silico central pattern generator: Oscillator, entrainment, motor neuron
adaptation and biped mechanism control. Biological Cybernetics, 88(2):137–
151, 2003.
[122] M. A. Lewis, A. H. Fagg, and G. A. Bekey. Genetic algorithms for gait synthesis
in a hexapod robot. In Y. F. Zheng, editor, Recent Trends in Mobile Robotics,
pages 317–331. World Scientific, New Jersey, 1994.
[123] M. A. Lewis, F. Tenore, and R. Etienne-Cummings. Cpg design using inhibitory
networks. In IEEE International Conference on Robotics and Automation, pages
3682–3687, 2005.
225
[124] G. E. Loeb. Learning from the spinal cord.
533(1):111–117, 2001.
The Journal of Physiology,
[125] W. P. Lombard and F. M. Abbott. The mechanical effects produced by the
contraction of individual muscles of the thigh of the frog. American Journal of
Physiology – Legacy Content, 20(1):1–60, 1907.
[126] B. Lucas and T. Kanade. An iterative image registration technique with an
application to stereo vision. In Proceedings of Image Understanding Workshop,
pages 121–130, 1981.
[127] A. Lundberg. Half-centres revisited. In Regulatory Functions of the CNS. Motion and Organization Principles., page 155167. Pergamon Akademiai Kiado,
Adv. Physiol. Sci., Budapest, Hungary, 1981.
[128] M. Lungarella and L. Berthouze. Robot bouncing: On the synergy between
neural and body-environment dynamics. In F. Iida, R. Pfeifer, L. Steels, and
Y. Kuniyoshi, editors, Embodied Artificial Intelligence, volume 3139 of Lecture
Notes in Computer Science, pages 629–629. Springer Berlin / Heidelberg, 2004.
[129] J. N. MacLean and B. J. Schmidt. Voltage-sensitivity of motoneuron nmda
receptor channels is modulated by serotonin in the neonatal rat spinal cord.
Journal of Neurophysiology, 86(3):1131–1138, 2001.
[130] A. Mahboobin, P. J. Loughlin, M. S. Redfern, S. O. Anderson, C. G. Atkeson, and J. K. Hodgkins. Sensory adaptation in balance control: Lessons for
biomimetic robotic bipeds. Neural Networks, 21(4):621–627, 2008.
[131] P. Manoonpong, T. Geng, B. Porr, and F. Worgotter. The runbot architecture
for adaptive, fast, dynamic walking. In Circuits and Systems, 2007. ISCAS
2007. IEEE International Symposium on, pages 1181 –1184, may 2007.
[132] E. Marder and D. Bucher. Central pattern generators and the control of rhythmic movements. Current Biology, 11(23):R986–R996, 2001.
[133] D. Marr. A theory of cerebellar cortex. Journal of Physiology, 202:437–470,
1968.
[134] K. Matsuoka. Sustained oscillations generated by mutually inhibiting neurons
with adaptation. Biological Cybernetics, 52(6):367–376, 1985.
[135] K. Matsuoka. Mechanisms of frequency and pattern control in the neural
rhythm generators. Biological Cybernetics, 56(5-6):345–353, 1987.
226
[136] T. Matsuraba, J. Morimoto, J. Nakanishi, M. Saro, and K. Doya. Learning
cpg-based biped locomotion with a policy gradient method. Robotics and Autonomous Systems, 54:911–920, 2006.
[137] J. Mccarthy and P. J. Hayes. Some Philosophical Problems from the Standpoint of Artificial Intelligence. In B. Meltzer and D. Michie, editors, Machine
Intelligence 4, pages 463–502. Edinburgh University Press, 1969.
[138] D. A. McCrea. Spinal circuitry of sensorimotor control of locomotion. The
Journal of Physiology, 533(1):41–50, 2001.
[139] D. A. McCrea and I. A. Rybak. Organization of mammalian locomotor rhythm
and pattern generation. Brain Research Reviews, 57(1):134 – 146, 2008.
[140] W. McCulloch and W. Pitts. A logical calculus of the ideas immanent
in nervous activity. Bulletin of Mathematical Biology, 5:115–133, 1943.
10.1007/BF02478259.
[141] T. McGeer. Passive dynamic walking. International Journal of Robotics Research, 9(2):62–82, 1990.
[142] T. McGeer. Passive walking with knees. In Robotics and Automation, 1990.
Proceedings., 1990 IEEE International Conference on, pages 1640 –1645 vol.3,
may 1990.
[143] R. K. Mehra. On the identification of variances and adaptive kalman filtering.
IEEE Transactions on Automatic Control, 15(2):175–184, 1970.
[144] T. Mergner, C. Maurer, and R. Peterka. A multisensory posture control model
of human upright stance. In Y. R. C. Prablanc, D. Pelisson, editor, Neural
Control of Space Coding and Action Production, volume 142 of Progress in
Brain Research, pages 189 – 201. Elsevier, 2003.
[145] T. Mergner, G. Schweigart, and L. Fennell. Vestibular humanoid postural control. Journal of Physiology-Paris, 103(3-5):178 – 194, 2009.
[146] H. Michalska and D. Mayne. Receding horizon control of nonlinear systems.
In Decision and Control, 1989., Proceedings of the 28th IEEE Conference on,
pages 107 –108 vol.1, dec 1989.
[147] M. Minsky and P. Seymour. Perceptrons. M.I.T. Press., Cambridge, MA, 1969.
[148] A. Mnard, H. Leblond, and J.-P. Gossard. Sensory integration in presynaptic
inhibitory pathways during fictive locomotion in the cat. Journal of Neurophysiology, 88(1):163–171, 2002.
227
[149] S. Mochon and T. A. McMahon. Ballistic walking: an improved model. Mathematical Biosciences, 52(3-4):241 – 260, 1980.
[150] M. Molinari. Plasticity properties of cpg circuits in humans: Impact on gait
recovery. Brain Research Bulletin, 78(1):22 – 25, 2009. Special Issue: Gait
recovery after spinal cord injury. From bench to bed.
[151] T. Mori, Y. Nakamura, M.-A. Sato, and S. Ishii. Reinforcement learning for
a cpg-driven biped robot. In Proceedings of the 19th national conference on
Artifical intelligence, AAAI’04, pages 623–630. AAAI Press, 2004.
[152] J. Morimoto, G. Cheng, C. Atkeson, and G. Zeglin. A simple reinforcement
learning algorithm for biped walking. In Robotics and Automation, 2004. Proceedings. ICRA ’04. 2004 IEEE International Conference on, volume 3, pages
3030 – 3035 Vol.3, april-1 may 2004.
[153] J. Morimoto, G. Endo, J. Nakanishi, S. Hyon, G. Cheng, D. Bentivegna, and
C. Atkeson. Modulation of simple sinusoidal patterns by a coupled oscillator
model for biped walking. In Robotics and Automation, 2006. ICRA 2006. Proceedings 2006 IEEE International Conference on, pages 1579 –1584, may 2006.
[154] K. A. Myers and B. D. Tapley. Adaptive sequential estimation with unknown
noise statistics. IEEE Transaction on Automatic Control, 21(4):520–523, 1976.
[155] Y. Nakamura, T. Mori, M. Sato, and S. Ishii. Reinforcement learning for a
biped robot based on a cpg-actor-critic method. Neural Networks, 20(6):723 –
735, 2007.
[156] J. Nakanishi, J. Morimoto, G. Endo, G. Cheng, S. Schaal, and M. Kawato.
Learning from demonstration and adaptation of biped locomotion. Robotics
and Autonomous Systems, 47(2-3):79 – 91, 2004.
[157] M. Narici, G. Roi, and L. Landoni. Force of knee extensor and flexor muscles
and cross-sectional area determined by nuclear magnetic resonance imaging.
European journal of applied physiology and occupational physiology, 57(1), 1988.
[158] A. Newell and H. Simon. The logic theory machine–a complex information
processing system. Information Theory, IRE Transactions on, 2(3):61 –79,
september 1956.
[159] N. Ogihara and N. Yamazaki. Generation of human bipedal locomotion by
a bio-mimetic neuro-musculo-skeletal model. Biological Cybernetics, 84:1–11,
2001.
228
[160] K. Oie, S. Carver, T. Kiemel, J. Barela, and J. Jeka. The dynamics of sensory
reweighting: a temporal asymmetry. Gait and Posture, 21(1):S29, 2005.
[161] K. S. Oie, T. Kiemel, and J. Jeka. Mulisensory fusion: simultaneous reweighting of vision and touch for the control of human posture. Cognitive
Brain Research, 14(1):164–167, 2002.
[162] M. Oussalah and J. D. Schutter. Adaptive kalman filter for noise identification. In Proceedings of the International Conference on Noise and Vibration
Engineering, pages 1225–1232, 2000.
[163] C. Paul, M. Bellotti, S. Jezernik, and A. Curt. Development of a human neuromusculo-skeletal model for investigation of spinal cord injury. Biological Cybernetics, 93:153–170, 2005. 10.1007/s00422-005-0559-x.
[164] K. Pearson and J. Gordon. Spinal reflexes. In E. R. Kandel, J. H. Schwartz,
and T. M. Jessel, editors, Principles of Neural Science. McGraw-Hill, New York,
fourth edition, 2000.
[165] K. Pearson, rjan Ekeberg, and A. Bschges. Assessing sensory function in locomotor systems using neuro-mechanical simulations. Trends in Neurosciences,
29(11):625 – 631, 2006.
[166] K. G. Pearson. Common principles of motor control in vertebrates and invertebrates. Annual Review of Neuroscience, 16(1):265–297, 1993.
[167] K. G. Pearson. Proprioceptive regulation of locomotion. Current Opinion in
Neurobiology, 5(6):786 – 791, 1995.
[168] K. G. Pearson and D. F. Collins. Reversal of the influence of group ib afferents
from plantaris on activity in medial gastrocnemius muscle during locomotor
activity. Journal of Neurophysiology, 70(3):1009–1017, 1993.
[169] R. Peterka. Sensorimotor integration in human postural control. Journal of
Neurophysiology, 88(3):1097–1118, 2002.
[170] R. Peterka and M. Benolken. Role of somatosensory and vestibular cues in
attenuating visually induced postural sway. Experimental Brain Research,
105(1):101–110, 1995.
[171] R. J. Peterka and P. J. Loughlin. Dynamic regulation of sensorimotor integration in human postural control. Journal of Neurophysiology, 91(1):410–423,
2004.
229
[172] M. Phillipson. Lautonomie et la centralization dans le systeme nerveux des
animaux. Travails du Laboratoire de la Physiologie, Institut Solvey, 7:1208,
1905.
[173] C. Pias. Analog, digital, and the cybernetic illusion. Kybernetes, 34:543 – 550,
2005.
[174] J. Pratt, C.-M. Chew, A. Torres, P. Dilworth, and G. Pratt. Virtual model
control: An intuitive approach for bipedal locomotion. International Journal
of Robotics Research, 20(2):129–143, 2001.
[175] J. Pratt and G. Pratt. Intuitive control of a planar bipedal walking robot. In
Robotics and Automation, 1998. Proceedings. 1998 IEEE International Conference on, volume 3, pages 2014 –2021 vol.3, may 1998.
[176] S. D. Prentice, A. E. Patla, and D. A. Stacey. Simple artificial neural network models can generate basic muscle activity patterns for human locomotion at different speeds. Experimental Brain Research, 123:474–480, 1998.
10.1007/s002210050591.
[177] B. I. Prilutsky and V. M. Zatsiorsky. Tendon action of two-joint muscles: Transfer of mechanical energy between joints during jumping, landing, and running.
Journal of Biomechanics, 27(1):25 – 34, 1994.
[178] B. I. Prilutsky and V. M. Zatsiorsky. Optimization-based models of muscle
coordination. Exercise and Sport Sciences Reviews, 30(1):32–8, 2002.
[179] A. Prochazka. Comparison of natural and artificial control of movement. Rehabilitation Engineering, IEEE Transactions on, 1(1):7 –17, mar 1993.
[180] A. Prochazka. Proprioceptive Feedback and Movement Regulation. John Wiley
& Sons, Inc., 2010.
[181] A. Prochazka, D. Gillard, and D. J. Bennett. Implications of positive feedback
in the control of movement. Journal of Neurophysiology, 77(6):3237–3251, 1997.
[182] T. Prokop, M. Schubert, and W. Berger. Visual influence on human locomotion:
Modulation to changes in optic flow. Experimental Brain Research, 114(1):63–
70, 1997.
[183] U. Proske and S. C. Gandevia. The kinaesthetic senses. The Journal of Physiology, 587(17):4139–4146, 2009.
[184] D. Purves. Neuroscience. Sinauer Associates, Inc, third edition, 2007.
230
[185] T. Reil and P. Husbands. Evolution of central pattern generators for bipedal
walking in a real-time physics environment. Evolutionary Computation, IEEE
Transactions on, 6(2):159 –168, apr 2002.
[186] L. Righetti and A. J. Ijspeert. Programmable central pattern generators: An
application to bipeg locomotion control. In Proceedings of the 2006 IEEE Conference on Robotics and Automation, 2006.
[187] D. Robinson, J. Pratt, D. Paluska, and G. Pratt. Series elastic actuator development for a biomimetic walking robot. In IEEE/ASME International Conference
on Advanced Intelligent Mechatronics, pages 561–568, 1999.
[188] Robotis Co. Ltd. Robotis Dynamixel RX-28 User’s Manual.
[189] S. Rossignol, R. Dubac, and J. Gossard. Dynamic sensorimotor interactions in
locomotion. Physiological Reviews, 86:89–154, 2006.
[190] L. Roussel, C. C. de Wit, and A. Goswami. Generation of energy optimal complete gait cycles for biped robots. In Proc. of the IEEE International Conference
on Robotics and Automation, pages 2036–2041, 1998.
[191] I. A. Rybak, D. G. Ivashko, B. I. Prilutsky, J. Chapin, and M. A. Lewis. Modeling neural control of locomotion: Integration of reflex circuits with cpg. In
Int. Conf. On Artificial Neural Networks, 2002.
[192] I. A. Rybak, K. Stecina, N. A. Shevtsova, and D. A. McCrea. Modelling spinal
circuitry involved in locomotor pattern generation: insights from the effects of
afferent stimulation. The Journal of Physiology, 577(2):641–658, 2006.
[193] T. Saidouni and G. Bessonnet. Generating globally optimised sagittal gait cycles
of a biped robot. Robotica, 21:199–210, March 2003.
[194] Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki, and K. Fujimura. The intelligent asimo: System overview and integration. In IEEE/RSJ
International Conference on Intelligent Robots and Systems, Lausanne, Switzerland, 2002.
[195] E. Schuitema, D. Hobbelen, P. Jonker, M. Wisse, and J. Karssen. Using a
controller based on reinforcement learning for a passive dynamic walking robot.
In Humanoid Robots, 2005 5th IEEE-RAS International Conference on, pages
232 –237, dec. 2005.
[196] A. L. Schwab and M. Wisse. Basin of attraction of the simpliest walking model.
In Proceedings of the International Conference on Noise and Vibration. ASME,
2001.
231
[197] A. Seyfarth, H. Geyer, R. Blickhan, S. Lipfert, J. Rummel, Y. Minekawa, and
F. Iida. Running and walking with compliant legs. In M. Diehl and K. Mombaur,
editors, Fast Motions in Biomechanics and Robotics, volume 340 of Lecture
Notes in Control and Information Sciences, pages 383–401. Springer Berlin /
Heidelberg, 2006.
[198] A. Seyfarth, M. Gnther, and R. Blickhan. Stable operation of an elastic threesegment leg. Biological Cybernetics, 84:365–382, 2001.
[199] D. C. Shapiro, R. F. Zernicke, R. J. Gregor, and J. D. Diestel. Evidence for
generalized motor programs using gait pattern analysis. Journal of Motor Behavior, 13(1):33–47, 1981.
[200] C. S. Sherrington. Integrative Action of the Nervous System. Charles Scribner’s
Sons, 1906.
[201] M. Shik, F. Severin, and G. Orlovskii. Control of walking and running by means
of electrical stimulation of the mid-brain. Biophizika, 11(4):659–66, 1966.
[202] G. Shöner. Dynamic theory of action-perception patterns: the “moving room”
paradigm. Biological Cybernetics, 64(6):455–462, 1991.
[203] P. Smolensky. On the proper treatment of connectionism. Behavioral and Brain
Sciences, 11:1–74, 1988.
[204] J. H. Solomon, M. Wisse, and M. J. Hartmann. Fully interconnected, linear
control for limit cycle walking. Adaptive Behavior, 18(6):492–506, 2010.
[205] Y. Sqalli-Houssaini, J. R. Cazalets, and F. Clarac. Oscillatory properties of the
central pattern generator for locomotion in neonatal rats. Journal of Neurophysiology, 70(2):803–813, 1993.
[206] R. B. Stein, J. E. Misiaszek, and K. G. Pearson. Functional role of muscle
reflexes for force generation in the decerebrate walking cat. The Journal of
Physiology, 525(3):781–791, 2000.
[207] R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning). MIT Press, Cambridge, 1998.
[208] G. Taga. Emergence of bipedal locomotion through entrainment among the
neuro-musculo-skeletal system and the environment. Physica D: Nonlinear Phenomena, 75(1-3):190 – 208, 1994.
[209] G. Taga. A model of the neuro-musculo-skeletal system for human locomotion.
Biological Cybernetics, 73:113–121, 1995. 10.1007/BF00204049.
232
[210] G. Taga. A model of the neuro-musculo-skeletal system for anticipatory adjustment of human locomotion during obstacle avoidance. Biological Cybernetics,
78:9–17, 1998. 10.1007/s004220050408.
[211] G. Taga, Y. Yamaguchi, and H. Shimizu. Self-organized control of bipedal
locomotion by neural oscillators in unpredictable environment. Biological Cybernetics, 65(3):147–159, 1991.
[212] F. Tan, C. Fu, and K. Chen. Biped blind walking on changing slope with reflex
control system. In Robotics and Automation (ICRA), 2010 IEEE International
Conference on, pages 1709 –1714, may 2010.
[213] R. Tedrake, T. Zhang, M. fai Fong, and H. Seung. Actuating a simple 3d
passive dynamic walker. In Robotics and Automation, 2004. Proceedings. ICRA
’04. 2004 IEEE International Conference on, volume 5, pages 4656 – 4661 Vol.5,
april-1 may 2004.
[214] R. Tedrake, T. Zhang, and H. Seung. Stochastic policy gradient reinforcement
learning on a simple 3d biped. In Intelligent Robots and Systems, 2004. (IROS
2004). Proceedings. 2004 IEEE/RSJ International Conference on, volume 3,
pages 2849 – 2854 vol.3, sept.-2 oct. 2004.
[215] E. Thelen and D. W. Cooke. Relationship between newborn stepping and later
walking: A new interpretation. Developmental Medicine & Child Neurology,
29(3):380–393, 1987.
[216] E. Thelen, K. D. Scala, and J. S. Kelso. The dynamic nature of early coordination: Evidence from bilateral leg movements in young infants. Developmental
Psychology, 23(2):179–186, 1987.
[217] E. Thelen and L. Smith. A Dynamic Systems Approach to the Development of
Cognition and Action. Journal of Cognitive Neuroscience, 7(4):512–514, Oct.
1995.
[218] E. Tytell, P. Holmes, and A. Cohen. Spikes alone do not behavior make: why
neuroscience needs biomechanics. Current Opinion in Neurobiology, In Press,
Corrected Proof:–, 2011.
[219] H. W. A. A. Van de Crommert, T. Mulder, and J. Duysens. Neural control of
locomotion: sensory control of the central pattern generator and its relation to
treadmill training. Gait and Posture, 7:251–263(13), 1998.
[220] H. van der Kooij, R. Jacobs, B. Koopman, and F. van der Helm. An adaptive
model of sensory integration in a dynamic environment applied to human stance
control. Biological Cybernetics, 84(2):103–115, 2001.
233
[221] H. van der Kooij, R. Jacobs, B. Koopman, and F. van der Helm. An alternative
approach to synthesizing bipedal walking. Biological Cybernetics, 88:46–59,
2003. 10.1007/s00422-002-0330-5.
[222] R. Q. van der Linde. Passive bipedal walking with phasic muscle contraction.
Biological Cybernetics, 81:227–237, 1999. 10.1007/s004220050558.
[223] B. van der Pol. Forced oscillations in a circuit with non-linear resistance. Philosophical Magazine Series, 3:65–80, 1927.
[224] E. D. Vaughan, E. D. Paolo, and I. R. Harvey. The evolution of control and
adaptation in a 3d powered passive dynamic walker. In Artificial Life IX: Proceedings of the Ninth International Conference on the Simulation and Synthesis
of Living Systems. MIT Press, 2004.
[225] B. Verdaasdonk, H. Koopman, and F. Van Der Helm. Energy efficient and
robust rhythmic limb movement by central pattern generators. Neural Networks,
19(4):388–400, 2006.
[226] B. Verdaasdonk, H. Koopman, and F. Van Der Helm. Energy efficient walking
with central pattern generators: from passive dynamic walking to biologically
inspired control. Biological Cybernetics, 101(1):49–61, 2009.
[227] M. Vukobratovic and B. Borovac. Zero-movement point - thirty-five years of
it’s life. International Journal of Humanoid Robotics, 1(1):157–173, 2004.
[228] M. Vukobratovic, A. A. Frank, and D. Juricic. On the stability of biped locomotion. Biomedical Engineering, IEEE Transactions on, BME-17(1):25 –36,
Jan. 1970.
[229] T. Wadden and r. Ekeberg.
comotion: single leg control.
10.1007/s004220050468.
A neuro-mechanical model of legged loBiological Cybernetics, 79:161–173, 1998.
[230] W. G. Walter. An imitation of life. Scientific American, 182(5):42–45, 1950.
[231] W. H. Warren Jr., B. A. Kay, W. D. Zosh, A. P. Duchon, and S. Sahuc. Optic
flow is used to control human walking. Nature Neuroscience, 4(2):213–216,
2001.
[232] B. Webb. Can robots make good models of biological behaviour? Behavioral
and Brain Sciences, 24:10331050, 2001.
[233] P. J. Whelan. Shining light into the black box of spinal locomotor networks. Philosophical Transactions of the Royal Society B: Biological Sciences,
365(1551):2383–2395, 2010.
234
[234] P.-B. Wieber. Trajectory Free Linear Model Predictive Control for Stable Walking in the Presence of Strong Perturbations. In IEEE-RAS International Conference on Humanoid Robots, Genova, Italie, 2006.
[235] U. Windhorst. Muscle proprioceptive feedback and spinal networks. Brain
Research Bulletin, 73(4-6):155 – 202, 2007.
[236] D. A. Winter. Biomechanics and Motor Control of Human Movement. John
Wiley and Sons, Inc., New York, 1990.
[237] M. Wisse, D. Hobbelen, R. Rotteveel, S. Anderson, and G. Zeglin. Ankle springs
instead of arc-shaped feet for passive dynamic walkers. In Humanoid Robots,
2006 6th IEEE-RAS International Conference on, pages 110 –116, dec. 2006.
[238] M. Wisse and J. van Frankenhuyzen. Design and construction of mike: a 2d
autonomous biped based on passive dynamic walking. In Adaptive motion of
animals and machines, page 143. Springer, 2006.
[239] D. M. Wolpert, R. Miall, and M. Kawato. Internal models in the cerebellum.
Trends in Cognitive Sciences, 2(9):338 – 347, 1998.
[240] S. Yakovenko, V. Gritsenko, and A. Prochazka. Contribution of stretch reflexes
to locomotor control: a modeling study. Biological Cybernetics, 90:146–155,
2004. 10.1007/s00422-003-0449-z.
[241] J. F. Yang, M. J. Stephens, and R. Vishram. Infant stepping: a method to study
the sensory control of human walking. The Journal of Physiology, 507(3):927–
937, 1998.
[242] K. Yin, K. Loken, and M. van de Panne. Simbicon: simple biped locomotion
control. In ACM SIGGRAPH 2007 papers, SIGGRAPH ’07, New York, NY,
USA, 2007. ACM.
[243] F. E. Zajac, R. R. Neptune, and S. A. Kautz. Biomechanics and muscle coordination of human walking: Part i: Introduction to concepts, power transfer,
dynamics and simulations. Gait & Posture, 16(3):215 – 232, 2002.
[244] E. P. Zehr and R. B. Stein. What functions do reflexes serve during human
locomotion? Progress in Neurobiology, 58(2):185 – 205, 1999.
[245] T. Zielinska. Coupled oscillators utilised as gait rhythm generators of
a two-legged walking machine. Biological Cybernetics, 74:263–273, 1996.
10.1007/BF00652227.
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement