IHMC- Quarterly Report
Jan - March 2006
1) Progress Against Planned Objectives
Our planned objectives from the previous reporting period and their
status is as follows:
a) Finish the Version1 algorithm and test it on various
Terrains. Done. Version1 algorithm was finished and tested on various
terrains with holes, steps, and rolling terrain.
b) Start the Version2 algorithm and incorporate some of our
development goals. Version2 has been started and some of the goals
accomplished. We are still finishing the development of Version2.
c) Develop a hierarchical learning algorithm for a Footstep Generator
over two-dimensional Terrain in which a “swath” is first
determined using a low-resolution map of the Terrain and then the
Footstep sequence is determined inside that swath using a
high-resolution map of the Terrain. Still under development.
d) Use Reinforcement Learning techniques in the Stance
Generator. Done. We implemented a Dynamic Programming Stance Generator
that seems to work quite well.
e) Develop a control algorithm that does not use traditional stiff
trajectory tracking, but instead uses more compliant force-control
methods. Think about the feasibility of using such methods on the
LittleDog robot, which does not have force-controllable actuators. We
have developed the mathematics for such a technique and tested it in
simulation and on the real robot. In simulation it works very well. On
the real robot, as expected, it doesn't work so well due to joint
friction. However, in some cases it seems to work ok and we are now
investigating the potential of using a combination of force control
and position control in further algorithms.
f) Develop a Trot-Walking algorithm for fast walking on smooth
Terrain. Still under development. We have significantly sped up our
crawl gait walk and will continue working on a trot gait to increase
the speed.
g) Develop user input methods for selecting Footsteps by clicking on
them in the Simulation GUI. Still under development. We have
implemented some other GUI functionality, such as viewing the
kinematically reachable area of the center of mass, given the footstep
locations and the use of a MIDI slider panel for inputting user input
to the algorithm.
h) Continue the development of our Reinforcement Learning Toolbox. The
Reinforcement Learning toolbox, particularly the Dynamic Programming
tools are now quite stable and work quite well. Since DP has been
working so well for us, we have not required the use of the other
methods too much so far.
i) Get the Simulated LittleDog to walk over fairly rough
two-dimensional Terrain. Under development.
j) Add noise to the simulations. We have not done this yet, since the
ground springiness adds a sort of noise and we are just starting to
look into what we can expect from the LittleDog and Motion Capture
sensors.
k) Develop a methodology for measuring the performance of a given
algorithm. Right now we are using the government testing formula for
measuring performance, but may revisit this later.
l) Acquire our LittleDogRobot and get it working. Done.
-----------------------------------------------------------------------------
2) Technical Accomplishments
A) Summary: During this period, we developed a total Center of Mass
calculator for the LittleDog; developed an iteratve Inverse Kinematic
method for positioning the Center of Mass (as opposed to the Center of
Body); developed a kinematic reachability and position plotter;
finished our Version1 simulation algorithm and got the simulated
LittleDog to walk over rolling terrain, steps, and gaps; acquired our
LittleDog robot and Motion Capture system; set up the truss system and
cameras for the Motion Capture system; reworked the forward and
inverse kinematics for the new LittleDog configuration; developed a
fast reactive crawl gait walking algorithm; and implemented the
low-level software for using our development environment, software
tools, and GUIs with the real robot.
B) Center of Mass calculator and iterative Inverse Kinematics Algorithm
The center of mass of the LittleDog robot is a function of the mass of
each link and the joint angles of the joints. We derived, implemented,
and tested the math for calculating the COM relative to the center of
the body. For stability of the robot, it is important to be able to
position the COM relative to the feet. However, the joint angles are
needed to determine the location of the COM and the joint angles are
unknown. Therefore, we developed an iterative inverse kinematics
procedure: 1. Guess a location of the body center based on the desired
COM location. 2. Calculate the resulting joint angles. 3. Calculate
the resulting COM. 4. Compare the result of step 4 with the desired
COM. If the error is less than a threshold, stop. If not, refine the
guess location of the body center and go back to step 2.
Using this algorithm, we can position the COM within 0.0005 m of the
desired position in typically two iterations.
C) Kinematic Reachability and Position Plotter.
The positions that the center of the body can be moved to are a
function of the foot locations. Given a set of foot locations, the
center of the body must lie in an closed volume of space. To help
visualize this volume we developed a Kinematic Reachability plotter
that determines and graphs this volume given a set of footsteps. For a
footstep sequence, these volumes must overlap so that the robot can
kinematically reach each set of footsteps. Originally, we computed
this volume by computing the Inverse Kinematics for a grid of points
in the volume and checked whether an inverse kinematic solution
existed. However, this check is very computationally expensive so we
have recently been determining much quicker checks for computing which
positions are reachable.
D) Version1 Simulation Algorithm
The main addition to the Version1 simulation algorithm from the
previous reporting period is the inclusion of a Dynamic Programming
Stance Generator. Given a sequence of Footsteps, the Stance Generator
determines a sequence of Stances (body x, y, z, roll, pitch, and
yaw). For this algorithm, we confined the roll and yaw to be zero and
the pitch to match the angle from the front feet to the hind feet. The
height above the ground was set to be a constant height above the
average of the feet locations. x and y were then the state variables
to be optimized. The action vector was the distance to move in x and
in y.
To increase efficiency, we precomputed all of the state-action-reward
transition pairs and stored them in a table. States were pruned that
were not kinematically reachable. Then Dynamic Programming was used on
this table to find the optimal Stances. For a typical terrain, 3
minutes of computation time was required to find a sequence of Stances
over the terrain. The majority of the computation time was required
for determining if a state was kinematically reachable. Once the table
was precomputed, only 3 seconds or so were required to solve for the
optimal. We are currently working on speeding up the test of whether a
Stance is kinematically reachable, and believe we can speed the
Dynamic Programming Stance Generator up by a factor of 10 or so.
We tested the Version1 algorithm on a number of simulated terrains
that included rolling terrain, holes and gaps, and steps. Gap sizes
were on the order of 5cm and steps were on the order of 4cm. The
walking was extremely slow as we were concentrating on static walking
and accurate foot placement. The Dynamic Programming planners were
very good at finding a sequence of steps across the terrain, often
requiring the body to move backward a little in order to allow a foot
to clear a gap. We are very pleased with this Dynamic Programming
algorithm and will continue to push its development on 2D terrain.
E) Fast Reactive Crawl Gait Algorithm
For moderate terrain, we developed a purely reactive static crawl gait
algorithm. This algorithm does not use information about the terrain
to walk. It only uses the internal sensors and foot sensors to walk
over the terrain.
The joints of each leg are controlled by a Cartesian velocity
controller. For each leg, a desired velocity of the foot is
determined. from the desired velocity, desired position, and time
step, the next desired foot position is determined: x_des(k+1) =
x_d(k) + v_des(k) * delta_t
Where x_des(i) is the desired hip to foot position vector at the ith
time step, v_des is the desired velocity vector of the foot, and
delta_t is the control loop time step. Using inverse kinematics, the
joint angles for the leg are determined from x_des.
The desired velocity for the leg depends if it is a stance leg or a
swing leg. For the stance leg, the desired velocity comes from a
series of controllers. The output of each controller is leg
velocities, which are then added together.
a) The stance height controller tries to have the z component of the
hip to foot vector (in body coordinates) equal to a desired value. If
it is not, then it moves the foot in the desired direction. The
velocity that this controller outputs is low, and thus the leg will
drift to the desired position over the course of a second or so;
b)The center of mass (COM) position controller moves the COM to a
position relative to the feet. It does this be determining how the COM
should move, and then moves all of the feet in the opposite
direction. The desired COM position is determined by the triangle
formed by the three feet, with the to be swing leg not included. The
target COM position is adjusted slightly forward or backward depending
on the pitch of the body. Once the COM is in position, the swing leg
starts swinging.
When a leg is swinging, the position of the swing leg is determined to
regulate the stance length and width and the stepping x and y
distances. The X step length and the Y step length are user input
parameters. The X value causes it to move forward and back, and the Y
value causes it to move left and right. The Stance length controller
and the Stance width controller control the spacing of the feet
relative to each other. This makes sure that the feet are spread out
according to user set desired values. When swinging, the leg moves up
by a set amount (step height), then it moves a displacement from its
starting point in the X and Y directions by Xdes and Ydes. When it has
moved this amount, it moves down until the foot switch registers a
contact.
The reactive walker has a fixed gait sequence of: hind right, front
right, hind left, front left, and then back to hind right again. When
the swing leg touches down, the next swing leg is picked from the
list, and a new COM target position is determined. The COM moves to
position, the leg swings, and the process repeats. Using this
controller, we have achieved walking on flat ground and over stairs
with the simulated LittleDog.
F) Low level software for LittleDog robot integration.
To stay consistent with our simulation work and to provide, in our
view, the best development environment, we have developed low level
software for using the same development tools on the LittleDog robot
as we have been for simulation. We have figured out how to develope
code for the robot in both C++ and in Java. We have created base
classes so that the real robot and the simulated robot are fairly
seemlessly integrated. In fact, we now use the exact same control file
on the real robot as on the simulated robot. We are now expanding
these classes to include information from the IMUs and the motion
capture system. We also use the simulation GUI as a GUI for the real
robot. In that way we can see real time plots of data of any variable
while the robot is running, rather than wait for the end of a run to
visualize data. We have also gotten a MIDI slider board to work on the
LittleDog host computer so that we can get user input while the robot
is walking. This is useful for things like tuning parameters on the
fly.
-----------------------------------------------------------------------------
3) Improvements to Prototypes
None.
-----------------------------------------------------------------------------
4) Significant Changes to Technical Approach
None.
-----------------------------------------------------------------------------
5) Planned Activities
a) Finish incorporating the Motion Capture and IMU signals for use in
our control algorithms. Gain familiarity with both systems.
b) Continue investigating compliant force control techniques and how
they can be used on the LittleDog, perhaps in combination with
position control techniques.
c) Develop a Trot-Walking algorithm for fast walking on smooth
Terrain.
d) Get the Simulated LittleDog to walk over fairly rough
two-dimensional Terrain.
e) Get the real LittleDog to walk over increasingly rough terrain.
f) Develop a hierarchical learning algorithm for a Footstep Generator
over two-dimensional Terrain in which a swath is first
determined using a low-resolution map of the Terrain and then the
Footstep sequence is determined inside that swath using a
high-resolution map of the Terrain.
g) Develop user input methods for selecting Footsteps by clicking on
them in the Simulation GUI.
-----------------------------------------------------------------------------
6) Issues or Concerns
None.