Simplifying Tool Usage in Teleoperative Tasks

University of Pennsylvania
ScholarlyCommons
Technical Reports (CIS)
Department of Computer & Information Science
July 1993
Simplifying Tool Usage in Teleoperative Tasks
Thomas Lindsay
University of Pennsylvania
Richard P. Paul
University of Pennsylvania
Follow this and additional works at: http://repository.upenn.edu/cis_reports
Recommended Citation
Thomas Lindsay and Richard P. Paul, "Simplifying Tool Usage in Teleoperative Tasks", . July 1993.
University of Pennsylvania Department of Computer and Information Science Technical Report No. MS-CIS-93-68.
This paper is posted at ScholarlyCommons. http://repository.upenn.edu/cis_reports/495
For more information, please contact libraryrepository@pobox.upenn.edu.
Simplifying Tool Usage in Teleoperative Tasks
Abstract
Modern robotic research has presented the opportunity for enhanced teleoperative systems. Teleprogramming
has been developed for teleoperation in time-delayed environments, but can also lead to increased
productivity in non-delayed teleoperation.
Powered tools are used to increase the abilities of the remote manipulator. However, tools add to the
complexity of the system, both in terms of control and sensing. Teleprogramming can be used to simplify the
operators interaction with the manipulator/tool system. Further, the adaptive sensing algorithm of the remote
site system (using an instrumented compliant wrist for feedback) simplifies the sensory requirements of the
system. Current remote-site implementation of a teleprogramming tool-usage strategy that simplifies tool use
is described in this document.
The use of powered tools in teleoperation tasks is illustrated by two examples, one using an air-powered
impact wrench, and the other using an electric winch. Both of these tools are implemented at our remote site
workcell, consisting of a Puma 560 robot working on the task of removing the top of a large box.
Comments
University of Pennsylvania Department of Computer and Information Science Technical Report No. MSCIS-93-68.
This technical report is available at ScholarlyCommons: http://repository.upenn.edu/cis_reports/495
Simplifying Tool Usage In Teleoperative Tasks
MS-CIS-93-68
GRASP LAB 353
Thomas Lindsay
Richard P. Paul
University of Pennsylvania
School of Engineering and Applied Science
Computer and Information Science Department
Philadelphia, PA 19104-6389
July 1993
Simplifying tool usage in teleoperat ive taskst
Thomas Lindsay and Richard P. Paul
GRASP Laboratory, University of Pennsylvania
3401 Walnut Street, Room 301C, Philadelphia, PA 19104-6228
ABSTRACT
Modern robotic research has presented the opportunity for enhanced teleoperative systems. Teleprogramming
has been developed for teleoperation in time-delayed environments, but can also lead to increased productivity in
non-delayed teleoperation.
Powered tools are used to increase the abilities of the remote manipulator. However, tools add to the complexity
of the system, both in terms of control and sensing. Teleprogramming can be used to simplify the operators
interaction with the manipulator/tool system. Further, the adaptive sensing algorithm of the remote site system
(using an instrumented compliant wrist for feedback) simplifies the sensory requirements of the system. Current
remote-site implementation of a teleprogramming tool-usage strategy that simplifies tool use is described in this
document.
The use of powered tools in teleoperation tasks is illustrated by two examples, one using an air-powered impact
wrench, and the other using an electric winch. Both of these tools are implemented a t our remote site workcell,
consisting of a Puma 560 robot working on the task of removing the top of a large box.
t ~ ~ ~ > eina the
r s Proceedings of Telemanipulator Technology, a session of the SPIEIOE Technology '92 Conference, Boston, M A ,
November 15-20. 1992
Simplifying tool usage in teleoperative taskst
Thomas Lindsay and Richard P. Paul
GRASP Laboratory, University of Pennsylvania
3401 Walnut Street, Room 301C, Philadelphia, PA 1 9 1 0 4 - 6 2 2 8
ABSTRACT
Modern robotic research has presented the opportunity for enhanced teleoperative systems. Teleprogramming
has been developed for teleoperation in time-delayed environments, but can also lead to increased productivity in
non-delayed teleoperation.
Powered tools are used to increase the abilities of the remote manipulator. However, tools add to the complexity
of the system, both in terms of control and sensing. Teleprogramming can be used to simplify the operators
interaction with the manipulator/tool system. Further, the adaptive sensing algorithm of the remote site system
(using an instrumented compliant wrist for feedback) simplifies the sensory requirements of the system. Current
remote-site implementation of a teleprogramming tool-usage strategy that simplifies tool use is described in this
document.
The use of powered tools in teleoperation tasks is illustrated by two examples, one using an air-powered impact
wrench, and the other using an electric winch. Both of these tools are implemented at our remote site workcell,
consisting of a Puma 560 robot working on the task of removing the top of a large box.
INTRODUCTION
Three major elements of teleprogramming [2, 41 (a form of supervisory control [I]) allow potential improvements
over a direct teleoperative system. First, the operator interacts with a virtual remote site, providing opportunities
for artificial operator aids, including visual a n d kinesthetic cues. Second, communication between t h e master and
remote sites is in t h e form of symbols instead of signals, allowing bandwidth limited a n d time-delayed teleoperations,
as ~vellas increasing t h e intelligence of communication between master a n d slave. Finally, t h e remote site executes
successive commands autonomously, allowing the remote site manipulator t o interact more intelligently with the
environment .
A robot manipulator is greatly limited in its strength t o accuracy ratio. For a relatively low-strength robot t o
execute tasks which require more strength, powered tools m a y be used in conjunction with t h e robot. T h e precision
of t h e low-power robot is coupled with t h e strength of less-precise heavy tools. For this research, tools include a n
impact wrench which delivers more torque t h a n the robot, a n d a winch with more lifting power t h a n the payload
capability of the robot.
T h e tool/manipulator system is complex from t h e point of control and sensing. E x t r a degrees of freedom lead to
a r e d u ~ i d a n tsystem, difficult t o control a n d difficult for t h e operator t o operate. Tool usage requires sensing, which
may further complicate the system. However, using teleprogramming, a simplified tool control a n d sensing structure
can be implen~ent~ed.
IVhen a powered tool is selected for use, the teleprogramming command UseTool(too1name) is sent t o the remote
site to select parameters and default conditions specific t o t h a t tool. A task (control) frame is created wherein the
tAppears in the Proceedings of Telemanipulator Technology, a session of the SPIE/OE Technology '92 Conference, Boston, M A ,
November 15-20, 1992
coordinate axes align with the natural degree(s) of freedom of the tool. The tool is then controlled instead of the
corresponding degree of freedom of the robot manipulator, which becomes passive. Control of the tool by the operator
is accomplished simply by motion of the master manipulator in the specific degree of freedom which the tool controls.
The addition of sensors for each and every tool would be difficult from an engineering standpoint and would add
unnecessary complexity t o the system. In our system, the wrist force/torque sensor of the remote robot manipulator
is used for tool sensing. General purpose guarded moves [5], incorporated in the teleprogramming language, are used
to monitor tool actions, and to determine successful tool operation.
The control and sensing structure outlined in this paper is implemented using an impact wrench and a winch.
Experimental results are presented.
2
CONTROL OF POWERED TOOLS
Powered tools add degrees of freedom t o the manipulator/tool system. For example, the impact wrench attached
to the end of the manipulator adds a rotational degree of freedom to the system (the natural rotation axis of the
wrench). The winch adds a translational degree of freedom in the direction of gravity. These extra degrees of
freedom of the system are redundant. Choosing the directions t o control and those t o become passive is normally
an optimization problem. A generalized redundant system would be difficult t o control in real time. However, the
tool's functionality prescribes that control by the tool of its own natural degree of freedom is necessary for proper
usage. This degree of freedom is automatically chosen as a direction t o control.
M'hen using a specific tool, a task frame can be assigned that aligns one Cartesian direction of the frame with
the natural axis of the tool, and this direction is controlled by the tool. For the robot manipulator, this direction
becomes passive, and the system redundancy is removed. Because of the functionality of the powered tool, and the
ability to use arbitrary task frames, control of the redundant manipulator/tool system is simplified.
The functionality of the tool is also important in the way the passive degree of freedom is treated. For example,
the impact wrench is a force controlled device. The corresponding passive degree of freedom in the manipulator
must be position controlled, preventing motion. The winch, on the other hand, is a position controlled device. The
corresponding degree of freedom in the manipulator is force controlled with a force preload, so that the manipulator
\\-ill comply with the motion of the winch and keep the cable taut.
From the human operator's station, the control of the manipulator/tool system requires that a specific tool is
chosen for use1. After this is accomplished, the operator can move the master arm around and see/feel the tool
working in the virtual remote site environment. Motions by the operator in the natural axis of the tool are used to
control tlle tool; no additional actions are needed. Some subtle changes are imposed on the feedback t o the master
a.rm in order to aid the operator. or example, control of the impact wrench is based on rotations about the z-axis of
the inaster manipulator's tool frame. In order t o prevent unwanted, accidental use of the wrench, a small amount
of resistance to motion in this direction is programmed into the master arm controller. Therefore, only deliberate
motions to control tlle tool are accepted as input. The winch control is designed similarly, with a further constraint
t liat the velocity in the z-direction of the base frame is forced to be constant when deliberate motions in this direction
are sensed, si~nulatingtlic constant velocity of tlle winch. The human operator does not have t o specifically control
tile tools; i11put.sof motion in tlle tool direction is all that is needed.
After the tool has been specified for use, commands are automatically generated in the same form as for non-tool
act.ions. Rlotions, changes in contact states, guards, etc. are in the same form. The remote site, notified that a
specific tool is in use, interprets these commands properly for the task.
Tool usage a t the master site has not yet been implemented in the experimental testbed.
3
SENSING FOR POWERED TOOLS
The addition of a new sensor brings added complexity to the system. It must be calibrated and properly implemented,
and the system will depend on the new sensor t o work properly. Too many sensors become redundant, and sensor
fusion techniques become necessary to correlate sensor inputs. This is expensive in terms of time and computational
power, as well as expensive in terms of the physical sensor devices. In order to simplify the problem, and to
reduce cost, all tool sensing feedback comes from the existing force/torque sensor of the remote manipulator, the
instrumented compliant wrist [3].
The use of sensory inputs for tool control is similar t o the implementation in the basic teleprogramming system.
Guarded moves terminate properly when expected sensory events are encountered within the tolerance limits. Otherwise, motion is terminated in an error state. Guarded moves are generated automatically by the master system as
required, in the same manner as a motion without tool usage. An adaptive sensing algorithm is used t o determine
sensor events from normal operating noise, which changes with different tools and operating conditions. Therefore,
the same algorithm can be used with or without tools in the system.
EXPERIMENTAL RESULTS
4.1
Impact wrench
An impact wrench has advantages over a conventional wrench used by the robot. It can deliver much greater torque
than the robot can by itself. The impact wrench is easier t o use than the conventional wrench, because no complex
movements are needed (however, most robot systems that use wrenches use electric or pneumatic wrenches which
also do not require motion of the robot). Drawbacks t o using the impact wrench include vibrations caused by the
tool, and the need for a power source. The use of the impact wrench is characteristic of other tools the robot may
use, such as drills and screwdrivers, which have the same natural axis of motion.
The impact wrench used for this research is a 318" drive pneumatic wrench. Mounted internally on the instrumented compliant wrist (see figure l(a)), it creates an extra degree of freedom with the same axis of rotation as
the z-axis of the end effector frame (T6). By specifying the wrench in the UseTool() command, rotation about the
z-axis of T 6 implies control of the wrench. The corresponding z-axis in the manipulator is position controlled with
no specified motion, and thus acts as a locked joint (no motion allowed).
Inserting a bolt involves either sensing torques about the axis of rotation, or sensing velocity in the bolt feed
direction. A high torque, or zero feed velocity, may indicate a jammed bolt or a properly seated bolt, so tolerances
must be checked. Removing a bolt requires that the outward velocity be monitored. When the outward velocity goes
to zero, the bolt is assumed to be fully extracted.
\+'hen the impact wrench is specified for use, the following parameters are automatically assumed:
The z-rotation mode for the robot manipulator is set for position control. This assumes that the current task
frame is aligned with the tool frame of the manipulator (T6).
Gains and maximum allowable forces/velocities are changed t o reflect the increase in noise when the tool is in
use.
hlotions in the positive z-rotation direction are transformed t o turn the impact wrench clockwise. Similarly,
inot,ions in the negative z-rotation direction turn the impact wrench on counterclockwise.
The folloxving is a sample program that is used t o remove a bolt from the top of a box. A program such as this
117ould he autoil~aticallygenerated by the master system as the operator moves the master manipulator into contact
Figure 1: Impact wrench (a)surrounded by wrist, (b)removing bolt
with the bolt in the virtual remote site, and turns the master manipulator in the direction of unscrewing the bolt
an almost automatic action. Explanation and notes on the program will follow.
>> Execution Environment
-
#O
UseTool(Wrench)
DefineVector(CP;<O.O,0.0,220.0>:EE)
DefineVector(X;<1.0,0.O,O.O>:EE)
DefineVector(Y;<O.O,1.O,O.O>:EE)
DefineTaskFrame(N:EE;CP;X;Y;?)
UseFrame(N)
AssignMode(P,P,P,P,P,P)
GuardForce(<0.0,0.0,-1.0>;<0.0,0.0,0.0>)
Move(5.0;<0.0,0.0,4.0>;<0.0,0.0,0.0>)
AssignMode(P,P,F,P,P,P)
Force(<0.0,0.0,1.0>;<0.0,0.0,0.0>)
1.0
1.1
1.2
>> Execution Environment #1
GuardVelocity(<O.O,O.O,l.O>;<O.O,O.O,O.O>)
Move(2.0;<0.0,0.0,0.0>;<0.0,0.0,-2.0>)
Esecution environment #O contains all the information needed to move the impact wrench into contact with the
bolt, via a guarded move. After the impact wrench is designated for use, and a task frame is created with origin at
tlie tool tip (see figure l ( b ) ) and the z-axis aligned with the rotation of the wrench, the commands generated are
identic.al t.o a similar motion into contact without the use of a tool.
Esecution environment #1 contains tlie necessary commands to remove the bolt. A GuardVelocity() command
is used to non nit or tlie outward velocity of the bolt. The rotation indicated in the Move() command turns the impact
velocity (mmhmestep)
-------------.
z-velocity
mean z-velocity
,150
-.loo
I,
;
:'
,
,
I
'f!
Figure 2: Velocity of robot while removing bolt
wrench on, and the nut is removed.
Figure l ( b ) shows the robot-mounted impact wrench removing a bolt, as in the sample program above. The task
frame coordinate system is indicated in the figure, with the origin a t the tool tip.
In order t o sense when the bolt is fully removed, the GuardVelocity() command is used. Shown in figure 2 is the
actual sensed velocity (raw sensor data) in the direction of the bolt axis as the bolt is removed, as well as the mean
velocity based on 10 time steps of data2. The mean velocity model is used by the system for detection of sensor
events because of the noise level of the raw sensor data. Note that the velocity model is not started until 112 second
into the motion. This is to allow for motion transients t o abate. In this run, for example, the socket did not initially
seat over the head of the bolt. When the impact wrench is turned on, the socket moves down over the bolt head, as
indicat.ed by the initial positive velocity in the raw sensor data; this data does not interfere with detection of bolt
removal. As the robot complies with the force of the bolt, the feed rate of the bolt (determined by the bolt pitch)
gives rise to a negative velocity in the robot. The mean velocity is monitored, and motion stops when the velocity
returns t.o zero, indicating that the bolt is no longer feeding. A slight delay in sensing is caused as a result of the
averaging of the velocity, but this does not adversely affect the task execution.
Each time step is 20ms, the rate that the remote site computer communicates with the robot controller
(a>
(b)
Figure 3: (a) System controlled winch, (b) winch removing box top
4.2
Winch
A winch can increase, the load-carrying capacity of the robot manipulator. The need for power t o offset the force of
gravity on payloads, which usually accounts for a large fraction of the total power used by the robot, is eliminated.
Because tlie payload capacity of a manipulator is usually inversely proportional t o its accuracy, a robot with higher
accuracy can be used for manipulation of heavy objects when a winch is used.
The winch is sensorless, and thus relies on the robot for sensing. The winch also adds a degree of freedom to the
system: a prismatic joint in the world z-coordinate. Motion in this direction is naturally controlled by the winch
when the winch is specified with the UseTool() command. The corresponding degree of freedom in the manipulator
is force co~ltrolledwith a force preload to keep the winch cable taut.
The winch used for this research, illustrated in figure 3(a) is small, and not much more powerful than the robot
itself3. However, knowledge gained about using the winch in conjunction with the robot can be applied to much
nlore powerful winches. The winch operates a t a fixed speed, and therefore the motions a t the operator's station
must be constrained t o this motion when the operator is using the winch.
\Yl~enthe command to use the winch is parsed, the following parameters are automatically set:
The z-direction mode for the robot manipulator is set for force control. The manipulator will then conform to
forces in the z-direction. This assumes that a task frame aligned with the kinematic base (world) coordinates
is used.
A force is set in the negative z-direction, to preload the winch cable (this keeps the cable taut).
Gains and maximum allowable forces/velocities are changed to reflect the increase in noise associated with the
constrained system.
R'lotions in the positive z-direction are transformed to raise the winch. Similarly, motions in the negative
z-direction lower the winch.
3 T l ~ payload
e
capacity of the winch is approximately 12 lbs.; the Puma robot payload is 5 lbs.
The following is a sample program that is used t o insert the winch hook into an eyebolt on the top of a box (see
figure 4). Explanation and notes on the program will follow.
> > Execution Environment #O
UseTool(Winch)
DefineVector(X;<O.O,1.O,O.O>:KB)
DefineVector(Z;<O.O,O.O,1.0>:KB)
DefineTaskFrame(TF:EE;WST;X;?;Z)
UseFrame(TF)
AssignMode(P,P,F,P,P,P)
GuardVelocity(<0.0,0.0,1.0>;<0.0,0.0,0.0>)
Move(9.O;<O.O,O.O,-15.0>;<0.0,0.0,0.0>)
1.0
1.1
>> Execution Environment #1
DefineVector(C;<0.0,30.0,160.0>:EE)
1.2
DefineTaskFrame(PP:EE;C;X;?;Z)
1.3
1.4
UseFrame(PP)
2.0
>> Execution Environment #2
2.1
2.2
2.3
AssignMode(P,F,F,P,P,P)
Move(l.5;<0.0,0.0,1.0>;<0.0,-0.4,0.0>)
GuardForce(<2.0,0.0,0.0>;<0.0,0.0,0.0>)
Move(5.0;<-12.0,0.0,0.0>;<0.0,0.0,0.0>)
>> Execution Environment
#3
3.0
3.1
DefineVector(C;<O.O,-52.0,180.0>:EE)
3.2
DefineTaskFrame(PP:KB;C;X;?;Z)
3.3
3.4
UseFrame(PP)
4.0
4.1
>> Execution Environment #4
Move(l.S;<O.O,O.O,O.0>;<0.0,-l.O,O.O>)
Move(9.0;<0.0,0.0,15.0>;<0.0,0.0,0.0>)
This simple set of instructions is typical of a program that would be automatically generated by the master
system from simple motions of the master arm by the human operator. Absent are any exploratory motions that
may be necessary to identify the location of the eye on top of the box.
Each execution environment in the above program contains the commands needed t o move between successive
steps of figure 4. In execution environment #0, command 0.1 informs the remote system that the winch is being
used. This implicitly results in a force applied in the negative z-direction. Commands 0.2 t o 0.5 define a task frame
t.o be used by the control. It is important t o note that when using the winch, the z-direction of the task frame must
correspond to the z-direction in kinematic base coordinates. Because the system will be moving, however, the frame
itself is defined t o be dynamic in command 0.4, by specifying the frame name TF t o be with respect to the dynamic
frame EE. After the frame has been designated, the commands are equivalent t o commands generated without the
use of a tool.
The robot is controlled in force mode in the z-direction with a force preload of approximately 2 lbs. on the winch
cable. Any notion specified in the z-direction is assumed t o have the velocity of the fixed-speed winch, and the winch
(f
Figure 4: Winch experiment
velocity (mmhbeskp)
.2-velocity
................----.
mean z-velocity
0.30
Figure 5: Hook velocity data for guarded move a-b
is controlled up or down as specified. The robot complies adequately t o this motion. All other motion directions are
controlled by the robot.
Sensing for the winch is more difficult than expected. Because the robot/tool system is constrained, internal
forces are greater, and more noise is produced. However, the adaptive sensing algorithm is able to determine sensor
events from the additional operating noise produced in this example.
Figure 5 shows the actual sensed velocity (raw data) and mean velocity (computed over 10 time steps) of the hook
in the z-direction for the motion commanded in execution environment #1 above (notice that there are two separate
scales on the y-axis of the graph, in order to make the data easier to read). The mean velocity model is not initialized
until 112 second into the move, to avoid any irregular or transient data when the move begins. The velocity data
is very noisy, and has a periodic frequency of approximately 3.3 Hz, which can be attributed the natural frequency
of the wristlwinch-cable mechanical system. The mean velocity data reduces the amplitude of this vibration and
delays the signal by about 1/10 second. Using the mean velocity data, the change in velocity caused by the collision
with the box is easily discerned. The delay of 1/10 second is acceptable.
Figure G shows the implicit force data for the move into contact with the eyebolt. The datashown is in the direction
of the motion. Note that the average force in the motion before contact (before approximately 755 timesteps on the
s-asis of the graph) is non-zero. This is caused by internal forces built up by the constraints of the system. The
........--..aundard deviation * 3.0
------------.
x-diition force
0.60
Figure 6: Hook force data for guarded move c-d
GuardForce command checks the deviation of the force data against the standard deviation model that is built up
over the previous 30 time steps (.6 second). If the deviation of the force data is above 3 times the standard deviation
for 4 consecutive time steps, a collision is detected. This rather stringent criteria is necessary in order to reject
spurious, non-collision related data. It does, however, add slightly to the time delay in detecting collisions.
5
CONCLUSIONS
The methods presented here to control tools in a teleoperative system have the advantage that they do not increase
the burden of work for the human operator. Except for the initial selection of a tool for use, the operation of tools
is transparent to the operator. Because they do not require extra sensory input, the problems of sensor fusion and
extra computational time needed to read and interpret additional signals are avoided, as well as avoiding the cost
and implementation of another sensor in the system.
A host of questions involving tool usage were raised during this research and not addressed. Among these are:
\,Irha.texploration techniques are necessary to find the bolt/eye/etc?
This question was not addressed in the examples presented here. Obviously, with a tolerance limit larger than the
size of the bolt (for example), some exploration will be necessary. This leads to another question:
\Vhat tolerances are necessary to be successful at removing a bolt, mating a hook and eye, etc.?
If the initial tolerance limits, defined by the accuracy of the initial imaging of the remote system, are not accurate
enough to find or make contact with the bolt, some method of refining the accuracy of the model will be necessary.
Efforts to use data collected at the remote site to refine the graphical model are currently in progress.
These first two questions can be more easily researched when the tool implementation at the master site is
operational.
Other questions that may be asked include:
Is the tool control implemented here too tooI-specific?
Have we chosen tools that are general enough to show usage characteristics of all powered tools?
The experiments presented here use only two powered tools. However, the methodology for tool usage appears to
applicable t o a wide range of tools that the robot can use. However, there are certain end effectors, namely grippers
and hands, that add extra degrees of freedom to the system that can not replace coincident degrees of freedom of
the robot manipulator.
6
ACKNOWLEDGMENTS
This work was supported by the National Science Foundation under Grant No. BCS-89-01352, "Model-Based Teleoperation in the Presence of Delay." Any opinions, findings, conclusions or recommendations expressed in this
document are those of the author and do not necessarily reflect the views of the National Science Foundation.
7
REFERENCES
[I] William R. Ferrell and Thomas B. Sheridan. Supervisory control of remote manipulation. IEEE Spectrum, 81-88,
October 1967.
[2] Janez Funda. Teleprogramming: Towards Delay-Invariant Remote Manipulation. PhD thesis, University of
Pennsylvania, 1991.
[3] Thomas Lindsay and Richard P. Paul. Design of a Tool-Surrounding Compliant Instrumented Wrist. Technical
Report RtS-CIS-91-30, GRASP LAB 258, University of Pennsylvania, April 1991.
[4] Richard P. Paul, Janez Funda, Thierry Simeon, and Thomas Lindsay. Teleprogramming for autonomous underwater manipulation systems. In Intervention '90, pages 91-95, The Marine Technology Society, June 1990.
[5] Peter M. Will and David D. Grossman. An experimental system for computer controlled mechanical assembly.
IEEE Transactions on Computers, C-24(9):879-888, 1975.
Download PDF