Autonomous Robot Feeding with Assistive Dexterous Arm (ADA)

PRESENTER: More than 12 million
non-institutionalized people living with a disability
in the United States alone require assistance
with one or more activities of daily living. Systems such as the
JACO robotic arm can be attached to a
motorized wheelchair and augment a person’s ability
to perform general purpose tasks that they can’t
complete on their own. The personal robotics laboratory
at the University of Washington aims to extend the capabilities
of general purpose robots to accomplish even
finer grained tasks. Eating free form food is one of
the most intricate manipulation tasks that humans perform. To initiate feeding, a person
would switch the robotic arm from teleoperation mode
to autonomous mode. The meal begins with
the robot picking up a custom 3D-printed fork that
is docked on the wheelchair when not in use. From that point on, ADA can
feed someone an entire meal with or without
human intervention. The system integrates
onboard sensors, electronics, and computation with the
arm and wheelchair enabling ADA to travel with the person. Robot-assisted feeding
requires robust non-prehensile manipulation of a deformable
target that is hard to model. Humans adapt their
approach to accommodate the shape and compliance
of each forkful of food. Our lab designed ADA to
follow a similar approach, using a combination of
sensing, perception, planning, and control to successfully
acquire and transfer various food items. The camera and tactile
sensor attached to the arm rely on a custom-built
perception module to wirelessly communicate
multi-modal sensing information to the robot. The haptic feedback
enables ADA to skewer with appropriate
force and confirm the attempt was successful. Autonomous feeding is
challenging for robots because it involves the
manipulation of deformable food items of various sizes, shapes,
textures, and compliance. A robot must solve the problem
of both where to skewer and how to skewer each
item of food, an act we refer to as “bite acquisition”. When deciding how
to skewer, the robot must learn at what angle
to approach the item, how to rotate the fork,
and what forces to apply. ADA takes all of these
factors into account when skewering a piece of food. For each item, whether it is
a cube of melon, a whole round strawberry, or a
soft slice of banana, the robot determines the
appropriate fork orientation and approach angle, then,
exerts the force required to successfully skewer and lift
the item without dropping it. While a piece of melon
can be skewered from above and safely lifted off of the
plate, a soft food like banana requires ADA to approach
from an angle that ensures the banana
will not slip off the end of the fork in mid-air. ADA must also
determine the best way to deliver the food to
the person’s mouth, which we refer to as “bite transfer”. The combination of bite
acquisition and bite transfer is straightforward
for a compact item with fairly uniform dimensions. But that is not the case with
longer items like a spear of celery or a baby carrot. For longer shapes, ADA will
spear the item closer to one end rather than in the center. While this is not particularly
necessary for successful bite acquisition, the
system is also looking ahead to what will
make the transfer easy in the next phase. At the point of
bite transfer, ADA will position the fork
so that the person can accept the food
from the opposite end to where it is skewered. This keeps the tines of the
fork away from the mouth, which as we learned
from user studies, is important for ease
of bite transfer. The system also angles
the fork in order to make it easier for the
person to take a bite. The arm mounted
camera enables ADA to detect when a
person has moved their head during the process
of delivering the food item. The robot uses visual
servoing to adjust its motion to the user’s head movement. This enables it to adjust the
path of the fork in mid-air to reach the person’s mouth. When the plate is empty
or when the person decides they have had enough to
eat, ADA returns the fork to its dock on the wheelchair
and the meal cycle is complete. At that point, the user
resumes tele-operation until they activate
the autonomous mode for the next meal. ADA represents a
significant step forward in solving some of the most
complex challenges in robot manipulation through a robust
system of perception, planning, and control that is informed
by multi-modal feedback from haptic and vision sensors. With ADA, we have
demonstrated the potential for a robot assisted
feeding system to increase independence
and improve quality of life for people living
with disabilities. For more information on the
robot assisted feeding project, visit us online at