How Can Robots Learn New Tasks? Practice, Practice, Practice

Babies learn how to identify and pick up objects in their world through trial and error. Now a computer scientist is trying the same approach with robots.

Robots have been around for decades, so you'd think by now they'd be able to do some useful things, such as put away the groceries or unload the dishwasher. It turns out these tasks are devilishly hard for robots.

So Stefanie Tellex is starting simple. Tellex is an assistant professor in the computer science department at Brown University. She's using a popular industrial robot built by Rethink Robotics. The company calls the robot Baxter.

It's typically used on an assembly line, but Tellex uses Baxter to perform the seemingly simple task of picking up objects from a table. It's an eclectic collection of objects, things like a 9-volt battery, a highlighter pen, a plastic kazoo. Baxter is supposed to pick an object up, shake its arm to make sure it has a good grip on it and set it back down on the table.

Watching Baxter trying to pick up these objects is a little like watching paint dry. It seems to take forever for the robot to reach the battery, and when it does, it misses it with its gripper and just knocks it over.

Frankly, it looks pathetic. Tellex doesn't disagree, but says there's a reason it seems so inept. "A lot of the stuff that is really, really, really hard for a robot to do, is almost effortless for a person to do," she says.

But, Tellex says, look at it from the robot's point of view. This isn't a factory where the robot has been programmed to do a very specific task. This Baxter doesn't know anything about batteries or kazoos. All it has is information from its cameras, but that information is just a bunch of numbers.

"It somehow has to look at that matrix of numbers, and then run a program to figure out where the object is and what it is and where it should put its gripper in order to not be so pathetic and actually pick it up effectively," Tellex says.

It takes a human baby two years or so to master these kinds of skills. Tellex thinks the way robots will get faster and smoother at picking up unfamiliar objects is to give them programs that let them learn from experience, just like a child would. After a robot has picked up a battery or a kazoo a couple dozen times, it will begin to recognize them reliably.

All this learning will take time, so Tellex's got her Baxter working round the clock picking up objects and putting them down, picking them up and putting them down ...

And she has an idea for speeding up the learning curve. "Everybody either has a Baxter, or has a friend with a Baxter, in robotics research right now," she says. A lot of the time, these robots aren't being used. "At night, for example, students go home, go to sleep, and the robot's just sitting there. And what it could be doing instead is collecting this type of data," Tellex says.

She is hoping to recruit some of these robots to do the same tasks as her Baxter to speed up the learning process — a kind of robotic version of many hands make light work.

Who knows? Maybe someday there'll be a Baxter to put away the groceries.