Video:There’s more than one way to skin a robot challenge.
What does ChatGPT have to do with the Internet?
As we look at the technological progress that we’ve made over the last few decades, Pulkit Agarwal reminds us that certain types of innovations have a lot in common. (Agarwal is an Assistant Professor in the department of Electrical Engineering and Computer Science (EECS) at MIT).
Using the term “digital intelligence,” he’s casting some light on significant innovations, and positing future advances in robotics, for both residential and commercial applications.
Agarwal is also using the term “artificial physical intelligence” or API, which might be slightly confusing to people who work with application programming interfaces, but is otherwise instructed in helping us see what used to be within the realm of science fiction – (he provides an image of the old Jetsons household robot from the end of the 20th century, when these sorts of designs were put purely theoretical).
Then there is a “reality check” on where we are, and where we are headed:
“What we have become really, really good at is building systems, which are very, very good at one particular task,” he says, going through some of the leaps and bounds evident in the hockey-stick graph of how we have pushed the envelope on technology through the end of the Moore’s law era.
Using the example of an “impression specialist,” Agarwal shows four-legged robots learning to walk on various terrains.
Examples from the DARPA challenge in 2015, he shows, are pretty primitive, and likely to fail in various ways. This illustrates the kinds of challenges that have to be overcome with complex physical tasks.
“It’s not just about … walking, but also, doing things which are very easy for us, for example, opening door knobs, so on and so forth,” he says.
Agarwal asks us to imagine which is harder: a backflip, or walking.
Both are hard, he notes, in their own ways.
“There are challenges associated with generalization which one needs to encounter,” he says. “And this is really a big challenge in making these systems work in the real world.”
A common approach, he reveals, involves human design models, and ML programs automatically learning skills from data simulations.
“The approach that we have been taking … (involves) how people have been building these controllers to use some task-specific knowledge, which allows them to work in a particular environment,” he says. “Can we use large amounts of data to (help programs) learn robust skills?”
First of all, we have the computational power: in three hours, Agarwal says, you can compile 100 days’ worth of data.
From there, he explains, a lot of generalization is possible, under the right conditions. Agarwal shows the example of a robot limping, but continuing to walk.
“That’s the kind of robustness that we want to see in our robotic systems,” he suggests.
Agarwal shows robotic designs learning to progressively run, then play, and describes how they can do things like learn to recover or maintain balance.
Then, he suggested, we proceed to teaching the programs about using tools.
“Could we get the kind of dexterity that human hands have, …, sometimes we’re doing things for fun, you know, but many times, you know, this dexterity is very important, (in critical tasks like) eating food.”
Check out the part of the video where Agarwal displays terms like point cloud, joint positions and joint commands.
“These systems are by no means perfect, but (they’re) certainly a step in a promising direction,” he says.
This said, Agarwal contrasts different methods for learning, including simulations, tele-operation and automatic learning models.
“A human might demonstrate how to perform the task, which might be easier than…a robot collecting data, sometimes,” he says.
In a lab example, he explicates how robots can generate to unseen objects and out of distribution poses.
“This is, again…an exciting research direction, where, for example, the arm demonstrates (a) task, and the robot tries to repeat what it saw,” he says of a final example on-screen. “So, in summary, we are looking at how we can make robots more capable by collecting large amounts of data through different means, as a way to attempt and reach physical intelligence as API.”