columbia engineers develop robot that imagines and understands itself

a robot learns to imagine itself

Many health and fashion-conscious people are aware that our body image is not always accurate, yet it is still a crucial piece of information that influences our presence and function in the actual world. Humans develop body perception as infants, but what about machines and robots?

With this idea as a motivating factor, Columbia Engineering researchers created a robot that learns to imagine and understand itself rather than the environment around it. This first-of-its-kind robot can learn a model of its complete body from scratch without the assistance of humans. In their new study, the engineers show how their robot constructed a kinematic model of itself and then utilized that model to plan motion, attain goals, avoid obstacles in a range of settings, and how it even recognized and compensated for body damage automatically.

exploring the world like a child

Set inside a circle of five streaming video cameras, the robotic arm watched itself through them while it moved freely. The machine squirmed and contorted to learn how its body moved in response to different motor inputs, much like a child exploring itself in front of a mirror. The robot came to a halt after nearly three hours; it had completed learning the relationship between its motor movements and the volume it occupied, by using its own deep neural network.

‘We were really curious to see how the robot imagined itself,’ shares Hod Lipson, professor of mechanical engineering and director of Columbia’s Creative Machines Lab. ‘But you can’t just peek into a neural network; it’s a black box.’ After the engineers experimented with multiple visualization methods, the self-image gradually appeared. ‘It was a sort of gently flickering cloud that appeared to engulf the robot’s three-dimensional body,’ said Lipson. ‘As the robot moved, the flickering cloud gently followed it.’

the robot constructed a kinematic model of itself and then utilized it to plan motion, attain goals, and avoid obstacles

image courtesy of Columbia Engineering

paving the way to self-sufficient and self-aware machines

The ability of robots to develop self-awareness without the assistance of humans is critical for several reasons. Not only does it save labor, but it also allows the robot to keep up with its own effect of use, as well as detect and adjust for damage. The authors claim that this capability is important because we need autonomous systems to be increasingly self-sufficient. For example, a factory robot may detect if something isn’t moving properly and compensate or ask for help.

The research is part of Lipson’s long quest to find ways to give robots self-awareness. ‘Self-modeling is a primitive form of self-awareness,’ he explains. ‘If a robot, animal, or human has an accurate self-model, it can function better in the world, it can make better decisions, and it has an evolutionary advantage.’

The researchers are aware of the limitations, hazards, and controversies associated with giving machines greater autonomy via self-awareness. Lipson admits that the level of self-awareness revealed in this study is minimal compared to that of humans. ‘We have to go slowly and carefully, so we can reap the benefits while minimizing the risks,’ he notes.

snapshot of Hod Lipson’s MARS 2022 talk on building self-aware machines