Sing with the Telenoid

Abstract

We introduce a novel research proposal project aimed to build a robotic setup in which the Telenoid learns to improvise jazz singing in a duet with a human singer. In the proposed application, the Telenoid acts in teleoperated mode during the learning phase, while it becomes more and more autonomous during the working phase. A goal of the research is to investigate the essence of human communication which is based on gestures and prosody. We will employ an architecture for imitation learning that incrementally learns from demonstrations sequences of internal model activations, based on the idea of coupled forward- inverse internal models for representing musical phrases and the body sequences of singer movements. Possible applications are in the field of entertainment, where the Telenoid setup system acts as a musical game like Guitar Hero or Rock Band; in the field of music learning; for the analysis of mental disorders as Autism, and to build a new generation of companion robots for elderly care.