Chyi-Yeu Lin and colleagues at the National Taiwan University of Science and Technology in Taipei created a robot that first takes a photo of the music, which is notated with numbers and words, using cameras built into the eyes. An algorithm extracts pitch, rhythm, and lyrics from that image and sends the information to a voice synthesiser.
The synthesiser matches sounds in the Mandarin language with the Roman spellings of the lyrics. Then, in a sudden burst of activity, the robot begins to sing, its mouth opening and closing with the words.
I’ll sleep just fine tonight, thanks.