Press "Enter" to skip to content

Shimon the Robot Writes and Sings Songs

Researchers at Georgia Tech have been working to improve a musical robot called Shimon. Now Shimon doesn’t simply play music, he also writes the words to his own songs – and sings them.

Shimon is a four-armed robot with a ball-like head. He holds small mallets in his “hands” to play a kind of xylophone called a marimba. As Shimon plays, his head moves around in time to the music.

Richard Savery playing with Shimon, the musical robot.
Shimon is a four-armed robot with a ball-like head. He holds small mallets in his “hands” to play a kind of xylophone called a marimba. As Shimon plays, his head moves around in time to the music.
(Source: Gil Weinberg, Georgia Tech.)

Shimon has been around for quite a while as a musical robot. Even back in 2015, he was playing with other musicians at the Kennedy Center in New York. What Shimon could do back then was already pretty cool.

Using “deep learning”, Shimon was taught to write his own music. Deep learning – also known as Artificial Intelligence (AI) – means that computer programs sort deeply through huge amounts of information. This allows them to find patterns humans may not have noticed. The programs can then use those patterns in new and surprising ways.

Gil Weinberg and Richard Savery play with Shimon, the musical robot.
Using “deep learning”, Shimon was taught to write his own music. He can create his own music in real time, while playing with other musicians. This is called “improvising”. Above, project members Gil Weinberg (left) and Richard Savery play with Shimon.
(Source: Gil Weinberg, Georgia Tech.)

For Shimon, that meant he could not only make up his own music, he could do it in real time, while playing with other musicians. This is called “improvising”.

Now Shimon is back with a whole bunch of new tricks. He can write the words to his own songs, and sing them.

Closeup of Shimon, the musical robot, singing.
Now Shimon is back with a whole bunch of new tricks. He can write the words to his own songs, and sing them.
(Source: Gil Weinberg, Georgia Tech.)

Shimon learned to write the words for the songs the same way he learned to write music – by being “fed” thousands and thousands of examples. Shimon was trained on the lyrics (words) to 50,000 songs.

Georgia Tech professor Gil Weinberg, who leads the Shimon project, gets Shimon going with a starting idea. Shimon then writes the lyrics based around that idea. Mr. Weinberg usually provides much of the music for the songs, but Shimon helps out there, too.

Gil Weinberg, the leader of the Shimon project.
Georgia Tech professor Gil Weinberg, who leads the Shimon project, gets Shimon going with a starting idea. Shimon then writes the lyrics based around that idea.
(Source: Gil Weinberg, Georgia Tech.)

Richard Savery, one of the researchers, says, “You’ll get a word like ‘storm’, and then it’ll generate a whole bunch of related words, like ‘rain’.” He says Shimon comes up with many ideas, keeps the good ones, and builds on them.

Singing the songs is another story. To give Shimon a voice, the Georgia Tech team worked with experts at Pompeu Fabra University in Spain. The voice was created using AI and sounds very much like a man.

Shimon’s face has also gotten new features. Shimon’s mouth now moves smoothly in time as he sings. He also has eyebrows, which allow his face to show more emotion.

Closeup of Shimon, the musical robot.
Shimon’s face has also gotten new features. Shimon’s mouth now moves smoothly in time as he sings. He also has eyebrows, which allow his face to show more emotion.
(Source: Gil Weinberg, Georgia Tech.)

To show off Shimon and his new abilities, the researchers have recorded one of Shimon’s songs, called “Into Your Mind”. They have made a video for the song, which is out on YouTube.

Later this spring, the team plans to put out an album of about 8-10 of Shimon’s songs. The album will be released on the music streaming service Spotify. There are also plans for Shimon to go on tour with a band to play and sing his songs live.

Gil Weinberg, Richard Savery, and others play with Shimon, the musical robot.
To show off Shimon and his new abilities, the researchers have recorded one of Shimon’s songs, called “Into Your Mind”. They have made a video for the song, which is out on YouTube (above).
(Source: Gil Weinberg, Georgia Tech.)

For Mr. Weinberg, that’s the main goal behind the Shimon project – not to have robots take over, but to have robots and humans make something beautiful together.



Did You Know…?

By chance, Shimon’s new song has some words of hope for people stuck at home because of the coronavirus : “Home is where the rainbow comes.”

Screenshot from video for "Into Your Mind", a song by Shimon, the musical robot.
Screenshot from video for “Into Your Mind”, by Shimon.
(Source: Gil Weinberg, Georgia Tech.)
Share:

Settings

Most news on NewsForKids.net is appropriate for all ages. When there is news that may not be suitable for all ages, we try to tag it. You can use the setting below to control whether content tagged in this manner is shown.