Smackathon Project – Obamatron
Smackathon Project – Obamatron
Obamatron is the Smackathon project created by Team America, the upstanding, patriotic team of engineers including Mark Godfrey, Ian Simon, Randal Leistikow, and Parag Chordia.
Inspired in part by Jimmy Fallon’s, “Brian Williams Raps Snoop Dog,” we strove to create an app which would allow Obama to sing any of the 700+ songs in the catalog of our karaoke app Sing! A second, complementary mode lets Obama sing any lyrics one types into the app.
We started by downloading the set of all of President Obama’s weekly addresses from the White House website and running speech recognition software on each video to identify the timing of all words. (The corpus contains approximately 7500 unique words.) We then developed a language model capable of mapping the lyrics to be sung with the best-matching sequence of words in the source videos, automatically finding closest-sounding substitute words whenever exact matches were never said by Obama in the available videos. By combining audio and video processing components from our apps Songify, AutoRap, and CineBeat, we were able to align the selected sequence of video clips to the rhythm of the song and shift the pitch of the voice to the song’s melody.
This technology could be modified to have any person be the singer, so long as sufficient source video recordings are available; the more quantity and variety of high-quality recordings available, the easier it will generally be to find a good match. In the 48-hour timeframe, we did not have time to perfect the language model and sequencing, so there are several steps we’ve sketched out to improve the realism and overall quality of the result. This was a really fun project, so we hope to revisit this work in the future. We also look forward enthusiastically to the next Smackathon. USA! USA! USA!
