Introducing Smuush: Crowdsourced Music Videos Made Easy
Introducing Smuush: Crowdsourced Music Videos Made Easy
Today, we’re announcing Smuush by Smule, a new way of visualizing the world connecting through music in the most authentic way possible – by using Vine to capture the exact moment at which people are experiencing a song and combining all these moments together.
Smuush was initially created as part of Smule’s annual Smackathon, a 48-hour marathon of hacking, smashing and smacking. Smuush is one of over a dozen crazy ideas that we were able to turn into reality. (You’ll be hearing about more of those projects later on this blog.)
So where did the idea for Smuush come from? Well the initial concept hit me hard about six months ago after a marketing department brainstorm. We were trying to figure out if Smule should build a presence on social sites like Vine, Instagram, etc. The traditional approach to creating a brand presence on these types of networks would be to create a profile with Smule branding and then curate content on a regular basis that captures the essence of our brand. This approach is time consuming and the potential return on investment would be uncertain at best. There had to be a better way.
So after doing some research I came up with a hypothetical equation.
In other words, if we combine the things innovative companies like Airbnb, and top musical acts like Bruno Mars are doing on the various social platforms with the essence of what makes Vine work so well – “When Vine videos are consumed in bulk, they are greater than the sum of their parts” – and then put all that in the context of what Smule does best…you get Smuush
Enough blah blah blah, what is Smuush?
Well, simply put, Smuush is a new way of visualizing the world connecting through music in about the most authentic way possible. It captures the exact moments when people are experiencing a song and combines all these moments together into a music video. It uses Vine as an endless and constantly updating repository of 6 second video clips that are easily searchable by hashtag, and Smule technology as the thread that ties them together in a musical and, at times, culturally relevant way. Just feed the application an audio file like an mp3 of a popular song (such as “Classic” by MKTO) and a commonly used Vine hashtag that is associated with that audio (such as “#Classic”), and a minute or two later, out pops a music video with people lip syncing, dancing to the beat, and otherwise expressing themselves as the song plays.
That’s it, pretty simple. And every time you run the app, the video could be completely different depending on how many people have created a video of themselves listening to that song since the last time you ran it!
But how does the technology work?
So glad you asked! The designer who helped make Smuush cute AND functional (Nick Kruge) made a pretty cool graphical explanation of how the technology works, which you can see below, but here’s the quick-and-dirty text version of what’s happening:
1) Search Vine for the most recently recorded videos containing the hashtag given to Smuush and start downloading as many as possible (it has been capped it at 500 for now).
2) Compute a spectral analysis of the audio file given to Smuush.
3) Compute a spectral analysis of all the videos Smuush downloaded.
4) Identify which videos actually contain portions of the song through a comparison of their spectral analyses
5) Find the exact moment at which those videos were recorded during the progress of the song
6) Place the videos on a timeline at the exact moment they were recorded, and stitch them all together!
7) Save the file to your computer and BAM! You can now see what the world is doing while they listen to that song. Pretty cool party trick.
Something else to note about the technology is that it is extremely fast. The spectral analysis, where we compare the audio file with the video clips, happens instantaneously – technically, just under 10 milliseconds per clip.
It takes on average less than 2 minutes from the time you click “Smuush It” to the time the video is finished. This includes the time to download each video, compute the spectral analysis, and stitch the video together. And it is all thanks to the coding genius of Mark T. Godfrey and the technology backbone provided by Ian Simon. Much speed. So Talent. WOW!
Now for a live demo.
The Smuush Technology as an expressive medium and misheard Spanish lyrics in “Billie Jean”:
Philosophically, one of the things I like best about Smuush is the fact that the technology itself has a sort of expressive characteristic. Chances are that no two videos will turn out exactly the same even when given the same, or nearly the same input. It’s somewhat random how it chooses the top video to display in a section where there are more than one to choose from, and seeding it with “#MKTOclassic” will produce something totally different than “#ClassicMKTO”.
If you give it the audio file for “Billie Jean” and hashtag “#MichaelJackson” chances are you will get less videos of people enacting misheard Spanish lyrics than if you give it the hashtag “#BillieJean”. Wait a minute. Misheard spanish lyrics in a song sung entirely in English? Yes, apparently in Spanish, when MJ sings “The kid is not my son” it sounds like he’s saying “Tu quieres una manzana”, which translates back into English as roughly “You want an apple”. Hence people holding up the lyric written on a piece of paper, and then biting into an apple at that part of the song. Who knew? Probably every native Spanish speaker, but I likely wouldn’t have ever learned this.
LASTLY – A note about the hackathon experience and homage to CineBeat:
Within the walls of Smule there is enough audio technology knowledge to circle the world 3 times. It’s a fact. 🙂 It is also a fact that the world of mobile is evolving at an alarming rate, and sometimes the amazing underlying tech that powers Smule’s applications gets lost in the fray. CineBeat, Smule’s first foray into the world of video, is one such example. Although it was amazing and introduced some incredible tech into the universe, it wasn’t able to stand the test of time against apps like Vine or Instagram. Or perhaps it was ahead of its time. Who knows?
No matter though. CineBeat showed that really awesome things can be done with music and video on a mobile device, and definitely played an inspirational role in the production of Smuush.
Considering I am more of a marketer and not a real engineer (as much as I would like to be), I couldn’t possibly have built Smuush myself. And certainly not in 2 days. But that’s why hackathons are so great. They provide an open and collaborative atmosphere to hack together something never before possible with access to the technology and people that make your company so great. And in this case, they allow for the reuse of old technology that may not otherwise be put back into production. So much fun! And thanks to the amazing team that took my original idea, made it better, and gave it legs. Hopefully it will continue to walk.
Turner Kirk – The “Mule” at Smule
