You are using an older browser version. Please use a supported version for the best MSN experience.

Dango mind-melds with emoji using deep learning and suggests them while you type

TechCrunch TechCrunch 10/06/2016 Devin Coldewey

Dango is an emoji suggestion chatbot — wait, where are you going? Stay with me, this is actually pretty cool.

Okay, so Dango is one of those virtual assistants that lives in your chat apps, and this one is based on a neural network that has been trained with millions of examples to understand what emoji mean. So not only can it suggest an appropriate one, but it can translate entire sentences. Its icon is a weird piece of cute cake, which it sits above your keyboard watching you type. It’s free for Android right now, with an iOS version coming out eventually.

The Dango team (many of the folks who made Minuum) used a recurrent neural network to perform a deep learning task (it’s just something you can do now) examining how emoji are used. The network started by just guessing what emoji go with what words and phrases, then would check that against real-world examples, and every time it got something wrong, it adjusted its parameters.

Repeat millions of times, checking those guesses against a huge amount of data gathered from the web. Assemble the data into a huge, many-dimensional “semantic space” where emoji are closer or more distant from certain concepts and ideas and phrases. Translate that semantic space into a more practically designed database — and boom, you’re done! Simple, right?

And now you get this:

Others don’t turn out so well, but you get the idea.

The app itself doesn’t translate entire phrases yet, though. Mainly it automatically suggests a top emoji for the situation, with others available as runners-up, as well as stickers and GIFs (thankfully you can tell it not to load them until you tap the cake thing). I had a little trouble finding a place where the cake piece wouldn’t block any text or critical buttons in my messaging apps, but your mileage may vary.

Automatic emoji suggestions exist, of course, but the ones out there generally rely on something basic but functional: suggesting the storm or snow emoji when you use words like “snow,” “snowy,” “snowflake” and so on. This deeper dive into the way emoji are used — and combinations thereof that can create entirely new meanings — would seem to produce quite a powerful data set. I love the visualization they created of how various emoji are associated with one another:

Dango’s database is being regularly updated to keep up with the latest slang and memes, too, but you’ll have to wait for app updates to get access to them; the associational database is local so there’s no lag or call-outs to a server. That helps with privacy, too.

Unfortunately, there’s no API to call just yet, though Dango co-dounder Will Walmsley told TechCrunch they were open to it if people are interested.

Another potential issue I asked about was the ever-present threat when you look at data from real people: what happens when the system internalizes racist, sexist, and generally bad data?

Clearly sensing that this is a sore point among AIs that interact with humans, Walmsley gave a more thorough response.

Bad associations are a real, and subtle, concern since AI tools like Dango reflect back our human best and our worst. We’ve in the past blacklisted certain associations and we’ll keep doing this — not unlike what Google does for its search suggestions — but it will be an ongoing challenge to get right.

Now that we’re publicly launched we’re also looking at adding a way for people to report problematic results.

I also asked whether the functionality in the website’s demo box (seen above) will be coming to the app, and when we can expect an iOS version. I’ll update the post when he gets back to me (again).

More from TechCrunch

image beaconimage beaconimage beacon