Passionate about technologies that make culture accessible I stumbled upon this fantastic short demo of what could become an essential new app for the blind. Assistive technology in apps is not new for us and we have worked hard to make our devices and apps comply with the a11y rules of today. However, in this video, it’s not the app itself, or even the charming smile of the inspiring blind developer that triggered my interest, but the way the app works: first, it combines existing technologies in a smart and meaningful way with impressive artificial intelligence such as highly precise emotion recognition. Second, these technologies are available for any developer as cloud based APIs.
This demo, presented on the Microsoft Developers Conference in 2016, shows an example of some very exciting developments like machine vision, augmented reality and artificial intelligence. All the neat stuff you see in the demo, like emotion recognition and smart text to speech conversion, could be used in museum tour apps. Imagine the possibilities when a tour app cannot only help the blind navigate within a museum, but also understand the space around them and interpret art, translating sight into sound.
From a developers point of view it’s far easier to integrate vision, speech, language, knowledge, and search APIs into an app by using a cloud based web service than to buy an SDK or to develop it yourself. And these APIs work in iOS, Android or Windows apps and platforms. Microsoft is not the only company offering this kind of cloud based services but it looks like it’s becoming easier to add cognitive capabilities to apps, making culture more accessible than ever.