skype eliminates language barrier with real-time speech translation feature

skype eliminates language barrier with real-time speech translation feature
all images courtesy of skype

skype has announced the first phase of their ‘skype translator’ preview program which will initially launch on windows 8.1 or windows 10 with english and spanish as the first two languages. based off of microsoft’s machine learning platform, the highly sophisticated software is the result of decades of research in speech recognition and automated translations. by analyzing data during the preview stage, the software can learn to better recognize and translate the diversity of topics, accents and language variation of actual skype users. essentially, the more the technology is used, the smarter it gets.

video courtesy of skype

‘skype translator’s machine learning protocols train and optimize speech recognition (SR) and automatic machine translation (MT) tasks, acting as the glue that holds these elements together,‘ explain skype developers mo ladha and chris wendt. ‘this ‘glue’ transforms the recognized text to facilitate translation. this process includes the removal of disfluencies – such as ‘ahs’ and ‘umms’ as well as re-phrasings into sentences, as well as addition of punctuation and capitalization.’

‘the training data for speech recognition and machine translation comes from a variety of sources, including translated web pages, videos with captions, as well as previously translated and transcribed one-on-one conversations. skype translator records conversations in order to analyze the scripts and train the system to better learn each language. we have also had many people donate data from previous conversations, which we also analyze and use to create training material for the statistical models that teach the speech recognition and machine translation engines how to map the incoming audio stream to text, and then the text to another language. skype translator participants are all clearly notified as the call begins that their conversation will be recorded and used to improve the quality of microsoft’s translation and voice recognition services,’ they add.

the app is able to translate text from instant message conversations between over 40 languages

the MT portion of the program translates text from one language to another. skype has used the same technology that powers bing translator on the web, but is engineered specifically for a conversational type of language. additionally, a customized bot orchestrates the entire experience. the bot is responsible for creating the call, and sending audio streams to the speech engines in exchange for translation and transcription. the translator bot acts like a third participant in the call. it translates what you just said when you’ve finished talking, and it translates what the person you called said, when they finish talking.