{"id":1293,"date":"2017-11-15T11:50:14","date_gmt":"2017-11-15T11:50:14","guid":{"rendered":"http:\/\/blogs.kent.ac.uk\/unikentcomp-news\/?p=1293"},"modified":"2018-05-10T14:45:59","modified_gmt":"2018-05-10T13:45:59","slug":"explainer-how-the-latest-earphones-translate-languages","status":"publish","type":"post","link":"https:\/\/blogs.kent.ac.uk\/unikentcomp-news\/2017\/11\/15\/explainer-how-the-latest-earphones-translate-languages\/","title":{"rendered":"Explainer: how the latest earphones translate languages"},"content":{"rendered":"
Shutterstock<\/a><\/span><\/figcaption><\/figure>\n

Ian McLoughlin<\/a>, University of Kent<\/a><\/em><\/p>\n

In the Hitchhiker\u2019s Guide to The Galaxy<\/a>, Douglas Adams\u2019s seminal 1978 BBC broadcast (then book, feature film and now cultural icon), one of the many technology predictions was the Babel Fish<\/a>. This tiny yellow life-form, inserted into the human ear and fed by brain energy, was able to translate to and from any language.<\/p>\n

Web giant Google have now seemingly developed their own version<\/a> of the Babel Fish, called Pixel Buds. These wireless earbuds make use of Google Assistant<\/a>, a smart application which can speak to, understand and assist the wearer. One of the headline abilities is support for Google Translate which is said to be able to translate up to 40 different languages. Impressive technology for under US$200.<\/p>\n

So how does it work?<\/p>\n

Real-time speech translation consists of a chain of several distinct technologies \u2013 each of which have experienced rapid degrees of improvement over recent years. The chain, from input to output, goes like this:<\/p>\n

    \n
  1. Input conditioning<\/strong>: the earbuds pick up background noise and interference, effectively recording a mixture of the users\u2019 voice and other sounds. \u201cDenoising<\/a>\u201d removes background sounds while a voice activity detector<\/a> (VAD) is used to turn the system on only when the correct person is speaking (and not someone standing behind you in a queue saying \u201cOK Google\u201d very loudly). Touch control is used to improve the VAD accuracy.<\/li>\n
  2. Language identification (LID)<\/strong>: this system uses machine learning to identify what language is being spoken<\/a> within a couple of seconds. This is important because everything that follows is language specific. For language identification, phonetic characteristics alone are insufficient to distinguish languages (languages pairs like Ukrainian and Russian, Urdu and Hindi are virtually identical in their units of sound, or \u201cphonemes\u201d), so completely new acoustic representations had to be developed<\/a>.<\/li>\n
  3. Automatic speech recognition (ASR)<\/strong>: ASR<\/a> uses an acoustic model to convert the recorded speech into a string of phonemes and then language modelling is used to convert the phonetic information into words. By using the rules of spoken grammar, context, probability and a pronunciation dictionary, ASR systems fill in gaps of missing information and correct mistakenly recognised phonemes to infer a textual representation of what the speaker said.<\/li>\n
  4. Natural language processing<\/strong>: NLP<\/a> performs machine translation from one language to another. This is not as simple as substituting nouns and verbs, but includes decoding the meaning<\/em> of the input speech<\/a>, and then re-encoding that meaning as output speech in a different language – with all the nuances and complexities that make second languages so hard for us to learn.<\/li>\n
  5. Speech synthesis<\/strong> or text-to-speech (TTS): almost the opposite of ASR, this synthesises natural sounding speech from a string of words (or phonetic information). Older systems used additive synthesis, which effectively meant joining together lots of short recordings of someone speaking different phonemes into the correct sequence. More modern systems use complex statistical speech models<\/a> to recreate a natural sounding voice.<\/li>\n<\/ol>\n