At Just the Right Time!

The vision of Wycliffe is to see God’s Word made accessible to all people in the language that speaks to their heart. Usually this might mean distributing a printed book. If there is a low literacy rate, then access to an audio version is important.

Audio versions of the Bible are recorded (often by Faith Comes By Hearing) and can be distributed on dedicated audio players like the Proclaimer. With the availability of smart phones, now a Scripture app created with Scripture App Builder can be distributed with the text and audio of the Bible bundled together.

What about early readers in the people group? Wouldn’t it be great if the text of the passage could be highlighted while the audio is being played?

In order to accomplish this, special timing files have to be created to let the software know which portions of the text and audio go together. Early attempts at this would take 45-60 minutes per chapter! For the Bible en Songoy de Gao App, this took weeks for a team of volunteers (listening to a language they didn’t know) to create these timing files.

Colleagues at Wycliffe searched for another way. Could software automate this process? Praise God, a solution was found! The Aeneas Library allows for the automated creation of the timing files. What used to take almost an hour can now be done in 3-5 seconds! How is this accomplished?

How does this thing work?

One word: Math

One Sentence (Layman Edition): A good deal of math and computer science, a handful of software engineering and some optimization tricks.

One Sentence (Pro Edition): Using the Sakoe-Chiba Band Dynamic Time Warping (DTW) algorithm to align the Mel-frequency cepstral coefficients (MFCCs) representation of the given (real) audio wave and the audio wave obtained by synthesizing the text fragments with a TTS engine, eventually mapping the computed alignment back onto the (real) time domain. 

 (source)

Phew! The result is that, now, a whole New Testament can be processed in less than an hour instead of weeks. Wycliffe is preparing to publish hundreds of Scripture apps. Many of them already have audio. This new software process came at just the right time!

[one_half]D923A05C-3E0D-46B1-898D-40756D077F58

Above: Bible Dedication of the Soŋay Language of Gao, Mali included the Bible en Songoy de Gao App

Right: John 3:16 in Soŋay Language of Gao with text highlighting synchronized with audio

[/one_half]

[one_half_last]Songoy_John_3_16

[/one_half_last]

 

Comments

  1. Hi Chris,

    I can’t tell you how excited I am to read this. I am a missionary in Togo working with numerous language groups. If I were to have the ability to see the text highlighted along with the audio, it would allow me to create compilations of audio passages in order to build a Bible study library for new village churches. It’s very rare to find people who can actually read their own language, so audio ressources are a must.

    One question: where do I find these apps you talked about which show the highlighted text as the audio is played? I have already tried the Bible.is app for Android and iOS to no avail. There’s text and audio, but not synced highlighting. If you could point me in the right direction, that would be fantastic! Thanks.

    • Hello Jesse,

      Sorry for the delay in responding. Your comment came in during a busy month! You can build your own Scripture App using Scripture App Builder. You will need access to the Paratext scripture text (and permissions to share it through an app).

      You can also use Reading App Builder to create picture books, health and community development materials, song books, illustrated stories and libraries of easy-to-read books for new readers.

      Each of these have the ability to synchronize text and audio. There is documentation on the website on how to use these applications which are Windows or Linux (Mac version in beta) desktop apps which generate an Android App that can be shared via Wifi, SD Card, or published to the Google Play Store.

      Grace & peace to you,
      Chris

Speak Your Mind

*