(this project involves a generation component by video concatenation as a class project for my Introduction to NLG class at FaMAF)

In this project, we seek to build a limited domain Android application with a limited domain automatic speech recognition system and a domain specific language generation targeting sign language.

For a first pass, we want to focus on the use of video concatenation as a (suboptimal) way of generating sign language and medical visits as the domain.

The target language is the Argentina sign language or LSA (Lengua de Señas Argentina). The input language is Spanish.

Tasks

  • Transform the doctor phrases into predicates for NLG (Sergio)
  • Write paraphrases for the doctor phrases (DrDub)
  • Record doctor phrases and train domain dependent ASR
  • Find missing vocabulary with respect to signing dictionary
  • Record missing vocabulary

Development

  1. Re-code videos in 3gp format (DrDub, done)
    1. ffmpeg -i farmacia1.swf -s qcif -r 25 -ab 32 farmacia1.3gp
    2. http://developer.android.com/guide/appendix/media-formats.html
    3. http://goinggnu.wordpress.com/2007/02/13/convert-avi-to-3gp-using-ffmpeg/
  2. Make simple Android app that shows the video (DrDub, done)
    1. see /proj/wip/sonajero-0.2.zip
    2. http://developer.android.com/resources/samples/ApiDemos/src/com/example/android/apis/media/VideoViewDemo.html
  3. Show video + camera view
    1. http://developer.android.com/reference/android/hardware/Camera.html
  1. Show video blended with camera view (with transparencies)