DrDubWiki

Printable version | Disclaimers | Privacy policy

Sign language generation on smartphones

From DrDubWiki

(this project involves a generation component by video concatenation as a class project by Sergio Canchi, a student of my Introduction to NLG class at FaMAF)

In this project, we seek to build a limited domain Android application with a limited domain automatic speech recognition system and a domain specific language generation targeting sign language.

For a first pass, we want to focus on the use of video concatenation as a (suboptimal) way of generating sign language and medical visits as the domain.

The target language is the Argentina sign language or LSA (Lengua de Señas Argentina). The input language is Spanish.

Tasks

  • Transform the doctor phrases into predicates for NLG (Sergio)
  • Write paraphrases for the doctor phrases (DrDub)
  • Record doctor phrases and train domain dependent ASR
  • Find missing vocabulary with respect to signing dictionary
  • Record missing vocabulary

Media:Example.ogg

Development

  1. Re-code videos in 3gp format (DrDub, done)
    1. <verbatim>ffmpeg -i farmacia1.swf -s qcif -r 25 -ab 32 farmacia1.3gp</verbatim>
    2. http://developer.android.com/guide/appendix/media-formats.html
    3. http://goinggnu.wordpress.com/2007/02/13/convert-avi-to-3gp-using-ffmpeg/
  2. Make simple Android app that shows the video (DrDub, done)
    1. see /proj/wip/sonajero-0.2.zip
    2. http://developer.android.com/resources/samples/ApiDemos/src/com/example/android/apis/media/VideoViewDemo.html
  3. Show video + camera view
    1. http://developer.android.com/reference/android/hardware/Camera.html
  1. Show video blended with camera view (with transparencies)

Links


Find

Browse
Main page
Recent changes
Edit
View source
Editing help
This page
Discuss this page
New section
Printable version
Context
Page history
What links here
Related changes
My pages
Log in / create account
Special pages
New pages
File list
Statistics
More...