Studies on Visual Semantic-Structural Parsing of Sign Languages

Sign languages are visually perceived, which describes the parts of an object and their relations based on the invariable structure. It is realized by disassembling the whole into parts and subsequently recombining them. In our research, the salient visual regions were firstly calculated in the views of the hearing-impaired person by simulating their cognitive mode about sign languages. Afterwards, the visual seeds of focus were concentrated on the hands to improve its effectiveness. Finally, we obtained the sub-sign language by making the generated semantic superpixels as the parts, which were mapped into the finger skeleton, or shrinking the palm to a node. By disassembling the images (states) of sign languages according to their anatomical structures, the visual semantic-structural parsing is preliminarily realized for a set of words of the similar sign language.