Bilingual Sign Language Dictionary - Semantic Scholar

6 downloads 5248 Views 872KB Size Report
truly bilingual (Spanish Sign Language-Spanish) electronic dictionaries for the .... term by LSE, the system is based on the analysis of formative parameters ( ...
Bilingual Sign Language Dictionary José L. Fuertes1, Ángel L. González1, Gonzalo Mariscal2, and Carlos Ruiz2 1

Facultad de Informática CETTICO, Facultad de Informática UPM, Campus de Montegancedo, 28660 Boadilla del Monte, Madrid, Spain {jfuertes, agonzalez}@fi.upm.es, {gmariscal, cruiz}@cettico.fi.upm.es 2

Abstract. The Spanish Sign Language Dictionary (DILSE) is one of the first truly bilingual (Spanish Sign Language-Spanish) electronic dictionaries for the deaf community. The properties of this format are perfectly matched to a visual language such as sign language, which uses space as a means of expression. Additionally, two-way searches for word entries are possible from either Spanish or signs. The signs have been previously classified according to sign language-based linguistic criteria. Furthermore, the system presented here includes different geographical varieties of Spanish Sign Language.1

1 Background Official recognition of sign language is important for the deaf community. A lot of work has gone into encouraging the dissemination, research and teaching of Spanish Sign Language in particular. Lack of knowledge of this language can lead, on the one hand, to it having a low social status and, on the other, to deaf people not receiving the proper information and training, resulting in functional isolation and thereby preventing this community’s full integration into society. The main goal of the Spanish Sign Language-Spanish Dictionary (DILSE: DIccionario de Lengua de Signos Española) is to help people with impaired hearing cope with this adversity. Additionally, it is an aid for anyone wanting to learn or improve their knowledge of Spanish Sign Language (LSE: Lengua de Signos Española). This dictionary works like any other bilingual dictionary in that the user can search words by LSE or by Spanish. To achieve the proposed objective, two clearly separate lines of research had to be set up. The goal of the first was to identify and classify the signs that were to be included in the dictionary. This then raised the need to establish a mechanism by means of which to catalogue and, therefore, be able to search by signs on the basis of their properties. The sign searching method had to be eminently visual, because LSE is first and foremost a visual language. The second line investigated how to develop software that could be used to create the dictionary, the essential requirement of which was that it would allow searching by sign based on mostly graphical features, the most natural form of expression of which is by means of pictures. 1

The Spanish National Confederation of Deaf People (CNSE), the ONCE Foundation, the Institute of Migrations and Social Services (IMSERSO) and the Centre of Computing and Communications Technology Transfer (CETTICO) have participated in this R&D project.

K. Miesenberger et al. (Eds.): ICCHP 2006, LNCS 4061, pp. 599 – 606, 2006. © Springer-Verlag Berlin Heidelberg 2006

600

J.L. Fuertes et al.

2 Related Work One of the first activities undertaken by deaf communities is the compilation of a sign language dictionary [1]. Limited printed editions of modest-sized basic dictionaries were originally developed. Now, however, thanks to Information and Communication Technology, dictionaries can be developed that take advantage of the whole range of multimedia capability. Representation in print is a weakness, because you are trying to represent statically something (sign language) that is essentially dynamic, and making them bilingual is a tricky point. Additionally, the use of technology not only means taking advantage of the opportunities offered by multimedia, but also making the best use of a wide range of possibilities that databases offer, such as support for two-way searching (LSE ↔ Spanish) and low-cost storage capacity. A number of types of sign dictionaries are defined according to their technological evolution in [1], beginning with printed and ending with multimedia dictionaries, which is the current development trend. One example of the use of printed dictionaries to create multimedia tools can be found in Australia, where we have "Signs of Australia" [2]. This is an Australian Sign Language (Auslan) dictionary based on the printed edition [3]. It contains about 4000 signs recorded on video and includes sections on the history of Auslan, an introduction to its grammar, and synonyms and antonyms. The dictionary can be searched by English words, by topic or by a subset of phonological properties (hand shape + hand arrangement + location). The National Deaf Children’s Society in the United Kingdom is a clear example of what efforts are being made at standardising the use of sign languages [4]. This organisation has a lot of multimedia tools for use by children and young people. The creation of a multimedia fingerspelling dictionary for Slovenian Sign Language has led to a standardisation of the use of this sign language [5]. The ASL (American Sign Language) online dictionary is unquestionably a very interesting initiative. It includes over 2500 signs, a figure that grows weekly [6]. Additionally, it is a meeting point for people who want to learn and communicate using ASL, as it includes lessons, a kids zone, ASL-based literature, information directories, travel guides, etc. The online dictionary for British Sign Language [7] is a similar scheme, and now includes around 500 signs. As regards Spanish Sign Language (LSE) dictionaries, there are a number of editions of printed dictionaries [8], [9]. Although they amounted to a big step forward at the time, they are flawed because they are paper editions. The "Signos 97-98" dictionary [10] is a LSE dictionary designed to support a sign language learning course. It contains grammatical and usage information and takes an educational approach. It includes about 1000 terms rendered with all multimedia characteristics, and limited bilingual searching is enabled. To search for a Spanish term by LSE, the system is based on the analysis of formative parameters (hand shapes, location, direction, movement and non-manual components). The “Mis Primeros Signos” dictionary [11] is the first children’s dictionary in LSE with around 500 terms. It is a bilingual tool (sign language and spoken language) designed for uptake by Spanish educational system bilingual schools, targeting children aged from 3 to 8 years. The signs are portrayed by children educated in the bilingual methodology, making it a unique piece of material from this viewpoint.

Bilingual Sign Language Dictionary

601

These latest developments were designed as support for LSE courses and, while they are effective in this context, they cannot be considered full-blown dictionaries because they only include terms pertaining to the course lessons. Another important point to be taken into account is that they are not as mature as lexicons developed for other languages, which is reflected by the fact that there is either no or only limited twoway search. Additionally, taking into account the diversity of sign languages within each country, a dictionary needs to include a compilation of the geographical varieties of each sign to be considered complete. Finally, and this applies worldwide, a vital point is that development efforts are very much dependent on teams of developers outside deaf communities, which means that ideally these communities should be supplied with the tools they need to add new or modify existing signs. These four points summarise the primary goals of DILSE [12,13] as a bilingual LSE dictionary for the Spanish deaf community, using multimedia to render sign languages’ visual properties.

3 Study and Compilation of Signs A fundamental task to assure that a dictionary’s content is both accurate and realistically reflects a language is to study this language and compile the most significant words. The words are the actual signs in DILSE. One key factor to be taken into account in the case of LSE is that there is no flexible and dynamic governing body to decide which words should be included in LSE and which should not. This, together with the deaf community’s need to be able to communicate in their mother tongue (LSE), often calls for the creation of new signs. The development of such new signs is linked to the appearance of new concepts in everyday life (Internet, Web, mobile telephony, scanner, magnetic resonance, etc.). Because there is no governing body, a number of signs may emerge to represent the same concept. Before quality audio-visual communication systems (broadband videoconferencing, UMTS, etc.) became widespread, the dissemination of these new signs was confined to the region in which they appeared. This has led to what are termed geographical varieties of a sign. A start was made to the work on compiling terms by thoroughly revising existing LSE dictionaries [8,9]. These dictionaries had several shortcomings. One weakness was that the format is not based on LSE as a self-contained language; it referred back to Spanish in all cases. Additionally, a sizeable number of the signs have fallen into disuse. Also examining some sign languages dictionaries put together in other European countries [14], we found that the signs were ordered by and search criteria were based on the hand shapes of each sign. This point was adopted as a criterion for building DILSE. The last point was to compile the geographical varieties of LSE for each word. We analysed major Spanish dictionaries and second language teaching/learning material to make a preliminary selection of vocabulary. Then we contacted associations of the deaf from different Spanish regions and asked them to compile the signs that they were then using to translate the terms on the list we had put together. Signs that were commonly used in most regions were classed as “national”.

602

J.L. Fuertes et al.

4 Description of the Signs In written languages, words are described by the letters of which they are composed. Obviously, we needed some way of describing a sign to be able to organise a bilingual LSE-Spanish dictionary. The initial hand shape before starting the sign and the movements made by the hands were chosen as a basis for classification, leading the CNSE [15] to establish the following characteristic factors as describing a sign: • Active hand shape: This establishes the arrangement of the hand that does most towards forming the sign. • Use of two hands: Some signs require the use of one hand and others involve both. • Passive hand shape: The shape of the second, or passive hand, is also involved in the description of a two-handed sign. • Body contact: This indicates whether or not the hands make contact with the body. • Hand movement: The relationship between the shape of and the movement made by each hand plays a role in two-handed signs. The different LSE words were then analysed and catalogued according to these characteristic factors, and the organisation of the sign-based search criteria was defined. We adopted the initial active hand shape as the first criterion for starting a search, because it is the key feature for representing a sign. For this purpose, we first had to select the initial active hand shapes. We started by examining the two existing Spanish classifications [16,17] and how initial active hand shapes were arranged in other countries [18], which, although based on different languages, was a potential source of underlying criteria. At the end of this thoroughgoing process, we concluded that there were 49 active hand shapes for the signs entered in the dictionary, grouped, according to linguistic criteria, into 11 hand-shape families. Initial active hand 1 hand with contact

with contact

without contact

2 hands ≠ shape

= shape

= movement

≠ movement

without contact

with contact

Initial passive hand without contact

with contact

without contact

Fig. 1. Signs description

We were then able to define the order in which other sign classification criteria, such as whether or not there is body contact, the use of two hands, whether or not the movement and the hand shape of both hands is the same, were to be considered. These criteria can identify the properties of each sign, without the need of any previous linguistic knowledge (see Fig. 1).

Bilingual Sign Language Dictionary

603

5 Software Development One of the essential aspects of system software development was to provide an interface tailored to the characteristics of users and the highly visual content of the information that was to be entered in the dictionary. Another vital point was to build a system that could include new functionalities and easily extend the dictionary contents, thereby giving the deaf community as much independence as possible and get them to identify with the tool. 5.1 Framework It was decided to use the Constructor of Relations with the User (COREUS: Constructor de Relaciones con el Usuario) framework [19] to undertake the development project. This decision was made because this is a good framework for developing applications in which the interface is likely to undergo modifications at development time, because it keeps the application software separate from the interface. Also the user interface can be tailored to the specific features of the user. To this end, COREUS works on the principle of concept-based interfacing [20], according to which each item appearing in the interface represents a concept. The interface representation (visual, audio, haptic, etc.) will depend on parameters like the users’ mother tongue, the application domain knowledge (in this case LSE), etc. Because LSE is a visual language, it is only logical that part of the information required to consult the dictionary should be graphical. This led to the definition of an interface based primarily on icons and videos to set out the information required for LSE-based searching. Interface development is influenced by the information required to run such a search. 5.2 Search Algorithm LSE-based searching depends above all on how the hands are positioned and the hand movements made to form the sign. Fig. 1 shows what sign classification criteria have been used to develop the search algorithm, and the interface that was to be used to gather information from users. Select. Family

Select Active Hand Search

Select Parameters

Select Contact

Select Two-Handed

TwoHanded?

Select Passive Hand

No

Search

Yes Initial A. Hand Shape =Initial P. Hand Shape Search

No

Yes

Select =M ovement

Passive Hand

Fig. 2. Search algorithm

Fig. 2 is a diagram of the algorithm that is responsible for running the search process. In the first place, the signs from the database are filtered by the selected family. Then the user enters the appropriate sign parameters. Finally, a list of all the signs that match the selected search pattern is retrieved from the database.

604

J.L. Fuertes et al.

6 Spanish Sign Language Dictionary (DILSE) This section briefly discusses the two search types that can be run: search a word to get its respective sign and locate a sign to get its Spanish translation. 6.1 Searching by Spanish Users type in a Spanish word to search for its respective sign or, alternatively, they can select the term from a word list. A results window is then displayed (Fig 4.b), showing all the information related to the located sign. This window displays the Spanish word, Spanish meaning and its context, Definition including an explanation in Spanish and a LSE video or an illustrative figure, Example of use, Lexical equivalences and Geographical varieties with links to other varieties. This window also includes an alphabetical listing of Spanish words, which can be used to search for other signs. 6.2 Searching by Sign To run a search based on a sign, users have to go through a number of windows to gradually define the exact target sign. First, a window showing the initial active hand shape families is displayed (Fig 3.a). Users have to choose the family to which the sign hand shape belongs. Then another window requesting information on the sign properties, according to the above-mentioned criteria, is displayed (Fig. 3.b).

Exit

Go to Spanish Sign Description

Credits

Select Hand Shape

Select Parameters

Back Select Family

Figure a

Next

Go to Games

Figure b

Fig. 3. Selection of family and search parameters

The next window offers a selection of signs that share the selected properties (Fig 4.a). Upon selecting one of these signs, a list of the Spanish words that can be used to translate the sign, together with the geographical variety to which the sign belongs is displayed. Additionally, users can view a video showing the formation of the selected sign. Information on the sign parameters (initial active and passive hand shapes, sign location, palm direction, movement, hand direction and facial expression) can be gathered at any time. If users select one of the words with its geographical variety, one last window (similar to the one viewed in Spanish-based searches) is displayed (Fig 4.b).

Bilingual Sign Language Dictionary

605

Select Sign

Meaning/Variety List

Sign Information

Figure a

Figure b

Fig. 4. Sign selection and search results

6.3 Morphological Information An additional DILSE utility is that it provides morphological information (related to classifiers and directional verbs). This information accompanies signs that undergo some sort of inflection that is dependent on the context in which they are used and has an effect on their formal structure. This means that a sign can have a different parametric structure depending on whether it is uninflected (for example, “avisar” -to warn-) or inflected (“os aviso” -I warn you-). This type of information is used for vocabulary learning through speech use contexts. An icon (representing a “C” if it is a classifier and a “V” if it is a directional verb) displayed in the results window indicates that morphological information associated with a sign is available. The classifiers window will contain a photo of the initial hand shape of the classifier and two videos (with the contextualised classifier and with an animation related to this classifier). The directional verbs window contains a list with some Spanish inflections of the selected verb and a video window showing the selected inflection in LSE.

7 Conclusions DILSE is one of the first LSE-Spanish bilingual dictionaries for deaf people whose content can be accessed from the description of the LSE sign. It uses for this purpose an interface enabling users to select the graphical properties of the sign. The interface was developed on the basis of COREUS, which was used to build a platform for creating and maintaining sign language-based dictionaries. The DILSE dictionary of neologisms [13] and the DILSE basic dictionary [12] have been developed thanks to this platform. Apart from the technological headway made thanks to platform development, another point worth considering is just how important it is to put together up-to-date and updateable electronic sign language dictionaries that are tailored to the characteristics and needs of this language and the user community. Exhaustive scientific research is a prerequisite for the standardization and full acceptance of sign language. Additionally, the only way of removing the communication barriers that the deaf community comes up against is to encourage the study, research and recognition

606

J.L. Fuertes et al.

of sign language. Bilingual dictionaries play an indispensable role in successfully achieving these goals. The platform developed on the basis of the COREUS framework has opened the door to updating and developing sign language-based bilingual dictionaries leading to the complete integration of deaf people.

References 1. Oviedo, A.: Diccionarios de Lenguas de Señas. Grupo de Estudios de Lingüística de las Lenguas de Señas, Universidad de los Andes, Mérida, Venezuela. Available at: http://www.insor.gov.co/orientaciones_generales/diccionario.htm (last visited 7-3-2006) 2. Johnston, T.: Signs of Australia on CD-ROM. AUSLAN Dictionary. Deafness Resources, Australia (1997) 3. Johnston, T.: “AUSLAN Dictionary. A Dictionary of the Sign Language of the Australian Deaf Community. Deafness Resources, Australia (1989) 4. National Deaf Children’s Society. Available at: http://www.ndcs.org.uk/ (last visited 7-32006) 5. Solina, F., Krapez, S., Jaklic, A., Komac, V.: Multimedia Dictionary and Synthesis of Sign Language. In: Rahman, S. M. (ed.): Design and Management of Multimedia Information Systems. Idea Group Publishing (2001) 268-281 6. Sign Language Online Dictionary and Lessons, ASL. Available at: http://handspeak.com, (last visited 7-3-2006) 7. British Language Online Dictionary, BSL. Available at: http://www.britishsignlanguage. com (last visited 7-3-2006) 8. Frigola, J.: Lenguaje de signos manuales. Ed. Dossat, 2nd ed., Barcelona (1998) 9. Pinedo, F. J.: Nuevo Diccionario Gestual Español. CNSE, Madrid (1988) 10. Rodrigo, J.: Signos 97-98. Diccionario de Lengua de Signos Española. Alicante (1998) 11. CNSE: Mis primeros Signos. Primer Diccionario infantil en lengua de signos. Spain (2005) 12. Martínez, F.: Diccionario de la Lengua de Signos Española (Diccionario Básico). CNSE, Spain (2000) 13. Martínez, F.: Diccionario de la Lengua de Signos Española (Diccionario de Neologismos). CNSE, Spain (2000) 14. Brien, D.: Dictionary of British Sign Language/English. Faber and Faber, London (1992) 15. Frutos, S., González, Á. L., Minchero, A., Nieto, M. J.: Diccionario de la Lengua de Signos Española (DILSE). Congreso Iberoamericano Iberdiscap 2000, Madrid, Spain (2000) 16. Rodríguez, M. A.: Lenguaje de signos. CNSE y Fundación ONCE, Madrid, Spain (1992) 17. Muñoz, I.: Aportación al estudio de la articulación de la Lengua de Signos Española en la Comunidad Valenciana. Undergraduate dissertation, University of Alicante, Spain (1996) 18. Prillwitz, S.: Hamburg Notation System (HamNoSys). Signum Press, Hamburg (1989) 19. Frutos, S., González, Á. L., Martínez, L. A., Montes, C.: Adapting Computer-Human Interaction in Real Time. In: Proc. of the IEEE International Conference on Systems, Man, and Cybernetics (SMC’98), San Diego, California, USA (1998) 1038-1043 20. González, Á. L.: Modelo para la generación y gestión en tiempo de ejecución de procesos de interacción hombre-máquina a partir de un lenguaje de especificación de relaciones con el usuario. PhD Dissertation, Universidad Politécnica de Madrid, Madrid, Spain (2003)