Purdue ASL Database

for automatic recognition of American Sign Language

This database consists of several videos of a set of ASL (American Sign Language) signs. The database is divided in three main parts: i) a set of videos of signs with distinct motion patterns, ii) a set of words with distinct handshapes, and iii) several ASL sentences to study prosody and sentence structure.

Fourteen native signers participated in the acquisition of the data. This data, for five of the participants, is now available. Due to its large size, the database will only be available on removable HDs or DVDs.

The Database

The database consist of the following three (3) parts:

PART 1: This dataset is specifically designed for the analysis of motion in ASL. The table below summarizes the different word/sentences signed by the participants.

1. up

2. down

3. away to target-center

4. diagonal away to target-left

5. left to right (probably arc)

6. right to left (probably arc)

7. away from signer-center to target-center

8. center-addressee toward signer-center

9. diagonal from signer-center to target-left

10. diagonal from signer-center target-right

11. ?diagonal addressee-center to target-left

12. ?diagonal addressee-center to target-right

13. handshape change; close to open; facing down

14. handshape change; close to open; facing away

15. orientation vertical to orientation horizontal

16. alternating left-right arc swing

17. toward signer (from unspecified start)

18. Z-shape (zig-zag) [3 strokes – straight to right, diagonal down to left, straight to right]

19. elbow-pivot circle in vertical plane

20. elbow-pivot circles in vertical plane

21. elbow-pivot orientation out to orientation in

22. away [two-handed; cf 3 above]

23. toward [two-handed; cf 17]

24. up [two-handed; cf 1]

25. down [two-handed; cf 2]

26. to left [two-handed; cf 4, 9]

27. to right [two-handed; cf 10]

28. away from signer-head; handshape change; close to open [two-handed; cf loc & mvt 22; handshape change 13; orientation 14)

29. toward signer-face [two-handed; cf 17]

30. left to right with handshape change open to close at signer-forehead

31. diagonal down upper left to lower right at signer-shoulder to waist

32. elbow-pivot orientation up to orientation down [two-handed; cf 21]

33. elbow-pivot orientation out to orientation in [lower cheek]

34. elbow-pivot orientation out to orientation in [upper cheek]

35. contact leftside-nose to rightside-nose [cf 31]

36. contact center-forehead down to contact center-chin [cf 31, 35]

37. diagonal away from vertical basehand [fingertips oriented up]

38. bounce from mvt down to contact with vertical basehand [fingertips oriented out]

39. mvt down to contact with vertical basehand [fingertips oriented out]

PART 2: This second set of videos is composed of signs based on a limited number of handshapes. Most of the signs do not include a handshape change, but the place of articulation, direction and motions might vary.

1. Vinegar

Dive

2. Rope

Ronnie Wilbur

3. Honor

Hard-of-hearing

4. Water

Wednesday

5. Same

Forever

6. Deaf-school

Interview

7. House

Present

8. You-jerk!

Sick

9. Fruit

Cat

10. Water

Wednesday

11. East

Elevator

12. Daily

Girl

13. Father

Tree

14. None

Empty-head

15. Drink

16. Exactly

Picky-picky

17. Must

Apple

PART 3: This final set corresponds to a group of sentences useful for the study of prosody and structure in ASL. Below are the transcriptions for each of the ten (10) sentences signed by each of the participants.

NEWSPAPER READ 1-p, AWFUL #STORM IN #FLA. HOMES, CARS, TREES, DESTROY. PEOPLE ABOUT 25 DIE, ABOUT NOT SURE.

LONG-AGO 1-p LITTLE-CHILD, ENJOY CLASSES ALL. CLASS FAVORITE, LANGUAGE. BEST!

SORRY! 1-p DOWNSTAIRS. CLOTHES HEAP DIRTY HAVE-TO WASH. 1-P NOT SEE LIGHT-FLASH.

KNOW-THAT “E(MILY)” BORN GIRL BABY. HAPPEN TWO-DAYS-AGO. FIRST GIRL.

SHOCK 1-p! DISCOVER GOOD FRIEND DIE. THINK HEART-ATTACK. NOT-KNOW …SEEM SICK HE, NOT-KNOW 1-p.

FRIEND BUY COMPUTER, HAVE EVERYTHING. SEEM TO ME, IMPRESS 1-P NOTHING. EXPENSIVE!

ALWAYS MY DAUGHTER, TAP-SHOULDER, ICE-CREAM, POP. 1-p BLEW-UP, TELL-HER LATER, DON’T BOTHER ME.

WRONG YOU. MEETING STARTS TIME TWO, YOU-TELL-ME NOON. WHAT’S-THE-MATTER-WITH-YOU?

YESTERDAY FUN. FAMILY … HUSBAND, DAUGHTER. SON LEAVE GO #LAKE, SWIM ALL-DAY.

KNOW-THAT DEAF SCHOOL HAVE NEW DORM. OLD NOT DESTROY, SURPRISE ME. BUILD NEW NEXT-TO BUILD. BEAUTIFUL.

English translations:

I read in the newspaper about a bad storm in Florida. Homes, cars, and trees were all destroyed. I think there were about 25 people killed.

A long time ago when I was a child, I enjoyed all my classes. My favorite class was language.

I'm sorry I didn't see the light flashing. I was downstairs doing the laundry.

Guess what, Emily had a baby girl two days ago. It's her first daughter.

I am so shocked to learn that my good friend died. I think it was a heart attack, but I'm not sure. I didn't know that he had been sick.

Apparently, my friend bought a computer that has everything. I'm not that impressed because it's so expensive!

My daughter always bothers me for ice cream or soda. I lost my temper and told her to not bother me till later.

You were wrong! The meeting started at 2 pm. You told me that it was at noon. What's wrong with you?

Yesterday was fun. My family, including my husband, son and daughter, went to the lake and swam all day.

Guess what, our Deaf school has new dorm! The old one was not destroyed which really surprised me. The beautiful new dorm was built right next to the old one.

The Videos

The videos are stored in a set of DVDs.

A first set of 42 DVDs, which includes 5 of the 14 subjects, will be available soon. These will be copied in a single HD.

Every subject was asked to place the arms down before and after each sign. Each sign has been extracted from each session and stored in a separate file. The first frame of the video is the previous frame to the one where motion starts (arms still down). The last frame is the one were the relevant motion stops; i.e., when the arms are completely down again.

Some On-line Sample Videos

Please, respect the copyright of these videos. The use of these video samples (or any part) without written permission is unauthorized and can be legally prosecuted. You can get permission to use the videos shown here by following the instructions described below. If you want to use these videos, you will also need to comply with the subjects requirements as stated in their consent forms.

· Video sample 1 (Motion example)

· Video sample 2 (handshape examples)

· Video sample 3 (Sentence example)

How to obtain the database

Please contact Prof. Ronnie Wilbur at Purdue University.

Documentation

You might want to read the following report to get more insights and details about our database:

· “The RVL-SLLL ASL Database”

A.M. Martínez, R.B. Wilbur, R. Shay and A.C. Kak

Proc. IEEE International Conference Multimodal Interfaces, 2002

pdf

Acknowledgments

This research was supported by US National Science Foundation, NSF grant No. 99-05848-BCS. This database was made possible through a collaboration between the Robot Vision Lab (School of Electrical and Computer Engineering) and the Sign Language Linguistics Lab (Linguistics -- AUS dept.) at Purdue University. The dataset could not have been collected without these collaborative efforts and the contribution of the Deaf Community in Lafayette and Indianapolis.