I’ve started to put some serious thought into how to actually design the deaf transcriber app:

Next steps:

• Flesh out each mode’s scenario. (And think a bit deeper with the multi-device mode’s scenario.)
• Start exploring branding/naming and visual design options. (Also decide how subtext will be conveyed; through emoticon, color, and/or typography.)

  Filed under: Degree Project, KCAI, Learning
  Comments: Comments Off on Week 4: Sketches!

Since I’m in an “anything goes” mindset with this project, I figured it’d be good to share some of the concept devices or peripherals that I’ve stumbled across. I will pull from some of these and apply it to my project in a way that works with my thesis question.

Concept phone that translates what the caller is saying:

Enable Talk Sign Language Gloves

Heads-Up Display Captions

Sign/Voice Language Translator (SVLT)

One thing that was suggested the other day was a way of using gloves or rings to detect ASL and send it to translation software. I really like this idea; “smart” rings (and/or conductive fingernail polish??) that can send movement information to a translator. Maybe this, combined with an unobtrusive facial recognition camera (to capture facial expression and emotion) could be a solution. And maybe on the Spoken English-to-Deaf side, there could be a heads-up display (either external, like a Google Glass-style overlay, or internal/invisible, like a contact lens display) that displays transcribed speech.

I am starting to gravitate toward the idea of a mobile app that is supplemented by peripherals, instead of just an app by itself using default phone hardware. Making layers of technology invisible and/or well-designed is not only appealing and exciting to me as a geeky designer; it also excites me because I want to create fewer barriers (not more!) between hearing and deaf people. Using technology that can be successfully integrated or work behind-the-scenes in daily life is a very important aspect to consider. It would make all parties involved in the deaf/hearing conversation feel a lot less awkward. And I think it should be my focus.

  Filed under: Degree Project, Find&Share, KCAI, Learning
  Comments: Comments Off on Find & Share: Deaf Device Concepts

I summed up my previous couple of weeks’ progress in the form of a presentation. Some feedback I received includes:

• Be careful about turning this project into a cold, sterile, pragmatic, primarily geeky endeavor. Remember that these are people I am designing for. (On of my solutions to this is to make conveying emotion and inflection a priority in the UI.)
• Maybe instead of the phone/tablet-based solution seen in the sketches below, I should pursue a different type of technology, such as an ASL-detecting peripheral (hello, MX!). However, the biggest problem there would be the same issue with just plain texting; aspects of natural language, such as emotion and inflection are lost, not in translation, but in the capturing of the content. So maybe I find a way to capture facial expression too?
• Show emotion, emphasis, and subtext formally, through various “designery” things, such as expressive typography, color/gradients in the messages themselves, or icons/emoticons.

Next steps:

• Sketch some possible visual UX/UI directions for the app as a whole.
• Develop scenarios for each aspect of the app.
• In-depth sketches/wireframes of each scenario (a few directions).

And here’s an updated schedule/timeline for this project:

  Filed under: Degree Project, KCAI, Learning
  Comments: Comments Off on Week 3: Sketches and Presentation

I’ve gotten to the point where I’ve pretty much narrowed down what my app will be; a two-way ASL and spoken English translator, where both of these inputs (either ASL captured with a camera, or speech captured with a microphone) output to text on a mobile device (phone or tablet). Most of the things that I’ve researched as comparables have been limited: one-way, and not very useful (like converting text to ASL… Why not just type a message to the deaf/HoH person? They can read. You don’t have to try to fake your way through a language you don’t know… Ugh.).

Anyway, because I’m the type of person who likes to know if what I’m designing for is even feasible (which makes it all the more exciting for me), I searched around for some similar projects. Well, lo and behold, I found something creepy-exciting! (I actually shrieked when I saw this!)

Computer scientists in Scotland are developing technology that can translate sign language into text through a simple webcam. The technology, which has the potential to run on variety of camera-enabled digital devices, could be a big help to speech-impaired users trying to communicate, especially with members of the non-signing world.

[…]

PSLT [Portable Sign Language Translator] aims to help young learners with speech difficulties, to empower mobility- and speech-challenged users to issue commands to their appliances and devices, and to allow people with speech difficulties to customize their language settings to a variety of regional variations and personal preferences.

The creepiest (most serendipitous/coincidental) part is this (and I swear to the design gods that I had never seen this before today):

“The aim of the technology is to empower sign language users by enabling them to overcome the communication challenges they can experience, through portable technology.”

Isn’t that, like, exactly my thesis statement? I guess that means that something like this is ready to be designed!!!

  Filed under: Degree Project, Find&Share, KCAI, Learning
  Comments: Comments Off on Creepy-Exciting Logistics

Over the weekend, I went to the movies with my hard-of-hearing (HoH) friend. Since the theater has changed their caption policy, we could no longer view open-captioned films. This change was actually more convenient for us, because it meant we were no longer limited to specific and rare showtimes. However, instead of simply buying a movie ticket for the open-captioned screening, we had to wait at the front desk to acquire a pair of special closed-captioned glasses (which my friend tells me she’s had a problem with before). The glasses have to be programmed for each movie in the theater, so the glasses display the captions for the correct movie (because of this, we had to get to the theater a bit early to make sure it was all set up). However, captions don’t appear for previews, so if there’s a problem with the device, you don’t notice until the movie has already started. In a previous visit to the movie theater, my friend experienced exactly this, and had to walk out to the front desk to complain and get it fixed while missing the first few minutes of her movie.

The point of this story is to bring to light the issue that the Deaf community faces; being an afterthought. Many companies and businesses adhere to ADA guidelines for people with obvious physical disabilities, but deaf and HoH people are often overlooked or their accommodations are half-assed. The thing with the movie caption glasses is such a clunky process, and reveals a fine line between convenience and annoyance. And it crosses that line all the time.

Deeply inspired by this experience, I later went full-force on my personas and visual/verbal audit. I felt like it needed to be done before I could move forward with designing a solution. The users of this project are those who feel marginalized or isolated on a regular basis.

To add to all this, here is my timeline for the next 10 weeks. I’ll be adding more items as things start to progress and get more specific. My goal is to not only design an app (luckily, only for one size), but to also do a demo video or some sort of walkthrough video showcasing the app as well.

  Filed under: Degree Project, KCAI, Learning
  Comments: 1

Inspiration:

• Vocre Translate: I love the idea of the translated messages being displayed in opposite directions for each user facing opposite ends of the device. This is, of course, limited to only spoken language with a text output.
• Swype Keyboard: Typing is gesture-based, predictive, super-fast, and error-proof. This allows for effective and efficient communication. Swype is on this list because it revolutionized typing on devices and changed the way users think about communicating on a mobile device.
• The Noun Project: Their goal is very similar to mine, in that they want to create an entire visual vocabulary to act as a universal language that transcends cultural barriers.

Web Research:

• http://en.wikipedia.org/wiki/Deaf_culture: Being Deaf (with a capital D) is a different concept than just having a lack of hearing. It’s a whole culture and community of people.
• http://en.wikipedia.org/wiki/Non-verbal_communication
• http://deafnotes.com/ubb/: Message board for the Deaf community. It hasn’t been updated in a long time, but it still has a lot of great content about social issues in the community.
• http://www.ehow.com/list_6023189_issues-deaf-people-face.html: This really confirms that what I’ve noticed with my hard-of-hearing friend is typical of the community.
• http://www.ehow.com/how_7371407_teach-students-using-concept-mapping.html: This gives some insight into how to teach the concepts of vocabulary to a deaf student.
• http://www.ehow.com/how_7381085_teach-writing-deaf-children.html and http://www.ehow.com/how_5079105_teach-deaf-students-read.html: These are reminders that English and ASL are very different languages, and that there is a bit of a gap between the two.

Observations I’ve Already Made:

One of my best friends is hard-of-hearing (very much so, I might add), and spending some time observing how she handles everyday situations is fascinating and disheartening at the same time. While she can hear most mid-range sounds, she is completely deaf to higher-pitched, higher-frequency sounds like sirens, spoken consonants, cell phones, and most television sounds (which she tells me are picked up differently in her hearing aids than real-life sounds). Something as simple as ordering a drink at Starbucks can be an uphill battle for her. She can communicate her order just fine, but understanding a barista’s questions like, “do you want whipped cream?” is always met with a reply of “I’m hard of hearing; can you repeat that?” or a questioning look in my direction (since we’re close friends, she’s used to my speech patterns, and I will sometimes throw some ASL in if possible to make things easier, although we’re both a bit rusty in our practice of it).

Some things that stood out in this experience:

• Hearing people mumble a lot and speak very quickly. We don’t realize that we do it, but it can be impossible for a hard-of-hearing person to understand, especially in a noisy environment or when someone isn’t directly facing the recipient. Inversely, I noticed that when I talk to my hard-of-hearing friend, I make up for this by enunciating clearly and speaking at a very conscious speed.
• Not everyone who is deaf (note the lowercase d) or hard-of-hearing knows ASL fluently. Even if a hearing person can communicate in this way, it might be of little help.
• Everyday errands and “small talk” are a struggle. When you can’t understand the person talking to you, you feel isolated and even alienated from mainstream culture.
• Texting and emailing have helped bridge the language gap immensely. It might seem trite, but texting is a great step forward and a powerful tool of mobile communication.

Other People to Talk to and Observe:

• Kansas School for the Deaf: Aren’t we lucky to have a Deaf school in Olathe? (Fun fact: Olathe ASL seems to be its own dialect. Pretty cool, huh?)
• Deaf Cultural Center: Also in Olathe, and also a good resource to have access to.
• Deaf Expression interpreters: It’d be interesting to see what kinds of obstacles they have noticed in communication between ASL and English.
• Message board members: It would be good to get some perspectives from people outside of the KC/Olathe area.
• People I already know. For awhile I took an ASL “class” from an interpreter/advocate. He’d be a great person to talk to. Also, one of his best friends is completely deaf since birth (unlike my close friend who is hard-of-hearing for only the past decade or so) and a member of Olathe’s large Deaf community (also unlike my close friend). My brother-in-law minored in ASL, and I’m sure he’d have some interesting perspectives as well.

  Filed under: Degree Project, KCAI, Learning
  Comments: Comments Off on Inspiration, Research, and Obervations

Problem: There is a language barrier between spoken English and American Sign Language (ASL). We need a conversational solution for when the use of interpreters, writing, and texting aren’t available, effective, or efficient enough for the situation.

Possible solutions:

• A mobile app that, on the Deaf side, uses a set of nouns, verbs, emotions, and gestures that can be assembled into sentences quickly by the Deaf user. It would output through text on the device’s screen or text-to-speech software through the device’s speakers to “translate” to the hearing recipient. On the hearing/speaking side, it would have “live captions”, turning spoken language into text, and inflections and mood into a UI change or icons (or some other way of depicting emotional changes, possibly through typography) on the screen.
• A kind of messenger app that strictly uses an icon-based system, without relying on text/verbal language. This would be a kind of universal language between users, eliminating the need for translation, instead of facilitating it. Like texting, but with a non-verbal vocabulary (because ASL and English aren’t directly translatable).

Perhaps the most exciting thing I’ve discovered so far is that nothing quite like what I have in mind exists at the moment (a two-way tool specifically designed to replace or supplement interpreters, lip-reading, writing things down, or even translation itself). In addition to overcoming language barriers between ASL and spoken English, I also want to find a way to maintain nonverbal elements of communication, like emotion/expression, subtext, inflection, etc. These are very important elements of both spoken English and ASL, but they are used/manifested in very different ways that may not translate as easily as basic verbal language does. My ultimate goal is to lose as little as possible in the translation between (or replacement of) these two methods of communication and to make communication easier.

Next Steps:

• Verbal and visual audit. I’ve already done a little bit of this, but I need to organize what I’ve already got and explore a bit more as well. I’ll be talking with some interpreters and members of the Deaf community soon.
• Communication model. This will show exactly where and how communication between hearing and non-hearing people breaks down and give insight into how it can be remedied.
• Process timeline. I need to plot out a manageable way of making all this stuff happen.

  Filed under: Degree Project, KCAI, Learning
  Comments: Comments Off on Problem, Solutions, and Next Steps

Senior Degree Project Process Book Layout by Erika Goering

A note about my decisions:

I’m using bright, cheerful colors to convey a lively, energetic, positive feel. I don’t want Deaf/hearing language obstacles to be melancholy or drab. The upbeat colors also give it a friendly feel, like a nice chat with friends.

My two very different typefaces represent the two languages, English and ASL, playing off of each other in a complementary way.

The grid has 9 columns, to force a type of balance and dynamism that takes careful consideration, just like when translating between languages.

  Filed under: Degree Project, KCAI, Learning
  Comments: Comments Off on Process Book Layout

Before the break, I settled on a project question:

How can a handheld/portable device assist the Deaf and hard-of-hearing community in better communicating with and understanding hearing people, and thus supplementing or reducing the need for hearing aids or interpreters?

Kinda wordy, right? Well, I’ve since refined it a bit and made it more digestible:

How can a portable tool be used to transcend language barriers between hearing and non-hearing people?

This refinement makes the project a bit more open, so that it doesn’t necessarily have to rely on a mobile app, and it’s not limited to just being a caption machine (I don’t necessarily want to create something that “hears” for a person). It can now work both ways; Deaf-to-hearing and hearing-to-Deaf. The focus is no longer only on the Deaf side; it’s in the flow and meaning of conversation itself, so that everyone understands what is being said (or signed).

  Filed under: Degree Project, KCAI, Learning
  Comments: 1

How can a handheld/portable device assist the Deaf and hard-of-hearing community in better communicating with and understanding hearing people, and thus supplementing or reducing the need for hearing aids or interpreters?

ASL and English are two different languages. How can I bridge that gap visually, through a user experience? That line of thinking alone excites the designer in me. This is gonna be awesome.

Also, a little relevant find & share: http://www.youtube.com/watch?v=sjln9OMOw-0

  Filed under: Degree Project, KCAI, Learning, Visual Advocacy
  Comments: Comments Off on Degree Project Question