TAPS, or Trigger-Activated Personal aSsistant, (formerly TACS), gives a voice to people who have limited mobility or difficulty speaking, including those with ALS, Cerebral Palsy or brain injuries. The user can simply tap wearable Bluetooth triggers to speak customizable phrases through an app on their phone. It could also be used as a hands-free phone interface: controlling lights & door locks, screen readers, and other apps like music players and the camera.

DrumPants Triggers and Control Box

The triggers are wearable or can be mounted to a wheelchair or cane, offering larger targets to hit than touchscreen devices, which is helpful for those with limited motor control. For quadriplegics, we successfully mounted the DrumPants pressure-sensitive foot pedals to the headrest of a wheelchair. The user could then trigger actions by pressing their head in various directions.

How to Wear Diagram

The prototype is built upon the open-source DrumPants wearable controller, and we've gotten valuable feedback and overall enthusiasm for the possibilities of using these controllers to help people in all situations: from the tracheotomy patient communicating with their loved ones, to the mother tapping to call to her child in the other room, we continue to hear more use cases from everyone we've talked to.

Solutions for the Exemplars

Jason DaSilva

Jason is a filmmaker, and is looking for solutions to assist his creative expression.

TAPS can trigger the camera on your phone, to capture still photographs or video in the moment. Beta tester Ryan carries his phone on his backpack’s strap, so the camera is facing whatever he’s seeing when he’s out and about.

Whenever he sees something interesting, he just taps the sensor on his leg to instantly snap a photo. This allows him to capture moments that are gone too quickly to fiddle with your phone, and allows him to capture candid moments that would be ruined by a camera.

Try it: Use the “InstaCamera” preset in the DrumPants Android app. First you must install the InstaCamera app.

Gus Chalkias

From the video, Gus is using the TalkBack feature of his phone which speaks back each letter he types individually. In the future, DrumPants TAPS could be used as a typing tool to insert common words and phrases, or to quickly respond to emails and texts.

Using the standard DrumPants functionality–triggering sounds–you can tap your clothes to make noises on your phone so you can find it easily.

And of course the DrumPants existing functionality can be used for artistic expression, like drumming and performing live music. Combine DrumPants with the iVoxel vocoder app and you can have autotune karaoke sessions!

Try it: Use one of the DrumKit sound banks in the app to play sounds. For iOS, you can use the “MIDI Preset” in the app to control the 3rd party iVoxel app.

Xian Horn

“I should be able to all Siri-like things without taking my phone out of my bag. Perhaps a button I can press on my canes or body would be ideal.” Source

Users can attach triggers to canes or wheelchairs for access to shortcuts on their phone: answer texts, turn on the lights, unlock your door, pause/play/skip your music player or TV, etc. Currently, the Android app can activate the voice recognition (Siri equivalent) by just tapping a DrumPad, so you don’t have to pull your phone out of your bag to give voice commands.

From initial testing, the DrumPants control box is perhaps too large to fit well on a cane, but in the future we hope to design more streamlined Bluetooth sensors for this use case.

Try it: Use the “Android Commander” preset in the Android app to activate voice recognition and control the music player app on your phone.

Paul Kotler

Paul types out the phrases he wishes to speak on his touchscreen. However, sometimes that may be cumbersome when the responses are short or too common to take the time to type them. The DrumPants triggers can be programmed with the phrases he uses most often, so he can activate them with a single tap, without even having to look at or type on the screen.

“Paul relies on a synthetic voice to speak the words he types and wishes he could speak with his own voice.”

You can record samples of your own voice and trigger their playback with the DrumPants app. We also are exploring adding different voices using a proprietary speech synthesis engine.

Try it: For text-to-speech, use the “Talker” preset in the iOS or Android app.

Adding custom sounds is only supported on the iOS DrumPants app at the moment. Here are instructions on how to record and use your own sounds: http://youtu.be/tFQfuYYgBco

Feedback We’ve Received

Many users are excited for the possibilities that a lightweight, wireless, wearable controller can offer. Here are a few ideas we’ve received from beta testers:

Customizable voices: a male voice for male users (one user even asked for a Darth Vader voice!)
- We are exploring adding a proprietary voice synthesis engine instead of the female voice that comes standard with Android and iPhone.
Using the DrumPants as a music therapy device for people with autism to communicate and socialize with others using the power of live music and improvisation.
Answering phone calls and reading texts and emails with a tap.
- Currently, Android does not allow other apps control over answering the phone. However, we hope that with the visibility and credibility this Connectability Challenge can give us, we can negotiate a solution with Google that opens this to everyone.
- Reading texts and emails is possible by integrating with Android’s screen reader system, however it is a significant programming challenge we hope to tackle in the future.
Compatibility with other assistive devices.
- There are other switch systems available that may be able to use the DrumPants wearable triggers by themselves, as a wearable upgrade to existing systems.
- We are also investigating integrating with the open-source Tecla software to expand their scope to wearable controllers.
Waterproofing the sensors and making them smaller.
- We have early designs for removing the wires and DrumPants control box from the equation, and storing the Bluetooth chip and battery in the DrumPad fabric strips themselves. With enough interest and feedback from users like you, we can raise investment to make this a reality!

How We Will Use the Prize Money

As we are completely bootstrapped at this moment, we do not have the luxury of investor money to spend on development and production costs. All development and testing thus far has been by volunteers in their spare time.

The prize money will help us give more DrumPants hardware to a wider variety of testers at universities and clinics in North America, as well as implementing the most requested features in the app, like licensing a proprietary voice synthesis engine for male voices.

We hope to create a solution that is less expensive than the other similar communication systems available, and one that is easy to transport and use in everyday situations. For instance, a popular Bluetooth switch system is the AbleNet Blue2. However, it is bulky, and costs the same price as DrumPants, but it only offers two switches instead of the DrumPants’ six.

As the DrumPants are open-source hardware, others can also use them to create other unique solutions, and the community is constantly coming up with new uses and games for wearable controllers. We hope this Challenge will inspire others to create wearable solutions catering to users with different abilities.