On Autism Awareness Day, App Developers Are Tackling the Really Hard Problems

PORTLAND, OR, April 4, 2016 (Newswire.com) - ​​​​The iPad has been a game-changer in autism intervention ever since it was launched — but ambitious startups are taking aim at harder and harder challenges, more than ever before.

April 2nd is Autism Awareness Day, and it has become a rallying point for communities to show solidarity with people with autism - from wearing a blue ribbon to work, to lighting up the Empire State Building in blue. Online, too, Autism Awareness Day feeds a frenzy of interest in autism - and particularly in autism-related apps. Ever since the iPad came out in 2010, developers have been creating a bewildering variety of apps that address nearly every aspect of autism intervention - speech, communication, social awareness, academics, and even online speech therapy.

"Our big breakthrough came from personally observing parents communicating with kids with autism. We realized that if we prompted a child with the right set of questions every time they picked a word, we could build a model of the meaning they were trying to convey — even if they couldn't express it verbally."

Ajit Narayanan, CEO, AVAZ

However, a new generation of ambitious developers is taking on even bigger challenges — challenges once thought to be ‘too big’ for apps to handle. The FreeSpeech app, currently featured as a Best New App on the Mac and iPad App Store, tackles the problem of teaching children with autism how to put words together to create complex thoughts. The app lets children move picture tiles around — like building blocks — and uses powerful AI-inspired algorithms to construct perfectly grammatical sentences from the picture map. 

Lucas Steuber is a speech therapist and applied linguist who runs Portland LanguageCraft — a  clinic in Portland, OR specializing in assistive technology for children with autism. He helped develop the FreeSpeech app, and is blown away by what the app has been able to do for children on his caseload. “Some of them have difficulties stringing together more than two words to form sentences,” he says. “And that limits the complexity of thoughts that they are able to express. When I put these kids in front of an iPad with FreeSpeech on it, they are suddenly able to create 7, 8-word sentences expressing incredibly complex ideas. We’re seeing first-hand how powerfully and capably these kids can communicate, if only they’re given the right assistive technology. It’s not just changing the lives of these children — it’s changing society’s perception of their abilities.”

Communication apps were among the first apps that appeared on the App Store when the iPad was first launched. Called Augmentative and Alternative Communication (AAC) apps, these apps used pictures as a more concrete (and therefore more accessible) way for children with autism to pick out words and string them together. AAC apps typically also feature synthesized voices that convert word sequences into speech, so they become, literally, a ‘voice’ for children with complex communication needs. 

Ajit Narayanan is the CEO of Avaz Inc, the company that built the FreeSpeech app. Narayanan and his team are based in India, and had earlier built Avaz — one of the first AAC apps on the App Store. 

“Language is more than just words - the true power of language comes from grammar, which is the ability to put these words together in billions of ways to create sentences that capture exactly what you want to say,” says Narayanan. “While linguists like Noam Chomsky have been studying the structure of grammar for decades now, no one had really built a ‘grammar algorithm’ that kids with disabilities can use to put pictures together.”

Narayanan and his team started looking at the problem of computers generating language from pictures back in 2012, based on feedback they were getting from Avaz users. Narayanan says they often despaired of solving an ‘unsolvable’ problem. “When we started working on grammar, it was 10% engineering and 90% science,” he says. “Our big breakthrough came from personally observing parents communicating with kids with autism. We realized that if we prompted a child with the right set of questions every time they picked a word, we could build a model of the meaning they were trying to convey — even if they couldn’t express it verbally.”

Narayanan described his ideas in a TED talk in 2013 - a talk that went viral with a million-plus views - but it took many years of research to assemble those ideas into a working app validated by clinical studies. In a world where many popular apps are developed in weeks, or even days, a 4-year-long gestation period is unusual, to say the least. But it’s paid off: FreeSpeech, which was launched in February, was Apple’s #1 Best New App on the App Store, and the Mac version, launched yesterday, also featured right at the top of the Mac App Store. 

More importantly, the fundamental scientific advances in FreeSpeech are helping speech therapists dramatically improve intervention outcomes. Alaina Kelley is a speech therapist with the Center for Speech, Language and Learning Inc, in Oakdale, MN. She’s an early user of FreeSpeech, having been a part of its long pre-release testing process. “Kids I work with had a huge difficulty with tense or agreement,” she says. “When I ask them ‘Have you eaten?’, some of them reply ‘I eat’ — they don’t really understand what the word ‘have’ is doing in that sentence. Others have trouble understanding why you would say ‘He eats’ and not ‘He eat’ — these concepts, which come so naturally to native English speakers, are challenging for kids with language impairments.”

When Kelley started using FreeSpeech with her kids, she could see an immediate response. “I can create a sentence like ‘I eat’ - and with a single tap on a picture, I can change that to ‘I have eaten’. Or I can drag the word ‘he’ and drop it on ‘I’ to show them the contrast between ‘eat’ and ‘eats’."

Steuber concurs with Kelley's assessment. "FreeSpeech is a game-changer in how kids with special needs view and manipulate language," he says. "It’s definitely a new generation of apps, and it's wonderful to see researchers and inventors - not just app developers - jump into the app world and try to make a difference."