Sign In

Communications of the ACM

ACM TechNews

Facebook's AI Can Caption Photos For the Blind on Its Own

View as: Print Mobile App Share:
From left, Matt King, Jeff Wieland, and Shaomei Wu of Facebook's Accessibility Team.

Facebook's Accessibility Team is working on an artificial intelligence tool that will automatically describe photos to the visually impaired.

Credit: Facebook

Facebook's Accessibility Team says it is developing an artificial intelligence tool that will be able to automatically describe photos posted on the social network to blind users.

Matt King, a member of the team who lost his sight in college, says the tool already offers several capabilities. He notes it can describe the objects in a photo, whether it is set inside or outdoors, and to some extent what the people in the scene are doing; for example, whether or not they are smiling. Although it is still limited, King says the current progress is promising.

The tool is built on deep-learning techniques such as those used by Facebook to identify people in pictures, and is part of the Accessibility Team's broader goal of making Facebook easier to use for those with disabilities.

The lab also is working on closed-caption technologies and tools such as mouth-controlled joysticks, which enable people who cannot use their hands to use Facebook.

Facebook researchers also are trying to open up the development process to those with disabilities, and they recently modified Facebook's open source app development tool React to work with text-to-speech readers and other software that aids the disabled.

From Wired
View Full Article


Abstracts Copyright © 2015 Information Inc., Bethesda, Maryland, USA


No entries found

Sign In for Full Access
» Forgot Password? » Create an ACM Web Account