Facebook has unveiled a new AI-driven tool that helps visually-impaired users ‘see’ photos. Katie Wright reports on how technology is making social media more accessible than ever

FOR more than 15 years, screen readers have used text-to-speech technology to allow people who are blind or visually impaired to hear text on their computer screens. But, until now, deciphering images has been much more difficult.

Emojis are accessible with screen readers because they have a Unicode definition attached, but photos aren't so simple - the sometimes hilarious results you get when using captionbot.ai, the app that tries to guess what's happening in your photos, can attest to that - which is why Facebook's latest unveiling is so significant. Previously, a screen reader would just announce the word 'photo' and the name of the person who shared it, but now 'automatic alternative text' can recognise things like babies and beards, sports, all kinds of foods, and whether a landscape is snowy, mountainous or a beach (so you can be sure exactly what kind of holiday your friend is bragging about).

Scroll over a photo of a forest, for example, and you'll hear "this photo may contain: outdoor, cloud, foliage, plant, tree". Ten months in the making, the tool has been made possible because of developments in the social network's object recognition technology and is the first big project engineer Matt King has been part of at Facebook.

King, who lost his sight as a result of a degenerative condition called retinitis pigmentosa, told the BBC: "On Facebook, a lot of what happens is extremely visual. And, as somebody who's blind, you can really feel like you're left out of the conversation, like you're on the outside."

For the 285 million people in the world who are blind or have severe visual impairment, the feature offers a lot more inclusivity: around two billion photos that are shared on Facebook, Instragram and WhatsApp every day. Driven by Artificial Intelligence, the system has been 'taught' using millions of example images, and tested thoroughly, and it'll continue to get better, providing richer and more accurate descriptions as it grows.

Available initially on iOS screen readers in English, software engineer Shaomei Wu says that automatic alt text will be rolled out to other languages and platforms soon. "While this technology is still nascent, tapping its current capabilities to describe photos is an important step toward providing our visually impaired community the same benefits and enjoyment that everyone else gets from photos."

GIVING VOICE TO AUTISM

TO mark World Autism Acceptance Month, Apple has released a pair of short films proving how technology can be such a powerful aid for those with the condition, which leads to difficulties with communication and social interaction. Focusing on Dillan Barmache, the clips show how the 16-year-old, who is non-verbal, uses an iPad to articulate the words he cannot. "So many people can't understand that I have a mind, all they see is a person who is not in control. But now you can hear me," Barmache writes, and his mother describes how incredible it is to hear her child's voice, albeit via computer, for the first time. Watch the moving videos on the Apple YouTube channel at youtube.com/apple.

IKEA'S I-KITCHEN

GREAT news for anyone who would rather run a marathon bare-foot over hot coals than spend a day at IKEA: the Swedish flat-pack specialist is trialling an app that lets you design your new kitchen using virtual reality. The IKEA VR Experience app, available on entertainment platform Steam using the HTC Vive headset, lets you explore three kitchen settings, altering the viewpoint to get a child's-eye view and changing the colour and finish of the fittings. "We hope that users will contribute to our virtual reality development, by submitting ideas on how to use virtual reality and how to improve the virtual kitchen," says Martin Enthed, IT manager at IKEA. You've got until August to submit your feedback - or beg for them to build an entire VR store. Download the app at store.steampowered.com.

INSTAGRAM'S BAKING BOOB

THE 'Free the Nipple' campaign argues that women should be legally allowed to go topless just as much as men, and its supporters often try to get round strict nudity rules on sites like Instagram. Which might explain why Sue Moseley's account was suspended recently - but it's not because she's a staunch FTN advocate. Fiona Mosely tweeted that her mother's account was halted after she posted a picture of a traditional Easter simnel cake. What's that got to do with mammaries? Actually, nothing. Instagram's overzealous standards software thought the little toasted marzipan balls were nipples and automatically deleted the pic. Sue's account was eventually reinstated after the Moseleys started a campaign of their own, called #FreetheSimnel.