Facebook's AI tech can spot hate speech by reading text from photos and videos

'Rosetta' system uses machine learning to understand text in images and videos so the social network can monitor hateful content

penguin meme - text being recognised

Facebook has created an artificial intelligence-powered system for reading text from photos, that it will use to spot hateful and inappropriate memes.

The system is called 'Rosetta' and uses machine learning to identify text in images and videos that can be then transcribed into something that is machine readable.

The social network has described the system in great detail in a blog post and say it is being put to use to "automatically identify content that violates our hate-speech policy".

"Understanding the text that appears on images is important for improving experiences, such as a more relevant photo search or the incorporation of text into screen readers that make Facebook more accessible for the visually impaired," Facebook said.

"Understanding text in images along with the context in which it appears also helps our systems proactively identify inappropriate or harmful content and keep our community safe."

Facebook has over two billion users and monitoring all the content posted daily is virtually impossible. However, a significant number of its users (and social media users in general) post content with written text overlaid, such as a meme, and the social network has built and deployed what it calls a "large-scale machine learning system".

The social network giant says the system can extract text from more than a billion public Facebook and Instagram images and video frames, in a number of languages, and then use a text recognition model to understand the context of the text and image together.

Rosetta has been trained on a vast data set of what Facebook calls "human-annotated" public images with text and their locations and also text on wider public images such as posters and menus.

Facebook says that Rosetta has already been widely adopted within various teams at the company and with Instagram, and noted that the text extracted from images is also being used in various machine learning models, such as those to improve its photo searching features.

Featured Resources

Key considerations for implementing secure telework at scale

Identifying the security risks and advanced requirements of a remote workforce

Download now

The State of Salesforce 2020

Your guide to getting the most from Salesforce

Download now

Fast, flexible and compliant e-signatures for global businesses

Be at the forefront of digital transformation with electronic signatures

Download now

Rethink your cybersecurity strategy for the new world

5 steps to secure the enterprise and be fit for a flexible future

Download now

Recommended

WhatsApp counters fake viral messages with new fact-check tool
communications

WhatsApp counters fake viral messages with new fact-check tool

4 Aug 2020
ByteDance investors value TikTok at $50 billion in acquisition bid
social media

ByteDance investors value TikTok at $50 billion in acquisition bid

29 Jul 2020
Michael Seibel to replace Alexis Ohanian on Reddit board of directors
chief executive officer (CEO)

Michael Seibel to replace Alexis Ohanian on Reddit board of directors

10 Jun 2020
Facebook launches Messenger app for desktop
facebook at work

Facebook launches Messenger app for desktop

3 Apr 2020

Most Popular

How to find RAM speed, size and type
Laptops

How to find RAM speed, size and type

3 Aug 2020
How to use Chromecast without Wi-Fi
Mobile

How to use Chromecast without Wi-Fi

4 Aug 2020
How do I fix the Windows 10 Start Menu if it's frozen?
operating systems

How do I fix the Windows 10 Start Menu if it's frozen?

3 Aug 2020