Google to combat online toxicity with AI

Google's Perspective software uses machine learning to filter out abusive comments

In an attempt to tackle online trolling, Google has launched new software that uses machine learning to filter out comments deemed too toxic for civilised discussions.

In a partnership with technology incubator Jigsaw, Google has launched Perspective, a program designed to help publishers and website admins with comment moderation by reviewing their content and providing ratings based on how unpleasant the language is.

An online demo of the technology is available here. Type in something like "You're a stupid idiot", and the software will return a rating based on how many other people viewed the comment as toxic, in this case 98%.

Advertisement - Article continues below

According to a recent report by the US Centre for Innovative Public Health, 72% of American internet users have witnessed online harassment, with almost half (47%) experiencing toxicity first hand.

"News organisations want to encourage engagement and discussion around their content, but find that sorting through millions of comments to find those that are trolling or abusive takes a lot of money, labor, and time," said Jigsaw president Jared Cohen, speaking in a blog post.

"As a result, many sites have shut down comments altogether. But they tell us that isn't the solution they want. We think technology can help."

Advertisement - Article continues below

The technology has already been applied to The New York Times, where traditionally a dedicated team trawls through "an average of 11,000 comments every day", and that the sheer volume of moderation required means only 10% of its articles have published comments.

Perspective has so far examined "hundreds of thousands of comments that had been labeled by human reviewers". Each time Perspective reads a comment, or receives corrections from users, the technology becomes more accurate at scoring for toxicity, according to Google.

Advertisement - Article continues below

The API has been made available on the Cloud Machine Learning Platform, Google's hub for machine learning services available to developers, as well as open source library TensorFlow.

While in its current form it only sifts out toxic comments, Google has said it plans to improve the technology to include other languages, and the ability to spot comments that may be unsubstantiated or off-topic.

Featured Resources

Preparing for long-term remote working after COVID-19

Learn how to safely and securely enable your remote workforce

Download now

Cloud vs on-premise storage: What’s right for you?

Key considerations driving document storage decisions for businesses

Download now

Staying ahead of the game in the world of data

Create successful marketing campaigns by understanding your customers better

Download now

Transforming productivity

Solutions that facilitate work at full speed

Download now


Google Android

Google Maps is testing traffic lights

9 Jul 2020
email providers

How to share your Google Calendar

8 Jul 2020
web browser

Google Chrome 86 update could add 28% to your battery life

6 Jul 2020

Privacy groups warn against Google's acquisition of Fitbit

2 Jul 2020

Most Popular

Google Android

Over two dozen Android apps found stealing user data

7 Jul 2020

How to find RAM speed, size and type

24 Jun 2020

The road to recovery

30 Jun 2020