Google to combat online toxicity with AI

Google's Perspective software uses machine learning to filter out abusive comments

In an attempt to tackle online trolling, Google has launched new software that uses machine learning to filter out comments deemed too toxic for civilised discussions.

In a partnership with technology incubator Jigsaw, Google has launched Perspective, a program designed to help publishers and website admins with comment moderation by reviewing their content and providing ratings based on how unpleasant the language is.

An online demo of the technology is available here. Type in something like "You're a stupid idiot", and the software will return a rating based on how many other people viewed the comment as toxic, in this case 98%.

According to a recent report by the US Centre for Innovative Public Health, 72% of American internet users have witnessed online harassment, with almost half (47%) experiencing toxicity first hand.

Advertisement - Article continues below
Advertisement - Article continues below

"News organisations want to encourage engagement and discussion around their content, but find that sorting through millions of comments to find those that are trolling or abusive takes a lot of money, labor, and time," said Jigsaw president Jared Cohen, speaking in a blog post.

"As a result, many sites have shut down comments altogether. But they tell us that isn't the solution they want. We think technology can help."

The technology has already been applied to The New York Times, where traditionally a dedicated team trawls through "an average of 11,000 comments every day", and that the sheer volume of moderation required means only 10% of its articles have published comments.

Perspective has so far examined "hundreds of thousands of comments that had been labeled by human reviewers". Each time Perspective reads a comment, or receives corrections from users, the technology becomes more accurate at scoring for toxicity, according to Google.

The API has been made available on the Cloud Machine Learning Platform, Google's hub for machine learning services available to developers, as well as open source library TensorFlow.

While in its current form it only sifts out toxic comments, Google has said it plans to improve the technology to include other languages, and the ability to spot comments that may be unsubstantiated or off-topic.

Featured Resources

What you need to know about migrating to SAP S/4HANA

Factors to assess how and when to begin migration

Download now

Your enterprise cloud solutions guide

Infrastructure designed to meet your company's IT needs for next-generation cloud applications

Download now

Testing for compliance just became easier

How you can use technology to ensure compliance in your organisation

Download now

Best practices for implementing security awareness training

How to develop a security awareness programme that will actually change behaviour

Download now


cloud computing

Google adds partners to real-time translation tools

8 Jan 2020

The IT Pro Products of the Year 2019: All the year’s best hardware

24 Dec 2019
search engine optimization (SEO)

Google is getting worse as it does more

21 Dec 2019

The IT Pro Podcast: Is the future multi-cloud?

29 Nov 2019

Most Popular

data governance

Brexit security talks under threat after UK accused of illegally copying Schengen data

10 Jan 2020
cyber security

If not passwords then what?

8 Jan 2020
web browser

What is HTTP error 503 and how do you fix it?

7 Jan 2020
Policy & legislation

GDPR and Brexit: How will one affect the other?

9 Jan 2020