Google to combat online toxicity with AI

Google's Perspective software uses machine learning to filter out abusive comments

In an attempt to tackle online trolling, Google has launched new software that uses machine learning to filter out comments deemed too toxic for civilised discussions.

In a partnership with technology incubator Jigsaw, Google has launched Perspective, a program designed to help publishers and website admins with comment moderation by reviewing their content and providing ratings based on how unpleasant the language is.

An online demo of the technology is available here. Type in something like "You're a stupid idiot", and the software will return a rating based on how many other people viewed the comment as toxic, in this case 98%.

Advertisement - Article continues below

According to a recent report by the US Centre for Innovative Public Health, 72% of American internet users have witnessed online harassment, with almost half (47%) experiencing toxicity first hand.

"News organisations want to encourage engagement and discussion around their content, but find that sorting through millions of comments to find those that are trolling or abusive takes a lot of money, labor, and time," said Jigsaw president Jared Cohen, speaking in a blog post.

"As a result, many sites have shut down comments altogether. But they tell us that isn't the solution they want. We think technology can help."

Advertisement
Advertisement - Article continues below

The technology has already been applied to The New York Times, where traditionally a dedicated team trawls through "an average of 11,000 comments every day", and that the sheer volume of moderation required means only 10% of its articles have published comments.

Perspective has so far examined "hundreds of thousands of comments that had been labeled by human reviewers". Each time Perspective reads a comment, or receives corrections from users, the technology becomes more accurate at scoring for toxicity, according to Google.

Advertisement - Article continues below

The API has been made available on the Cloud Machine Learning Platform, Google's hub for machine learning services available to developers, as well as open source library TensorFlow.

While in its current form it only sifts out toxic comments, Google has said it plans to improve the technology to include other languages, and the ability to spot comments that may be unsubstantiated or off-topic.

Advertisement

Recommended

Visit/infrastructure/network-internet/355233/russian-isp-intercepted-traffic-from-internet-giants-like
Network & Internet

Russian ISP intercepted traffic from AWS, Facebook, Google and more

6 Apr 2020
Visit/cloud/cloud-backup/355025/google-backup-and-sync-review-that-syncing-feeling
cloud backup

Google Backup and Sync review: That syncing feeling

17 Mar 2020
Visit/mobile/google-android/354816/android-11-developer-access-arrives-earlier-than-expected
Google Android

Android 11 developer access arrives earlier than expected

20 Feb 2020
Visit/policy-legislation/data-protection/354814/google-to-shift-uk-user-data-to-the-us-post-brexit
data protection

Google to shift UK user data to the US post-Brexit

20 Feb 2020

Most Popular

Visit/development/application-programming-interface-api/355192/apple-buys-dark-sky-weather-app-and-leaves
application programming interface (API)

Apple buys Dark Sky weather app and leaves Android users in the cold

1 Apr 2020
Visit/data-insights/data-management/355170/oracle-cloud-courses-are-free-during-coronavirus-lockdown
data management

Oracle cloud courses are free during coronavirus lockdown

31 Mar 2020
Visit/security/privacy/355211/google-releases-location-data-to-showcase-effectiveness-of-coronavirus
privacy

Google releases location data to show effectiveness of coronavirus lockdowns

3 Apr 2020