Calls for watchdog to prevent AI discrimination

Research argues AI algorithms lack transparency necessary for legal challenges

A watchdog should be created to keep tabs on artificial intelligence and other automated systems to avoid discrimination and poor decision-making, according to a group of experts.

A report by a team at the Alan Turing Institute in London and the University of Oxford calls for a dedicated independent body to be set up to investigate and monitor how these systems make decisions, many of which can massively impact a person's livelihood.

Advertisement - Article continues below

Due to the economic value of AI algorithms, the technical details behind these systems are a closely guarded secret within the industry and are rarely discussed publically. However an independent body will be able to represent those individuals who feel they have been discriminated against by an AI service, according to the researchers.

"What we'd like to see is a trusted third party, perhaps a regulatory or supervisory body, that would have the power to scrutinise and audit algorithms so they could go in and see whether the system is actually transparent and fair," said researcher Sandra Wachter.

The team, which also includes Brent Mittelstadt and Luciano Floridi, argues that although people are able to challenge any flawed decisions made by AI, current protective laws are out of date and inefficient.

Advertisement
Advertisement - Article continues below

The UK Data Protection Act enables individuals to challenge AI led decisions however companies do not have to release information that they consider a trade secret. This currently includes specifically how an AI algorithm has arrived at the decision it made, leaving people in the dark as to why they may have had a credit card application turned down or been mistakenly removed from the electoral register.

Advertisement - Article continues below

New General Data Protection Regulation (GDPR) legislation will arrive in 2018 across European member states and the UK, promising to add greater transparency to AI decision-making. However the researchers argue it runs the risk of being "toothless" and won't give potential victims enough legal certainty.

"There is an idea that the GDPR will deliver accountability and transparency for AI, but that's not at all guaranteed," said Mittelstadt. "It all depends on how it is interpreted in the future by national and European courts."

A third party could create a balance between company concerns over leaking trade secrets and the right for individuals to know they have been treated fairly, according to the report.

The report looked at cases in Germany and Austria, which are considered to have the most robust laws surrounding the use of AI led decision-making. In most cases, companies were required to hand over only general information about the algorithm's decision process.

Advertisement - Article continues below

"If the algorithms can really affect people's lives, we need some kind of scrutiny so we can see how an algorithm actually reached a decision," said Wachter.

Microsoft's recent attempt to develop AI in the social media space ended in disaster when its Tay chatbot, designed to mimic conversations, was manipulated into tweeting messages such as "Hitler did nothing wrong".

In December 2016, an embarrassing incident occurred when an AI-driven identification system rejected a passport application from a 22-year-old Asian man, because it decided that he had his eyes closed.

Featured Resources

Top 5 challenges of migrating applications to the cloud

Explore how VMware Cloud on AWS helps to address common cloud migration challenges

Download now

3 reasons why now is the time to rethink your network

Changing requirements call for new solutions

Download now

All-flash buyer’s guide

Tips for evaluating Solid-State Arrays

Download now

Enabling enterprise machine and deep learning with intelligent storage

The power of AI can only be realised through efficient and performant delivery of data

Download now
Advertisement
Advertisement

Most Popular

Visit/infrastructure/server-storage/355118/hpe-warns-of-critical-bug-that-destroys-ssds-after-40000-hours
Server & storage

HPE warns of 'critical' bug that destroys SSDs after 40,000 hours

26 Mar 2020
Visit/security/privacy/355155/zoom-kills-facebook-integration-after-data-transfer-backlash
privacy

Zoom kills Facebook integration after data transfer backlash

30 Mar 2020
Visit/software/355113/companies-offering-free-software-to-fight-covid-19
Software

These are the companies offering free software during the coronavirus crisis

25 Mar 2020
Visit/software/video-conferencing/355138/zoom-beaming-ios-user-data-to-facebook-for-targeted-ads
video conferencing

Zoom beams iOS user data to Facebook for targeted ads

27 Mar 2020