Businesses blame lack of resources for not explaining AI decisions

Research finds there's no 'one-size-fits-all' approach as context behind decisions matters more than anything

AI brain

Companies using artificial intelligence (AI) more often than not cite non-tech reasons like high costs as an excuse for not fully explaining to people why these systems have come to certain conclusions.  

The main challenges that businesses face in explaining the rationale to end users behind decisions made by AI-powered systems hinge mostly on logistics and resources, rather than anything technical, the Information Commissioner's Office (ICO) has revealed.

Advertisement - Article continues below

Its interim report, published this week in collaboration with the Alan Turning Institute, was commissioned to explore how businesses approach explaining AI decisions to people affected by them, and how this can improve in future.

The fact that technical feasibility wasn't cited as a major stumbling block came as a source of relief to the ICO, as it means organisations responsible for deploying AI are confident the technology can be explained.

The UK's data regulator also claims its findings highlight a need for raising the profile of explaining AI decisions at board-level. This is so the right budget and personnel allocations can be made to address these issues. Businesses also face difficulties due to the lack of any standardised approach for internal accountability for explainable AI decisions.

"If an AI system makes a decision about an individual, should that person be given an explanation of how the decision was made?" said the ICO's senior policy officer Alex Hubbard.

Advertisement - Article continues below
Advertisement - Article continues below

"Should they get the same information about a decision regarding criminal justice as they would about a decision concerning healthcare?

"Industry roundtable participants generally felt confident they could technically explain the decisions made by AI. However, they raised other challenges to 'explainability' including cost, commercial sensitivities (eg infringing intellectual property) and the potential for 'gaming' or abuse of systems."

As part of its research, the ICO and the Alan Turing Institute hosted several roundtable discussions with industry representatives from the public, private and third sectors. Individuals and consumers were also invited to 'citizen juries' to discuss these themes.

Beyond the main challenges in explaining AI decisions, the report found there's a desire for education and awareness-raising activities to inform the public on the use and benefits of AI. It's not, however, clear which section of society should take responsibility for engaging the public on these issues.

There are also risks, the report claims, that awareness-raising can "simply serve to normalise the use of AI decisions, disproportionately emphasising its benefits" so people are less likely to question its use.

Advertisement - Article continues below

However, the strongest message to emerge, according to the ICO, is that context matters more than anything else when it comes to the expectations by individuals.

Factors such as the importance or urgency of the decision, or the power of the user to change factors influencing the decision, and even the scope for bias, play a huge role in what people expect from organisations that deploy these technologies.

Generally, people who participated in the research expected an explanation from an AI decision in the same way they would expect an explanation from human judgement. But they also questioned whether AI decisions should be held to higher standards, given the possibility for humans to harbour ulterior or selfish motives.

Overall, this suggests there's no "one-size-fits-all" approach; rather the content and delivery of explanations must adapt to the audience and context around the AI-based decisions.

"The ICO has said many times that data protection is not a barrier to the use of innovative and data-driven technologies," Hubbard continued. "But these opportunities cannot be taken at the expense of being transparent and open with individuals about the use of their personal data.

Advertisement - Article continues below

"The guidance will help organisations to comply with data protection law but will not be limited to this. It will also promote best practice, helping organisations to foster individuals' trust, understanding, and confidence in AI decisions."

The interim report will feed directly into the ICO's guidance for organisations, which will go out for public consultation over the summer before being published in full in the autumn.

Featured Resources

Top 5 challenges of migrating applications to the cloud

Explore how VMware Cloud on AWS helps to address common cloud migration challenges

Download now

3 reasons why now is the time to rethink your network

Changing requirements call for new solutions

Download now

All-flash buyer’s guide

Tips for evaluating Solid-State Arrays

Download now

Enabling enterprise machine and deep learning with intelligent storage

The power of AI can only be realised through efficient and performant delivery of data

Download now


artificial intelligence (AI)

MIT develops AI tech to edit outdated Wikipedia articles

13 Feb 2020

Most Popular

video conferencing

Zoom beams iOS user data to Facebook for targeted ads

27 Mar 2020
Server & storage

HPE warns of 'critical' bug that destroys SSDs after 40,000 hours

26 Mar 2020

These are the companies offering free software during the coronavirus crisis

25 Mar 2020
high-performance computing (HPC)

IBM dedicates supercomputing power to coronavirus research

24 Mar 2020