AI recruitment tool pulled by Amazon for sex bias

Recruitment software was not favourable towards women

Brain, network, network intelligence, AI, computing, connected, artificial intelligence, machine learning

Amazon has been forced to scrap an artificial intelligence system used by the company for recruitment after it was found the technology was biased against female applicants.

The AI tech was first developed at Amazon in 2014 as a way of quickly filtering out most candidates and providing the firm with the top five people. However, by 2015, it was realised that the system was not rating applicants in a gender-neutral way.

The problem lay in how the system was trained. It was fed CVs to detect patterns in recruiting over a 10-year period. Most of the applications came from men. According to a report from Reuters, the system taught itself that male candidates were preferable to women. It downgraded CVs if found words such as "women's" and penalised graduates of all-female colleges.

While Amazon recoded the software to make the AI neutral to these terms, this did not guarantee that the technology would find other methods of being discriminatory against women, the report said.

The team, set up in Amazon's Edinburgh engineering hub, created 500 models concentrated on detailed job functions and locations. The system was also taught to recognise around 50,000 terms that showed up on past candidates' CVs.

Advertisement
Advertisement - Article continues below
Advertisement - Article continues below

The technology learned to assign little importance to skills common across IT applicants, favouring terms more commonly found on male engineers' resumes, such as "executed" and "captured," the report said.

The model used in the AI system had other problems that led to unqualified candidates being recommended for a variety of unsuitable jobs.

This eventually led to Amazon pulling the plug on the team as executives "lost hope" over the project, anonymous sources told Amazon. The tool could not be solely relied upon to sort candidates.

The firm now uses a "much-watered down version" of the recruiting engine to carry out "rudimentary chores".

Featured Resources

What you need to know about migrating to SAP S/4HANA

Factors to assess how and when to begin migration

Download now

Your enterprise cloud solutions guide

Infrastructure designed to meet your company's IT needs for next-generation cloud applications

Download now

Testing for compliance just became easier

How you can use technology to ensure compliance in your organisation

Download now

Best practices for implementing security awareness training

How to develop a security awareness programme that will actually change behaviour

Download now
Advertisement

Recommended

Visit/careers/28212/a-guide-to-cyber-security-certification-and-training
Careers & training

A guide to cyber security certification and training

13 Jan 2020
Visit/email-clients/19598/hotmail-outlookcom-upgrades-your-questions-answered
Software

Hotmail.co.uk migration to Outlook.com: Qs answered

11 Nov 2019
Visit/careers/28219/it-manager-job-description-what-does-an-it-manager-do
Careers & training

IT manager job description: What does an IT manager do?

28 Oct 2019
Visit/business-strategy/31780/the-it-pro-panel
Business strategy

The IT Pro Panel

28 Oct 2019

Most Popular

Visit/policy-legislation/data-governance/354496/brexit-security-talks-under-threat-after-uk-accused-of
data governance

Brexit security talks under threat after UK accused of illegally copying Schengen data

10 Jan 2020
Visit/microsoft-windows/32066/what-to-do-if-youre-still-running-windows-7
Microsoft Windows

What to do if you're still running Windows 7

14 Jan 2020
Visit/hardware/laptops/354533/dell-xps-13-new-9300-hands-on-review-chasing-perfection
Laptops

Dell XPS 13 (New 9300) hands-on review: Chasing perfection

14 Jan 2020
Visit/operating-systems/25802/17-windows-10-problems-and-how-to-fix-them
operating systems

17 Windows 10 problems - and how to fix them

13 Jan 2020