IT Pro is supported by its audience. When you purchase through links on our site, we may earn an affiliate commission. Learn more

AI recruitment tool pulled by Amazon for sex bias

Recruitment software was not favourable towards women

Brain, network, network intelligence, AI, computing, connected, artificial intelligence, machine learning

Amazon has been forced to scrap an artificial intelligence system used by the company for recruitment after it was found the technology was biased against female applicants.

The AI tech was first developed at Amazon in 2014 as a way of quickly filtering out most candidates and providing the firm with the top five people. However, by 2015, it was realised that the system was not rating applicants in a gender-neutral way.

The problem lay in how the system was trained. It was fed CVs to detect patterns in recruiting over a 10-year period. Most of the applications came from men. According to a report from Reuters, the system taught itself that male candidates were preferable to women. It downgraded CVs if found words such as "women's" and penalised graduates of all-female colleges.

While Amazon recoded the software to make the AI neutral to these terms, this did not guarantee that the technology would find other methods of being discriminatory against women, the report said.

The team, set up in Amazon's Edinburgh engineering hub, created 500 models concentrated on detailed job functions and locations. The system was also taught to recognise around 50,000 terms that showed up on past candidates' CVs.

The technology learned to assign little importance to skills common across IT applicants, favouring terms more commonly found on male engineers' resumes, such as "executed" and "captured," the report said.

The model used in the AI system had other problems that led to unqualified candidates being recommended for a variety of unsuitable jobs.

This eventually led to Amazon pulling the plug on the team as executives "lost hope" over the project, anonymous sources told Amazon. The tool could not be solely relied upon to sort candidates.

The firm now uses a "much-watered down version" of the recruiting engine to carry out "rudimentary chores".

Featured Resources

Accelerating AI modernisation with data infrastructure

Generate business value from your AI initiatives

Free Download

Recommendations for managing AI risks

Integrate your external AI tool findings into your broader security programs

Free Download

Modernise your legacy databases in the cloud

An introduction to cloud databases

Free Download

Powering through to innovation

IT agility drive digital transformation

Free Download

Recommended

A guide to cyber security certification and training
Careers & training

A guide to cyber security certification and training

16 Jun 2022
World’s biggest four-day working week trial kicks off in UK
flexible working

World’s biggest four-day working week trial kicks off in UK

6 Jun 2022
Pushing cloud AI closer to the edge
machine learning

Pushing cloud AI closer to the edge

1 Jun 2022
Europe's tech sector struggles to find employees with AI skills
Careers & training

Europe's tech sector struggles to find employees with AI skills

25 Apr 2022

Most Popular

FCC commissioner urges Apple and Google to remove TikTok from app stores
data protection

FCC commissioner urges Apple and Google to remove TikTok from app stores

29 Jun 2022
Former Uber security chief to face fraud charges over hack coverup
data breaches

Former Uber security chief to face fraud charges over hack coverup

29 Jun 2022
Internet providers look to ease cost of living crisis with cheaper broadband
broadband

Internet providers look to ease cost of living crisis with cheaper broadband

29 Jun 2022