Microsoft’s new AI tells stories based on your photos

But the AI struggles to describe pictures as anything but “awesome”

Microsoft's latest AI system can automatically caption photos based on the people and objects in them, and the context they were taken in.

In development at Microsoft Research, the aim of the project is to explain what appears in a picture, as well as what seems to be happening and how it might potentially make a person feel, the researchers told Live Science.

Computers are already at work identifying what is in photos, such as Facebook's auto-tagging and Google's image search, which is capable of matching similar images based on colour, objects and even postures.

Facebook's own AI development group, FAIR, last year revealed its own AI can recognise images in photos, while in April this year it started captioning photos for blind people via its own Automatic Alternative Text engine.

Advertisement
Advertisement - Article continues below

In order to build its own system, Microsoft's researchers used deep learning methods, for instance, getting the computer to learn how to identify cats in photos by analysing thousands of examples of cats.

Scientists fed more than 8,100 new images into its storytelling system to analyse what stories it generated.

An image captioning program might take five images and suggest: "This is a picture of a family; this is a picture of a cake; this is a picture of a dog; this is a picture of a beach."

But Microsoft's storytelling program could take those same images and suggest: "The family got together for a cookout; they had a lot of delicious food; the dog was happy to be there; they had a great time on the beach; they even had a swim in the water."

The system is still in it early stage for the time being.

Researchers had to gauge how effective the system was at generating stories. Because of the large number of stories the computer generated, they deicide to use an automated method. They found that this automated method rated the computer storyteller as performing almost as well as human storytellers.

However, the system still has a tough time differentiating words, and in the initial tests, has been prone to describe everything as being "awesome".

Margaret Mitchell, study senior author and a computer scientist at Microsoft Research, said the future for this technology is to "help people share their experiences while reducing nitty-gritty work that some people find quite tedious".

This is the latest AI experiment from Microsoft's research arm, which also let loose the disastrous Tay chatbot on Twitter earlier this year.

Amazon recently built out its own AI division with the purchase of Orbeus, an image recognition firm.

Featured Resources

The IT Pro guide to Windows 10 migration

Everything you need to know for a successful transition

Download now

Managing security risk and compliance in a challenging landscape

How key technology partners grow with your organisation

Download now

Software-defined storage for dummies

Control storage costs, eliminate storage bottlenecks and solve storage management challenges

Download now

6 best practices for escaping ransomware

A complete guide to tackling ransomware attacks

Download now
Advertisement

Recommended

Visit/mobile/23617/the-best-smartphones-to-buy
Mobile

Best smartphone 2019: Apple, Samsung and OnePlus duke it out

20 May 2019

Most Popular

Visit/cloud/microsoft-azure/354230/microsoft-not-amazon-is-going-to-win-the-cloud-wars
Microsoft Azure

Microsoft, not Amazon, is going to win the cloud wars

30 Nov 2019
Visit/mobile/mobile-phones/354273/pablo-escobars-brother-launches-budget-foldable-phone
Mobile Phones

Pablo Escobar's brother launches budget foldable phone

4 Dec 2019
Visit/network-internet/wifi-hotspots/354283/industrial-wi-fi-6-trial-reveals-blistering-speeds
wifi & hotspots

Industrial Wi-Fi 6 trial reveals blistering speeds

5 Dec 2019
Visit/hardware/354237/five-signs-that-its-time-to-retire-it-kit
Sponsored

Five signs that it’s time to retire IT kit

29 Nov 2019