Artificial intelligence can choose recipes after looking at food images
Researchers have developed an
"In computer vision, food is mostly neglected because we don't have the large-scale datasets needed to make predictions," says Yusuf Aytar, an MIT post-doc who co-wrote the paper with MIT Professor Antonio Torralba.
But seemingly useless photos on social media can actually provide valuable insight into health habits and dietary preferences.
In a new paper with the Qatar Computing Research Institute (QCRI), the team trained an AI system called "
Given a photo of a food item, 'Pic2Recipe' could identify ingredients like flour, eggs and butter, and then suggest several recipes that it determined to be similar to images from the database.
"You can imagine people using this to track their daily nutrition, or to photograph their meal at a restaurant and know what's needed to cook it at home later," said Christoph Trattner, assistant professor at MODUL University Vienna who was not involved in the paper.
The system did particularly well with desserts like cookies or muffins, since that was a main theme in the database.
However, it had difficulty determining ingredients for more ambiguous foods, like sushi rolls and smoothies.
In the future, the team hopes to be able to improve the system so that it can understand food in even more detail.
The paper is set to be presented at the 'Computer Vision and Pattern Recognition' conference in Honolulu later this month.