(2024-09-03) Why Ai Can Push You To Make The Wrong Decision At Work

Why AI Can Push You to Make the Wrong Decision at Work. Chatbots help craft emails and act as customer support. Algorithms help banks decide loan eligibility and assist in reviewing job applications. AI is gaining a foothold in many workplaces, aggregating information and helping people make smarter decisions. At least that’s how AI is sold.

Designed to reduce ambiguity and simplify decisions, AI algorithms can instead lead us to doubt years of expertise and make the wrong decision because of a cognitive shortcut — automation bias. Simply put, it’s our human tendency to reduce our vigilance and oversight when working with machines

Automation bias is one of many cognitive shortcuts, called heuristics, people use to make faster, more efficient decisions. People might assume automated systems like AI are more accurate, consistent, and reliable than humans and defer to those suggestions.

people are afraid to question it,” says Nyalleng Moorosi, senior researcher at the Distributed AI Research Institute. “I think the way they've been marketed is such that people really doubt themselves when faced with a decision: a disagreement between what they think is right, and what the system thinks is right.”

Automation bias can also impact clinical decisions. Usually, two radiologists are assigned to view a mammogram to assess whether someone has breast cancer. A recent study in Radiology recruited 27 radiologists to test whether an AI assistant could replace the second set of eyes. The AI assistant conducted a Breast Imaging Reporting and Data System (BI-RADS) assessment on each scan. Researchers knew beforehand which mammograms had cancer but set up the AI to provide an incorrect answer for a subset of the scans. When the AI provided an incorrect result, researchers found inexperienced and moderately experienced radiologists dropped their cancer-detecting accuracy from around 80% to about 22%. Very experienced radiologists’ accuracy dropped from nearly 80% to 45%.