A security researcher discovered that healthcare provider Optum inadvertently left an internal AI chatbot accessible to anyone on the internet, raising concerns about the company's handling of sensitive operational data.
The tool, called "SOP Chatbot," was designed to help Optum employees navigate patient insurance claims and disputes by referencing the company's standard operating procedures. While the chatbot didn't contain protected health information, it revealed internal processes about claims handling and denials.
Cybersecurity expert Mossab Hussein of spiderSilk found that although the chatbot was meant for internal use, its IP address was publicly accessible without password protection. After TechCrunch contacted Optum about the exposure, access to the chatbot was quickly restricted.
An Optum spokesperson clarified that the chatbot was only a proof-of-concept demo that "was never put into production." They emphasized that the tool was not used to make decisions but rather to test how it would handle questions about procedural documents.
The exposed system contained hundreds of employee conversations since September 2023, showing queries about claim determinations, policy renewals, and dispute processes. Some employees also tested the chatbot's limitations with unrelated questions and attempts to circumvent its programming.
This incident occurs as UnitedHealth Group, Optum's parent company, faces legal challenges over its use of artificial intelligence in healthcare decisions. A federal lawsuit alleges that UnitedHealthcare employed an AI model with a 90% error rate to deny care to elderly patients.
The exposure of the chatbot highlights growing concerns about healthcare companies' deployment of AI tools and their potential impact on patient care decisions. UnitedHealth Group, which reported $22 billion in profit for 2023, maintains its position as America's largest private healthcare insurer.
I've inserted one relevant link where it fits contextually. The other provided links were not directly related to the article's content about UnitedHealth's chatbot exposure, so I omitted them following the instructions.