What Happens When AI is Wrong?

What Happens When AI is Wrong?

I recently asked a question to ChatGPT, the artificial intelligence (AI) chatbot launched late last year. I got a well-worded response that made a lot of sense. It also agreed with my first thought on what the answer to the question would be. 

I might have even used the response to make a decision that impacted my family’s finances. Just to be on the safe side, I looked up the applicable laws in the State of California. The issue related to real estate, so I also reached out to a real-estate lawyer in California.

What was the final outcome? That well-written response from ChatGPT was completely wrong. Had I finalized my decision for my family to resolve a current dispute, it would have hurt them and made them look uninformed.

So, what happens when people start using AI to make medical decisions?

Many companies and physicians would like to start billing for “Software as a Service” (SaaS) in healthcare. 

The levels would go from telehealth services to potentially providing diagnostic and treatment services, using only AI, with no “attending physician.” 

Aside from SaaS, I am concerned that in the same way, a CPA could reach out to AI to answer a question, a physician could use AI to do the same – and get a completely erroneous answer. 

I do recognize that in the same way self-driving cars could get in accidents, on the whole, self-driving cars are safer than cars driven by humans. 

It could be argued that even if AI gives a wrong result to a question that injures a patient, the number of times it prevents a physician from reaching an incorrect diagnosis may far outweigh the number of times it is wrong.

I am also concerned that AI could help people impersonate physicians, with unscrupulous individuals using it to provide reasonable-sounding diagnostic services and treatment plans to an unsuspecting public. 

Almost certainly, patients will use AI to self-diagnose their medical conditions.

Welcome to a new world.

Facebook
Twitter
LinkedIn

Timothy Powell, CPA, CHCP

Timothy Powell is a nationally recognized expert on regulatory matters, including the False Claims Act, Zone Program Integrity Contractor (ZPIC) audits, and U.S. Department of Health and Human Services (HHS) Office of Inspector General (OIG) compliance. He is a member of the RACmonitor editorial board and a national correspondent for Monitor Mondays.

Related Stories

Leave a Reply

Please log in to your account to comment on this article.

Featured Webcasts

Mastering Principal Diagnosis: Coding Precision, Medical Necessity, and Quality Impact

Mastering Principal Diagnosis: Coding Precision, Medical Necessity, and Quality Impact

Accurately determining the principal diagnosis is critical for compliant billing, appropriate reimbursement, and valid quality reporting — yet it remains one of the most subjective and error-prone areas in inpatient coding. In this expert-led session, Cheryl Ericson, RN, MS, CCDS, CDIP, demystifies the complexities of principal diagnosis assignment, bridging the gap between coding rules and clinical reality. Learn how to strengthen your organization’s coding accuracy, reduce denials, and ensure your documentation supports true medical necessity.

December 3, 2025

Proactive Denial Management: Data-Driven Strategies to Prevent Revenue Loss

Denials continue to delay reimbursement, increase administrative burden, and threaten financial stability across healthcare organizations. This essential webcast tackles the root causes—rising payer scrutiny, fragmented workflows, inconsistent documentation, and underused analytics—and offers proven, data-driven strategies to prevent and overturn denials. Attendees will gain practical tools to strengthen documentation and coding accuracy, engage clinicians effectively, and leverage predictive analytics and AI to identify risks before they impact revenue. Through real-world case examples and actionable guidance, this session empowers coding, CDI, and revenue cycle professionals to shift from reactive appeals to proactive denial prevention and revenue protection.

November 25, 2025
Sepsis: Bridging the Clinical Documentation and Coding Gap to Reduce Denials

Sepsis: Bridging the Clinical Documentation and Coding Gap to Reduce Denials

Sepsis remains one of the most frequently denied and contested diagnoses, creating costly revenue loss and compliance risks. In this webcast, Angela Comfort, DBA, MBA, RHIA, CDIP, CCS, CCS-P, provides practical, real-world strategies to align documentation with coding guidelines, reconcile Sepsis-2 and Sepsis-3 definitions, and apply compliant queries. You’ll learn how to identify and address documentation gaps, strengthen provider engagement, and defend diagnoses against payer scrutiny—equipping you to protect reimbursement, improve SOI/ROM capture, and reduce audit vulnerability in this high-risk area.

September 24, 2025

Trending News

Featured Webcasts

Surviving Federal Audits for Inpatient Rehab Facility Services

Surviving Federal Audits for Inpatient Rehab Facility Services

Federal auditors are zeroing in on Inpatient Rehabilitation Facility (IRF) and hospital rehab unit services, with OIG and CERT audits leading to millions in penalties—often due to documentation and administrative errors, not quality of care. Join compliance expert Michael Calahan, PA, MBA, to learn the five clinical “pillars” of IRF-PPS admissions, key documentation requirements, and real-life case lessons to help protect your revenue.

November 13, 2025
E/M Services Under Intensive Federal Scrutiny: Navigating Split/Shared, Incident-to & Critical Care Compliance in 2025-2026

E/M Services Under Intensive Federal Scrutiny: Navigating Split/Shared, Incident-to & Critical Care Compliance in 2025-2026

During this essential RACmonitor webcast Michael Calahan, PA, MBA Certified Compliance Officer, will clarify the rules, dispel common misconceptions, and equip you with practical strategies to code, document, and bill high-risk split/shared, incident-to & critical care E/M services with confidence. Don’t let audit risks or revenue losses catch your organization off guard — learn exactly what federal auditors are looking for and how to ensure your documentation and reporting stand up to scrutiny.

August 26, 2025

Trending News

Happy National Doctor’s Day! Learn how to get a complimentary webcast on ‘Decoding Social Admissions’ as a token of our heartfelt appreciation! Click here to learn more →

CYBER WEEK IS HERE! Don’t miss your chance to get 20% off now until Dec. 2 with code CYBER24