What Happens When AI is Wrong?

What Happens When AI is Wrong?

I recently asked a question to ChatGPT, the artificial intelligence (AI) chatbot launched late last year. I got a well-worded response that made a lot of sense. It also agreed with my first thought on what the answer to the question would be. 

I might have even used the response to make a decision that impacted my family’s finances. Just to be on the safe side, I looked up the applicable laws in the State of California. The issue related to real estate, so I also reached out to a real-estate lawyer in California.

What was the final outcome? That well-written response from ChatGPT was completely wrong. Had I finalized my decision for my family to resolve a current dispute, it would have hurt them and made them look uninformed.

So, what happens when people start using AI to make medical decisions?

Many companies and physicians would like to start billing for “Software as a Service” (SaaS) in healthcare. 

The levels would go from telehealth services to potentially providing diagnostic and treatment services, using only AI, with no “attending physician.” 

Aside from SaaS, I am concerned that in the same way, a CPA could reach out to AI to answer a question, a physician could use AI to do the same – and get a completely erroneous answer. 

I do recognize that in the same way self-driving cars could get in accidents, on the whole, self-driving cars are safer than cars driven by humans. 

It could be argued that even if AI gives a wrong result to a question that injures a patient, the number of times it prevents a physician from reaching an incorrect diagnosis may far outweigh the number of times it is wrong.

I am also concerned that AI could help people impersonate physicians, with unscrupulous individuals using it to provide reasonable-sounding diagnostic services and treatment plans to an unsuspecting public. 

Almost certainly, patients will use AI to self-diagnose their medical conditions.

Welcome to a new world.

Facebook
Twitter
LinkedIn

Timothy Powell, CPA, CHCP

Timothy Powell is a nationally recognized expert on regulatory matters, including the False Claims Act, Zone Program Integrity Contractor (ZPIC) audits, and U.S. Department of Health and Human Services (HHS) Office of Inspector General (OIG) compliance. He is a member of the RACmonitor editorial board and a national correspondent for Monitor Mondays.

Related Stories

Leave a Reply

Please log in to your account to comment on this article.

Featured Webcasts

Trending News

Featured Webcasts

Ask Dr. Hirsch: Clarifying Medicare’s Most Misunderstood Rules – Part 2

Medicare regulations are complex and even seasoned professionals struggle to apply them consistently. Due to overwhelming demand, Dr. Hirsch returns for Part 2 of Ask Dr. Hirsch: Clarifying Medicare’s Most Misunderstood Rules to answer even more of Medicare’s most misunderstood questions, covering inpatient status, observation, SNF access, Medicare Advantage denials, and more. Join Dr. Hirsch as he provides clear, referenced answers to real-world questions submitted by your peers, helping you navigate Medicare compliance with confidence and clarity.

June 18, 2026

Reengineering Utilization Management: Building an Adaptive Model for the New Payer Era

Traditional utilization management models can no longer keep pace with regulatory shifts, payer scrutiny, and operational pressures. In this webcast, Tiffany Ferguson, LMSW, CMAC, ACM, ACPA-C, introduces an Adaptive Model strategy that modernizes UM through role specialization, technology-driven workflows, and proactive, team-based processes. Attendees will learn how to restructure programs to improve efficiency, strengthen clinical collaboration, and enhance financial performance in a rapidly changing healthcare environment.

May 20, 2026

Compliance for the Inpatient Psychiatric Facility (IPF-PPS): Minimizing Federal Audit Findings by Strengthening Best Practices

Federal auditors are intensifying their focus on inpatient psychiatric facilities, using advanced data analytics to spotlight outliers and pursue high‑dollar repayments. In this high‑impact webcast, Michael Calahan, PA, MBA, Compliance Officer and V.P., Hospital & Physician Compliance, breaks down what regulators are really targeting in IPF-PPS admissions, documentation, treatment and discharge planning. Attendees will learn practical steps to tighten processes, avoid common audit triggers and protect reimbursement and reduce the risk of multimillion-dollar repayment demands.

April 9, 2026

Mastering MDM for Accurate Professional Fee Coding

In this timely session, Stacey Shillito, CDIP, CPMA, CCS, CCS-P, CPEDC, COPC, breaks down the complexities of Medical Decision Making (MDM) documentation so providers can confidently capture the true complexity of their care. Attendees will learn practical, efficient strategies to ensure documentation aligns with current E/M guidelines, supports accurate coding, and reduces audit risk, all without adding to charting time.

March 31, 2026

Trending News

Celebrate Lab Week with MedLearn! Sign up to win one year of our Laboratory All Access Pass! Click here to learn more →

Have a Medicare regulation question you’d love Dr. Hirsch to answer? Now is your chance! CLICK HERE to learn more→

Happy National Doctor’s Day! Learn how to get a complimentary webcast on ‘Decoding Social Admissions’ as a token of our heartfelt appreciation! Click here to learn more →

This Memorial Day, we honor those who gave all for our freedom. Take 20% off sitewide through May 29 with code MEMORIAL26 at checkout

CYBER WEEK IS HERE! Don’t miss your chance to get 20% off now until Dec. 1 with code CYBER25

CYBER WEEK IS HERE! Don’t miss your chance to get 20% off now until Dec. 2 with code CYBER24