Must-Ask Questions for AI Vendors

At the recent HCIF conference in Napa, I led a roundtable on a critical question: how do you vet AI vendors before they ever touch your clinical documentation? With every startup claiming to be the next AI breakthrough, it’s on you to separate game-changers from landmines. This checklist comes from frontline experience running an agency and building Apricot—here’s what I’d ask before trusting anyone with your patient data.

1. Make Them Sign a BAA. Non-negotiable. If they hesitate on a Business Associate Agreement, walk. Period.
   
   
2.  Is it Just a Flashy Demo? If they can’t show you real, live demos with actual workflows and clinician users in the field, it’s a red flag. Home health isn’t a playground for flashy “move fast and break things” tech bros.
   
   
3. Do They Record Ambiently in Patient Homes? This is a compliance landmine. If the answer is yes—run. What happens when a patient says no? What’s the fallback? How do they reconcile discrepancies between recordings and documentation? Recording isn’t documentation. Relying on raw transcription alone is risky and almost certainly noncompliant. “But they delete the recording” - run faster.
   
   
4. Is 90%+ of the Tool Actual AI? If it’s mostly rule-based logic or glorified RPA, it’s not AI. No machine learning or LLM? No sale.
   
   
5. Do They Really Understand Home Health? If their explanations of your space feel off, they probably don’t get it. That’s a liability.
   
   
6. Can They Name 3 Customers—and Can You Call One? No references, no trust.  Bonus points if they’ll let you do a ride-along with a clinician using their platform in the field.
   
   
7. Can They Explain Their Compliance Posture in Detail? If they can’t walk you through CMS rules, documentation standards, and anti-upcoding policies—or can’t bring in a credible compliance expert—you’re handing a sharp object to a baby.
   
   
8. Do They Have BAAs With Their Vendors? They must have signed BAAs with every vendor, including LLM providers like Claude or OpenAI. Training data restrictions included.
   
   
9. Will They Submit to a Security Review? If they hide behind legalese or dodge your IT team’s questions, it’s a red flag. Demand transparency on architecture and data flow.
   
   
10. Who Signed Off on Their Compliance? If no one has formally approved their workflows against CoPs, anti-fraud policies, and documentation standards, you’re playing with liability.
    
    

Bottom line

If you’re serious about AI in home health, demand vendors who respect compliance, privacy, and clinical nuance. Anything less is a risk you can’t afford.