Recent Bluesight research found that 35% of diversion staff already use general-purpose tools like ChatGPT or Microsoft Copilot to analyze controlled substance data. Most of them are doing it without a clear picture of how those tools were built, where queries go, or what happens to data after it’s submitted.
Adopting AI in a clinical environment is not primarily a technology decision. It is a data governance decision. The compliance officers and CMOs making that call need a working understanding of AI model architecture to ask the right questions of any vendor they evaluate.
What Drug Diversion in Healthcare Has to Do With AI
Drug diversion in healthcare sits at the center of this conversation because it is where sensitive data and AI adoption overlap most directly.
Drug diversion in healthcare refers to any act that redirects a controlled substance from its intended path between the manufacturer and the patient. The medication diversion meaning in practice is broader than theft. It includes tampering with injectable drugs, substituting saline for fentanyl, and diverting opioids for personal use while continuing to provide care.
According to the CDC, outcomes include patients denied essential pain medication, providers delivering substandard care while impaired, and infection outbreaks from tampered injectables.
An estimated 10–15% of healthcare professionals have chemical dependency at rates comparable to the general population, and substance use disorder is a primary driver of diversion activity.
Diversion investigations pull from some of the most sensitive data in a health system:
- Automated dispensing cabinet transactions
- Controlled substance waste records
- ADT (admission, discharge, transfer) data
- HR and credentialing files
- Pharmacy dispensing logs
AI tools that analyze this data for behavioral anomalies and diversion patterns generate real compliance exposure if the underlying model was not designed for it. What is drug diversion in healthcare from a compliance standpoint? A patient safety problem that generates highly sensitive data. The AI tools used to detect it must be evaluated on that basis.
How General-Purpose AI Models Are Actually Trained
General-purpose models like GPT-4 and Gemini were not built for healthcare. Their training pipeline explains why.
Stage 1: Pre-Training on a Massive Corpus
The first phase of training feeds the model an enormous collection of text pulled from the internet: web pages, digitized books, code repositories, forums, and articles, totaling hundreds of billions of words. The model learns by predicting the next word in a sequence, billions of times across the full dataset.
The model does not store text the way a file system stores documents. Statistical patterns are encoded into billions of numerical parameters called weights. Those weights represent relationships across language, not retrievable records.
That architecture does not eliminate risk.
Because patterns are absorbed from everything in the training corpus, the model can surface verbatim fragments of training data when prompted in specific ways. Researchers have demonstrated that adversaries can elicit long, verbatim training sequences from deployed models.
Most vendors do not disclose the full provenance of their pre-training corpus, which means buyers cannot know what sensitive material was ingested.
Stage 2: Fine-Tuning and Feedback Loops
After pre-training, the model goes through a refinement process called Reinforcement Learning from Human Feedback (RLHF). Human annotators rate model outputs; those ratings train a reward model, and the base model is updated to favor higher-rated responses. This is the stage where user inputs can enter the training pipeline.
Whether they do depends on each vendor’s data retention settings. Using OpenAI as an example, the defaults vary by account tier:
| Account Tier | Trains on Inputs by Default | Data Retained | Notes |
| Consumer (Free/Plus/Pro) | Yes, unless opted out | Up to 30 days | Opt-out does not remove data from completed training runs |
| Team / Enterprise / API | No | Up to 30 days for abuse monitoring | Thumbs-up/down feedback can still opt specific conversations in |
| Zero Data Retention (API) | No | None | Available only to qualifying enterprise customers by request |
A signed agreement is not the same as verified configuration. Staff using personal ChatGPT accounts on work devices, or providing feedback ratings inside a business account, can create exposure that no BAA covers. Auditing account tier, retention settings, and actual employee behavior is required.
The PHI Problem
General-purpose models have no native HIPAA compliance. Using them without a Business Associate Agreement (BAA) and specific configuration risks HIPAA violation, exposes the organization to legal liability, and erodes patient trust.
A common rationalization is that administrative tasks fall outside the scope of PHI. Under HIPAA, any information that identifies a patient and relates to their care is PHI, regardless of whether the query looks clinical. That includes:
- Scheduling context tied to a specific patient
- Billing inquiries that reference a care episode
- HR records connected to a patient interaction
The use case determines the classification, not the intent of the user.
The Questions Every AI Vendor Should Be Able to Answer
A compliance officer evaluating any AI vendor for use with healthcare data should expect direct answers to the following:
| Question | Why It Matters |
| Does this model train on my inputs? | Determines whether submitted queries feed future training runs |
| Who configures data retention, and at what tier? | Reveals whether default settings expose data without active opt-out |
| Where does my data go when I submit a query? | Confirms whether data leaves a controlled environment |
| What is the model’s training corpus? Was PHI included? | Establishes baseline risk from pre-training data ingestion |
| Is there a signed BAA, and what does it cover specifically? | Distinguishes contractual protection from architectural protection |
| How does the vendor handle a breach involving model outputs? | Tests incident response readiness for a scenario unique to LLM deployment |
A general-purpose vendor may offer a BAA. That matters, but it does not resolve the training corpus question or the configuration risk. The BAA governs how the vendor handles data going forward. It does not change what the model was trained on.
Why “De-Identified” Is Not a Safe Harbor
Stripping identifying information before submitting data to a general-purpose model does not resolve the compliance risk.
HIPAA de-identification requires satisfying one of two methods, both defined by HHS:
- Expert Determination: A qualified professional applies statistical and scientific methods to certify that re-identification risk is very small
- Safe Harbor: Removal of all 18 specified identifier types, including dates, geographic data, device identifiers, and account numbers, with no remaining actual knowledge that the data could identify an individual
Both methods are manual and complex. A single missed identifier constitutes a breach.
De-identification also does not protect against inference. Research has demonstrated that LLM agents can reconstruct real-world identities by combining anonymized data with contextual cues and publicly available information, without explicit re-identification being the goal.
The data only needs to be rich enough for the model to make the connection.
Data that appears non-clinical can still expose identifiable individuals when processed by a model with broad inferential capabilities. That includes:
- Scheduling queries with timing patterns tied to specific staff or patients
- Staffing records that narrow identity by role, shift, and department
- Billing context that links a care episode to a specific individual
What a Purpose-Built Healthcare AI Looks Like
Bluesight’s Prism Assistant was built for pharmacy and compliance teams specifically, not adapted from a general-purpose tool.
It operates entirely within Bluesight’s secure environment, drawing from data already integrated into ControlCheck. That includes automated dispensing cabinet logs, controlled substance transactions, waste records, and behavioral patterns. That data does not leave the platform.
Prism Assistant was not pre-trained on internet corpora and refined through public feedback loops. It was configured for diversion-specific workflows from the start. The training corpus risk and the retention configuration risk that apply to general-purpose tools do not apply here.
The performance outcomes from teams currently using it:
- Recurring reports: completed 96% faster, from roughly six hours of manual assembly to 15 minutes
- Pre-investigation triage: 90% faster, from three hours to approximately 10 minutes
- Controlled substance variance analysis: 97% faster, from 30 minutes to under a minute
Healthcare data security software built this way removes the vendor-model-training risk at the architecture level. The data boundary is structural. It does not depend on correct configuration by individual staff members or enforcement of an opt-out policy that most users do not know exists.
What This Means for Healthcare Data Security Software Decisions
A BAA is a necessary starting point, not a sufficient endpoint. The training lineage of the underlying model, where data travels when a query is submitted, how retention settings are configured by default, and whether a breach involving model outputs has ever been tested are the questions that determine actual exposure.
The C-suite checklist for any AI vendor evaluation:
| Item | What to Verify |
| BAA status | Signed, covers the specific use case, reviewed by legal |
| Data residency | Confirmed that data does not leave a controlled environment |
| Training corpus | Vendor can document what was and was not included |
| Default retention settings | Reviewed at the account tier the organization actually uses |
| Sandbox or isolation | Queries processed in an isolated environment, not a shared model |
| Breach response | Vendor has a documented protocol for model output exposure |
General-purpose tools require layered policy controls to approach safe use in a healthcare setting: opt-out configuration, employee training, BAA management, and ongoing auditing. Each layer introduces human error risk.
Purpose-built tools remove that stack by resolving the risk at the architecture level.
The question for any compliance officer evaluating AI right now is whether the tool was designed for this data in the first place.
Request a demo to see how Bluesight’s Prism Assistant handles controlled substance data within a secured environment.